An abundance of literature reviews, including systematic reviews, is published each year. How can the reader be sure they are reliable and that, even for experts, conscious and unconscious bias has not crept into the findings and recommendations? Cochrane reviews try to overcome these concerns about reliability and bias.
Cochrane reviews, often long and sometimes impenetrable, are rigorous systematic reviews generated through hard work and, occasionally, tears. These reviews are conducted under the supervision of the Cochrane Collaboration using Cochrane’s review manager software and following a predefined protocol to answer a research question.1
As with all systematic reviews, Cochrane reviews may or may not include a meta-analysis, which uses statistical methods to combine the results of included studies to provide a summary or pooled result. This result should be more precise than the result of any individual included study.1 Further characteristics of systematic reviews are listed in Box 1. The layout of all Cochrane reviews is standardised and starts with the abstract, followed by the plain language summary, summary of findings (SoF) tables, background, methods, results (including study flow diagram and risk of bias table), discussion, authors’ conclusions (including implications for clinical practice and research), graphs, tables and references.
|
Box 1. Characteristics of a systematic review (reproduced from the Cochrane review handbook)1
|
|
The key characteristics of a systematic review are:
-
a clearly stated set of objectives with pre-defined eligibility criteria for studies;
-
an explicit, reproducible methodology;
-
a systematic search that attempts to identify all studies that would meet the eligibility criteria;
-
an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and
-
a systematic presentation, and synthesis, of the characteristics and findings of the included studies.
|
|
Reproduced with permission from Green S, Higgins JPT, Alderson P et al. In: Higgins JPT, Green S (editors), Cochrane handbook for systematic reviews of interventions. Chichester, UK: John Wiley & Sons, 2016.
|
If you find Cochrane reviews boring and difficult to read, then you are in good company. Greenhalgh asks, ‘Why are they are so boring?’2 Greenhalgh applauds the founding and growth of the Cochrane Collaboration, but is concerned that the process of reducing clinical problems to a focused experimental question removes much of the contextual information in which the patient’s presentation to healthcare is situated. Furthermore, objective analysis of the readability of Cochrane reviews revealed that understanding even the plain language summaries required a university education.3 Add to this the description of all the technical processes required to reduce bias and the result can, indeed, be a product that is impenetrable for the end‑user.2
The constant work of the Cochrane Collaboration to reduce sources of bias in their reviews also means they have rightly become one of the most trusted sources of evidence we have for our remedies and management plans in clinical practice. The knowledge produced by Cochrane reviews informs the guidelines that shape our responses to clinical presentations in general practice. Where Cochrane reviews collate empirical evidence to answer a specific research question, guidelines are developed by experts who gather and synthesise the relevant scientific evidence, often including systematic reviews, to make recommendations on clinical care.4 Conducting or participating in the construction of a Cochrane review under the supervision of experienced researchers has become a rite of passage into clinical research for medical students and junior doctors.5 Those who survive the journey learn a great deal about how clinical knowledge is produced, and many continue to do research using their newly acquired skills.5 In this article, we ask: How do we find a Cochrane review when we need one? And, what do we do with a Cochrane review once we have found it?
Clinical scenario
Dr AC, a general practice registrar, has just diagnosed Mr TR, aged 55 years, with onychomycosis of the toenail. The registrar looks up the electronic Therapeutic Guidelines (eTG) at the suggestion of her supervisor and finds that both oral terbinafine and azoles (itraconazole and fluconazole) are recommended.6 The guidelines note that terbinafine is more effective but has more adverse effects. Given this information, Dr AC wonders where she can find information that might be of interest to her patient Mr TR, such as the effectiveness of the treatment and the risk of adverse effects. Her mind turns to the Cochrane Library.
The value of Cochrane reviews
The Cochrane Collaboration notes that Cochrane reviews are internationally recognised as the gold standard for high-quality information.7 Authors of Cochrane reviews often spend months (sometimes years) synthesising all the relevant evidence about a clinical question in the one paper. Each part of the process – including searching, appraising and collating – is peer reviewed and follows a rigorous reporting standard. Reassuringly, the Cochrane Collaboration policy on conflict of interest declares that Cochrane reviews ‘must be independent of conflicts of interest associated with commercial sponsorship’.8
However, Cochrane reviews are not always the best sources for answers to our clinical questions. They generally do not deal with questions of prognosis, aetiology or diagnosis. Furthermore, because they are time-consuming to conduct, there is not always a Cochrane review for every intervention question, and clinicians sometimes must settle for lower level evidence such as a randomised controlled trial.
How to find a Cochrane review
A Google search using the term ‘Cochrane’ takes Dr AC to the Cochrane Collaboration homepage, which has a Cochrane Library tab. On arrival at the library page, she bookmarks ‘www.cochranelibrary.com’. Intuitively, the obvious next step is to type ‘onychomycosis’ into the search bar at the top, and the top hit is the most relevant: ‘Oral antifungal medication for toenail onychomycosis’ (Figure 1).9 An alternative approach is to search through the topic list at the bottom of the Cochrane Library webpage. This review regarding onychomycosis is categorised under the heading ‘Skin Disorders’ and the sub-heading ‘Disorders of hair & nails’.
/Cochrane-reviews/AJGP-06-2018-Focus-van-Driel-Cochrane-Fig-1.jpg.aspx "AJGP-06-2018-Focus-van-Driel-Cochrane-Fig-1.jpg")
Figure 1. Search results for onychomycosis in the Cochrane Library
Sometimes the intuitive approach fails because the search terms used are not ideal or there are too many hits. In this situation, we need a more rigorous approach, such as generating search terms from our question using a population, intervention, comparator and outcome (PICO) approach. For our patient, we are looking at terbinafine (intervention), compared with azoles (comparator) for resolution of onychomycosis (outcome) in adults with onychomycosis (population). The Centre for Evidence-Based Medicine website provides a user-friendly explanation of how to ask focused PICO questions that facilitate finding the evidence for clinical questions relevant to general practice.10 We can use the ‘Advanced Search’ tab on the Cochrane Library home page and add each term to the search manager. Using onychomycosis, terbinafine and azoles, we narrow the search to two hits, both Cochrane reviews. The Cochrane Library is not restricted to Cochrane reviews, but also stores non-Cochrane reviews and randomised controlled trials, which are useful, especially if a Cochrane review is not available for a given topic. More searching information is available under the Help tab on the Cochrane home page, including a document titled ‘How to use the Cochrane Library’, which is particularly informative.11
How to read a Cochrane review
Clinicians approach Cochrane reviews in a variety of ways. Some go straight for the abstract, some for the plain language summary, and some for the conclusions. However, conclusions may be susceptible to author bias. Studies have shown that author conclusions are not always justified by the results, especially if results are not statistically significant.12 An innovation in Cochrane reviews is the SoF table, which has been shown to improve rapid retrieval and understanding of the review results.13,14
Dr AC notes that the abstract states the authors found 48 studies involving 10,200 participants, and she scrolls quickly to the SoF table. SoF tables have a convenient ‘Explanation’ hyperlink (Figure 2), which starts by saying that SoF tables,
provide the findings for the most important outcomes for someone making a decision. These include potential benefits and harms, whether the included studies provide data for these outcomes or not. Additional findings may be reported elsewhere in the review. 15
/Cochrane-reviews/AJGP-06-2018-Focus-van-Driel-Cochrane-Fig-2.jpg.aspx "AJGP-06-2018-Focus-van-Driel-Cochrane-Fig-2.jpg")
Figure 2. Summary of findings table for the Cochrane review, ‘Oral antifungal medication for toenail onychomycosis’
From this SoF table (Figure 2), Dr AC can see the raw data informing the eTG recommendations, and the SoF table facilitates a shared management decision with Mr TR. The clinical cure row shows that 575 out of 1000 participants on terbinafine are cured of onychomycosis (57.5% success), compared with 471 out of 1000 for participants using azoles (47.1% success).9 The difference is 104 out of 1000. When this fraction is inverted, the resulting number needed to treat with terbinafine (over azoles) is just under 10. These results are based on 15 randomised controlled trials involving over 2000 participants. The authors used Grading of Recommendations, Assessment, Development and Evaluation (GRADE) criteria4 to judge the evidence as moderate quality. GRADE assessments are formed using information inputted during the construction of the SoF table to provide an overall assessment of the quality of the evidence for an outcome. As the footnote explains, this outcome lost quality points for including studies that were unblinded and lacked a description of the randomisation process, including allocation concealment. Dr AC notes the ‘adverse events’ row in the same table that shows similar rates of adverse events (346 per 1000). The abstract states that the most common side effects are headaches, viral infections and rashes.9 The next SoF table in the review compares terbinafine with placebo. The placebo results in clinical cure in 62 out of 1000 participants (6.2%). If Mr TR decides not to treat his toenails, they are unlikely to improve by themselves. However, somewhat disappointingly, over 40% of participants did not achieve clinical cure with azoles and terbinafine either.9 After discussing Mr TR’s options and preferences, as well as treatment efficacy, adverse effects, cost and duration, he and Dr AC decide to try terbinafine 250 mg orally, once daily for 12 weeks.
Other ways of learning Cochrane review findings
The Cochrane Collaboration is well aware of the challenges its readers, and particularly busy clinicians, face in accessing the key messages of their reviews. Including consumers on the author teams may enhance relevance to consumers, and having editors who can scrutinise the plain language summary, which is the review’s window to real-life application, will improve readability and comprehension. An increasing number of reviews are accompanied by a ‘podcast’ voicing that summarises the findings of the review and its implications for clinical practice in less than five minutes.16 Podcasts are available in up to 30 languages, facilitating access for non–English speaking health professionals and consumers. For those who prefer a journal format, summaries (usually no more than two printed pages) of some reviews are published in highly respected journals; examples include a summary of topical antibiotics for surgical wounds17 and antifungals for onychomycosis.18 The Cochrane Collaboration has agreements with a number of journals to make dissemination easier. Finally, in addition to SoF tables, some readers find the visual representation of meta-analysis in a forest plot, found in the graphs section of a Cochrane review, useful in making their own judgement of an intervention’s effectiveness. A friendly introduction to Forest Plots has been published by Students 4 Best Evidence.19
Conclusions
Clinicians who invest time in digesting a Cochrane review by reading it, absorbing the SoF table or using an alternative medium, such as a journal summary or podcast, will become familiar with all the relevant research for that clinical question. Furthermore, the Cochrane review gives clinicians valuable clinically relevant information beyond what is available in guidelines; clinicians are therefore better equipped to come to a shared management decision with patients.