Professional
Volume 47, Issue 6, June 2018

Unlocking the value of Cochrane reviews for general practitioners

Geoffrey Spurling    Ben Mitchell    Mieke L van Driel   
doi: 10.31128/AJGP-01-18-4477   |    Download article
Cite this article    BIBTEX    REFER    RIS

Background
Cochrane reviews are difficult to construct and may be difficult to read, but they produce trusted, high-quality research responses to common clinical questions.
Objective
The objective of this article is to help clinicians navigate the Cochrane Library and Cochrane reviews. Using a common clinical scenario, we illustrate how to find the information required to guide evidence-based decision making with patients.
Discussion
Clinicians looking for answers to clinical questions often turn first to guidelines. However, these may not provide enough background to balance the pros and cons of a treatment. Cochrane reviews often inform guidelines and contain more in-depth clinical information for shared decision making. The introduction of Summary of Findings (SoF) tables has made the studies in Cochrane reviews more accessible. In this paper, we discuss how to read and interpret these SoF tables. Additional resources, such as journal summaries and podcasts, have also improved the accessibility of Cochrane review findings.
ArticleImage

An abundance of literature reviews, including systematic reviews, is published each year. How can the reader be sure they are reliable and that, even for experts, conscious and unconscious bias has not crept into the findings and recommendations? Cochrane reviews try to overcome these concerns about reliability and bias.

Cochrane reviews, often long and sometimes impenetrable, are rigorous systematic reviews generated through hard work and, occasionally, tears. These reviews are conducted under the supervision of the Cochrane Collaboration using Cochrane’s review manager software  and following a predefined protocol to answer a research question.1

As with all systematic reviews, Cochrane reviews may or may not include a meta-analysis, which uses statistical methods to combine the results of included studies to provide a summary or pooled result. This result should be more precise than the result of any individual included study.1 Further characteristics of systematic reviews are listed in Box 1. The layout of all Cochrane reviews is standardised and starts with the abstract, followed by the plain language summary, summary of findings (SoF) tables, background, methods, results (including study flow diagram and risk of bias table), discussion, authors’ conclusions (including implications for clinical practice and research), graphs, tables and references.
 

Box 1. Characteristics of a systematic review (reproduced from the Cochrane review handbook)1

The key characteristics of a systematic review are:

  • a clearly stated set of objectives with pre-defined eligibility criteria for studies;

  • an explicit, reproducible methodology;

  • a systematic search that attempts to identify all studies that would meet the eligibility criteria;

  • an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and

  • a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

Reproduced with permission from Green S, Higgins JPT, Alderson P et al. In: Higgins JPT, Green S (editors), Cochrane handbook for systematic reviews of interventions. Chichester, UK: John Wiley & Sons, 2016.


If you find Cochrane reviews boring and difficult to read, then you are in good company. Greenhalgh asks, ‘Why are they are so boring?’2 Greenhalgh applauds the founding and growth of the Cochrane Collaboration, but is concerned that the process of reducing clinical problems to a focused experimental question removes much of the contextual information in which the patient’s presentation to healthcare is situated. Furthermore, objective analysis of the readability of Cochrane reviews revealed that understanding even the plain language summaries required a university education.3 Add to this the description of all the technical processes required to reduce bias and the result can, indeed, be a product that is impenetrable for the end‑user.2

The constant work of the Cochrane Collaboration to reduce sources of bias in their reviews also means they have rightly become one of the most trusted sources of evidence we have for our remedies and management plans in clinical practice. The knowledge produced by Cochrane reviews informs the guidelines that shape our responses to clinical presentations in general practice. Where Cochrane reviews collate empirical evidence to answer a specific research question, guidelines are developed by experts who gather and synthesise the relevant scientific evidence, often including systematic reviews, to make recommendations on clinical care.4 Conducting or participating in the construction of a Cochrane review under the supervision of experienced researchers has become a rite of passage into clinical research for medical students and junior doctors.5 Those who survive the journey learn a great deal about how clinical knowledge is produced, and many continue to do research using their newly acquired skills.5 In this article, we ask: How do we find a Cochrane review when we need one? And, what do we do with a Cochrane review once we have found it?

Clinical scenario

Dr AC, a general practice registrar, has just diagnosed Mr TR, aged 55 years, with onychomycosis of the toenail. The registrar looks up the electronic Therapeutic Guidelines (eTG) at the suggestion of her supervisor and finds that both oral terbinafine and azoles (itraconazole and fluconazole) are recommended.6 The guidelines note that terbinafine is more effective but has more adverse effects. Given this information, Dr AC wonders where she can find information that might be of interest to her patient Mr TR, such as the effectiveness of the treatment and the risk of adverse effects. Her mind turns to the Cochrane Library.

The value of Cochrane reviews

The Cochrane Collaboration notes that Cochrane reviews are internationally recognised as the gold standard for high-quality information.7 Authors of Cochrane reviews often spend months (sometimes years) synthesising all the relevant evidence about a clinical question in the one paper. Each part of the process – including searching, appraising and collating – is peer reviewed and follows a rigorous reporting standard. Reassuringly, the Cochrane Collaboration policy on conflict of interest declares that Cochrane reviews ‘must be independent of conflicts of interest associated with commercial sponsorship’.8

However, Cochrane reviews are not always the best sources for answers to our clinical questions. They generally do not deal with questions of prognosis, aetiology or diagnosis. Furthermore, because they are time-consuming to conduct, there is not always a Cochrane review for every intervention question, and clinicians sometimes must settle for lower level evidence such as a randomised controlled trial.

How to find a Cochrane review

A Google search using the term ‘Cochrane’ takes Dr AC to the Cochrane Collaboration homepage, which has a Cochrane Library tab. On arrival at the library page, she bookmarks ‘www.cochranelibrary.com’. Intuitively, the obvious next step is to type ‘onychomycosis’ into the search bar at the top, and the top hit is the most relevant: ‘Oral antifungal medication for toenail onychomycosis’ (Figure 1).9 An alternative approach is to search through the topic list at the bottom of the Cochrane Library webpage. This review regarding onychomycosis is categorised under the heading ‘Skin Disorders’ and the sub-heading ‘Disorders of hair & nails’.


AJGP-06-2018-Focus-van-Driel-Cochrane-Fig-1.jpg
Figure 1. Search results for onychomycosis in the Cochrane Library

Sometimes the intuitive approach fails because the search terms used are not ideal or there are too many hits. In this situation, we need a more rigorous approach, such as generating search terms from our question using a population, intervention, comparator and outcome (PICO) approach. For our patient, we are looking at terbinafine (intervention), compared with azoles (comparator) for resolution of onychomycosis (outcome) in adults with onychomycosis (population). The Centre for Evidence-Based Medicine website provides a user-friendly explanation of how to ask focused PICO questions that facilitate finding the evidence for clinical questions relevant to general practice.10 We can use the ‘Advanced Search’ tab on the Cochrane Library home page and add each term to the search manager. Using onychomycosis, terbinafine and azoles, we narrow the search to two hits, both Cochrane reviews. The Cochrane Library is not restricted to Cochrane reviews, but also stores non-Cochrane reviews and randomised controlled trials, which are useful, especially if a Cochrane review is not available for a given topic. More searching information is available under the Help tab on the Cochrane home page, including a document titled ‘How to use the Cochrane Library’, which is particularly informative.11

How to read a Cochrane review

Clinicians approach Cochrane reviews in a variety of ways. Some go straight for the abstract, some for the plain language summary, and some for the conclusions. However, conclusions may be susceptible to author bias. Studies have shown that author conclusions are not always justified by the results, especially if results are not statistically significant.12 An innovation in Cochrane reviews is the SoF table, which has been shown to improve rapid retrieval and understanding of the review results.13,14

Dr AC notes that the abstract states the authors found 48 studies involving 10,200 participants, and she scrolls quickly to the SoF table. SoF tables have a convenient ‘Explanation’ hyperlink (Figure 2), which starts by saying that SoF tables,

provide the findings for the most important outcomes for someone making a decision. These include potential benefits and harms, whether the included studies provide data for these outcomes or not. Additional findings may be reported elsewhere in the review. 15
 


AJGP-06-2018-Focus-van-Driel-Cochrane-Fig-2.jpg
Figure 2.
Summary of findings table for the Cochrane review, ‘Oral antifungal medication for toenail onychomycosis’

From this SoF table (Figure 2), Dr AC can see the raw data informing the eTG recommendations, and the SoF table facilitates a shared management decision with Mr TR. The clinical cure row shows that 575 out of 1000 participants on terbinafine are cured of onychomycosis (57.5% success), compared with 471 out of 1000 for participants using azoles (47.1% success).9 The difference is 104 out of 1000. When this fraction is inverted, the resulting number needed to treat with terbinafine (over azoles) is just under 10. These results are based on 15 randomised controlled trials involving over 2000 participants. The authors used Grading of Recommendations, Assessment, Development and Evaluation (GRADE) criteria4 to judge the evidence as moderate quality. GRADE assessments are formed using information inputted during the construction of the SoF table to provide an overall assessment of the quality of the evidence for an outcome. As the footnote explains, this outcome lost quality points for including studies that were unblinded and lacked a description of the randomisation process, including allocation concealment. Dr AC notes the ‘adverse events’ row in the same table that shows similar rates of adverse events (346 per 1000). The abstract states that the most common side effects are headaches, viral infections and rashes.9 The next SoF table in the review compares terbinafine with placebo. The placebo results in clinical cure in 62 out of 1000 participants (6.2%). If Mr TR decides not to treat his toenails, they are unlikely to improve by themselves. However, somewhat disappointingly, over 40% of participants did not achieve clinical cure with azoles and terbinafine either.9 After discussing Mr TR’s options and preferences, as well as treatment efficacy, adverse effects, cost and duration, he and Dr AC decide to try terbinafine 250 mg orally, once daily for 12 weeks.

Other ways of learning Cochrane review findings

The Cochrane Collaboration is well aware of the challenges its readers, and particularly busy clinicians, face in accessing the key messages of their reviews. Including consumers on the author teams may enhance relevance to consumers, and having editors who can scrutinise the plain language summary, which is the review’s window to real-life application, will improve readability and comprehension. An increasing number of reviews are accompanied by a ‘podcast’ voicing that summarises the findings of the review and its implications for clinical practice in less than five minutes.16 Podcasts are available in up to 30 languages, facilitating access for non–English speaking health professionals and consumers. For those who prefer a journal format, summaries (usually no more than two printed pages) of some reviews are published in highly respected journals; examples include a summary of topical antibiotics for surgical wounds17 and antifungals for onychomycosis.18 The Cochrane Collaboration has agreements with a number of journals to make dissemination easier. Finally, in addition to SoF tables, some readers find the visual representation of meta-analysis in a forest plot, found in the graphs section of a Cochrane review, useful in making their own judgement of an intervention’s effectiveness. A friendly introduction to Forest Plots has been published by Students 4 Best Evidence.19

Conclusions

Clinicians who invest time in digesting a Cochrane review by reading it, absorbing the SoF table or using an alternative medium, such as a journal summary or podcast, will become familiar with all the relevant research for that clinical question. Furthermore, the Cochrane review gives clinicians valuable clinically relevant information beyond what is available in guidelines; clinicians are therefore better equipped to come to a shared management decision with patients.

Competing interests: Dr van Driel is an (unpaid) editor for the Cochrane Acute Respiratory Infections Group.
Provenance and peer review: Commissioned, externally peer reviewed.
References
  1. Cochrane Collaboration. Cochrane Handbook for systematic reviews of interventions. Chichester, UK: John Wiley & Sons, 2011. Available at http://handbook-5-1.cochrane.org [Accessed 1 March 2018]. Search PubMed
  2. Greenhalgh T. Outside the box: Why are Cochrane reviews so boring? Br J Gen Pract 2012;62(600):371. doi: 10.3399/bjgp12X652418. Search PubMed
  3. Luis C, Jeffrey G, Hickey B. Readability of Cochrane Breast Cancer Group plain language summaries. Better Knowledge for Better Health | Un meilleur savoir pour une meilleure santé. Abstracts of the 21st Cochrane Colloquium. Quebec: John Wiley & Sons, 2013. Search PubMed
  4. van Driel ML, Spurling G. Guidelines and systematic reviews: Sizing up guidelines in general practice. Aust Fam Physician 2017;46(6):438–40. Search PubMed
  5. van Driel M. Growing research – Involving students in Cochrane reviews. Aust Fam Physician 2011;40(10):803. Search PubMed
  6. Expert Group for Dermatology. Onychomycosis (tinea of the nails). In: eTG complete [Internet].Melbourne: Therapeutic Guidelines Limited, 2018. Search PubMed
  7. Cochrane Collaboration. What is Cochrane evidence and how can it help you? London: Cochrane Collaboration, 2018. Available at www.cochrane.org/what-is-cochrane-evidence [Accessed 1 January 2018]. Search PubMed
  8. Cochrane Collaboration. Conflicts of interest and Cochrane reviews. London: Cochrane Collaboration, 2018. Available at http://community.cochrane.org/editorial-and-publishing-policy-resource/ethical-considerations/conflicts-interest-and-cochrane-reviews [Accessed 1 January 2018]. Search PubMed
  9. Kreijkamp-Kaspers S, Hawke K, Guo L, et al. Oral antifungal medication for toenail onychomycosis. Cochrane Database Syst Rev 2017;7:CD010031. doi: 10.1002/14651858.CD010031.pub2. Search PubMed
  10. Centre for Evidence-Based Medicine. Asking focused questions. Oxford: University of Oxford, 2018. Available at www.cebm.net/2014/06/asking-focused-questions [Accessed 1 March 2018]. Search PubMed
  11. Cochrane Collaboration. How to use the Cochrane Library. London: John Wiley & Sons, 2018. Available at http://olabout.wiley.com/WileyCDA/Section/id-390001.html [Accessed 1 January 2018]. Search PubMed
  12. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 2010;303(20):2058–64. doi: 10.1001/jama.2010.651. Search PubMed
  13. Wallace J, Byrne C, Clarke M. Improving the uptake of systematic reviews: A systematic review of intervention effectiveness and relevance. BMJ Open 2014;4(10):e005834. doi: 10.1136/bmjopen-2014-005834. Search PubMed
  14. Rosenbaum SE, Glenton C, Oxman AD. Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. J Clin Epidemiol 2010;63(6):620–26. doi: 10.1016/j.jclinepi.2009.12.014. Search PubMed
  15. Cochrane Collaboration. Explanations for Cochrane Summary of Findings tables. London: John Wiley & Sons, 2018. Available at www.cochranelibrary.com/about/explanations-for-cochrane-summary-of-findings-sof-tables.html [Accessed 1 January 2018]. Search PubMed
  16. Cochrane Collaboration. The latest Cochrane podcasts. London: Cochrane Collaboration, 2018. Available at www.cochrane.org/multimedia/podcasts [Accessed 1 January 2018]. Search PubMed
  17. Heal CF, Banks JL, Lepper P, Kontopantelis E, van Driel ML. Meta-analysis of randomized and quasi-randomized clinical trials of topical antibiotics after primary closure for the prevention of surgical-site infection. Br J Surg 2017;104(9):1123–30. doi: 10.1002/bjs.10588. Search PubMed
  18. Kreijkamp-Kaspers S, Hawke K, van Driel ML. Oral medications to treat toenail fungal infection. JAMA 2018;319(4):397–98. doi: 10.1001/jama.2017.20160. Search PubMed
  19. Hung TQ. How to read a Forest Plot. Oxford: Students 4 Best Evidence, 2016. Available at www.students4bestevidence.net/forest-plot/?utm_content=buffer050f4&utm_medium=social&utm_source=facebook.com&utm_campaign=buffer [Accessed 1 March 2018]. Search PubMed

Cochraneevidence-based medicine

Download article