Do homeopathy clinical trials lead to reporting bias?

A recently published BMJ paper presents an interesting study.
 
The premise was to determine the magnitude of reporting bias in clinical trials of homeopathy by examining the differences in efficacy between trials that are unregistered, compared to trials that are registered, and also examining the proportion of trials that remained unpublished.
 
In 2005, the policy of only publishing trials that were prospectively registered on a transparent trial registry was adopted. The authors reported that overall, only 46.6% of published trials on homeopathy were registered.
 
The authors then took data from a 2017 systematic review and re-analysed it according to whether trials were registered or unregistered. The effect size from unregistered trials was large, and from registered trials was small and not statistically significant.
 
The authors conclude that the true effect of homeopathy may be overestimated by unregistered trials.
 
The issue of unregistered and unpublished trials is not unique to homeopathy and the authors have acknowledged this in their discussion.
 
An analysis of trials in 2012−14 showed that a third were retrospectively registered. In the BMJ paper, about half were retrospectively registered.
 
It is clear that we need to pay attention to the methodological quality of all of the evidence that is presented to us as clinicians, in order to determine how confident we are that the evidence presents the truth.
 
It is encouraging to see a statistically significant trend in increasing registration rates over time, with the trend line at 75% registered in 2021.
 
Accounting for varying trial quality is usually done by a formal risk of bias assessment of individual trials, and then using Grading of Recommendations Assessment, Development and Evaluation (GRADE).
 
Using the GRADE process would allow for issues such as non-registration and retrospective registration of trials to be considered when making a recommendation around the certainty of evidence.
 
It’s curious that the authors did not take this widely used and gold-standard approach to their meta-analysis. Further, they did not extract data from recent trials themselves; they chose to use data from a previously published meta-analysis which was published in 2017 and searched trials that were published up to 2014.
 
Given it can take several years to publish results, these findings are very out of date as trials may have been completed in 2011 or 10 years ago. A far better approach would have been to update the meta-analysis then do a sensitivity analysis excluding unregistered trials.
 
It’s definitely an unusual approach the authors have taken and not consistent with usual practice with meta-analyses.
 
It is also important to realise that their meta-analysis was only based on a small (19) subset of trials between 2002−14 and does not reflect the overall body of evidence that may have been published up until 2022.
 
Nevertheless, their findings, and those of the original meta-analysis are important.
 
The original 2017 meta-analysis reported that the overall quality of the body of evidence was low. Analysis of a subset of trials that were considered to be reliable evidence did not find a difference between homeopathy and placebo.
 
This meta-analysis pooled findings from 75 trials that looked at 48 different clinical conditions.
 
When they conducted sub-group analyses, a statistically significant effect was observed for influenza, irritable bowel syndrome and seasonal allergic rhinitis. Once the lowest rated trials (ie the poorest quality trials) were removed, no effect was seen.
 
What does this mean for GPs?
The evidence from clinical trials on homeopathy suggests that the inclusion of poor-quality trials over-inflates the reported efficacy, and analyses of higher quality trials − those without significant reporting bias − has not detected a difference between groups.
 
It’s important to note that these trials used non-individualised homeopathy, and that the evidence presented by the BMJ paper authors is not the most up to date given they decided to re-analyse data from a 2017 meta-analysis, which searched for trials published up until 2014.
 
Should we be discussing homeopathy use with our patients? Absolutely.
 
We should be taking a patient-centred and non-judgemental approach whenever patients disclose use of any non-medical therapies.
 
The RACGP’s position statement on homeopathy ‘supports the concept of patient-centred practice, in which the values, preferences, and personal healthcare philosophy of the patient are respected and individuals play an important role in their own healthcare’.
 
It goes on to say that it is important that patients feel comfortable discussing the use of complementary medicines with their treatment team, and that medical practitioners should ‘initiate conversations with patients about their use of or intention to use homeopathy, and assist patients to think critically about the efficacy and safety of homeopathy so that they may make informed healthcare decisions’. 
 
It is important to remember that for some patients, for example people from India, homeopathy may have been a well-accepted therapy in their home country.
 
GPs can play a vital role in helping patients make properly informed decisions about healthcare, and being able to interpret and convey the evidence clearly is key.
 
In this case, evidence from high-quality trials published prior to 2014 does not show a difference compared to placebo.
 
We should be presenting this information to our patients in the process of assisting them to make evidence-informed decisions – but it’s important to do this respectfully, taking into account their values and preferences.
 
When we engage with our patients in a collaborative and non-judgemental way, we are providing optimal care and building trust. This trust is crucial in creating a successful doctor−patient relationship and improving health outcomes as a result.
 
Log in below to join the conversation.

clinical trials complementary medicine homeopathy reporting bias