General practice research addresses gaps in evidence regarding aspects of care delivered by general practitioners (GPs) and general practices. It encompasses all health professionals working within this setting and can be considered a subsection of primary care research.1,2 While 85% of the Australian population visit their GPs annually, general practice research is underrepresented in the amount of research funding it receives when compared with tertiary healthcare settings.1,3 Research conducted in tertiary care may not generate findings applicable to general practice patients.1,4–6 However, it is crucial that clinical practice be informed by general practice research so that GPs have tools and guidelines appropriate for their patients.3,7 Barriers and enablers to conducting general practice research have been documented;8 these barriers can be overcome through collaboration with academic departments and larger centres, particularly for larger studies.9 Research about issues that GPs consider to be of clinical importance, and those that their patients consider to be relevant, is more likely to succeed.9
The aim of this study was to establish a set of general practice research priorities to guide resource allocation and inform a general practice research agenda that facilitates delivery of optimal patient care.
Method
An ‘internal expert group’ of GPs who conduct research provided broad project guidance, while a smaller subgroup led the project and made methodological decisions.
A literature review was initially conducted to determine the different methodological approaches used to establish research priorities.10–14 The evidence-based Delphi method was selected. Its approach facilitates effective group communication between experts that addresses the complexity of identifying priority areas in healthcare research.15,16 The Delphi method entails the structuring of a group of experts to address a complex problem asynchronously, often via the anonymity and relative convenience of an online platform, in an iterative process that elicits the views of the expert panel involved.15,16 It discloses how participant peers responded to the same information in the previous survey/s, providing participants with an opportunity to reconsider their decisions.15,16 After two or more survey rounds, a consensus is reached.15,16
The Delphi method was previously used in Australia to identify general practice research priorities.14 It has also been used to assess educational needs of GPs17 and to develop core competencies in evidence-based practice for health professionals.18 The Delphi process is commonly modified and, on the basis of available evidence, the researchers selected a modified two-round Delphi study as most appropriate for this study.19–22
Delphi method
The approach used in this study differed slightly from the classically described Delphi method, where a qualitative first- round survey may occur, followed by more than one round of surveying.15,16 In place of the qualitative first round, the researchers conducted a literature review of current published evidence to establish a master list of priorities.21 The use of a literature review in place of the qualitative first round has been documented as an acceptable alternative and was fit for purpose in view of time and funding limitations.19–21
The Royal Australian College of General Practitioners (RACGP) National Research and Evaluation Ethics Committee (NREEC) granted ethics approval on 19 April 2018 (NREEC No. 18-005).
Construction of a master list of priorities
To align with the study aims, the scope of the project was limited to ‘general practice’ rather than ‘primary care’ research. The researchers also integrated a horizontal (system-wide) and vertical (disease-related) approach to establishing a list of master priorities. First, a list of priorities was compiled, then the results were cross-referenced to identify relevant items. Sources included National Health and Medical Research Council research priorities;23 Primary Health Care Research, Evaluation and Development research priorities;14 and RACGP Foundation grant topics. Frequently presenting conditions in general practice were also considered,24–26 as were the recently determined top 10 international primary care research priorities.27 The internal expert group then reviewed the list of priorities.
A total of 63 priority items were identified and categorised under the following groups: disease-related, population health, process of care, healthcare management and other.
Selection of expert panel members
A set of accepted criteria – experience, knowledge and policy influence – was used as inclusion criteria for selecting the 83 expert panel invitees.28 The researchers also considered the general practice stakeholders most likely affected by the decision-making process. The resultant expert panel members included funders; RACGP members, (then) council and representatives of expert committees; representatives from university departments of general practice; representatives from other general practice organisations; general practice education providers; representatives from primary care networks; allied health including pharmacy and nursing; rural health, carer and consumer organisations; and community and philanthropic organisations.
Questionnaire
For the first-round survey, external expert panellists were emailed invitations to participate. Consent was implied if invitees chose to participate. They were asked to rate the 63 items as ‘very important’, ‘desirable’ or ‘non-essential’ in the context of the need for general practice–based research to improve patient care, and on the basis of the management of the item in general practice. In addition to rating items, panellists were asked to provide qualitative comments about the wording of items and nominate additional items that should be included in the list. It was determined that the items reaching a threshold of >69% or <30% on the ‘very important’ rating would be considered as reaching consensus for high or not high priority respectively and would be excluded from the second round.15,20,21
The validity and usefulness of the final list of items was discussed at a meeting of current and past members of the RACGP Expert Committee – Research (REC-R) acting as a consensus group.
Results
Response rate
Twenty-three of 83 invitees completed the first-round survey (27.7%), and 28 of 78 completed the second round (35.9%). Five of the original invitees were removed from the invitation list for the second-round survey as each indicated they did not wish to participate in the survey process.
Ten respondents completed only survey one, 15 respondents completed only survey two, and 13 respondents completed both surveys (a total of 38 different respondents). Appendix 1 outlines information about the respondents’ roles/affiliations.
Of the original 63 items in the first-round survey, 17 items were excluded from the second-round survey because they reached the consensus for high priority in the first round (>69%), and 12 items were excluded because they reached consensus for not high priority (<30%) in the first round (Figure 1). These items were returned to the final list following the second survey. The remainder of the items were re-scored in the second round. On the basis of external panellist feedback, four new items were added to the second-round survey, and four items from the first-round survey were rephrased and included in the second-round survey for re-rating (Figure 1).
Figure 1. Flowchart of the Delphi process to identify general practice research priorities
Priority items
Priority items are presented as a combined group in rank order in Table 1. The top 21 items are highlighted. Items by group are presented in Tables 2, 3, 4, 5 and 6.
| Table 1. Ranked combined priorities |
|
Overall priority number
(Items 1–21 of 67)
|
Item |
| 1 |
Quality of care |
| 2 |
Evidence-based practice |
| 3 |
Models of primary care delivery |
| 4 |
Consumer focus |
| 5 |
Multimorbidity management |
| 6 |
Mental health |
| 7 |
Collaborative care |
| 8 |
Avoiding hospitalisations |
| 9 |
Chronic pain |
| 10 |
Quality use of medicines |
| 11 |
Use of technology in primary care delivery |
| 12 |
Alcohol and substance abuse disorders |
| 13 |
Aged care and ageing |
| 14 |
Dementia (new listing in round two) |
| 15 |
Obesity |
| 16 |
Health promotion and illness prevention |
| 17 |
Social determinants of health |
| 18 |
Aboriginal and Torres Strait Islander peoples |
| 19 |
Family violence |
| 20 |
Non-pharmacological treatments (including exercise and counselling techniques such as active listening) |
| 21 |
Use of electronic data (eHealth records, data linkage) |
| Table 2. Disease-related priorities |
Group priority number
(1–10 of 24 items) |
Item |
| 1 |
Mental health |
| 2 |
Chronic pain |
| 3 |
Alcohol and substance abuse disorders |
| 4 |
Dementia (new listing in this group in round two) |
| 5 |
Obesity |
| 6 |
Cardiovascular health (including hypertension, management of cardiovascular disease) |
| 7 |
Cerebrovascular health (including stroke) |
| 8 |
Diabetes mellitus (including diabetic retinopathy) |
| 9 |
Cancer (including diagnosis and treatment cancer, cancer survivor support) |
| 10 |
Arthritis and musculoskeletal conditions (including back pain, sprain, tendonitis, osteoporosis, fracture management and prevention) |
| Table 3. Process of care priorities |
Group priority number
(1–5 of 7 items) |
Item |
| 1 |
Evidence-based practice |
| 2 |
Collaborative care |
| 3 |
Health promotion and illness prevention |
| 4 |
Non-pharmacological treatments (including exercise and counselling techniques such as active listening) |
| 5 |
Antimicrobial stewardship in primary care (new listing in round two) |
| Table 4. Population health priorities |
Group priority number
(1–5 of 13 items) |
Item |
| 1 |
Aged care and ageing |
| 2 |
Social determinants of health |
| 3 |
Aboriginal and Torres Strait Islander peoples |
| 4 |
Family violence |
| 5 |
Rural and remote populations (including telehealth) |
| Table 5. Healthcare management priorities |
Group priority number
(1–5 of 19 items) |
Item |
| 1 |
Quality of care |
| 2 |
Models of primary care delivery |
| 3 |
Consumer focus |
| 4 |
Multimorbidity management |
| 5 |
Avoiding hospitalisations |
| Table 6. Other general practice issues |
Group priority number
(All items) |
Item |
| 1 |
Use of electronic data (eHealth records, data linkage) |
| 2 |
Data extraction tools, use of routinely collected data (Medicare Benefits Scheme, Pharmaceutical Benefits Schedule, National Prescribing Service, MedicineWise, MedicineInsight) |
| 3 |
General practice coding systems including Doctor Command Language (DOCLE), which translates terminology used in general practice into code (new listing in this group in round two) |
| 4 |
Virtual online communities of practice |
Consensus group meeting
A consensus meeting identified several concerns including perceived low response rate and low absolute number of expert panel members, which could produce results that did not reflect true opinion. The group also noted that GPs and others might mistakenly assume that lower ranked topics were considered of lesser importance by participants, which could result in negative press. They also observed that a Delphi process essentially applies a quantitative score to a process that in some respects requires a qualitative approach.15,16 Some internal panel members reflected that their concerns about the Delphi survey results may be at least partially influenced by their relative inexperience with the Delphi process, and that this response might be replicated by others reading the results if they are unfamiliar with the Delphi process and its legitimate applications.
As a result of the consensus group meeting, it was decided to present the items in rank order rather than revealing scores. It was also decided there should be no set number of priority items. However, as there was a drop in the percentage score after 21 items, forming a natural gap in the data, these items were highlighted.
Discussion
Establishing a list of research priorities is important for informing a general practice research agenda, prioritising funding and, ultimately, optimising the delivery of best patient care.
Highly ranked items are broad in terms of research scope. Quality of care, for instance, encompasses many aspects of general practice, as do ‘evidence-based practice’, ‘models of primary care delivery’ and ‘multimorbidity’. It is unsurprising, given the broad scope of work undertaken in general practice, that population health topics such as ‘health promotion and illness prevention’ and ‘social determinants of health’ were ranked highly. Disease-related topics that ranked highly included mental health, chronic pain, alcohol and substance abuse disorders, aged care and ageing, dementia and obesity; such conditions reflect the burden of disease in Australia.29,30 Many research items that had a lower rank could be re-categorised or reframed in the context of one of the top-priority topics. For instance, antimicrobial stewardship could be considered part of ‘quality care of medicines’. Other topics are contextually relevant in the conduct of all types of research, for instance, ‘consumer focus’, ‘rural and remote’ and ‘practice-based research networks’. In the evolving world of electronic data, use of technology in primary care delivery, as well as use of routinely collected general practice data, ranked highly.
A recent study identified the top 10 international primary care research priorities; however, in contrast to the methodology used in the current study, this study grouped topics together under broad categories.27 Although most of the top 10 international primary care research priorities featured within the top 21 priorities considered in this study, the differing methodologies might account for some differences between the two priority listings.
When all 67 items were grouped together and presented in rank order, many ‘process of care’ items appeared near the top of the list, compared with ‘disease-related’ items. This may be a reflection of the number and diversity of disease-related topics in comparison to process of care topics, causing dilution in consensus. It is important, therefore, to consider the list of priorities in the context of the original groups in which they had been categorised, as well as in combination.
The Delphi method can answer questions that may not be possible or feasible with alternative methodologies.31 Consensus methods, which are based on expert opinion, are considered a weaker type of evidence than, for example, the gold standard consistent randomised controlled trial in the hierarchy of evidence-based medicine.32 This view is, however, an oversimplification and underestimation of the value of expert opinion and the consensus method applied in an appropriate context.
Viewed in the context of the entire study, the internal panel considered all items, which were identified by sourcing the best available evidence, to be important topics. However, the Delphi process was used to identify topics the external panellists considered important priority areas for general practice research, rather than topics of global importance.
Strengths and limitations
The diversity of research topics and the variety of contexts in which they may apply make it challenging to produce a priority list of research topics for Australian general practice. Although the research comprises different views from a range of general practice research stakeholders, they may not adequately represent the full spectrum of views held by the individuals whom they represent. Participants may also have particular research interests that could have affected their prioritisation of topics.
Furthermore, although only 23 of 83 invitees took part in round one and 28 of 78 in round two, the final number of participants was adequate for this method of research.20 There are no firm guidelines on the size of a Delphi panel;33 however, findings are more likely to be stable with a larger panel, with individual responses having less influence on overall findings. In one study, a panel of 23 experts produced stable results;34 others have shown stability with rounds of approximately ≥20 members.35,36 When considering the absolute number of responses involved and the response rate, it should be remembered that this is an expert panel consensus method rather than a cross-sectional survey. Participants were chosen purposefully, and each identified stakeholder represented a larger group. Unlike a cross-sectional survey, the data were not analysed using descriptive or inferential statistics, and therefore rules regarding absolute numbers responding, response rates and the representativeness of the responders do not apply.
Conclusion
Identifying current Australian general practice research priorities is complicated as the methodological process and results will be scrutinised by a wide variety of stakeholders, some of whom may conclude that their particular views are not captured.
The list of priorities identified in the modified Delphi study is not exhaustive, but it does reflect the input of a diverse range of stakeholders and highlights both very broad and quite specific areas of research for Australian general practice.
The researchers were satisfied that the Delphi method chosen was the evidence-based approach best suited to identify the priorities.
Implications for general practice
The results of the study will provide a useful reference point to inform a research agenda, prioritise funding and, ultimately, optimise the ability of Australian general practice research to deliver the best patient care on the basis of current evidence.
The results are under consideration by the Commonwealth to help inform funding decisions regarding future medical research.
| Appendix 1. Delphi respondent affiliations, rounds one and two |
| Role/affiliation |
Round one |
Round two |
| GP academic |
14 |
16 |
| RACGP |
1 |
3 |
| Primary Health Network |
0 |
1 |
| Consumer organisation |
1 |
1 |
| Allied health |
1 |
2 |
| CALD population representation |
2 |
2 |
| Funder |
2 |
0 |
| GP educator |
2 |
2 |
| Rural representation |
0 |
1 |
| CALD, culturally and linguistically diverse; GP, general practitioner; RACGP, Royal Australian College of General Practitioners |