Research
Volume 48, Issue 11, November 2019

General practice research priority setting in Australia: Informing a research agenda to deliver best patient care

Clare Heal    Gail Roberts   
doi: 10.31128/AJGP-05-19-4928   |    Download article
Cite this article    BIBTEX    REFER    RIS

Background
General practice research is the subsection of primary care research that addresses gaps in evidence about care delivered in general practice. Despite delivering care to 85% of the Australian population annually, general practice receives a paucity of government health research funding when compared with tertiary healthcare settings. However, general practitioners (GPs) require evidence-based tools and guidelines applicable to their patients.
Objective
The aim of this study was to establish a set of general practice research priorities to guide resource allocation, and to inform a research agenda that optimises the delivery of the best patient care.
Methods
A comprehensive literature review was conducted, followed by a modified two-round Delphi survey of general practice stakeholders.
Results
Sixty-seven general practice research priorities were identified. Priorities included quality care and evidence-based practice.
Discussion
The results provide a contemporary reference point for an Australian general practice research agenda that helps prioritise and advocate for funding, and enables delivery of evidence-based patient care.
 

General practice research addresses gaps in evidence regarding aspects of care delivered by general practitioners (GPs) and general practices. It encompasses all health professionals working within this setting and can be considered a subsection of primary care research.1,2 While 85% of the Australian population visit their GPs annually, general practice research is underrepresented in the amount of research funding it receives when compared with tertiary healthcare settings.1,3 Research conducted in tertiary care may not generate findings applicable to general practice patients.1,4–6 However, it is crucial that clinical practice be informed by general practice research so that GPs have tools and guidelines appropriate for their patients.3,7 Barriers and enablers to conducting general practice research have been documented;8 these barriers can be overcome through collaboration with academic departments and larger centres, particularly for larger studies.9 Research about issues that GPs consider to be of clinical importance, and those that their patients consider to be relevant, is more likely to succeed.9

The aim of this study was to establish a set of general practice research priorities to guide resource allocation and inform a general practice research agenda that facilitates delivery of optimal patient care.


Method

An ‘internal expert group’ of GPs who conduct research provided broad project guidance, while a smaller subgroup led the project and made methodological decisions.

A literature review was initially conducted to determine the different methodological approaches used to establish research priorities.10–14 The evidence-based Delphi method was selected. Its approach facilitates effective group communication between experts that addresses the complexity of identifying priority areas in healthcare research.15,16 The Delphi method entails the structuring of a group of experts to address a complex problem asynchronously, often via the anonymity and relative convenience of an online platform, in an iterative process that elicits the views of the expert panel involved.15,16 It discloses how participant peers responded to the same information in the previous survey/s, providing participants with an opportunity to reconsider their decisions.15,16 After two or more survey rounds, a consensus is reached.15,16

The Delphi method was previously used in Australia to identify general practice research priorities.14 It has also been used to assess educational needs of GPs17 and to develop core competencies in evidence-based practice for health professionals.18 The Delphi process is commonly modified and, on the basis of available evidence, the researchers selected a modified two-round Delphi study as most appropriate for this study.19–22

Delphi method

The approach used in this study differed slightly from the classically described Delphi method, where a qualitative first- round survey may occur, followed by more than one round of surveying.15,16 In place of the qualitative first round, the researchers conducted a literature review of current published evidence to establish a master list of priorities.21 The use of a literature review in place of the qualitative first round has been documented as an acceptable alternative and was fit for purpose in view of time and funding limitations.19–21

The Royal Australian College of General Practitioners (RACGP) National Research and Evaluation Ethics Committee (NREEC) granted ethics approval on 19 April 2018 (NREEC No. 18-005).

Construction of a master list of priorities

To align with the study aims, the scope of the project was limited to ‘general practice’ rather than ‘primary care’ research. The researchers also integrated a horizontal (system-wide) and vertical (disease-related) approach to establishing a list of master priorities. First, a list of priorities was compiled, then the results were cross-referenced to identify relevant items. Sources included National Health and Medical Research Council research priorities;23 Primary Health Care Research, Evaluation and Development research priorities;14 and RACGP Foundation grant topics. Frequently presenting conditions in general practice were also considered,24–26 as were the recently determined top 10 international primary care research priorities.27 The internal expert group then reviewed the list of priorities.

A total of 63 priority items were identified and categorised under the following groups: disease-related, population health, process of care, healthcare management and other.

Selection of expert panel members

A set of accepted criteria – experience, knowledge and policy influence – was used as inclusion criteria for selecting the 83 expert panel invitees.28 The researchers also considered the general practice stakeholders most likely affected by the decision-making process. The resultant expert panel members included funders; RACGP members, (then) council and representatives of expert committees; representatives from university departments of general practice; representatives from other general practice organisations; general practice education providers; representatives from primary care networks; allied health including pharmacy and nursing; rural health, carer and consumer organisations; and community and philanthropic organisations.

Questionnaire

For the first-round survey, external expert panellists were emailed invitations to participate. Consent was implied if invitees chose to participate. They were asked to rate the 63 items as ‘very important’, ‘desirable’ or ‘non-essential’ in the context of the need for general practice–based research to improve patient care, and on the basis of the management of the item in general practice. In addition to rating items, panellists were asked to provide qualitative comments about the wording of items and nominate additional items that should be included in the list. It was determined that the items reaching a threshold of >69% or <30% on the ‘very important’ rating would be considered as reaching consensus for high or not high priority respectively and would be excluded from the second round.15,20,21

The validity and usefulness of the final list of items was discussed at a meeting of current and past members of the RACGP Expert Committee – Research (REC-R) acting as a consensus group.


Results 

Response rate

Twenty-three of 83 invitees completed the first-round survey (27.7%), and 28 of 78 completed the second round (35.9%). Five of the original invitees were removed from the invitation list for the second-round survey as each indicated they did not wish to participate in the survey process.

Ten respondents completed only survey one, 15 respondents completed only survey two, and 13 respondents completed both surveys (a total of 38 different respondents). Appendix 1 outlines information about the respondents’ roles/affiliations.

Of the original 63 items in the first-round survey, 17 items were excluded from the second-round survey because they reached the consensus for high priority in the first round (>69%), and 12 items were excluded because they reached consensus for not high priority (<30%) in the first round (Figure 1). These items were returned to the final list following the second survey. The remainder of the items were re-scored in the second round. On the basis of external panellist feedback, four new items were added to the second-round survey, and four items from the first-round survey were rephrased and included in the second-round survey for re-rating (Figure 1).

AJGP-11-2019-Research-Roberts-GP-Research-Priority-Fig-1.jpg

Figure 1. Flowchart of the Delphi process to identify general practice research priorities


Priority items

Priority items are presented as a combined group in rank order in Table 1. The top 21 items are highlighted. Items by group are presented in Tables 2, 3, 4, 5 and 6.

Table 1. Ranked combined priorities
Overall priority number
(Items 1–21 of 67)
Item
1 Quality of care
2 Evidence-based practice
3 Models of primary care delivery
4 Consumer focus
5 Multimorbidity management
6 Mental health
7 Collaborative care
8 Avoiding hospitalisations
9 Chronic pain
10 Quality use of medicines
11 Use of technology in primary care delivery
12 Alcohol and substance abuse disorders
13 Aged care and ageing
14 Dementia (new listing in round two)
15 Obesity
16 Health promotion and illness prevention
17 Social determinants of health
18 Aboriginal and Torres Strait Islander peoples
19 Family violence
20 Non-pharmacological treatments (including exercise and counselling techniques such as active listening)
21 Use of electronic data (eHealth records, data linkage)
 
Table 2. Disease-related priorities
Group priority number
(1–10 of 24 items)
Item
1 Mental health
2 Chronic pain
3 Alcohol and substance abuse disorders
4 Dementia (new listing in this group in round two)
5 Obesity
6 Cardiovascular health (including hypertension, management of cardiovascular disease)
7 Cerebrovascular health (including stroke)
8 Diabetes mellitus (including diabetic retinopathy)
9 Cancer (including diagnosis and treatment cancer, cancer survivor support)
10 Arthritis and musculoskeletal conditions (including back pain, sprain, tendonitis, osteoporosis, fracture management and prevention)
 
Table 3. Process of care priorities
Group priority number
(1–5 of 7 items)
Item
1 Evidence-based practice
2 Collaborative care
3 Health promotion and illness prevention
4 Non-pharmacological treatments (including exercise and counselling techniques such as active listening)
5 Antimicrobial stewardship in primary care (new listing in round two)
 
Table 4. Population health priorities
Group priority number
(1–5 of 13 items)
Item
1 Aged care and ageing
2 Social determinants of health
3 Aboriginal and Torres Strait Islander peoples
4 Family violence
5 Rural and remote populations (including telehealth)
 
Table 5. Healthcare management priorities
Group priority number
(1–5 of 19 items)
Item
1 Quality of care
2 Models of primary care delivery
3 Consumer focus
4 Multimorbidity management
5 Avoiding hospitalisations
 
Table 6. Other general practice issues
Group priority number
(All items)
Item
1 Use of electronic data (eHealth records, data linkage)
2 Data extraction tools, use of routinely collected data (Medicare Benefits Scheme, Pharmaceutical Benefits Schedule, National Prescribing Service, MedicineWise, MedicineInsight)
3 General practice coding systems including Doctor Command Language (DOCLE), which translates terminology used in general practice into code (new listing in this group in round two)
4 Virtual online communities of practice
Consensus group meeting

A consensus meeting identified several concerns including perceived low response rate and low absolute number of expert panel members, which could produce results that did not reflect true opinion. The group also noted that GPs and others might mistakenly assume that lower ranked topics were considered of lesser importance by participants, which could result in negative press. They also observed that a Delphi process essentially applies a quantitative score to a process that in some respects requires a qualitative approach.15,16 Some internal panel members reflected that their concerns about the Delphi survey results may be at least partially influenced by their relative inexperience with the Delphi process, and that this response might be replicated by others reading the results if they are unfamiliar with the Delphi process and its legitimate applications.

As a result of the consensus group meeting, it was decided to present the items in rank order rather than revealing scores. It was also decided there should be no set number of priority items. However, as there was a drop in the percentage score after 21 items, forming a natural gap in the data, these items were highlighted.


Discussion

Establishing a list of research priorities is important for informing a general practice research agenda, prioritising funding and, ultimately, optimising the delivery of best patient care.

Highly ranked items are broad in terms of research scope. Quality of care, for instance, encompasses many aspects of general practice, as do ‘evidence-based practice’, ‘models of primary care delivery’ and ‘multimorbidity’. It is unsurprising, given the broad scope of work undertaken in general practice, that population health topics such as ‘health promotion and illness prevention’ and ‘social determinants of health’ were ranked highly. Disease-related topics that ranked highly included mental health, chronic pain, alcohol and substance abuse disorders, aged care and ageing, dementia and obesity; such conditions reflect the burden of disease in Australia.29,30 Many research items that had a lower rank could be re-categorised or reframed in the context of one of the top-priority topics. For instance, antimicrobial stewardship could be considered part of ‘quality care of medicines’. Other topics are contextually relevant in the conduct of all types of research, for instance, ‘consumer focus’, ‘rural and remote’ and ‘practice-based research networks’. In the evolving world of electronic data, use of technology in primary care delivery, as well as use of routinely collected general practice data, ranked highly.

A recent study identified the top 10 international primary care research priorities; however, in contrast to the methodology used in the current study, this study grouped topics together under broad categories.27 Although most of the top 10 international primary care research priorities featured within the top 21 priorities considered in this study, the differing methodologies might account for some differences between the two priority listings.

When all 67 items were grouped together and presented in rank order, many ‘process of care’ items appeared near the top of the list, compared with ‘disease-related’ items. This may be a reflection of the number and diversity of disease-related topics in comparison to process of care topics, causing dilution in consensus. It is important, therefore, to consider the list of priorities in the context of the original groups in which they had been categorised, as well as in combination.

The Delphi method can answer questions that may not be possible or feasible with alternative methodologies.31 Consensus methods, which are based on expert opinion, are considered a weaker type of evidence than, for example, the gold standard consistent randomised controlled trial in the hierarchy of evidence-based medicine.32 This view is, however, an oversimplification and underestimation of the value of expert opinion and the consensus method applied in an appropriate context.

Viewed in the context of the entire study, the internal panel considered all items, which were identified by sourcing the best available evidence, to be important topics. However, the Delphi process was used to identify topics the external panellists considered important priority areas for general practice research, rather than topics of global importance.

Strengths and limitations

The diversity of research topics and the variety of contexts in which they may apply make it challenging to produce a priority list of research topics for Australian general practice. Although the research comprises different views from a range of general practice research stakeholders, they may not adequately represent the full spectrum of views held by the individuals whom they represent. Participants may also have particular research interests that could have affected their prioritisation of topics.

Furthermore, although only 23 of 83 invitees took part in round one and 28 of 78 in round two, the final number of participants was adequate for this method of research.20 There are no firm guidelines on the size of a Delphi panel;33 however, findings are more likely to be stable with a larger panel, with individual responses having less influence on overall findings. In one study, a panel of 23 experts produced stable results;34 others have shown stability with rounds of approximately ≥20 members.35,36 When considering the absolute number of responses involved and the response rate, it should be remembered that this is an expert panel consensus method rather than a cross-sectional survey. Participants were chosen purposefully, and each identified stakeholder represented a larger group. Unlike a cross-sectional survey, the data were not analysed using descriptive or inferential statistics, and therefore rules regarding absolute numbers responding, response rates and the representativeness of the responders do not apply.


Conclusion

Identifying current Australian general practice research priorities is complicated as the methodological process and results will be scrutinised by a wide variety of stakeholders, some of whom may conclude that their particular views are not captured.

The list of priorities identified in the modified Delphi study is not exhaustive, but it does reflect the input of a diverse range of stakeholders and highlights both very broad and quite specific areas of research for Australian general practice.

The researchers were satisfied that the Delphi method chosen was the evidence-based approach best suited to identify the priorities.

Implications for general practice

The results of the study will provide a useful reference point to inform a research agenda, prioritise funding and, ultimately, optimise the ability of Australian general practice research to deliver the best patient care on the basis of current evidence.

The results are under consideration by the Commonwealth to help inform funding decisions regarding future medical research.

Appendix 1. Delphi respondent affiliations, rounds one and two
Role/affiliation Round one Round two
GP academic 14 16
RACGP 1 3
Primary Health Network 0 1
Consumer organisation 1 1
Allied health 1 2
CALD population representation 2 2
Funder 2 0
GP educator 2 2
Rural representation 0 1
CALD, culturally and linguistically diverse; GP, general practitioner; RACGP, Royal Australian College of General Practitioners
Competing interests: None.
Provenance and peer review: Not commissioned, externally peer reviewed.
Funding: The work reported in this paper was funded by the RACGP.
Acknowledgements
The authors thank all participants for the generous donation of their time and the provision of their expert opinion to make this study possible. They also thank the RACGP Expert Committee – Research (REC-R) and the RACGP GP Research Priority Setting Working Group members for the valuable ideas that assisted in the development of the project. Special thanks are reserved for the previous Chair of the REC-R, Professor Tania Winzenberg, who was a catalyst for the development of the project that led to this paper, and to REC-R previous member, Professor Paul Glasziou, who provided consistent and expert guidance throughout the development of the project regarding its design and implementation, and assisted with data analysis. The authors also acknowledge the insights, comments and technical editing of the Working Group members, Associate Professors Sanjyot Vagholkar and Jan Radford, Adjunct Associate Professor Bianca Brijnath, and RACGP Research Program Manager, Ms Tania Lim, in the development of this paper. We are indebted to you all.
References
  1. Winzenberg TM, Gill GF. Prioritising general practice research. Med J Aust 2016;205(2):55–57. doi: 10.5694/mja16.00578. Search PubMed
  2. Magin P, Pirotta M, Farrell E, Van Driel M. General practice research: Training and capacity building. Aust Fam Physician 2010;39(5):265–66. Search PubMed
  3. Heal C, Banks J, Divakaran P, Buttner P. Building a bridge from the swamp to the ivory tower: Conducting randomised controlled trials in general practice. Aust J Gen Pract 2018;47(3):153–57. Search PubMed
  4. Austad B, Hetlevik I, Mjølstad BP, Helvik AS. Applying clinical guidelines in general practice: A qualitative study of potential complications. BMC Fam Pract 2016;17:92. doi: 10.1186/s12875-016-0490-3. Search PubMed
  5. Steel N, Abdelhamid A, Stokes T, et al. A review of clinical practice guidelines found that they were often based on evidence of uncertain relevance to primary care patients. J Clin Epidemiol 2014;67(11):1251–57. doi: 10.1016/j.jclinepi.2014.05.020. Search PubMed
  6. Scullard P, Abdelhamid A, Steel N, Qureshi N. Does the evidence referenced in NICE guidelines reflect a primary care population? Br J Gen Pract 2011;61(584):e112–17. doi: 10.3399/bjgp11X561177. Search PubMed
  7. The Royal Australian College of General Practitioners. Media release: GPs urge Federal government to extend its commitment to our nation’s health in 2018–2019 budget. East Melbourne, Vic: RACGP, November 2017. Available at www.racgp.org.au/gp-news/media-releases/2017-media-releases/november-2017/gps-urge-federal-government-to-extend-commitment [Accessed 18 September 2019]. Search PubMed
  8. Gunn J, Pirotta M. Research in general practice. Aust Fam Physician 2017;46(3):166–67. Search PubMed
  9. Lockery JE, Collyer TA, Abhayaratna WP, et al. Recruiting general practice patients for large clinical trials: Lessons from the aspirin in reducing events in the elderly (ASPREE) study. Med J Aust 2019;210(4):168–73. doi: 10.5694/mja2.12060. Search PubMed
  10. Yoshida S. Approaches, tools and methods used for setting priorities in health research in the 21(st) century. J Glob Health 2016;6(1):010507. doi: 10.7189/jogh.06.010507. Search PubMed
  11. Bryant J, Sanson-Fisher R, Walsh J, Stewart J. Health research priority setting in selected high income countries: A narrative review of methods used and recommendations for future practice. Cost Eff Resour Alloc 2014;12:23. doi: 10.1186/1478-7547-12-23. Search PubMed
  12. Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: Nine common themes of good practice. Health Res Policy Syst 2010;8:36. doi: 10.1186/1478-4505-8-36. Search PubMed
  13. Sibbald SL, Singer PA, Upshur R, Martin DK. Priority setting: What constitutes success? A conceptual framework for successful priority setting. BMC Health Serv Res 2009;9:43. doi: 10.1186/1472-6963-9-43. Search PubMed
  14. Kalucy L, Beacham B, Raupach J, Dwyer J, Pilotto L. Priorities for Primary Health Care Research, Evaluation and Development in Australia. Final Report Priority Setting Process: Stage One. Adelaide: Primary Health Care Research and Information Service, 2001. Search PubMed
  15. Linstone HA, Turoff M. The Delphi method: Techniques and applications. Ann Arbor, MI: Addison-Wesley, 1975. Search PubMed
  16. Turoff M, Hiltz SR. Computer based Delphi processes. In: Adler M, Ziglio E, editors. Gazing into the oracle: The Delphi method and its application to social policy and public health. London: Jessica Kingsley Publishers, 1996; p. 56–59. Search PubMed
  17. Broomfield D, Humphris GM. Using the Delphi technique to identify the cancer education requirements of general practitioners. Med Educ 2001;35(10):928–37. doi: 10.1111/j.1365-2923.2001.01022.x. Search PubMed
  18. Albarqouni L, Hoffmann T, Straus S, et al. Core competencies in evidence-based practice for health professionals: Consensus statement based on a systematic review and Delphi survey. JAMA Netw Open 2018;1(2):e180281. doi: 10.1001/jamanetworkopen.2018.0281. Search PubMed
  19. Linstone HA, Turoff M. Delphi: A brief look backward and forward. Technol Forecast Soc Change 2011;78(9):1712–19. doi: 10.1016/j.techfore.2010.09.011. Search PubMed
  20. Okoli C, Pawlowski SD. The Delphi method as a research tool: An example, design considerations and applications. Inform Manag 2004;42(1):15–29. doi: 10.1016/j.im.2003.11.002. Search PubMed
  21. Custer RL, Scarcella JA, Stewart BR. The modified Delphi technique – A rotational modification. JVTE 1999;15(2):50–58. doi: 10.21061/jcte.v15i2.702. Search PubMed
  22. Hoekstra D, Mütsch M, Kien C, Gerhardus A, Lhachimi SK. Identifying and prioritising systematic review topics with public health stakeholders: A protocol for a modified Delphi study in Switzerland to inform future research agendas. BMJ Open 2017;7(8):e015500. doi: 10.1136/bmjopen-2016-015500. Search PubMed
  23. National Health and Medical Research Council. Research priorities. Canberra: NHMRC, 2018. Available at www.nhmrc.gov.au/research-policy/research-priorities [Accessed 2 August 2019]. Search PubMed
  24. Britt H, Miller GC, Bayram C, et al. A decade of Australian general practice activity 2006–07 to 2015–16. General practice series no 41. Sydney: Sydney University Press; 2016. Search PubMed
  25. Britt H, Miller GC, Henderson J, et al. A decade of Australian general practice activity 2006–07 to 2015–16. General practice series no. 40 Sydney, NSW: Sydney University Press, 2016. Search PubMed
  26. Murtagh J, Rosenblatt J, Coleman J, Murtagh C. Research and evidence-based meidicine. In: Murtagh’s General Practice. 7th edn. Sydney, NSW: McGraw-Hill Education Australia, 2018; p. 108–09. Search PubMed
  27. O’Neill B, Aversa V, Rouleau K, Lazare K, Sullivan F, Persaud N. Identifying top 10 primary care research priorities from international stakeholders using a modified Delphi method. PLoS One 2018;13(10): e0206096. doi: 10.1371/journal.pone.0206096. Search PubMed
  28. Sylvester S, Magin P, Sweeney K, Morgan S, Henderson K. Procedural skills in general practice vocational training – What should be taught? Aust Fam Physician 2011;40(1–2):50–54. Search PubMed
  29. Australian Institute of Health and Welfare. Australia’s health 2018: In brief. Cat. no. AUS 222. Canberra: AIHW, 2018. Search PubMed
  30. Australian Institute of Health and Welfare. Impact of alcohol and illicit drug use on the burden of disease and injury in Australia: Australian Burden of Disease Study 2011. Cat. no. BOD 19. Canberra: AIHW, 2018. Search PubMed
  31. Ziglio E. The Delphi method and its contribution to decision-making. In: Adler M, Ziglio E, editors. Gazing into the oracle: The Delphi method and its application to social policy and public health. London: Jessica Kingsley Publishers, 1996; p. 3–33. Search PubMed
  32. Centre for Evidence-Based Medicine. Oxford Centre for Evidence-based Medicine – Levels of Evidence (March 2009). Oxford, CEBM, 2009. Available at www.cebm.net/2009/06/oxford-centre-evidence-based-medicine-levels-evidence-march-2009 [Accessed 2 August 2019]. Search PubMed
  33. Jorm AF. Using the Delphi expert consensus method in mental health research. Aust N Z J Psychiatry 2015;49(10):887–97. doi: 10.1177/0004867415600891. Search PubMed
  34. Akins RB, Tolson H, Cole BR. Stability of response characteristics of a Delphi panel: Application of bootstrap data expansion. BMC Med Res Methodol 2005;5:37. doi: 10.1186/1471-2288-5-37. Search PubMed
  35. Ross AM, Kelly CM, Jorm AF. Re-development of mental health first aid guidelines for suicidal ideation and behaviour: A Delphi study. BMC Psychiatry 2014;14:241. doi: 10.1186/s12888-014-0241-8. Search PubMed
  36. Hutchings A, Raine R, Sanderson C, Black N. A comparison of formal consensus methods used for developing clinical guidelines. J Health Serv Res Policy 2006;11(4):218–24. doi: 10.1258/135581906778476553. Search PubMed

General practiceResearch

Download article