Primary health networks (PHNs) were established in Australia in 2015 with the goal of increasing the efficiency and effectiveness of medical services for patients, particularly those at risk of poor health outcomes, and improving the coordination of care to ensure patients receive the right care in the right place at the right time.1 To achieve these objectives, PHNs typically work with general practices in their regions on data-driven continuous quality-improvement (CQI) activities aligned to the collaborative model, which the Australian Commission on Safety and Quality in Health Care describes as implementing changes during defined, rapid quality-improvement cycles, characterised by measurable targets, with the aim of sustained care performance.2 However, the practical experience of most PHNs is that sustainable, measurable improvements in general practice care are challenging to achieve.3
The relative lack of formal CQI activities and inconsistency in clinician participation can lead to significant variation in patient care.4 There are well-known barriers to CQI, including time to participate, organisational culture, and a growing evidence-base that suggests continuing practice support, training and financial incentives are required to overcome these barriers.5 However, research on the best approaches to increase clinician participation in CQI is often subject to significant limitations, such as non-standardised and bundled interventions, and the findings might not therefore be generalisable.6
For some general practitioners (GPs) who are supported by their PHN, not being able to sustain quality improvement could be in part due to the fee-for-service environment of general practice, where ring-fenced time for GPs to engage in such activities is not funded through Medicare. Furthermore, fee for service might incentivise excessive services and unnecessary or inappropriate care, directly affecting the quality of care.7
From a PHN perspective, there is an expectation that their practice support must be more efficient and cost-effective, as highlighted by the Australian Department of Health,8 which recommends that the traditional approach to CQI of face-to-face meetings between PHN practice support team members and general practice staff might be at least be partly replaced through increased use of technology, including automated decision support.
Clinical decision support systems need to be built into the clinician’s workflow to have impact and increase efficiencies.9 Low response rates to decision support could be due to the type of content, with clinicians more likely to respond to prompts about younger patients, or to alerts about serious health issues.10 Therefore, engaging GPs in the design is important in ensuring acceptability of the solution.
As PHNs have experienced, data-driven improvement requires considerable effort by general practices. The commercially available IT tools that support CQI did not meet the Gold Coast PHN (GCPHN)’s requirements to bring quality improvement into the GP’s workflow during the patient’s consultation, or to target patients at most risk.11 The commercial tool GCPHN was using was also labour intensive for PHN staff, and it was challenging to produce evidence of quality improvement in general practices. To address these issues, a population health management tool with GP decision support functionality was developed with Department of Health (DoH) innovation funding, and as such, they own the intellectual property to Primary Sense.12 The DoH has no access or rights to the data; the only data provided to other agencies is the Practice Incentives Program Quality Improvement dataset to the Australian Institute of Health and Welfare. The tool is not open source, but is available to other PHNs on a cost-recovery basis; there is no cost to general practices. The authors have no financial interest or personal attachment to the product.
Aim
The role of GP decision support resulting in quality and safety interventions that are auditable being done at the point of care has not been widely reported in Australia. GP clinical information systems might provide alerts and prompts, but the responses are not recorded and so not reviewed for accuracy. Decision support systems can create risk, especially if not maintained and evaluated.13 Therefore, close monitoring, auditing and evidenced-based updates are part of the Primary Sense system. The aim of this research was to gain an understanding of the timeliness and clinical content to enable medication safety alerts and care prompts to be responded to. The influence of a range of factors affecting a GP’s decision to act was investigated to ascertain if timely, accurate and patient-specific information contributes to improved patient care without causing alert fatigue.
Methods
Development, testing and validation of decision support functionality
Primary Sense was developed in partnership with academics and local GPs, who were identified due to their leadership in medication safety or interest in the practical application of risk stratification in general practice. The group developed guiding principles that safeguard the use of the data. The medication safety indicators and key issues were screened for eligibility by two reviewers (both academic and local GPs). The findings were presented to the broader group, and disagreements were resolved through discussion until consensus was reached. Development of the prompts followed a similar process. In both cases, the functionality agreed was not necessarily performed by existing clinical software.
Our software is currently compatible with MedicalDirector and Best Practice Software. The tool de-identifies and encrypts data every five minutes before transmission for analysis in the database in Azure Australia.14 De-identification involves creating and extracting a unique patient identifier that is used to collate the patient’s information stored across various tables in our database; a similar identifier is also created for practice staff. The patient’s age is also extracted with anyone under one year old recorded as 0 and anyone over 90 years as 90+. Only data required for the reporting and notification logic are extracted. Encryption scrambles the data for transit, greatly reducing the likelihood of data being usable if hacked, which the system can also detect. Patient re-identification only occurs within the practice.
The system is Therapeutic Goods Administration approved and has had an external privacy impact assessment, architecture review, cyber security review and penetration testing. A data lifecycle is publicly available. There is a patient consent section in the desktop that allows patients to elect to not have their data extracted at all, for it just to be used for the primary purpose of sending risk factors back to the GP or for secondary purposes, including PHN planning. Only the GCPHN software developer and system auditor have access to the Primary Sense database.
The Johns Hopkins ACG System has been incorporated into Primary Sense, primarily for risk stratification to prioritise patients in prompts, practice reports and populations in aggregated PHN reports.15 The easy-to-understand patient-complexity scores reflect comorbidities and healthcare resources consumed, and the hospitalisation risk score predicts those with a greater than 80% risk in the next 12 months. A range of practice-level reports were designed to help teams better understand their populations and data quality.16
Medication safety alerts
Medication safety alerts were developed by a small group of academics and local GPs to target high-risk scenarios where there might be low GP awareness (Figure 1). Alerts appear as a pop-up message prior to printing a prescription (Table 1). The information in the alert is generated for each patient from their routinely recorded information (Figure 2). To reduce alert fatigue, if a GP clicks an action in the alert, other than ‘remind me next time’, the selection will hold off re-alerting for 12 months, even if the clinical intervention is not done.
Figure 1. Example of an alert
| Table 1. Medication safety alerts: July 2020 to June 2021 |
| Alert |
Occasions (n) |
Interventions
done (n) |
% |
| Azathioprine/mercaptopurine without thiopurine methyltransferase testing |
39 |
18 |
46 |
| Metformin, where latest estimated eGFR is <30 mL/min/1.73m2 |
36 |
4 |
11 |
| Biological medication without relevant lab tests within the past six months |
64 |
24 |
38 |
| Immunosuppressive drugs without relevant lab tests within the past six months |
399 |
199 |
50 |
| Antiplatelet drug for patient with history of peptic ulcer or gastrointestinal bleed and no gastroprotection |
57 |
36 |
63 |
| Antipsychotic medication without relevant lab tests within the past 12 months |
1,421 |
667 |
47 |
| Combined hormonal contraceptive where there is a history of migraine |
204 |
43 |
21 |
| Hypoglycaemics other than simple metformin in elderly patients (age ≥75 years) with HbA1c <53 mmol/mol (<7%) |
322 |
81 |
25 |
| Fentanyl patch for non-cancer pain |
146 |
55 |
38 |
| Digoxin, where latest eGFR is <45 mL/min/1.73m2 |
37 |
10 |
27 |
| Bisphosphonate medication for osteoporosis, where latest eGFR is <35 mL/min/1.73m2 |
6 |
4 |
67 |
| Hypoglycaemics other than simple metformin in patients (age <75 years) with HbA1c <48 mmol/mol (<6.5%) |
422 |
109 |
26 |
| eGFR, estimated glomerular filtration rate; HbA1c, glycated haemoglobin |
Figure 2. Medication safety alerts process. Click here to enlarge.
ATC, Anatomical Therapeutic Chemical; GP, general practitioner; ICPC, International Classification of Primary Care; LOINC, Logical Observation Identifiers, Names and Codes; LTC, long-term condition
Care prompts
Prompts are a more recent addition to the functionality, are aimed at potential gaps in care, and cover a broad range of patients cohorts and clinical scenarios (Figure 3). They were developed and prioritised by a group of local and academic GPs to ensure the right clinical scenarios would prompt for the appropriate at-risk groups, at a volume that would be manageable by GPs. Prompts provide curated information ‘pushed’ out at the start of the GP consultation containing up to three suggested interventions for consideration (Table 2). The prioritisation enables the most important interventions to be presented on the first prompt. For example, vaccinations on the first prompt, with subsequent interventions, such as a care plan appearing on the next prompt in a rolling manner as they are completed by the GP. Influenza vaccinations are a seasonal prompt that is disabled if there is not a vaccine available. The GP’s clinical software and other apps might also prompt for this; however, these systems lack the evidence base to identify high-risk patients. To reduce prompt fatigue, the same click action mechanisms as the alerts is used to reduce re-prompting.
Figure 3. Example of a prompt
| Table 2. Prompts: January 2021 to June 2021 |
| Prompts listed according to priority |
Occasions (n) |
Interventions
done (n) |
% |
| Due influenza vaccination: Complexity 4 or 5 or pregnant |
3,105 |
2,234 |
72 |
| Due pertussis vaccination in pregnancy after 20 weeks |
355 |
167 |
47 |
| Due meningococcal vaccination: Aboriginal and Torres Strait Islander children |
98 |
10 |
10 |
| Due hepatitis A vaccination: Aboriginal and Torres Strait Islander children |
165 |
16 |
10 |
| Consider haemochromatosis testing for raised ferritins ×2 or raised saturated transferrin |
1,949 |
198 |
10 |
| Missing CV risk medication (statin and antihypertensive) when CV score is >15% |
1,803 |
825 |
46 |
| Due Heart Health Check when CV risk is >15% and statin and antihypertensive are missing |
1,556 |
32 |
2 |
| Due Aboriginal and Torres Strait Islander health assessment when CV risk score >10% and statin or antihypertensive missing |
60 |
17 |
28 |
| Due annual microalbumin pathology in diabetes or CKD |
4,187 |
490 |
12 |
| Due care plan: Complexity 4 or 5 (or 3 if hospital risk is >80%) |
8,011 |
1,265 |
16 |
| Due mental health care plan with 2 or more mental health conditions |
796 |
172 |
22 |
| Due medication review when there are 7 or more current medications |
2,587 |
18 |
1 |
| CKD, chronic kidney disease; CV, cardiovascular |
User interface
The interface was designed to be intuitive and unobtrusive. End-user testing was limited due to the lack of an appropriate test environment to simulate a busy GP practice. However, when the alerts went live, GPs started interacting with them straight away. The desktop icon is discreet, about the size of a 20 cent coin, and by default, positioned on the right margin of the GP’s screen. The icon expands into a short, drop-down menu when clicked. Medication safety alerts appear on the right of the screen, are about the size of an iPhone, and stay visible until the GP selects an action or minimises the alert. Care prompts have a similar appearance to alerts, but they occur more frequently; they are timed to stay visible for only three minutes to prevent ‘stacking’ on the GP’s screen.
As the patient might see what appears on the GP’s computer screen, the presentation of the information was designed to be suggestive rather than directive or alarmist. Brief information and links to resources and patient handouts were incorporated to provide immediately helpful context.
The alerts and prompts provide a ‘one-click’ process in which the GP can dismiss the pop-up message by stating an intent to act, defer or disagree. There is also the ability to provide free-text comments in alerts, prompts and reports. Comments and feedback from GPs are used to further enhance accuracy and functionally of the decision support. GPs have the ability to opt in and out of receiving alerts and prompts in the desktop app.
Implementation
The tool was demonstrated to practices that were interested; only one practice declined an installation. The software is in approximately half (n = 80) of the general practices in the Gold Coast, with 487 distinct GP users; all practices have a data-sharing agreement with GCPHN, and have accepted the terms and conditions for its use. It has detected over one million patient records, and by assigning unique patient identifiers, highlights 703,800 individuals, of whom 519,000 have visited the practices since January 2020. There are an estimated 635,200 residents on the Gold Coast.17
As part of the installation process, practical demonstrations focused on using the tool for CQI were provided to general practice staff. However, due the project’s resource constraints, there was no specific training program or awareness raising developed for GPs on the decision support. Approximately 110–130 GPs use the app each day, and this number fluctuates based on GPs at work and who has access to the desktop app. The Chair of the National Health and Medical Research Council advised that ethics for the project would be required. Ethics approval (reference 16154) for the whole project as a quality improvement activity was granted by Bond University.
Implementation and ongoing use occurred before and during the current COVID-19 pandemic, which increased demand for COVID-19 testing and vaccinations, and introduced the use of phone consultations. In this article, we do not make any adjustments for the competing GP priorities of the pandemic over what would have been business as usual.
Data collection
The programming analyses all historical and new patient data routinely to highlight gaps in care and high-risk clinical scenarios for presentation in alerts and in prompts. The software checks the general practice’s database every two seconds for the need to trigger a patient alert, and every 10 seconds for the need to trigger a patient prompt. If the system identifies that the patient the GP is consulting with should receive a medication safety alert, or should receive a prompt, the relevant patient data are sent from our database to the desktop app, and the information appears on the GP’s screen within a couple of seconds. The trigger information and GP response, such as clicking options or lack of response, are captured in our database for analysis. GPs have entered close to 300 comments in the comments section of the alert and prompt, which are used to audit and refine the system.
Data analysis
Analysis involved Structured Query Language queries on our database from July 2020 to July 2021, comparing the trigger data with GP responses, and searching for the suggested interventions done for the patient on the day of the trigger (Figure 4). The only exception to ‘on the day’ is where the intervention is a Medicare Benefits Schedule billed item of care, as this might require a separate patient appointment, and therefore a two-week interval is allowed. As the system is live and data are constantly changing, the results were exported into Excel for in-depth analysis.
Figure 4. Process to assess if an intervention occurred following a medication safety alert. Click here to enlarge.
GP, general practitioner
Through the analysis we observed a potential association between prompts and alerts in terms of linking between the evidence-based intervention suggested in each. For example, an alert to conduct lipid tests for a patient on antipsychotic medication leads to automated recalculation of absolute cardiovascular risk if the pathology is done, and if above 15% and a statin and an antihypertensive medication are missing, the GP is prompted to consider further prescribing to manage the emergent cardiovascular risk. The system can then track individuals who receive the amended medication regimen. This shows how a medication safety alert can lead to reducing other seemingly unrelated but important risk factors for the patient.
Results
Alerts
Between July 2020 and June 2021, 312 out of 487 GPs received one or more medication safety alerts. In total, 93,164 patients’ medications were monitored for an alert, with 3153 alerts triggered for 2328 patients and 1250 of the suggested interventions being done (40%; Table 1). On average, 110 GPs per day interacted with the tool; however, only 6–8 received a medication alert. The rate at which GPs respond to medication alerts was 80%. When the GP selected that they intended to take action, the intervention was performed 70% of the time. Interestingly when the GP did not select an action, the intervention rate was still 70%. Interaction rates indicated there were no changes to the level of GP interaction over two years (Table 3).
| Table 3. General practitioners’ responses to medication safety alerts |
| Responses |
Three-month results
(July–September 2020) |
Six-month results
(July–December 2020) |
Nine-month results
(July 2020–March 2021) |
12-month results
(July 2021–June 2021) |
| Alerts responded to |
558/688 (81%) of alerts |
1,375/1,546 (81%) of alerts |
1,998/2,413 (83%) of alerts |
2,516/3,153 (80%) of alerts |
| Agree, I will take action |
219 (32%) |
839 (54%) |
1,094 (45%) |
1,294 (51%) |
| Agree, but override |
181 (26%) |
230 (15%) |
423 (18%) |
556 (22%) |
| Wrong for the patient |
55 (8%) |
52 (3%) |
82 (3%) |
122 (6%) |
| Remind me next time |
103 (14%) |
261 (16%) |
399 (17%) |
544 (22%) |
| No selection (ignored) |
130/688 (19%) |
171/1,546 (11%) |
399/2,413 (17%) |
637/3,153 (20%) |
Prompts
From January 2021 to June 2021, 333 out of 375 GPs received one or more prompts. On average, 85 GPs per day received one or more prompt. In total, 118,684 patients were monitored for a prompt, with 19,019 prompts containing 24,672 interventions triggered during a visit for 17,398 patients (15%) and 5444 interventions done (22%) for 4660 patients (27%). Some patients had more than one intervention done (Table 2). The number of times a GP selected an action was 1240 (7%). The GPs also updated the patient record, including ceasing four or more medications, archiving four or more conditions and changing smoking status for 284 patients (on the day of the cardiovascular risk prompt).
Discussion
Main findings
In this article, we described how medication alerts and care prompts can be associated with GPs engaging in opportunistic, CQI and mitigation of medication-related patient safety risks, reducing the need for PHNs to provide site visits and peer-to-peer education. Automated decision support at the point of care provides a sustainable and scalable way to support CQI and patient safety, a sentiment echoed by the Productivity Commission in its review of the approach.18 The volume of triggered prompts on a daily basis highlights that approximately 15% of patients who attend practices are high risk, with potential gaps in care.
The decision support functionality has indicated that >25% of changes in clinical care is possible when there is a good co-design process, resulting in the end user valuing and engaging with the information being presented. This is more favourable than a recent study in the Australian hospital system, where almost 50% of prescriptions generated an alert, but fewer than 20% were acknowledged, and no prescriptions were changed in response.19
Within the Primary Sense practices we were able to compare GPs who had the app to those who did not, and found that recording smoking status was done more frequently by those with the app (Figure 5). While both groups recorded smoking status less often during 2020–21 (perhaps an impact of COVID-19 vaccinations), for those with a desktop it dropped by 5% (from 51,631 to 49,283) and for GPs without it there was a 19% drop (from 16,698 to 13,584). The year 2021 saw the release of the cardiovascular risk prompt, which might have had some influence.
Figure 5. Average smoking recorded by general practitioners in 2020 and 2021 (n = 408)
GP, general practitioner
Practical implications
Response and intervention rates may have been affected by the COVID-19 pandemic, which was at its height during the sampling phase of this study. There was concern this would increase and change GP workload;20 a phone consultation would prevent a GP undertaking a prompted influenza intervention on the day. Rolling out the new prompt functionality on the existing alert functionality during the COVID-19 pandemic does not appear to have negatively affected intervention rates or increase alert fatigue, and only one GP opted out of the prompts.
Although alert fatigue is a well-described phenomenon, the reasons why are understudied and poorly understood.21 Frequent alerts, and particularly repeated alerts, have been found to cause fatigue.22 We found that alerts tailored to individual patients and about important safety issues reduced the volume of alerts, and the ability for GPs to stop repeated alerts has managed fatigue.
Presentation and relevance of the information is also important. Patient-specific information provided automatically on screen has been found to be more effective than other methods of decision support.23 The prompts provide additional patient-specific risk scores, which may have influenced the GP’s decision to intervene.
The alerts and prompts improved coding and data quality, and while this can be distracting if done during the consultation, the GPs decided it was important enough to do at that point.24 GPs interpret information for relevance to the individual patient, and have the chance to provide feedback about the perceived accuracy and helpfulness (or lack of) of the alert or prompt. This often results in changes to the patient’s history, medication list and risk scores, improving the accuracy of clinical records.
Strengths and limitations
An identified strength of the approach was operating in near real time, with data extracts and analysis occurring every few minutes, allowing relevant information about high-risk patients to be presented to the GP during a consultation.
A potential limitation of our study was the assumption that GP actions were attributed to receiving an alert or prompt, and if so, that it was from our decision support as opposed to from other clinical software. We attempted to identify GP intended actions by the use of a single-click response choice, but it is conceivable that GPs would have identified the missing action without any computer decision support.
A further weakness of our study was that the decision support was introduced without specific education or significant promotion, instead relying on GPs’ intuitive understanding of the value of alerts and prompts. Therefore, we were unable to determine if the impact of alerts and prompts would change if GPs had greater knowledge and more insight into the functionality.
Approximately 80% of all practices can use the decision support functionality due to limited compatibility with software other than MedicalDirector and Best Practice Software.
Next steps
It is unknown if GP interaction with alerts and prompts will have longer-term implications leading to pre-emptive patient management, and ultimately a reduced number of alerts and prompts. Further work is needed in exploring the relationship and synergy between alerts and prompts, and understanding the optimal mix to support improved quality and safe care. How presenting the ACG system risk scores influences GP behaviour may also be explored further. Future research will continue to validate the alerts and prompts, and then link results to patient outcomes and even their experience of care. We also need to understand key enabling factors and barriers to facilitate normalisation to maximise functionality of the GP decision support systems.
In the meantime, a promotional push is planned for the Gold Coast to inform GPs of the importance of decision support in managing their patients at most risk. Work will continue to make the functionality compatible with other GP clinical software systems. Additional support from PHNs is being sought to scale the decision support capability for use by GPs in other PHN regions, which should increase medication safety and quality care across the country.
Conclusion
There is a potential need for digital solutions to improve efficiency and effectiveness of healthcare. Automated decision support provides a potential solution. We found that GP decision support in the form of alerts and prompts that are timely, accurate and patient specific result in GPs performing quality and safety interventions at the point of care and do not appear to cause alert fatigue.
However, some of our GP users have said that it is challenging to address the gaps in care during a 15-minute consultation when the patient is there with their own health concerns. The GPs did understand that the prompts and alerts were patient specific, and had ideas about clinical aspects they would like added, which will be considered by the Clinical Governance Group.
Further research would help explore how best to understand how to optimise software and clinician interactions and how to normalise GP decision support functionality into business as usual. This research may be best placed to occur in general practice, as GPs interact with the tool.