Advertising

Viewpoint
Volume 53, Issue 10, October 2024

Inaccuracy of cardiovascular disease calculators in Australian primary healthcare software

Jason Agostino    Karl Skewes    Deborah Wong    Andrea Timothy    Rosemary Wyber    Kim Greaves   
doi: 10.31128/AJGP-09-23-6975   |    Download article
Cite this article    BIBTEX    REFER    RIS

Assessment of cardiovascular disease (CVD) risk is grounded in increasingly sophisticated risk algorithms. An updated Australian CVD risk algorithm was released in July 2023.1 This update replaces the previous 2012 guidelines, which made consensus-based recommendations about CVD risk assessment based on weak evidence relevant to Aboriginal and Torres Strait Islander people. The 2023 guideline is based on a contemporary population with a similar CVD risk profile to Australia’s, albeit in the ongoing absence of Aboriginal and Torres Strait Islander-specific information. It introduces new variables, including postcodes as a proxy for socioeconomic status and use of cardiovascular medications.1 Improvements in CVD algorithm accuracy offer opportunities for more targeted management. Successful implementation of CVD guidelines will require the integration of new algorithms into general practitioner (GP) software packages and workflow. Given that more clinicians use CVD risk calculators than read the guidelines,2 algorithm fidelity in CVD risk calculators is essential for high-quality CVD prevention.

To understand the status quo, we examined how the previous 2012 CVD risk assessment guidelines are implemented within three large GP clinical software products (CSP; Best Practice, Medical Director and Communicare) and a major data extraction tool (Pen CS CAT4).3 CVD risk calculators embedded in these CSP were audited against the variables required for assessing CVD risk in the 2012 guidelines.3 The audit was supplemented by analysis by the Australian Institute of Health and Welfare (AIHW) of national indicators relating to cardiovascular disease risk assessment in general practice.4,5

The 2012 CVD risk algorithm requires an initial check for conditions that automatically qualify a patient to be at ‘clinically determined high risk of CVD’ (eg patients with diabetes and microalbuminuria).3 If these conditions are not present, then absolute CVD risk is calculated using the Framingham risk equation (FRE). Previous research has established that clinically determined high-risk conditions are especially important for Aboriginal and Torres Strait Islander people, with 80% of people who are at high CVD risk identified through clinically determined high-risk conditions and 20% through the FRE.6

In our audit of CSP, although all the embedded calculators included variables within the FRE, conditions that conferred clinically determined high risk were missing or incomplete (Figure 1).


AJGP-10-24-6975-VP-Agostino-CVD-Calculators-Fig-1.png

Figure 1. Inclusion of variables in the 2012 CVD risk assessment guidelines in CVD risk calculators embedded within three general practice clinical software products.3

BP, blood pressure; CKD, chronic kidney disease; CSP, clinical software product; CVD, cardiovascular disease; ECG, electrocardiogram; HDL, high-density lipoprotein; LVH, left ventricular hypertrophy.


Given these findings, the AIHW reanalysed the 2020 National Key Performance Indicators (nKPI) results on CVD risk levels among Aboriginal and Torres Strait Islander people who had their CVD assessed in the past two years, by CSP type (Table 1).7 This analysis revealed a more than a three-fold variation in the proportion of patients identified as at high risk of CVD by each system: 9.6% high risk for CSP1 (111/1142, 95% confidence interval [CI] 8.0%, 11.5%) versus 34.3% high risk for CSP2 (4859/14,173, 95% CI 33.5%, 35.1%).

Table 1. Proportion of Aboriginal and Torres Strait Islander regular clients aged 35–74 years with no known history of CVD who had an absolute CVD risk assessment result recorded within the previous 24 months as either low, moderate or high risk, by electronic medical record type
CSP Age group (years) Low risk
(95% CI)
Moderate risk
(95% CI)
High risk
(95% CI)
CSP 1 35–44 (n=222) 94.6 (90.8, 92.1) 5.0 (2.5, 8.7) 0.5 (0.0, 2.5)
45–54 (n=417) 82.0 (78.0, 85.6) 13.2 (10.1, 16.8) 4.8 (3.0, 7.3)
55–64 (n=359) 59.9 (54.6, 65.0) 24.8 (20.4, 29.6) 15.3 (11.8, 19.5)
65–74 (n=144) 38.2 (30.2, 46.7) 38.2 (30.2, 46.7) 23.6 (16.9, 31.4)
Total 35–74
(n=1142)
72.0 (69.3, 74.6) 18.4 (16.2, 20.8) 9.6 (8.0, 11.5)
CSP 2 35–44 (n=4623) 79.7 (78.5, 80.9) 0.9 (0.7, 1.2) 19.4 (18.2, 20.5)
45–54 (n= 4957) 63.9 (62.5, 65.2) 6.4 (5.7, 7.1) 29.7 (28.5, 31.0)
55–64 (n=3195) 41.5 (39.8, 43.2) 11.4 (10.3, 12.6) 47.1 (45.4, 48.9)
65–74 (n=1398) 21.2 (19.1, 23.5) 8.3 (6.9, 9.9) 70.5 (68.0, 72.8)
Total 35–74
(n=14,173)
59.8 (59.0, 60.6) 5.9 (5.5, 6.3) 34.3 (33.5, 35.1)
CSP 3 35–44 (n=572) 85.3 (82.1, 88.1) 2.6 (1.5, 4.3) 12.1 (9.5, 15.0)
45–54 (n=828) 73.4 (70.3, 76.4) 12.0 (9.8, 14.4) 14.6 (12.3, 17.2)
55–64 (n=585) 50.9 (46.9, 55.1) 19.8 (16.7, 23.3) 29.2 (25.6, 33.1)
65–74 (n=266) 30.1 (24.6, 36.0) 12.4 (8.7, 17.0) 57.5 (51.3, 63.5)
Total 35–74
(n=2251)
65.5 (63.5, 67.5) 11.7 (10.4, 13.1) 22.8 (21.1, 24.6)
The figures in this table are derived from the June 2020 National Key Performance Indicators.7
CI, confidence interval; CSP, clinical software product; CVD, cardiovascular disease.

Although some variation was expected due to differences in patient populations, the CSP with the lowest proportion of patients with high CVD risk corresponded to the CSP with missing variables that conferred clinically determined high risk of CVD. Variation was also most pronounced in younger age groups where clinically determined high risk is of greater importance.

In our audit of CAT4, we found that both clinically determined high risk and FRE were used correctly to calculate CVD risk. However, there were inconsistencies in the definitions of patients clinically determined as at high risk of CVD and non-smokers.8 Of greater consequence was the absence of a date range filter that permitted inclusion of CVD risk variables such as blood pressure, cholesterol or diabetes assessment taken outside the period recommended by the 2012 guidelines. As a result, out-of-date clinical measurements, sometimes recorded more than 10 years previously, were used to calculate a patient’s risk of CVD.8

The potential effect of the absence of a cut-off date was highlighted in the AIHW’s second Practice Incentive Program Quality Improvement (PIP QI) report.9 Results for the quality improvement measure on the proportion of regular clients with CVD risk factors recorded ranged from 52% among CAT4 extracts to 37.5% for the alternative data extraction tool, POLAR, and could be explained by the use of a date range filter by POLAR only.

Together, these results show that a decade following the release of the 2012 CVD risk guidelines, unregulated implementation of risk algorithms has resulted in tools that might miss identifying patients at high CVD risk and cannot support accurate quality improvement.

The new CVD risk algorithm expands the variables in the 2012 CVD risk calculator. Additional variables are included for the standard risk calculator and for patients with diabetes. Several reclassification factors are also included for clinicians to consider. The risk algorithm is more complicated and prone to error if not validated and systematically implemented.

Avoidance of the same mistakes being repeated requires better governance of CVD risk algorithms. In Aotearoa New Zealand, developers of their CVD risk algorithm called for national leadership in implementing a unified national CVD risk calculator.10 In Australia, we propose similar action with the use of centralised cloud-based tools, embedded within CSP, to ensure the availability of validated and current equations for high-quality CVD risk assessment. Governance mechanisms could be developed to address Indigenous data sovereignty, data management and security considerations. Collaboration with the peak general practice colleges, Heart Foundation and software developers, led by the Australian Digital Health Agency, could assist in improving the governance and development of CVD risk calculators in GP CSP.

Competing interests: KG’s salary has been supported by the Medical Research Future Fund 2021 Genomics Health Futures Mission (Streams 1–2 and 6, Application 2015961) for ‘Preparing Australia for use of genomics in prevention of heart-disease: Focus On South Asian Australians’. JA, KS, DW, AT and RW have no competing interests to declare.
Provenance and peer review: Not commissioned, externally peer reviewed.
Funding: The work was supported by funding from the Australian Government Department of Health and Aged Care First Nations Health Division. The funder had no role in the study design, data collection and analysis, decision to publish or preparation of the manuscript.
Correspondence to: jason.agostino@anu.edu.au
Acknowledgements
The authors thank Deanna Pagini and Felicity Murdoch from the Australian Institute of Health and Welfare (AIHW) for supporting the additional analysis of AIHW data.
This event attracts CPD points and can be self recorded

Did you know you can now log your CPD with a click of a button?

Create Quick log
References
  1. Commonwealth of Australia as represented by the Department of Health and Aged Care. Australian guideline for assessing and managing cardiovascular disease risk. Commonwealth of Australia as represented by the Department of Health and Aged Care, 2023. Available at www.cvdcheck.org.au/?utm_source=website&utm_medium=referral&utm_content=guideline-page [Accessed 13 February 2024]. Search PubMed
  2. Bonner C, Fajardo MA, Doust J, McCaffery K, Trevena L. Implementing cardiovascular disease prevention guidelines to translate evidence-based medicine and shared decision making into general practice: Theory-based intervention development, qualitative piloting and quantitative feasibility. Implement Sci 2019;14(1):86. doi: 10.1186/s13012-019-0927-x. Search PubMed
  3. National Vascular Disease Prevention Alliance. Guidelines for the management of absolute cardiovascular risk. National Vascular Disease Prevention Alliance, 2012. Available at www.heartfoundation.org.au/getmedia/4342a70f-4487-496e-bbb0-dae33a47fcb2/Absolute-CVD-Risk-Full-Guidelines_2.pdf [Accessed 13 February 2024]. Search PubMed
  4. Australian Institute of Health and Welfare (AIHW). Aboriginal and Torres Strait Islander specific primary health care: Results from the OSR and nKPI collections. AIHW, 2023. Available at www.aihw.gov.au/reports/indigenous-australians/indigenous-primary-health-care-results-osr-nkpi/contents/summary [Accessed 13 February 2024]. Search PubMed
  5. Australian Government Department of Health and Aged Care. Practice Incentives Program Quality Improvement incentive – Guidance. Australian Government Department of Health and Aged Care, 2021. Available at www1.health.gov.au/internet/main/publishing.nsf/Content/PIP-QI_Incentive_guidance [Accessed 14 June 2023]. Search PubMed
  6. Calabria B, Korda RJ, Lovett RW, et al. Absolute cardiovascular disease risk and lipid-lowering therapy among Aboriginal and Torres Strait Islander Australians. Med J Aust 2018;209(1):35–41. doi: 10.5694/mja17.00897. Search PubMed
  7. Australian Institute of Health and Welfare (AIHW). Aboriginal and Torres Strait Islander specific primary health care: Results from the OSR and nKPI collections (archived version). AIHW, 2021. Available at www.aihw.gov.au/getmedia/88fc4b74-f0b4-4a80-ab21-46fe7af2e132/combined-nkpi-and-osr-report-archived-01-june-2021-release.pdf.aspx [Accessed 16 February 2024]. Search PubMed
  8. Greaves K. Applied epidemiology in cardiovascular disease and in COVID-19 in Queensland [Master’s thesis]. Australian National University, 2022. Available at https://openresearch-repository.anu.edu.au/bitstream/1885/261377/1/MAE%20Thesis%20Kim%20Greaves%20u6263507%20revised%20230222%20Tracked%20changes%20accepted.pdf [Accessed 16 February 2024]. Search PubMed
  9. Australian Institute of Health Welfare (AIHW). Practice Incentives Program Quality Improvement measures: Data update 2021–22. AIHW, 2022. Available at www.aihw.gov.au/reports/primary-health-care/practice-incentives-program-measures-2021-22/contents/about [Accessed 13 February 2024]. Search PubMed
  10. Kerr A, Wells S, Moffitt A, et al. A unified national cardiovascular disease (CVD) risk generator is required to address equity in the management of CVD risk in clinical practice in New Zealand. N Z Med J 2019;132(1500):89–94. Search PubMed

Health servicesHeart diseasePreventive medicineQuality of healthcare

Download article