Accreditation processes have been developed to enhance quality and safety outcomes across several healthcare sectors.1 Accreditation programs originated in hospital sectors; however, their implementation in general practice is now prevalent.2,3 Australian general practices have shown a high rate of engagement with accreditation (~84% were accredited in 2020), which might be partly due to economic incentives.4 However, concerns regarding levels of active implementation of quality assurance processes reflective of contemporary standards have been raised.5
Australian general practice accreditation is a multi-step process involving registration with an accrediting agency, completion of a self-assessment, an onsite surveyor audit according to the fifth edition standards,6 and remediation of any non-conformant indicators, within a three-year cycle.7 Registration with the accrediting agency might occur up to 18 months prior to a practice’s accreditation expiry date, allowing for an appropriate period for facilitated learning and communication between the practice and the accreditation agency.8 An ample period between assignation of an accrediting agency and the re-accreditation expiry provides sufficient opportunity to implement quality assurance processes in preparation for the site visit. The impact of preparation time between general practices and accrediting agencies with site-visit performance has not, until now, been empirically evaluated.
A measure of performance within the general practice accreditation process is the rate of indicator conformity at the site visit. Survey visits are completed by a two-person team that includes at least one general practitioner (GP),6 and determines conformance to 124 indicators set out by The Royal Australian College of General Practitioners (RACGP).6 Practices are then presented with a report reflecting their compliance with all indicators and provided with 65 businesses days to remediate any indicator non-conformity, prior to an accreditation certificate being issued. Performance at the site visit varies between practices,9–11 and it is of particular interest to investigate whether time to prepare for the site visit between practices and accrediting agencies partially explains this variability in performance.
Two further outcomes within the accreditation process reflective of performance after the site visit are available. These include: (1) time to remediate non-conformant indicators; and (2) the number of transactions between practices and the accreditation agency required to remediate non-conformant indicators. These metrics are critical to the operational capacity of accrediting agencies and extend accreditation timelines for general practices.5 It is important to identify predictors for practices that require a longer time and greater support to remediate non-conformant indicators, so resources (eg increased support from the agency) can by suitably distributed.
It is the aim of the current study to evaluate the association between several metrics reflective of the time to prepare for the site visit with: (1) indicator conformity at the site visit; (2) the time to remediate non-conformant indicators after the site visit; and (3) the level of assistance required to remediate non-conformant indicators after the site visit.
Methods
Data sources and study population
The data encompass consecutive Australian general practice accreditation cycles made between December 2020 and July 2022. Data were recorded from the practice prior to, during and soon after the accreditation site visit using a proprietary web-based application commissioned by the accrediting agency. As part of the National General Practice Accreditation Scheme, data are routinely reported to the Australian Commission on Safety and Quality in Health Care for performance monitoring.7 Surveillance visits, non-standard medical practices and after-hours practices were excluded from the analyses.
The Macquarie University Human Research Ethics Committee confirms that our project is exempt from ethical review.12
Study variables
Time variables
Three variables were synthesised as measures of the time to prepare for general practice accreditation. These variables represent key periods throughout the accreditation process, with less time potentially representing reduced ability to prepare and implement quality assurance processes important for obtaining positive outcomes from the accreditation processes. Registration reflects the initial process in assigning an accrediting agency by general practices. The specific time periods available include:
- the period between date of registration with the accreditation agency and current expiry date of the practice accreditation cycle (months). Practices who were new to the accreditation process (had no current expiry date) were removed from analyses utilising this variable
- the period between date of registration with the accreditation agency and the planned site visit (months)
- the period between submission of the self-assessment and the planned site visit (months).
Indicator conformity
Met and not met compliance (binary) scores for each indicator from the site visit were provided. Site visits were completed by a two-person team and includes at least one GP. The GP surveyor must have at least five years’ full-time equivalent experience as a vocationally registered GP and be working at least two sessions a week in face-to-face patient contact in an accredited practice.6
Time to remediate indicator non-conformity
Practices are provided with 65 business days to remediate any non-conformant indicators, with assistance from the accrediting agency. The actual number of business days required by practices to remediate all non-conformant indicators was counted. A longer time to remediate non-conformant indicators might be related to a higher number of non-conformant indicators or reduced compliance with the accreditation process.
Transactions required to remediate indicator non-conformity
Following the site visit, the accreditation agency will submit a report detailing the outcome of the assessment and provide recommendations to the practice on how to remediate indicators identified as non-conformant. The total number of transactions between the accreditation agency and practice were recorded. A higher number of transactions with the accreditation agency during this period represents greater assistance required by the practice to remediate non-conformant indicators.
Confounders
Available confounding variables were included in statistical models described below. These were the size of the practice (GP headcount), number of previous accreditation cycles made with the accreditation agency, and an urban or rural location.
Statistical analysis approach
Practice characteristics were displayed as counts (numbers) and percentages for categorical variables and means and standard deviations for quantitative variables. All analyses were conducted in STATA v17 (StataCorp).13
We evaluated the association between time variables prior to the site visits with indicator conformity assessed at the site visit using separate mixed-effect, multilevel logistic regression models. These were conducted with indicator conformity as the dependant variable and each of four time variables (registration expiry; registration COVID expiry; registration site visit; or self-assessment submission site visit) as the independent variables. In all statistical models, the primary sampling unit was the practice and included the three confounding variables. Multilevel models have been used as the data are viewed as arising from a multilevel sampling design in which repeated measures are taken on each practice, and a sample of practices has been recruited.
We evaluated the association between time variables prior to the site visit with compliance after the site visit. Separate regression models were conducted with the two measures recorded after the site visit (time to remediate non-conformant indicators and number of transactions required to remediate non-conformant indicators) as the dependant variables and the time variables described above as the independent variables. Both independent and dependant variables were standardised into the same time unit (months) to aid interpretation of model coefficients. Regression models were used to calculate practice performance post site visit (time to remediate and number of transactions) of all non-conformant indicators, as this is often viewed at a practice level.
Results
Of the 757 practices with data, 15 were removed from the analyses due to these practices transitioning over from another accreditation agency after their expiry date. Within this sample, 122 practices were new to the accreditation process, and therefore did not have an accreditation expiry date, thus were not included in the relevant statistical models. In addition, there were 88 (12%) practices that were conformant to all indicators at the site visit and therefore had their post site visit variables counted as zero. Table 1 shows available practice characteristics and a summary of all study variables.
| Table 1. Practice characteristics and study variables |
| Sample size (n) |
Variables |
Metrics |
| 742 |
Urban location, n / % |
476 / 64 |
| 742 |
GP head count, m (SD) |
5.92 (4.21) |
| 742 |
Number of previous GP accreditation cycles, m (SD) |
3.14 (1.79) |
| 742 |
Corporate-owned practices, n / % |
296 / 39.9 |
| 620 |
Registration – Expiry period, m (SD) |
11.43 (4.14) |
| 620 |
Registration – COVID-19 extension expiry period, m (SD) |
23.42 (4.14) |
| 742 |
Registration – Survey visit period, m (SD) |
17.22 (5.31) |
| 742 |
Submission of self-assessment – Survey visit period, m (SD) |
4.68 (3.38) |
| 742 |
Indicators non-conformant at survey visit, m (SD) |
7.00 (7.27) |
| 742 |
Days to remediate indicator non-conformity, m (SD) |
36.9 (25.13) |
| 742 |
Total transactions with accreditation agency to remediate indicator non-conformity, m (SD) |
21.93 (28.72) |
| Date range calculated as months unless otherwise specified. Transactions represent communication within the proprietary system after the site visit related specifically to indicator non-conformities. Bracket numbers indicate standard deviation (SD) of the mean (m) practice characteristics and study variables. |
Table 2 shows the association between time variables prior to the site visit with the rate of indicator conformity. A small, statistically significant and positive association was identified regarding the registration–accreditation expiry period, indicating practices with a larger period between registration with the accrediting agency and their current accreditation expiry experienced a higher number of non-conformant indicators.
| Table 2. Association between time variables prior to the site visit with the rate of indicator conformity at the site visit |
| |
Odds ratio (95% CI), z-score, P value |
| Registration – accreditation expiry |
1.04 (1.02–1.06), 3.26, 0.001 |
| Registration – site visit |
1.008 (0.99–1.03), 0.93, 0.35 |
| Self-assessment submission – site visit |
1.009 (0.99–1.03), 0.65, 0.52 |
| Time interval unit of measurement is months. Confounders included in the model include general practitioner headcount, number of previous accreditation cycles with accrediting agency and urban or rural location. Bold indicates statistically significant models. |
Table 3 shows the association between the time variables prior to the site visit with measures recorded after the site visit. A small, negative and statistically significant relationship was identified between the registration–accreditation expiry period and the time to remediate non-conformant indicators (Appendix 1). This indicates practices with a larger period between registration with the accrediting agency and their current accreditation expiry were associated with less time to remediate non-conformant indicators. A small, negative and statistically significant relationship was identified between the self-assessment submission–site visit period with the time to remediate non-conformant indicators (Appendix 2). This indicates practices who submitted the self-assessment early relative to their site visit exhibited a shorter time to remediate non-conformant indicators. A moderate, negative and statistically significant relationship was identified between the registration–accreditation expiry period and the number of transactions required to remediate non-conformant indicators, indicating practices who register early with the accrediting agency relative to their accreditation expiry require less assistance to remediate non-conformant indicators (Appendix 3).
| Table 3. Association between time variables prior to the site visit and post-site visit measures |
| |
β (Standard error), z-score, P value |
| Time to remediate non-conformities |
Number of transactions |
| Registration – accreditation expiry |
–0.02 (0.008), –3.07, 0.002 |
–0.66 (0.28), –2.40, 0.02 |
| Registration – site visit |
–0.006 (0.006), –1.06, 0.29 |
–0.19 (0.21), –0.90, 0.37 |
| Self-assessment submission – site visit |
–0.02 (0.009), –2.73, 0.006 |
–0.40 (0.31), –1.29, 0.20 |
| All time variables are standardised to months. Bold indicates statistically significant models. Confounders include general practitioner headcount, number of previous accreditation cycles with accrediting agency and urban or rural location. |
Discussion
This study sought to evaluate the relationship between several metrics reflective of preparation time with accreditation performance at, and after, the mandatory site visit. Considering the widely adopted uptake of accreditation practices in healthcare, there continues to be a lack of empirically driven research in this field.14 We identified that a greater period between registration with the accrediting agency and the practice accreditation expiry date was associated with higher indicator conformity at the site visit, as well as less time and less assistance required to remediate non-conformant indicators. Further, late submission of self-assessment prior to the site visit was associated with a longer period to remediate non-conformant indicators.
There were consistent results with respect to the period of registration with the accrediting agency and current general practice accreditation expiry date. Registration reflected the date at which general practices assigned an accrediting agency, which is the first point of contact in the (re)accreditation process.7 General practice staff perceptions and attitudes towards accreditation might impact the positive outcomes of, and engagement to, accreditation.8 Specifically, attitudes related to the accreditation process have consistently reported time constraints by support staff and practitioners in addition to their day-to-day roles.8,15,16 Accreditation-specific tasks such as registering for accreditation, completion of the self-assessment, implementation of quality assurances processes and logistical components related to organising site visits can take a considerable amount of time for general practices to complete. Although these tasks are required by all practices undergoing accreditation, our data highlight significant variation in the time general practices register for (re)accreditation with the accrediting agencies, as well as how long it takes to complete tasks, such as submission of self-assessment. Importantly, this variation in time to prepare for the accreditation site visit partially explains performance at the site visit, the time to remediate non-conformant indicators and the assistance required to remediate non-conformant indicators post site visit. Our results emphasise the importance of a larger time period to prepare for accreditation-specific tasks within the current three-year accreditation cycle for general practices, as well as potentially disincentivise late registration for re-accreditation or submission of the self-assessment.
There are several limitations to the current study. First, the study period overlaps with the COVID-19 pandemic and the associated impacts, and practices were provided with an extension to their accreditation expiry as a result. It is important to replicate these analyses in future periods without a global pandemic significantly impacting general practices. Second, the results presented reflect data from a single accreditation agency, and comparison with other providers is encouraged. Third, as this is a secondary analysis and we are limited in our investigation by the information currently measured, alterative and more specific metrics, such as self-reported questionnaires, might yield additional useful insights. Finally, our analyses assumed equal weighting to all non-conformant indicators, where in practice, non-conformant indicators are risk stratified as low, moderate, high or critical and this might be something that future research should consider.
Limited research exists investigating the relationship between general practice time to prepare for accreditation and performance outcomes. We identified that greater time to prepare within the accreditation process was associated with small-to-moderate improvements in performance at and after the mandatory site visit. These results provide an empirical examination of several components within the current Australian general practice accreditation framework and might guide the future implementation of the accreditation program.