Rapid cycle evaluation of a program to enhance the readiness and response to family violence in general practices

Family and domestic violence (FDV) is a prevalent public health and human rights issue. FDV is also highly gendered, with most victim-survivors being women and children. One in six girls in Australia experience physical and/or sexual abuse before the age of 15 years, with the same proportion of women experiencing physical and/or sexual violence by a current or previous partner from the age of 15 years.¹ Exposure to FDV causes long-term physical, mental and social health effects, with family violence being the leading cause of premature death and ill-health for women under the age of 45 years and the seventh biggest contributor in women of all ages.²

Victim-survivors face many barriers in accessing the support they need to optimise their safety and wellbeing.³ However, research confirms that victims want their general practitioner (GP) to enquire about FDV.^4,5 Victim-survivors frequently attend general practices,³ although most are not asked about their experiences of FDV, representing a significant lost opportunity for providing support.³ GPs and other practice staff also experience barriers in engaging with victim-survivors, including a lack of time, inadequate training and skills, and limited access to specialist services.^6,7 To help address these barriers, the Australian Government funded six Primary Health Networks (PHNs) across Australia to develop training packages for general practices. This research was conducted to evaluate a new program for practices in regional Victoria.

Methods

Ethical approval was obtained from The Royal Australian College of General Practitioners’ (RACGP’s) Human Research Ethics Committee (MG02933). The research was a mixed methods study using rapid-cycle evaluation (RCE) in parallel with a program co-design process. RCE is an approach for evaluating and enhancing a new intervention, both to optimise the program design and demonstrate early indicators of program effectiveness (eg changes in awareness, knowledge and confidence) to avoid larger investments of time and money in ineffective programs or unnecessary components within programs that occur with more traditional styles of program implementation where the evaluation occurs only at the end of a funding period.^8,9 It involves rapid testing and analysis over successive implementation cycles using short-term outcome data for early optimisation of the program, including the logistics, content and learning activities, to help maximise the desired longer-term outcomes.^9,10

The project team consisted of two GPs and two FDV specialists, all of whom had experience working with victim-survivors and facilitating FDV training programs, as well as a project manager. The co-design and research lead (MG) has 15 years’ experience in both program design and research, including RCE. The project team completed a literature review and needs assessment before developing the first version of the training program. The training was then implemented over three cycles, with data reviewed after each cycle by the project team and adjustments made to the program as indicated by the data. During these workshops, the project team also contributed observational data about the program, and these data were also included in the qualitative analysis.

The program consisted of two separate training sessions, including a one-hour lunchtime session for administrative staff and a two-hour evening session for clinical staff. Both sessions provided information and skills relevant to these groups and their respective roles in addressing FDV within the practice. Members of the project team facilitated the training in pairs, with either two FDV specialists or a GP and FDV specialist, and with other members attending the training as observers. This was followed by four practice visits by the FDV specialists to further support and capacitate the practices.

Recruitment and participants

The research took place between June and December 2022 in six general practices in regional Victoria. Purposive sampling was used to recruit practices, with information about the project disseminated online by the PHN. Practices who applied were provided with more detailed information and screened for program readiness before being selected. Staff were invited by the practice manager to participate in the training. However, staff could attend the training without participating in the research. The participant information sheet was reviewed verbally by program facilitators immediately after each training session, and any final concerns and questions addressed before the survey was completed by consenting attendees. Practices received an incentive package for program participation, with individual staff receiving gift cards for completing post-training surveys and individual interviews.

For the interviews, convenience sampling was used to recruit staff from different roles in the practice; staff were approached by the FDV specialists during their final practice visits. Members of the project team were involved in both the co-design and facilitation, as well as being research participants; observational data were collected from team workshops. To reduce any perceived pressure by team members to participate in the research, a PHN employee with research experience outside the project team who was not an authority figure conducted the consent process. Project team members could elect not to participate in the research while still having their specific feedback incorporated into the program design.

The project team recognised that FDV are highly sensitive, pervasive issues that can cause distress, especially for those with lived experience. This risk was acknowledged before each training session and interview, with information provided about support services. Voluntary participation was emphasised, with permission given for participants to leave the training or interview at any time. The FDV specialists are also experienced in detecting and responding to distress.

Data collection

Process data focused on program activities, and outputs were collected by the project team using an online form following each practice visit.

There were different surveys for each training session. These surveys consisted of approximately 20 questions, including basic demographic information; perceived changes in knowledge, attitudes and confidence; and feedback about the training, including both rating scales and short answer questions. Outcomes measures, including changes in awareness, knowledge and confidence, were adapted from existing survey tools to reflect the training content of each session. These short-term measures were selected as predictors of longer-term or sustained effects, reflecting the research goals of ensuring program ‘evaluability’ demonstrated by positive immediate outcomes. The survey was reviewed for clarity and usability by the project team and external academics prior to use. Program participants completed the survey on their mobile phones immediately following the training. Some survey questions asked participants to self-report their awareness and knowledge of FDV topics before and after the training in a single-survey instrument, encouraging them to make a comparison with their awareness and knowledge before the training to after its completion.

Face-to-face, semi-structured interviews of between 15 and 45 minutes were conducted during office hours at times convenient to the staff in a private clinic room. The interview guide was developed by MG and LG, who are experienced qualitative researchers, and structured to reflect the program content and goals.

The five project team workshops occurred either virtually or in an office location, ensuring privacy and confidentiality of the information shared. The interviews and workshops were audio recorded on the PHN’s secure Microsoft Teams account, with the participants’ consent, and saved in password‑protected folders on its server and the lead researcher’s computer. The recordings were auto-transcribed and cleaned by members of the project team. Transcribed data did not contain identifiable information, as codes were assigned to each respondent.

Data analysis

Quantitative data were analysed using SPSS software (IBM, Armonk, NY, USA). Participant background and program feedback were summarised using descriptive statistics, including counts, average scores and percentages. As the data were confirmed to be parametric, a paired samples t-test was used to assess mean pre- and post-training knowledge and attitude scores. Because pre and post data were collected simultaneously, participant scores were automatically matched.

Qualitative data from the individual interviews and workshops were analysed as a combined dataset, representing different but complementary perspectives on the same question. The initial familiarisation with the data revealed complementary themes arising from both the interviews and workshop transcripts, and thus a single coding approach was used. Themes and subthemes were identified based on recurring patterns, as outlined by Braun and Clarke.¹¹ The analysis was completed by MG with oversight by LG, who both have experience in qualitative research. The findings were then presented to the project team for feedback and validation. Regarding reflexivity, the team’s perspectives included experience in general practice, public health, management, the family violence sector, working directly with victim‑survivors, as well as lived experience of FDV.

Results

Process data

A total of six practices across five shires in urban and rural Victoria enrolled in the program. As seen in Table 1, 29 mostly administrative staff attended Session 1, with 36 GPs, nurses or practice managers attending Session 2. The FDV specialists conducted six visits to each practice; four following the training sessions.

Quantitative data

As shown in Table 2, practice staff rated both training sessions highly, with scores of more than nine out of 10 for facilitators’ competence and likelihood of recommending the program to other practices. Most participants reported the training to be highly relevant to their role.

When asked how much their knowledge, attitude and responses to FDV had changed, 93% of attendees for Session 1 reported this as ‘quite a bit’ or ‘extremely’ (Table 3). Similarly, most participants attending Session 2 reported perceived changes in knowledge and confidence as ‘quite a bit’ or ‘extremely’, including 92% for knowledge of support services, 97% for confidence in assessing risk, 89% for confidence in completing safety plans and 83% for confidence in providing long-term support to survivors (Table 4).

As shown in Table 5, pre-post survey scores for Session 1 demonstrated statistically significant increases in alertness and awareness to the possibility of patients experiencing FDV of between 1.58 and 2.86 (out of 10) (P<0.001). Session 2 participants reported large changes in perceived knowledge and confidence in identifying and supporting victim-survivors (Table 6). Pre-and post-training levels of alertness to FDV, awareness of clinical indicators and knowledge of how to respond to disclosures also increased by between 3.03 and 3.39 points (P<0.001). Importantly, the likelihood of clinical staff asking a patient about FDV, if indicated, increased from 5.31 to 8.39 (out of 10) following the training (P<0.001).

Qualitative data

Five themes emerged from the thematic analysis. These included navigating the multiple layers of complexity associated with implementing the FDV training program in a general practice environment, providing victim-centred care, tailoring the program to the needs of practices, the program’s influence on the practice staff and recommendations for program enhancement.

Theme 1. Navigating multiple layers of complexity

In designing and implementing the program, the project team navigated multiple layers of complexity. This included complexity within the practices, such as having multiple competing demands with GPs being time poor, affecting both their availability to participate in the training and provide care to victim-survivors. Beliefs and values about FDV both at an organisational and individual level also influenced the responsiveness to the program:

One GP in particular … he became quite escalated; he stood up over everybody … He had a very unique perspective of family violence and that our statistics were corrupt … The other GPs in the room were clearly uncomfortable. (Project team [PT] 3)

Similarly, the complexities and risks of FDV were recognised, with victim-survivors having long-term, multifaceted care needs requiring highly specialised knowledge and skills:

(It’s a) long-term journey that a GP might go on (with the victim-survivor) and what that might look like in terms of ongoing support and accessing different services or crisis accommodation or what happened in the end. (PT2)

Theme 2. Providing victim-centred care

To navigate these complexities appropriately, ensuring that practice staff did not inadvertently escalate a victim-survivor’s risk, the project team highlighted the need to keep the training client focused, placing the victim‑survivor as the expert of their experience and needs. Training participants’ responses suggested that this message was absorbed:

I want to dive in and jump in and help. As much as I see things happening around me, it reminded me that the person most affected is the one that I’m going to focus on. It’s not about me, and I have an agenda. I want to save you. But it’s not about me; it’s about how I help you. (Practice Manager [PM] 1)

Theme 3. Tailoring the program to the needs of general practices

To further address these complexities, the data highlighted the importance of the program being tailored to the needs of practices. This included customising the training content to address the additional time and workforce pressures of practising in regional Australia, including providing simple, consistent messages and quick reference materials and clinical tools while remaining responsive to the varying levels of readiness of each practice. This was reflected in participant feedback about the program:

It was great, it was short, it was to the point, and we didn’t feel like we were bogged down with too much information. It was just the right amount of time. And during a lunch break was fine. (Receptionist [R] 3)

Having clear role definitions and policies and procedures that each practice adapted to their specific needs, and localised referral processes and connections with the broader FDV sector were also seen as important to ensure staff felt supported and able to easily access specialist services:

It can seem really overwhelming if that’s not your core business. I think always reinforcing that, you know, you don’t have to do it on your own. This is a shared responsibility – seek help, get a secondary consult. (PT3)

Theme 4. Influencing the practice’s recognition and response to FDV

The practice staff and project team observed that practices were better able to recognise and respond to FDV following program participation. Both administrative and clinical staff reported being more alert to and aware of the signs of FDV within their respective roles:

There are so many more aspects to it (beyond physical violence) …. that don’t always come to the forefront of your thoughts … There’s the emotional, financial, and isolating and controlling. (Nurse [N] 5)

The project team observed that staff were more aware of indicators of FDV both in the clinic and community and were requesting secondary consultations, with several new disclosures:

(After the training) there were definitely red flags being noticed, and even some out in the community. (PT3)

This increased engagement by practices in FDV was enhanced by practice staff feeling more supported. This support came from multiple areas, including the information and resources provided, improved teamwork within their practice, the FDV specialist and the broader family violence sector.

Before the program, I didn’t know much about what to do or how to get around it. I do now; I’m more confident in my knowledge base and on where to direct victims for support and how to offer support to them, while they’re here… if I don’t have all the answers at the time for the person that comes in, I know where I can go to get them or where I can send them to get the answers that they need. (N6)

Theme 5. Recommendations for program enhancement

Recommendations were made to enhance the program content, with participants requesting more information on higher risk groups, such as the elderly and culturally and linguistically diverse populations. Although minor changes were made to the logistics, most discussion focused on addressing issues that arose from the responses of practice staff to the training, including resistance to the gendered nature of FDV, discomfort and emotional distress by those with lived experience. However, the strongest recommendation from practice staff was to ensure ongoing support was available to the practice by FDV specialists:

I think just the fact that we know that you are there, and I think if something comes up, we would seek help … If there was a gap that we identified, I think that’s when we approach you. (GP3)

Discussion

This research adds to the emerging body of literature that confirms training can improve health professionals’ readiness to respond to FDV. Readiness in this context has been described as including being motivated, understanding the importance of a health system response, taking an advocacy approach, working with others, and being supported with ongoing domestic violence and abuse training, clinical protocols, and tools and leadership in the health system.¹² The quantitative and qualitative findings confirm that awareness, alertness, knowledge, confidence, teamwork and feelings of being supported were strengthened by the training.

Although anecdotal evidence indicated that FDV might not have always been recognised or enquired about in practice before the training, the research confirmed that practice staff can be equipped to do this through the provision of simple, consistent messages, practical tools and access to quick reference materials and referral pathways. Although the training was relatively brief, the post-training surveys demonstrated large changes in perceived knowledge and confidence in identifying and supporting victims that was triangulated in the qualitative data. Although, as the research by Leung et al suggests,¹³ training of longer duration might have a greater effect, this might not be feasible within a busy practice environment. Furthermore, a recent meta-analysis concluded that a range of strategies and program ‘doses’ can be effective and that there is unlikely to be a ‘one-size-fits-all’ approach.¹⁴

There was evidence that practices and individuals within a practice were not equally receptive to the program, with relevant research suggesting that legitimisation of work by staff at an individual level and differences in the ethos and culture at a practice level can influence the effects of training.¹⁵ However, resistance towards FDV training or the gendered nature of FDV cannot be addressed solely in programs such as this but must also continue to be acknowledged and addressed by broader society and its key institutions.¹⁶

Limitations of the research included potential bias with the project team completing both the co-design and research arms. This is common to all internal program evaluations and conveys significant advantages. As described by Cooke et al,¹⁷ advantages include proximity to the implementation and program participants and a deeper understanding of the program dynamics, ensuring validity and translatability of the findings. Furthermore, as embedded within RCE, the purpose of the research was program enhancement by identifying opportunities for improvement rather than highlighting positive outcomes. However, to address the risk of bias, LG provided oversight and an external perspective. Similarly, the influence selection and social desirability biases within the interview responses of the practice staff is recognised. Triangulation of the interviews with qualitative data from the post-training survey and project team workshops demonstrated convergence of the findings and strengthened their validity.

The Australian Government has continued funding for the program over the next three years. The research findings suggest that the program is logistically suitable for the busy, demanding general practice environment and meets participants’ learning needs, as reflected in early indicators of enhanced readiness to respond to FDV. The program is thus ready for a more rigorous evaluation that measures longer-term, quantifiable outcomes, as was the aim of the research. The research recommendations can be addressed during the next funding cycle. To enable sustainability, emphasis will be placed on further strengthening the connections between the practices and local FDV services. This is critical because, as also described by Hegarty et al,¹² readiness to address FDV is mediated through feeling supported by the broader family violence sector. Using FDV specialists embedded within local FDV services to facilitate the program will also help to ensure the program is scalable.

Conclusion

FDV is a common and serious public health issue, with general practices being a critical component of a holistic health system response. The research confirmed that the project achieved its aim of developing an introductory level program to improve the readiness and capacity of practices to identify and support victim-survivors of FDV, which was specifically tailored to the needs of practices in regional Victoria.

Although there are multiple layers of complexity involved in recognising and responding to FDV that both training providers and practices need to navigate, our research demonstrates that there are practices that want to be part of the response and that programs like this can support them in doing so. This is particularly true when they understand their role, as described by the World Health Organization,¹⁸ as a ‘gateway to comprehensive care’ rather than having to carry the complexities of providing support to victim-survivors alone. However, more research is needed to determine the longer-term effects of the program and to confirm what support general practices need for sustained effects. Although beyond the scope of this article, further research and interventions addressing resistance to FDV content by some practices and staff are also needed but must be framed within broader efforts to address societal attitudes towards women and gender-based violence.