The only chance for a ‘bird’s-eye view’: General practice registrar experiences of direct observation

Registrars in the Australian General Practice Training (AGPT) Program commence patient consultations on their own from the outset of their placements.^1–4 In some training settings registrars are directed to contact their supervisor under specific circumstances; however, typically, registrars choose to only discuss with their supervisor those patient encounters they deem to be clinically challenging.^1,4,5 This differs from other countries with a comparable primary healthcare system,² and from international medical graduates commencing general practice training in Australia with level 1 supervision, where all patients need to be discussed with their supervisor.¹ Although a variety of measures are put in place to support registrars commencing in general practice, one of which is direct observation, the safety of this largely unobserved practice has been questioned.^1,3,6

To improve patient safety, the introduction of a mandated period of direct observation, particularly during transition from hospital to general practice, has been suggested.^5,7

Direct observation is defined as the purposeful observation of trainees in real time with feedback⁸ where the ‘master’ clinician watches and provides feedback to the ‘apprentice’. This serves two purposes: first, assessment at various stages of training during real clinical work at the top ‘does’ level of Miller’s pyramid;⁹ and second, supporting the learner with formative feedback and coaching, guiding them towards achieving their learning goals.¹⁰

There is ample evidence for the use of direct observation in assessment, but when it does occur its utility for learning is often questioned, especially by registrars.^3–6,11,12 Additionally, most literature on direct observation focuses on tertiary care settings and procedural skills rather than the primary care setting and consultation and communication skills.^8,10 It also lacks the perspectives of registrars.

Although direct observation is an important teaching technique in general practice training, evidence shows that it does not occur as frequently as expected by regional training organisations (RTOs) that currently deliver training and colleges that oversee training.^3,4,6,11,12 Implementing an initial mandatory period of direct observation, as has been suggested, may have unintended consequences and unsatisfactory outcomes if registrar preferences and engagement are not better considered or integrated.

RTOs require supervisors to undertake professional development, although the extent varies. The Victorian RTO where this study was conducted requires all supervisors within three years of their accreditation to complete a mandatory core module, ‘Observing Your Registrar’.

The aim of this study was to explore registrar experiences of direct observation to identify key considerations for using direct observation in general practice training.

Methods

General practice registrars’ experiences of direct observation during clinical practice were explored using a qualitative phenomenological approach.¹³

Purposeful sampling¹⁴ was used to recruit AGPT registrars from an RTO based in Victoria, Australia, across three general practice terms (ie GPT1–GPT3) and training in rural and metropolitan locations. Registrars were identified through the RTO’s database. All eligible participants (approximately 120) were sent email invitations by administration staff together with the plain language statement. The study was also advertised across all registrar workshops. All registrars who expressed interest within the study timeframes were followed up for interview. Sample size was guided by appraisal of information power whereby the more information the sample holds, relevant to the study focus, the fewer participants needed.¹⁵ While the richness of data was enough to meet study objectives, the study was also timebound, which limited the extent of further recruitment.

Semi-structured interviews (Box 1) were conducted by NA between August and December 2020 via videoconference. Interviews were audio-recorded, transcribed verbatim and de-identified. Interviews ranged 35–50 minutes, with the average being 39 minutes. Participants were sent the questions prior to the interview. Data were stored securely and managed using NVivo 12.
 

Thematic analysis^16,17 was used, guided by Braun and Clarke’s six-step method.^16,18,19 The initial coding framework was constructed by NA, and sensitised by the research questions and adult-learning²⁰ and work-based²¹ learning theories. Analysis was iterative, so findings from initial interviews were explored with latter participants. Categories were refined through further coding and interrogation of the data, and discussion during regular team meetings.

NA (a general practice supervisor and experienced medical educator) led the analysis, with BG (an experienced qualitative and primary healthcare researcher) and ST (an experienced primary healthcare medical education researcher and general practitioner [GP]) contributing to analysis and interpretation. Regular meetings between all authors enriched analysis and enhanced interpretive rigour.¹⁴ NA and BG held positions within the registrars’ RTO. ST was external to the organisation. NA was involved in teaching but had no involvement in the assessment of these registrars. BG had no prior relationship with participants.

Ethics approval was provided by the Medical Education Human Ethics Advisory Group (HEAG) at the University of Melbourne (ID: 2057233.1).

Results

Eight registrars responded to the invitation, and seven were interviewed. The eighth registrar did not respond to further efforts to follow up. Most (n = 5) were in their second general practice term, with one in their first and one in their third. Three identified as female. Three were in rural locations, and all of those identified as male.

All registrars reported direct observation to be valuable for their learning, with one registrar stating, ‘it should be a part of every registrar’s experience in general practice’ (Registrar 3). Direct observation was described as a ‘unique opportunity to be observed in their clinical setting …’ (Registrar 3) and potentially ‘the last time you’ve got a chance to get that support and to be observed’ (Registrar 5).

Four main themes were identified: registrar engagement, supervisor engagement, practice engagement and training organisation engagement.

The dynamic tension/interaction between each of these created both enablers and barriers to registrars’ learning from direct observation (Table 1).
 

Registrar engagement

Registrar attributes, attitudes and beliefs

Enthusiasm and motivation were key drivers for engaging with the direct observation process, with registrars initiating direct observation themselves gaining the most benefit and satisfaction. For example, registrars who proactively requested direct observation as an adjunct to routine teaching, organising direct observation sessions in liaison with the practice manager, were more likely to receive direct observation regularly throughout training. Positive outcomes of this included identification of learning needs around consultation skills, improved organisational skills and the formation of positive relationships with supervisors:

So, based on my experience in GPT1 when it’s actually something I pushed for in GPT2 … my supervisor has been very open to me sort of suggesting what I would like … I had to stay on top of it, so I had to be proactive … [Registrar 5]

I think that probably evolved more because I also did request it as well ... [Registrar 7]

Similarly, registrar inertia and lack of motivation had an impact on the direct observation experience:

It wasn’t something that they particularly pushed … was left up to me to initiate … it just got left on the back burner. [Registrar 1]

Registrars acknowledged the discomfort direct observation could create but perceived a rich learning experience from direct observation in terms of revealing their ‘unknown unknowns’. Engagement in direct observation by the registrar was enhanced for those who valued consultation and communication skills and who were aware of this potentially being a ‘last chance’ for observed feedback of these skills:

It’s the last time you’ve got a chance to get that support and to be observed … I feel like it doesn’t happen a lot in in medicine, apart from like clinical exams and then it’s sort of a bit too late. That’s something you need to fix before … [Registrar 5]

Although most registrars acknowledged well-developed communication as being a key skill to master in general practice, this skill is not often prioritised given the sheer volume of knowledge registrars felt they ‘needed to know’, particularly during their first general practice term. Communication and consultation skills were perceived by some as an ‘add-on’ (‘it wasn’t the first thing that came to mind’) and something better focused on towards the end of training. Registrars who prioritised clinical knowledge and skills over communication and consultation skills expressed that direct observation was of low value. They felt that the time required for reflecting and unpacking the consultation process might not yield what they considered to be ‘useful’:

It’s not as high value … if we use that same amount of time to, you know, talk about, say osteoarthritis, we would have covered a whole topic and including some patient examples already. So, I think that definitely is more time consuming for what you get at the end. [Registrar 3]

The anxiety-provoking nature of direct observation, the discomfort of being observed and the fear of ‘losing face’ in front of patients and its effect on rapport affected registrar attitudes towards direct observation. Direct observation improved performance or affected confidence:

… as a registrar you’re on your game a lot more when someone [is] standing there watching you. You certainly feel a little bit of added pressure. [Registrar 2]

It’s very nerve wracking … somebody is watching, don’t say the wrong words … [Registrar 3]

I think it’s a pride thing … [Registrar 4]

Registrar perceived benefits of direct observation

Registrars were more likely to engage positively with direct observation if they perceived a benefit in it for them. ‘What’s in it for me?’ was a key enabler of engagement in direct observation.

Table 2 highlights the benefits identified by registrars.
 

Supervisor engagement

Supervisors’ buy-in to direct observation was another key enabler. Registrars who experienced bi-directional direct observation – observing their supervisor consultations in addition to being observed themselves – found direct observation of their own practice to be less intimidating while helping to strengthen the registrar–supervisor relationship:

It was a real … masterclass on how to do a consultation, like a lot less intimidating, because I was like, right ‘Well, I’ve sat and watched you, so I mean it’s only fair that you get to sit and watch me now’. [Registrar 4]

Registrars identified that if their supervisors were under time pressure due to competing clinical demands in the practice, or were not being adequately remunerated for performing direct observation, they did not appear to ‘buy in’ to the process. Lack of supervisor or practice prioritisation of direct observation and buy-in were a barrier to the initiation of direct observation by the registrar:

… supervisors are obviously trying to do a job themselves and have limited time to give ... [Registrar 2]

… supervisor is sitting in with you, then they’re not seeing patients and I think that is the biggest one. [Registrar 5]

Other things took priority ... [Registrar 1]

Feedback was identified as a vital part of the direct observation session for learning.

To have perceived utility for learning, feedback needed to be timely and consist of reflection, collaboration between the supervisor and registrar, identification of strengths and areas for improvement based on observed behaviours, and lead to individualised learning planning.

While supervisor feedback such as ‘you are doing well, keep doing what you are doing’ boosted confidence for the registrar, immediate individualised constructive feedback post direct observation helped to further learning and progress for the registrars. Additionally, identifying learning points to review at the next direct observation session gave registrars something to work towards. This feedback cycle contributed to building a positive registrar–supervisor relationship.

Practice engagement

Creating a learning culture and embracing direct observation as the ‘norm’ was a key facilitator. These practices reduced the administrative burden by ensuring direct observation was scheduled in advance, blocking off mutual time for both supervisor and registrar, with time for feedback.

Patient willingness to participate in direct observation was also an enabling dimension of such practice cultures.

Having an observed consultation followed by time for discussion worked the best for registrar learning:

Set time works because it’s one less thing for me to do and think about in terms of planning it and putting it in/booking it in so I definitely like that factor that it’s not me trying to keep track of ‘When was the last time I did that? I should set that up, what days are we both here?’ The practice manager just does it. [Registrar 1]

Practice adaptability and embracing technology such as remote observation of consultations using video, particularly during the COVID-19 pandemic, appeared to contribute to overall satisfaction of the registrar experience.

Training organisation engagement

Promotion of direct observation

Promotion of direct observation by the RTO to supervisors and registrars was a key driver for engaging both parties in it. Most registrars noted that while they were provided with some information about direct observation, more could be done to encourage direct observation and highlight its purpose and benefit:

Explaining to registrars early on the purpose of it ... I think reframing that will be really helpful for enhancing the benefits to the registrars … [Registrar 4]

Registrars also suggested that supervisors be given more support and training in setting up and structuring direct observation as well as in feedback provision.

Some registrars expressed uncertainty and a lack of clarity about expectations from direct observation, particularly with frequency, length of direct observation and structure, creating difficulty for registrars to approach their supervisors and practices about direct observation. This limited registrars’ opportunities to learn about their ‘unconscious incompetence’. Registrars who did not experience any direct observation found it difficult to judge if it was a ‘potentially missed opportunity’ versus those that had an experience, albeit negative:

I don’t know if that’s something that other people have had a lot of or even if they get that or if I’ve missed out on potential benefit from not having that. [Registrar 1]

I think now it’s hard to say because if you don’t ever get it, you don’t know what you’re missing out on, in a way. So, if that happened say in GPT2 or 3 then I would probably raise that point and say that it is useful to me. [Registrar 7]

Mandating direct observation, setting competencies, embedding into curriculum

Mandating direct observation and embedding this into the curriculum with specific clinical competencies to be achieved during various stages of training was one suggestion for reducing the uncertainty around expectations. Benchmarking or standard-setting expected levels of performance was thought to allow for better utility of direct observation.

Without the competencies, registrars perceived mandatory direct observation as unlikely to improve engagement and had the potential to become prescriptive. This was perceived as counterproductive, with the potential of it becoming yet another hurdle activity required during training:

… maybe you just mandate it. I mean, you could make a lot more mandated in the early phases and then less in the latter stages … I think the way to make something valid is to enshrine it in curriculum. Otherwise it’s hard to validate it, I guess in some people’s eyes. [Registrar 5]

Structure for direct observation

Registrars identified some structural factors as yielding positive outcomes from direct observation. These included having planned sessions, normalising direct observation in the clinic and specifying frequency and duration of direct observation.

Registrars said that planned direct observation sessions yielded more benefits than ad hoc.

An unplanned or ad-hoc direct observation session that was poorly structured, with lack of explanation of the potential benefits, with limited or no reflection and feedback was seen as a barrier to registrar engagement:

It is generally during planned teaching time rather than just, oh, ‘I’m coming in today’. [Registrar 3]

Apart from GPT1 where the registrars felt it needed to be scheduled at the start, middle and end of semester, there was no uniformity on how often direct observation was thought to be useful. ‘Regular’ direct observation that was individualised to the registrars’ learning needs was considered beneficial. Some suggested direct observation at educationally meaningful points such as scheduled, formalised training organisation–driven feedback sessions during their term would be helpful. All noted that direct observation should not be overly regular or repetitive, with some stating weekly sessions would be excessive. One of the reasons given for less frequent observation was for self-regulation and learning, allowing the opportunity to learn from mistakes. Decisions about frequency were beneficial when made in collaboration with the registrar, considering registrar learning needs:

It can be done once every three months, that kind of thing and then it can be or may be more acceptable to supervisors and to registrars as well. [Registrar 7]

If there was maybe two more opportunities for … compulsory direct supervision … that would be part of either your week six or week 20 feedback. [Registrar 4]

Short sessions with two consultations – or a maximum of one hour for each session – were favoured. This allowed valued time after the consultation for reflection and feedback.

Discussion

This research explored the perspective of registrars and identified key enablers and barriers to direct observation from a registrar perspective. Until now, registrar experiences of direct observation have remained under-explored; this study highlights important contextual complexities of general practice training, and how these influence direct observation structure and experience. The value of direct observation as a learning tool was supported by this study.^3,4,6,11,12 Findings also highlight the need for registrars to know why and how direct observation can enhance their learning. Registrar engagement in direct observation is heavily influenced by their orientation to learning and the different skills that they see as valuable. Gaining insight into these factors can help us understand the key factors needed for learner engagement in direct observation.

General practice training is an apprenticeship-based model where learning predominantly occurs in practice by ‘doing’. Hingeing on this core principle, participating in the day-to- day workplace activities is, in itself, learning. Although registrar engagement is important, supervisor and practice buy-in (work environment) is also key to this experience. The principles of work-based learning²¹ provide insight into the learner’s experience and the challenging and complex dynamics of the work environment and how these influence registrar engagement in direct observation.^21–23

Various factors influence learning, such as the physical workplace environment, the type and duration of the learning activity, level of support and inclusivity and prior experience of all stakeholders.²⁴ The influence of positive support was highlighted in this study where supervisors who provided direct observation in a proactive, structured and reciprocal manner better enabled direct observation. The willingness of patients to participate also contributed positively to the direct observation experience for registrars.

The benefits of direct observation found in this study – such as improved consultation and communication skills through feedback and coaching, and improved patient safety, particularly when a registrar is transitioning from hospital- to community-based training – corresponds with benefits reported in the literature.^5,7,8,10

This study highlights the importance of workplace culture on facilitating individual participation and learning,²¹ and therefore the important role practices and supervisors have for normalising direct observation as an educational activity for registrars.

Limitations and strengths

This study was exploratory and sought to understand the experiences of participants in depth; thus, a sample size of seven was appropriate. This afforded a deeper exploration of registrar experiences and there was no expectation of data saturation. The findings represent the experiences of these participants²⁵ and it was not the intention to generalise the findings. There may be some degree of transferability to other registrars in the same RTO, or to registrars from other RTOs in Australia, though this will be judged by the reader familiar with these contexts. Further research with a broader sample of AGPT Program registrars across other RTOs would likely identify additional facilitators and barriers to direct observation.

The study did not explore how many sessions of direct observation the registrars had during practice. Sampling was drawn from participants self-selecting to be a part of the research, and results may reflect those with extremely positive or negative experiences of direct observation; however, this did not appear to be the case as registrars reflected on a non-polarised range of experiences of direct observation.

This study was focused on the experiences and perceptions of registrars. Experiences of other stakeholders – such as supervisors, practices and training organisations – were beyond the scope of the study.

Given NA’s involvement in medical education, rigour was enhanced through minimal involvement in direct recruitment of registrars and regular peer debriefing with co-authors throughout analysis. NA’s insight and relationship with registrars was highly beneficial for this study given the deep understanding of the context of the supervisor, registrar, practice and patient relationship.²⁶

Considerations for further exploration and research

The study identified several considerations from registrar perspectives for future practice when providing direct observation for learning. We suggest these as areas for further research, exploration and discussion (Table 3). This study was conducted in Victoria. Exploring registrar experiences of direct observation from other RTOs and states across Australia would add further insight.
 

Conclusion

This research highlighted registrar experiences of direct observation. It identified some key considerations when setting up direct observation at all levels within the general practice training system, centred around the engagement of registrar, supervisor, placement practice and training organisation.