Registrars in the Australian General Practice Training (AGPT) Program commence patient consultations on their own from the outset of their placements.1–4 In some training settings registrars are directed to contact their supervisor under specific circumstances; however, typically, registrars choose to only discuss with their supervisor those patient encounters they deem to be clinically challenging.1,4,5 This differs from other countries with a comparable primary healthcare system,2 and from international medical graduates commencing general practice training in Australia with level 1 supervision, where all patients need to be discussed with their supervisor.1 Although a variety of measures are put in place to support registrars commencing in general practice, one of which is direct observation, the safety of this largely unobserved practice has been questioned.1,3,6
To improve patient safety, the introduction of a mandated period of direct observation, particularly during transition from hospital to general practice, has been suggested.5,7
Direct observation is defined as the purposeful observation of trainees in real time with feedback8 where the ‘master’ clinician watches and provides feedback to the ‘apprentice’. This serves two purposes: first, assessment at various stages of training during real clinical work at the top ‘does’ level of Miller’s pyramid;9 and second, supporting the learner with formative feedback and coaching, guiding them towards achieving their learning goals.10
There is ample evidence for the use of direct observation in assessment, but when it does occur its utility for learning is often questioned, especially by registrars.3–6,11,12 Additionally, most literature on direct observation focuses on tertiary care settings and procedural skills rather than the primary care setting and consultation and communication skills.8,10 It also lacks the perspectives of registrars.
Although direct observation is an important teaching technique in general practice training, evidence shows that it does not occur as frequently as expected by regional training organisations (RTOs) that currently deliver training and colleges that oversee training.3,4,6,11,12 Implementing an initial mandatory period of direct observation, as has been suggested, may have unintended consequences and unsatisfactory outcomes if registrar preferences and engagement are not better considered or integrated.
RTOs require supervisors to undertake professional development, although the extent varies. The Victorian RTO where this study was conducted requires all supervisors within three years of their accreditation to complete a mandatory core module, ‘Observing Your Registrar’.
The aim of this study was to explore registrar experiences of direct observation to identify key considerations for using direct observation in general practice training.
Methods
General practice registrars’ experiences of direct observation during clinical practice were explored using a qualitative phenomenological approach.13
Purposeful sampling14 was used to recruit AGPT registrars from an RTO based in Victoria, Australia, across three general practice terms (ie GPT1–GPT3) and training in rural and metropolitan locations. Registrars were identified through the RTO’s database. All eligible participants (approximately 120) were sent email invitations by administration staff together with the plain language statement. The study was also advertised across all registrar workshops. All registrars who expressed interest within the study timeframes were followed up for interview. Sample size was guided by appraisal of information power whereby the more information the sample holds, relevant to the study focus, the fewer participants needed.15 While the richness of data was enough to meet study objectives, the study was also timebound, which limited the extent of further recruitment.
Semi-structured interviews (Box 1) were conducted by NA between August and December 2020 via videoconference. Interviews were audio-recorded, transcribed verbatim and de-identified. Interviews ranged 35–50 minutes, with the average being 39 minutes. Participants were sent the questions prior to the interview. Data were stored securely and managed using NVivo 12.
Box 1. Themes explored during interviews |
- Registrar’s background and what interested them to participate in the research
- Registrar thoughts on direct observation
- Registrar experiences of having consultations observed
- What was useful or otherwise about direct observations
- Thoughts about how direct observation improves practice (prompt for specific examples)
|
Thematic analysis16,17 was used, guided by Braun and Clarke’s six-step method.16,18,19 The initial coding framework was constructed by NA, and sensitised by the research questions and adult-learning20 and work-based21 learning theories. Analysis was iterative, so findings from initial interviews were explored with latter participants. Categories were refined through further coding and interrogation of the data, and discussion during regular team meetings.
NA (a general practice supervisor and experienced medical educator) led the analysis, with BG (an experienced qualitative and primary healthcare researcher) and ST (an experienced primary healthcare medical education researcher and general practitioner [GP]) contributing to analysis and interpretation. Regular meetings between all authors enriched analysis and enhanced interpretive rigour.14 NA and BG held positions within the registrars’ RTO. ST was external to the organisation. NA was involved in teaching but had no involvement in the assessment of these registrars. BG had no prior relationship with participants.
Ethics approval was provided by the Medical Education Human Ethics Advisory Group (HEAG) at the University of Melbourne (ID: 2057233.1).
Results
Eight registrars responded to the invitation, and seven were interviewed. The eighth registrar did not respond to further efforts to follow up. Most (n = 5) were in their second general practice term, with one in their first and one in their third. Three identified as female. Three were in rural locations, and all of those identified as male.
All registrars reported direct observation to be valuable for their learning, with one registrar stating, ‘it should be a part of every registrar’s experience in general practice’ (Registrar 3). Direct observation was described as a ‘unique opportunity to be observed in their clinical setting …’ (Registrar 3) and potentially ‘the last time you’ve got a chance to get that support and to be observed’ (Registrar 5).
Four main themes were identified: registrar engagement, supervisor engagement, practice engagement and training organisation engagement.
The dynamic tension/interaction between each of these created both enablers and barriers to registrars’ learning from direct observation (Table 1).
Table 1. Enablers and barriers to direct observation |
Key contributor |
Enablers |
Barriers |
Registrar |
Attributes
- Intrinsic motivation
- Reflectiveness
- Confidence
- Flexibility and adaptability
Attitudes and beliefs
- Perceived benefits of direct observation
- Assessing safety
- Uncovering the unknown unknowns
- Bridging the gap during transition
- Developing core skills in consultation and communication
Perceived learning potential
- Value for learning
- Individualised planning of learning
- Achieving excellence in consultation
- Prioritising application of knowledge over content
|
Attributes
- Inertia
- Increased anxiety related to being observed
- Overconfidence
- Fixed mindset
Attitudes and beliefs
- Perceived lack of value of direct observation
- Staged performance as a result of being observed
- Lack of returns for investment of time
Perceived limits to learning potential
- Negative experience or lack of experience
- Uncertainty of what to expect
- Observer effect
|
Supervisor |
Engagement
- Buy-in to concept
- Motivation for task
- Multiple supervisors providing variety of supervision styles
- Establishing an educational alliance
- Bidirectional direct observation
- Promotion, planning and facilitation of direct observation
- Provision of high-quality, valuable feedback
- Collaboration with general practice registrar and other general practitioners
- Longitudinal follow-through
|
Disengagement
- Unavailable or unapproachable
- Insufficient remuneration
- Poorly perceived legitimacy or value of direct observation
|
Practice and patient |
Engagement
- Entrenching direct observation in practice culture
- Making direct observation the ‘norm’
Supporting direct observation in the clinic
- Embracing technology
- Reducing the administrative burden
- Encouraging patient participation in feedback
|
Disengagement
- Disengaged practice culture for direct observation
- Lack of structured program
- Poor patient acceptance of an observer
- Practice rigid or unadaptable to challenges
- Pandemic – physical distancing
- Telehealth challenges – technology issues
|
Training organisation
|
Promotion of direct observation
- Explanation of benefits to stakeholders
- Supervisor support and training
- Mandating direct observation
- Structured program of direct observation
- Standard of competencies
|
Lack of promotion of direct observation
- Limited promotion of benefits
- No structure to direct observation sessions
- No setting of expectations around direct observation
- Lack of training provided to supervisors on feedback and facilitation of direct observation
|
Registrar engagement
Registrar attributes, attitudes and beliefs
Enthusiasm and motivation were key drivers for engaging with the direct observation process, with registrars initiating direct observation themselves gaining the most benefit and satisfaction. For example, registrars who proactively requested direct observation as an adjunct to routine teaching, organising direct observation sessions in liaison with the practice manager, were more likely to receive direct observation regularly throughout training. Positive outcomes of this included identification of learning needs around consultation skills, improved organisational skills and the formation of positive relationships with supervisors:
So, based on my experience in GPT1 when it’s actually something I pushed for in GPT2 … my supervisor has been very open to me sort of suggesting what I would like … I had to stay on top of it, so I had to be proactive … [Registrar 5]
I think that probably evolved more because I also did request it as well ... [Registrar 7]
Similarly, registrar inertia and lack of motivation had an impact on the direct observation experience:
It wasn’t something that they particularly pushed … was left up to me to initiate … it just got left on the back burner. [Registrar 1]
Registrars acknowledged the discomfort direct observation could create but perceived a rich learning experience from direct observation in terms of revealing their ‘unknown unknowns’. Engagement in direct observation by the registrar was enhanced for those who valued consultation and communication skills and who were aware of this potentially being a ‘last chance’ for observed feedback of these skills:
It’s the last time you’ve got a chance to get that support and to be observed … I feel like it doesn’t happen a lot in in medicine, apart from like clinical exams and then it’s sort of a bit too late. That’s something you need to fix before … [Registrar 5]
Although most registrars acknowledged well-developed communication as being a key skill to master in general practice, this skill is not often prioritised given the sheer volume of knowledge registrars felt they ‘needed to know’, particularly during their first general practice term. Communication and consultation skills were perceived by some as an ‘add-on’ (‘it wasn’t the first thing that came to mind’) and something better focused on towards the end of training. Registrars who prioritised clinical knowledge and skills over communication and consultation skills expressed that direct observation was of low value. They felt that the time required for reflecting and unpacking the consultation process might not yield what they considered to be ‘useful’:
It’s not as high value … if we use that same amount of time to, you know, talk about, say osteoarthritis, we would have covered a whole topic and including some patient examples already. So, I think that definitely is more time consuming for what you get at the end. [Registrar 3]
The anxiety-provoking nature of direct observation, the discomfort of being observed and the fear of ‘losing face’ in front of patients and its effect on rapport affected registrar attitudes towards direct observation. Direct observation improved performance or affected confidence:
… as a registrar you’re on your game a lot more when someone [is] standing there watching you. You certainly feel a little bit of added pressure. [Registrar 2]
It’s very nerve wracking … somebody is watching, don’t say the wrong words … [Registrar 3]
I think it’s a pride thing … [Registrar 4]
Registrar perceived benefits of direct observation
Registrars were more likely to engage positively with direct observation if they perceived a benefit in it for them. ‘What’s in it for me?’ was a key enabler of engagement in direct observation.
Table 2 highlights the benefits identified by registrars.
Table 2. Benefits of direct observation identified by registrars |
Key benefit identified by the registrar |
Reasoning for the key benefit |
Supporting quotes |
Safety (particularly during transition from hospital to general practice)
|
Transition from hospital to general practice as a significant time of change bringing with it anxieties about confidence, competence, support and safety
Bridged the gap during the transition in terms of support, and safety for both the patient and the registrar
Provided a platform to grasp new concepts such as uncertainty
Direct observation not often carried out in hospital
|
‘I feel like there was really not much direct observation and I would have liked to have somebody watch me and give feedback with consultations … think when you first start in general practice coming from the hospital, you do feel quite shaky in terms of it’s a lot to take on. It’s a really big transition and it is kind of good to have somebody watch you sometimes because otherwise it’s just you and the patient in the room and nobody ever checks in to sort of to see what you’re doing and it’s just you steer the ship.’ [Registrar 5]
‘There is somebody watching you to try to make sure you are doing the right thing.’ [Registrar 3]
|
Individualised plan for learning, boosting confidence, support
|
If done regularly, allowed the supervisor and registrar to obtain an assessment point of progress over time during the rotation
Provided opportunity for feedback
|
‘So it was a good way for [the supervisor] to gauge where I’m at … and then plan your further education thereafter.’ [Registrar 3]
‘… it can either be used just to inform the registrar of where they’re at with respect to the rest of the cohort.’ [Registrar 2]
‘[Supervisors] are able to comment on how I’ve changed … that’s probably been the most helpful part.’ [Registrar 7]
|
Unconscious incompetence or ‘unknown unknowns’ despite level of experience
|
One of the only modalities that allowed for registrar self-reflection with facilitated feedback on the observed encounter, uncovering the ‘unknown unknowns’
Value of the ‘bird’s-eye view’ of something registrars were not able to see when immersed in the encounter
|
‘I think when somebody observes you, they think of things that you may not see yourself. Somebody who is observing can just look at the whole thing, in a bird’s eye view that that you can’t when you are involved in it.’ [Registrar 5]
‘… learning to the next level.’ [Registrar 2]
‘… had the possibility of taking their consultation from A to A+.’ [Registrar 2]
|
Provided opportunities for specific consultation skill development
|
Helped develop specific consultation skills such as:
- building rapport
- agenda setting
- eliciting ideas, concerns and expectations
- time management
- shared decision making
- safety netting and follow-up
Provided opportunity to improve registrar’s clinical reasoning with the supervisor using the ‘think out loud’ technique for feedback
|
‘In terms of, you know, your, your clinical reasoning, but also how you approach that, you know the non-medical stuff [like] sort of communication skills and just seeing what the patient is trying to tell you.’ [Registrar 3]
‘The first direct observation … we worked on … the first minute kind of so letting the patient say what they are here for and then … get all the agenda out.’ [Registrar 7]
‘With the signposting for patients so to set up their expectations. I think [direct observation has] been quite good for that.’ [Registrar 7]
|
Preparation for Fellowship examinations
|
Examination practice as potentially the ‘carrot’ that increased the uptake
|
‘I think that’s all going to change right because the whole format of the clinical exam has changed so you know, the [remote clinical examination] has changed what time we’re going to be examined and assessed and I think that people are going to want more direct observation, because that’s relevant to how they are going to be assessed now.’ [Registrar 4]
‘Because we’re all doing the same set of exams under the same exact conditions, it’s probably something that we all need to train and prepare for.’ [Registrar 2]
|
Supervisor engagement
Supervisors’ buy-in to direct observation was another key enabler. Registrars who experienced bi-directional direct observation – observing their supervisor consultations in addition to being observed themselves – found direct observation of their own practice to be less intimidating while helping to strengthen the registrar–supervisor relationship:
It was a real … masterclass on how to do a consultation, like a lot less intimidating, because I was like, right ‘Well, I’ve sat and watched you, so I mean it’s only fair that you get to sit and watch me now’. [Registrar 4]
Registrars identified that if their supervisors were under time pressure due to competing clinical demands in the practice, or were not being adequately remunerated for performing direct observation, they did not appear to ‘buy in’ to the process. Lack of supervisor or practice prioritisation of direct observation and buy-in were a barrier to the initiation of direct observation by the registrar:
… supervisors are obviously trying to do a job themselves and have limited time to give ... [Registrar 2]
… supervisor is sitting in with you, then they’re not seeing patients and I think that is the biggest one. [Registrar 5]
Other things took priority ... [Registrar 1]
Feedback was identified as a vital part of the direct observation session for learning.
To have perceived utility for learning, feedback needed to be timely and consist of reflection, collaboration between the supervisor and registrar, identification of strengths and areas for improvement based on observed behaviours, and lead to individualised learning planning.
While supervisor feedback such as ‘you are doing well, keep doing what you are doing’ boosted confidence for the registrar, immediate individualised constructive feedback post direct observation helped to further learning and progress for the registrars. Additionally, identifying learning points to review at the next direct observation session gave registrars something to work towards. This feedback cycle contributed to building a positive registrar–supervisor relationship.
Practice engagement
Creating a learning culture and embracing direct observation as the ‘norm’ was a key facilitator. These practices reduced the administrative burden by ensuring direct observation was scheduled in advance, blocking off mutual time for both supervisor and registrar, with time for feedback.
Patient willingness to participate in direct observation was also an enabling dimension of such practice cultures.
Having an observed consultation followed by time for discussion worked the best for registrar learning:
Set time works because it’s one less thing for me to do and think about in terms of planning it and putting it in/booking it in so I definitely like that factor that it’s not me trying to keep track of ‘When was the last time I did that? I should set that up, what days are we both here?’ The practice manager just does it. [Registrar 1]
Practice adaptability and embracing technology such as remote observation of consultations using video, particularly during the COVID-19 pandemic, appeared to contribute to overall satisfaction of the registrar experience.
Training organisation engagement
Promotion of direct observation
Promotion of direct observation by the RTO to supervisors and registrars was a key driver for engaging both parties in it. Most registrars noted that while they were provided with some information about direct observation, more could be done to encourage direct observation and highlight its purpose and benefit:
Explaining to registrars early on the purpose of it ... I think reframing that will be really helpful for enhancing the benefits to the registrars … [Registrar 4]
Registrars also suggested that supervisors be given more support and training in setting up and structuring direct observation as well as in feedback provision.
Some registrars expressed uncertainty and a lack of clarity about expectations from direct observation, particularly with frequency, length of direct observation and structure, creating difficulty for registrars to approach their supervisors and practices about direct observation. This limited registrars’ opportunities to learn about their ‘unconscious incompetence’. Registrars who did not experience any direct observation found it difficult to judge if it was a ‘potentially missed opportunity’ versus those that had an experience, albeit negative:
I don’t know if that’s something that other people have had a lot of or even if they get that or if I’ve missed out on potential benefit from not having that. [Registrar 1]
I think now it’s hard to say because if you don’t ever get it, you don’t know what you’re missing out on, in a way. So, if that happened say in GPT2 or 3 then I would probably raise that point and say that it is useful to me. [Registrar 7]
Mandating direct observation, setting competencies, embedding into curriculum
Mandating direct observation and embedding this into the curriculum with specific clinical competencies to be achieved during various stages of training was one suggestion for reducing the uncertainty around expectations. Benchmarking or standard-setting expected levels of performance was thought to allow for better utility of direct observation.
Without the competencies, registrars perceived mandatory direct observation as unlikely to improve engagement and had the potential to become prescriptive. This was perceived as counterproductive, with the potential of it becoming yet another hurdle activity required during training:
… maybe you just mandate it. I mean, you could make a lot more mandated in the early phases and then less in the latter stages … I think the way to make something valid is to enshrine it in curriculum. Otherwise it’s hard to validate it, I guess in some people’s eyes. [Registrar 5]
Structure for direct observation
Registrars identified some structural factors as yielding positive outcomes from direct observation. These included having planned sessions, normalising direct observation in the clinic and specifying frequency and duration of direct observation.
Registrars said that planned direct observation sessions yielded more benefits than ad hoc.
An unplanned or ad-hoc direct observation session that was poorly structured, with lack of explanation of the potential benefits, with limited or no reflection and feedback was seen as a barrier to registrar engagement:
It is generally during planned teaching time rather than just, oh, ‘I’m coming in today’. [Registrar 3]
Apart from GPT1 where the registrars felt it needed to be scheduled at the start, middle and end of semester, there was no uniformity on how often direct observation was thought to be useful. ‘Regular’ direct observation that was individualised to the registrars’ learning needs was considered beneficial. Some suggested direct observation at educationally meaningful points such as scheduled, formalised training organisation–driven feedback sessions during their term would be helpful. All noted that direct observation should not be overly regular or repetitive, with some stating weekly sessions would be excessive. One of the reasons given for less frequent observation was for self-regulation and learning, allowing the opportunity to learn from mistakes. Decisions about frequency were beneficial when made in collaboration with the registrar, considering registrar learning needs:
It can be done once every three months, that kind of thing and then it can be or may be more acceptable to supervisors and to registrars as well. [Registrar 7]
If there was maybe two more opportunities for … compulsory direct supervision … that would be part of either your week six or week 20 feedback. [Registrar 4]
Short sessions with two consultations – or a maximum of one hour for each session – were favoured. This allowed valued time after the consultation for reflection and feedback.
Discussion
This research explored the perspective of registrars and identified key enablers and barriers to direct observation from a registrar perspective. Until now, registrar experiences of direct observation have remained under-explored; this study highlights important contextual complexities of general practice training, and how these influence direct observation structure and experience. The value of direct observation as a learning tool was supported by this study.3,4,6,11,12 Findings also highlight the need for registrars to know why and how direct observation can enhance their learning. Registrar engagement in direct observation is heavily influenced by their orientation to learning and the different skills that they see as valuable. Gaining insight into these factors can help us understand the key factors needed for learner engagement in direct observation.
General practice training is an apprenticeship-based model where learning predominantly occurs in practice by ‘doing’. Hingeing on this core principle, participating in the day-to- day workplace activities is, in itself, learning. Although registrar engagement is important, supervisor and practice buy-in (work environment) is also key to this experience. The principles of work-based learning21 provide insight into the learner’s experience and the challenging and complex dynamics of the work environment and how these influence registrar engagement in direct observation.21–23
Various factors influence learning, such as the physical workplace environment, the type and duration of the learning activity, level of support and inclusivity and prior experience of all stakeholders.24 The influence of positive support was highlighted in this study where supervisors who provided direct observation in a proactive, structured and reciprocal manner better enabled direct observation. The willingness of patients to participate also contributed positively to the direct observation experience for registrars.
The benefits of direct observation found in this study – such as improved consultation and communication skills through feedback and coaching, and improved patient safety, particularly when a registrar is transitioning from hospital- to community-based training – corresponds with benefits reported in the literature.5,7,8,10
This study highlights the importance of workplace culture on facilitating individual participation and learning,21 and therefore the important role practices and supervisors have for normalising direct observation as an educational activity for registrars.
Limitations and strengths
This study was exploratory and sought to understand the experiences of participants in depth; thus, a sample size of seven was appropriate. This afforded a deeper exploration of registrar experiences and there was no expectation of data saturation. The findings represent the experiences of these participants25 and it was not the intention to generalise the findings. There may be some degree of transferability to other registrars in the same RTO, or to registrars from other RTOs in Australia, though this will be judged by the reader familiar with these contexts. Further research with a broader sample of AGPT Program registrars across other RTOs would likely identify additional facilitators and barriers to direct observation.
The study did not explore how many sessions of direct observation the registrars had during practice. Sampling was drawn from participants self-selecting to be a part of the research, and results may reflect those with extremely positive or negative experiences of direct observation; however, this did not appear to be the case as registrars reflected on a non-polarised range of experiences of direct observation.
This study was focused on the experiences and perceptions of registrars. Experiences of other stakeholders – such as supervisors, practices and training organisations – were beyond the scope of the study.
Given NA’s involvement in medical education, rigour was enhanced through minimal involvement in direct recruitment of registrars and regular peer debriefing with co-authors throughout analysis. NA’s insight and relationship with registrars was highly beneficial for this study given the deep understanding of the context of the supervisor, registrar, practice and patient relationship.26
Considerations for further exploration and research
The study identified several considerations from registrar perspectives for future practice when providing direct observation for learning. We suggest these as areas for further research, exploration and discussion (Table 3). This study was conducted in Victoria. Exploring registrar experiences of direct observation from other RTOs and states across Australia would add further insight.
Table 3. Key considerations for exploration and further research for a program of direct observation |
Key contributor |
Key considerations |
Registrar |
- Active involvement and initiation of direct observation
- Recognition that discomfort creates powerful learning
- Engaging in a collaborative discussion for:
- developing a learning plan involving direct observation
- tailoring plan to the individualised need
- Recognition and prioritising of consultation skill development over content acquisition
|
Supervisor |
- Engaging in collaborative process with registrar:
- at the start of term to develop an individualised plan
- during the term for monitoring and adjusting plan for individualised learning needs
- offering direct observation and consistent structure
- Helping the registrar prioritise consultation and communication over content
- Providing bi-directional direct observation
- Providing high-quality feedback
|
Training environment/practice |
- Normalising direct observation
- Actively removing the administrative burden by scheduling regular sessions
- Prioritising direct observation
|
General practice training systems |
- Promoting direct observation, including information on its benefits
- Setting up a recommended structure for direct observation per rotation, which can be tailored to individual learning needs
- Setting standard for each level of training for specific competencies
- Giving consideration to incorporating direct observation into supervisor and registrar curriculum
- Supporting supervisor training/upskilling in delivery of direct observation and provision of feedback
|
Conclusion
This research highlighted registrar experiences of direct observation. It identified some key considerations when setting up direct observation at all levels within the general practice training system, centred around the engagement of registrar, supervisor, placement practice and training organisation.