What is AI? An introduction for clinicians

Artificial intelligence (AI) is increasingly present in clinical medicine, offering tools that can enhance practice efficiency and support clinical decisions. However, AI is fundamentally a collection of mathematical procedures that identify patterns in data – there is no true understanding or intelligence as we conceive it. This distinction is crucial for safe clinical implementation.¹

This article will first define key AI terminology relevant to general practice (Table 1), then it will explain how these technologies work using clinical examples, discuss their current and emerging applications in primary care and address important limitations including bias and the need for clinical oversight.

General practitioners (GPs) are already encountering AI in various forms: clinical documentation assistants (AI scribes), diagnostic support systems, risk prediction tools and administrative applications.² Understanding these technologies helps clinicians make informed decisions about their adoption and use.

Understanding AI fundamentals

Machine learning involves systems that improve their performance by analysing data patterns. Consider electrocardiogram (ECG) interpretation software: it examines features such as ST segments and P waves across thousands of expert-annotated ECGs, gradually learning to make similar assessments.³ The system adjusts its parameters to minimise differences between its predictions and expert diagnoses. This represents AI at its most transparent – we can understand what features it considers and why.

Neural networks and deep learning represent a more complex approach. These systems can identify patterns directly from raw data without predefined features. In our ECG example, although traditional approaches rely on measurements we specify, deep learning systems can discover subtle patterns in the raw signal that humans might not recognise. This capability comes with a significant trade-off: we cannot explain why the system made a particular decision. This ‘black box’ problem raises important questions about clinical accountability and patient safety.⁴

Generative AI creates new content on the basis of training data. Large language models (LLMs) are the text-generating subset most relevant to clinical practice. These power everything from clinical note generation to patient education materials, making them increasingly present in GP workflows.⁵

Current AI applications in clinical practice and research settings

Clinical documentation represents one of the most mature applications. AI scribes transcribe consultations in real time and generate structured notes.⁶ Although they reduce documentation burden, they require review for accuracy. In other countries, they are also increasingly used to extract key information to optimise revenue generated from the clinical encounter.⁶

Diagnostic support spans multiple specialties though is yet to achieve routine use in Australian general practice. This delayed uptake is justified, given concerns regarding both accuracy as well as explainability. Dermatology image analysis assesses skin lesions with improving accuracy, with Australian researchers pioneering efforts to improve explainability and therefore clinician trust.⁷ Ophthalmology applications screen for diabetic retinopathy in primary care, identifying at-risk patients who might miss specialist review.⁸ Radiology AI provides preliminary interpretations, prioritising urgent cases and flagging subtle findings.⁹

Risk prediction tools synthesise multiple data sources. Machine learning has been used by Australian researchers to improve the accuracy of cardiovascular risk prediction.¹⁰ Hospital readmission prediction identifies patients needing support, whereas falls risk assessment and mental health screening represent emerging preventive care applications.¹¹

Clinical decision support extends beyond drug interaction checking to providing guideline-based recommendations tailored to patient characteristics and differential diagnosis suggestions. These remain advisory tools requiring clinical judgment.

Administrative applications often provide quick returns on investment. Appointment optimisation reduces no-shows and improves patient satisfaction with their health system interaction.¹²

We now turn to a specific discussion of LLMs, as they have generated a lot of both public and professional interest, after which we discuss some challenges in clinical AI implementation.

Large language models

Understanding how LLMs function helps clinicians use them appropriately and recognise their limitations. These models process text by analysing patterns in how words and phrases appear together across vast amounts of training data. When a clinician types ‘Management of uncontrolled hypertension requires’, the model does not consult a medical textbook or recall physiological principles. Instead, it performs statistical analysis on patterns from millions of medical texts, research papers and clinical guidelines to predict what words typically follow this phrase.¹³

This process can produce remarkably coherent and seemingly knowledgeable responses. The model might complete the sentence with ‘careful consideration of patient comorbidities, lifestyle modifications and appropriate pharmacological intervention’, which sounds clinically appropriate.

However, this apparent sophistication masks a fundamental limitation: the model has no understanding of what hypertension actually is, how blood pressure affects organs or why certain medications work. LLMs are performing pattern matching at a scale and complexity that can produce text indistinguishable from expert clinical writing but without any comprehension of meaning or consequences. Indeed, modern LLMs could probably write both sides of the classic historic Pickering–Platt hypertension debate with tighter prose than either of the real interlocutors while understanding none of it.¹⁴

Sometimes the variability clinicians notice when using LLMs – where similar questions yield different responses – seems like a subtle change in model ‘understanding’. But these differences actually stem from how these models generate text. Rather than selecting the single most probable next word every time, models introduce controlled randomness to avoid repetitive outputs.¹⁵ This means asking ‘What are the treatment options for type 2 diabetes?’ might yield a response emphasising metformin and lifestyle changes on one occasion, whereas another query might lead with discussion of newer glucagon-like peptide-1 agonists. Neither response indicates the model ‘knows’ more in one instance than another; it is simply following different probability paths through its training patterns.

This characteristic explains why prompt engineering – the practice of carefully crafting queries to achieve desired outputs – has become important for clinical users. Small changes in phrasing can yield significantly different responses. For instance, asking for ‘evidence-based treatment options’ versus ‘current treatment options’ might activate different pattern associations in the model’s training data.¹⁶ Clinicians who understand this can better use these tools while recognising that variation in output quality reflects statistical processes, not genuine understanding or reasoning.

Understanding and addressing bias in clinical AI

The issue of bias in AI systems represents one of the most critical challenges for clinical implementation. All AI systems can perpetuate or amplify biases present in their training data, with potentially serious implications for patient care and health equity.¹⁷ Understanding these biases helps clinicians recognise when AI recommendations might be unreliable or inappropriate for specific patients.

Consider the use case of a neural network approach to skin lesion analysis.¹² The developers gave many naevus photos to the neural network in addition to the final histopathological diagnosis, allowing the program to develop its own approach to diagnosing the presence or absence of malignancy. The problem was that some of the naevi in the training data had ink marking around them as the photos were taken just prior to excision by a dermatologist, and the ones that were excised were more likely to be malignant. The model learnt that one parameter that was very predictive of malignancy was the presence of skin marker around the lesion – which is clearly not useful and an important example of the risks of unintentionally biasing the dataset.

Demographic bias poses perhaps the most widespread challenge for primary care applications. Many AI tools have been trained predominantly on data from specific populations, often overrepresenting younger, healthier, more affluent or ethnically homogeneous groups. Pulse oximetry algorithms provide a stark example: devices calibrated primarily on lighter skin tones can overestimate oxygen saturation in patients with darker skin, potentially missing hypoxemia.¹⁸ Voice recognition systems used in clinical documentation may struggle with accents or speech patterns underrepresented in training data, creating barriers to care for immigrant populations or those with speech differences, as well as for physicians whose accents differ from those represented in the training data.¹⁹

Socioeconomic bias emerges when AI systems train on data that do not reflect the full spectrum of clinical presentations.²⁰ An AI system trained primarily on data from well-resourced academic medical centres might learn patterns associated with patients who present early with classic symptoms and good health literacy. When deployed in community settings serving populations with limited healthcare access, these systems may miss atypical presentations or fail to account for the delayed presentations common when patients face barriers to care.^17,20 For instance, an AI system trained to recognise diabetic complications might underperform when patients present with advanced disease because they could not afford regular monitoring or medications.

Historical healthcare disparities embedded in training data create another layer of complexity. If an AI system learns from historical data where certain populations received different treatment patterns – whether because of bias, access issues or social determinants – it may perpetuate these disparities. An AI system might learn that certain ethnic groups are less likely to receive aggressive cardiac interventions, not because it is clinically appropriate, but because that is the pattern in historical data.¹⁷ Without careful attention, such systems could recommend continuing these disparate treatment patterns, effectively encoding past injustices into future care recommendations.

Mitigating these biases requires active, ongoing clinician engagement rather than passive acceptance of AI recommendations. Before implementing any AI tool, clinicians should investigate the composition of its training data. Does it include patients like yours? Were multiple ethnic groups, socioeconomic backgrounds and geographic regions represented? Has the tool been validated specifically in primary care settings or only in specialised academic centres?¹⁷ GPs and other medical practitioners already apply such scrutiny to clinical trials that may not adequately reflect their own patient populations. The skills needed here are similar and readily transferrable.

Practical implementation and clinical oversight

Successfully integrating AI into general practice requires more than simply purchasing software. It demands thoughtful evaluation, careful implementation and ongoing oversight to ensure these tools enhance rather than compromise patient care. The complexity of primary care, with its undifferentiated presentations and longitudinal relationships, creates unique challenges for AI implementation that might not be apparent in more specialised settings.

AI will increasingly be used to provide point-of-care clinical advice.²¹ Understanding the training data’s relevance to your specific patient population is crucial – an AI trained on urban emergency department presentations may perform poorly in rural general practice. Validation studies should specifically include primary care settings, not just specialist clinics or hospital environments. The transparency of the AI’s decision-making process varies significantly between tools; those that provide some explanation for their recommendations, even if simplified, allow for better clinical oversight than completely opaque systems.²¹

Workflow integration often determines whether an AI tool succeeds or creates more problems than it solves. The most sophisticated AI becomes a burden if it disrupts established clinical workflows or requires extensive workarounds. Successful implementation typically starts with identifying specific pain points in current practice – perhaps the hours spent on documentation or the challenge of keeping up with changing guidelines – then selecting AI tools that address these specific issues.⁶

Training requirements extend beyond simple software operation. Clinical staff need to understand not just how to use AI tools, but also their limitations and potential failure modes. A nurse using an AI-powered triage system must recognise when the AI’s recommendations do not align with clinical judgment and feel empowered to override them. Reception staff using AI-powered scheduling scheduling must understand when to involve clinical staff in complex booking decisions the AI system cannot handle appropriately.

Limitations and future directions

Current AI systems in medicine face fundamental limitations that will not be resolved simply through incremental improvements. The lack of true understanding or reasoning ability means AI cannot genuinely comprehend patient narratives, recognise subtle clinical patterns outside its training or apply judgment to novel situations.² When a patient presents with an unusual constellation of symptoms, an experienced GP draws on years of clinical experience, understanding of local disease patterns and knowledge of the individual patient. AI systems cannot replicate this integrative thinking, regardless of their training data volume.

The potential for convincing but incorrect outputs poses particular risks in healthcare. Unlike errors that are obviously wrong, AI can generate recommendations that seem plausible and are presented with apparent confidence, making them harder to identify.²² A medication recommendation might be pharmacologically reasonable but inappropriate for a specific patient’s circumstances in ways the AI cannot appreciate. This ‘confident incorrectness’ requires constant vigilance from clinicians.

Complex, multifaceted clinical scenarios remain beyond current AI capabilities. General practice often involves managing multiple chronic conditions, psychosocial factors and patient preferences simultaneously. An elderly patient with diabetes, mild cognitive impairment, limited family support and financial constraints requires holistic management that AI cannot adequately address. Although AI might optimise each condition individually, it cannot integrate these factors into coherent, person-centred care plans.

Future developments may address some current limitations through improved architectures and training methods. Proposed improvements include better explainability (so-called explainable AI, or ‘xAI’), allowing clinicians to understand AI reasoning; reduced bias through more representative training data and algorithmic adjustments; and enhanced ability to indicate uncertainty rather than always providing confident outputs.²² However, these remain technical improvements to pattern recognition systems, not steps toward genuine understanding or clinical reasoning. The fundamental need for clinical oversight will persist regardless of technical advances.

Conclusion

AI technologies offer valuable tools for enhancing primary care delivery, from reducing documentation burden to supporting diagnostic decisions and improving practice efficiency.² However, AI tools remain sophisticated pattern- recognition systems without true clinical understanding, judgment or ability to appreciate the full context of patient care. This limitation is not a temporary technical hurdle but a fundamental characteristic of current AI approaches.

The GPs who will thrive in an AI-enhanced healthcare landscape are those who embrace these tools thoughtfully, understanding that the art of medicine – with its emphasis on human connection, clinical judgment and individualised care – remains irreplaceable.