TGA weighs in on AI discussions

As the potential for artificial intelligence (AI) use in healthcare continues to be explored, the TGA has weighed in on the debate over how existing and emerging tools should be regulated.
 
In its report, Clarifying and strengthening the regulation of Medical Device Software including Artificial Intelligence, the TGA delivered findings from a review into whether existing legislation, regulations and guidance are appropriate to meet the future challenges of AI use in medical software and across the healthcare sector.
 
It also canvassed opinions on steps to clarify and strengthen existing regulation to mitigate risks and leverage opportunities.
 
Engaging more than 600 stakeholders and receiving 53 formal responses, including a submission from the RACGP, the review confirmed an existing role of AI in healthcare ‘driven by benefits including increased efficiency, improved patient health outcomes, cost reduction, improved accessibility and capability’.

This includes the use of AI scribe tools which can automate parts of the clinical documentation process for doctors.

However, there were ‘a number of issues associated with the use of these products, both observed and reported’.
 
This includes a lack of understanding about what products meet the definition of a medical device and are therefore regulated by the TGA, as well as the inappropriate use of AI-enabled products due to a lack of understanding or misinformation about their intended purpose.
 
Amongst the review’s findings is that AI scribes are being used in a way that deems them ‘medical devices’ and therefore should be subject to the same pre-market rules that other medical devices need to comply with.

‘Developers of digital scribes claim they are not a medical device as their intended purpose is to summarise clinical practice notes,’ the report said.

‘Users report digital scribes frequently propose diagnosis or treatment options for patients beyond the stated diagnosis or treatment a clinician has identified during consultations.

‘This functionality indicates digital scribes meet the definition of a medical device and require pre-market approval and inclusion in the Australian Register of Therapeutic Goods and are potentially being supplied in breach of the Act.’

The report recommends targeted action to improve compliance, including a review of digital scribes to determine whether they fall under the category of medical devices, and if so, whether they are compliant with regulation.

‘Stakeholders, including clinicians and consumers who use these kinds of products, have identified that the absence of humans, lack of transparency and failure to engage with existing regulatory requirements represent a combination of circumstances that may lead to patient harm,’ it says.

‘In many instances, users are not aware that AI or machine learning has been used in the development of software, or is used operationally within the clinical workflow.’

The TGA review also flagged stakeholder reports of an ‘unwillingness from some developers to provide the accountability, transparency and responsibility necessary for engagement with existing regulation’.
 
‘A further cultural issue is the pervading belief among some developers that software products don’t present a meaningful risk to consumers and users, particularly when they are integrated with the provision of healthcare, where a human is in the loop, or where outputs are information only.’
 
The majority (78%) of stakeholders requested the TGA continue to directly engage with the software sector to deliver improved education and guidance materials to help clarify existing regulatory obligations.

The TGA review received submissions from a range of stakeholders from within the healthcare and therapeutic goods sectors, including members of consumer representative organisations, health professional peak bodies, government entities and the medical device industry, including sponsors and software developers.
 
Most stakeholders agree that the TGA’s existing risk and principles-based regulatory framework is flexible, robust and largely fit for purpose to meet the current and emerging risks associated with AI technology.
 
Log in below to join the conversation.

AI artificial intelligence scribes TGA