Reviewed by Danielle Ellis, B.Sc.Nov 9 2023
GPT-4 conversational artificial intelligence (AI) can diagnose and triage health concerns on par with board-certified clinicians, and its performance is unaffected by patient race or ethnicity.
Image Credit: iStock
Background
The accuracy of this type of AI for diagnosis and triage, as well as whether the AI’s recommendations include potential racial and ethnic biases derived from that information, have not been examined even though GPT-4, a conversational artificial intelligence, “learns” from information on the internet. This is true even though the application of this technology in healthcare settings has increased recently.
Method
Using 45 typical clinical vignettes, the researchers compared the diagnostic and triage processes of GPT-4 and three board-certified physicians to see which method offered the most likely diagnosis and determined which of the triage levels—emergency, non-emergency, or self-care—was most appropriate.
The study has certain shortcomings. Although the clinical vignettes were based on actual instances, they only offered diagnostic summary information, which might not accurately represent clinical practice, which usually provides patients with more in-depth information.
Furthermore, the GPT-4’s answers can vary depending on how the questions are phrased, and it might have picked up tips from the clinical vignettes this study used. Furthermore, it is possible that the results are not applicable to other conversational AI systems.
Impact
The results can be used by health systems to implement conversational AI to enhance patient diagnosis and facilitate effective triage.
The findings from our study should be reassuring for patients because they indicate that large language models like GPT-4 show promise in providing accurate medical diagnoses without introducing racial and ethnic biases. However, it is also important for us to continuously monitor the performance and potential biases of these models as they may change over time depending on the information fed to them.
Dr Yusuke Tsugawa, Study Senior Author and Associate Professor, Medicine, Division of General Internal Medicine and Health Services Research, David Geffen School of Medicine University of California, Los Angeles
Naoki Ito, Sakina Kadomatsu, Kiyomitsu Fukaguchi, Mineto Fujisawa, Ryo Ishizawa, Naoki Kanda, Daisuke Kasugai, Mikio Nakajima, and Tadahiro Goto are the other study authors.