Generative AI Can Help Doctors Diagnose Patients — But Is it Biased?
A new study by Professor Damon Centola tested if AI tools could help improve medical care without increasing bias.
In the age of generative artificial intelligence (AI), healthcare workers and health communication scholars want to ensure that new AI diagnostic tools used by doctors do not exacerbate bias against underrepresented minority patients.
A new study published in Nature Communications Medicine, and co-authored by Damon Centola, Elihu Katz Professor of Communication, Sociology, and Engineering at the Annenberg School for Communication at the University of Pennsylvania, tested whether using AI would impact performance and bias during a physician's clinical decision-making when evaluating chest pain in patients.
During the study, 50 U.S.-licensed doctors watched a video of either a white male or Black female patient describing chest pain symptoms. The research team asked doctors to make medical decisions based on this video and test results. Participants then received suggestions from an AI system and could change their decisions based on these suggestions.
The research team found that doctors were willing to consider the AI’s suggestions and made more accurate medical decisions after receiving this help.
For the white male patient scenario, accuracy increased from 47% to 65%. For the Black female patient scenario, it rose from 63% to 80%. Notably, these improvements in decision-making happened equally for all patients, regardless of their race or gender.
“Our findings suggest that, in this case, AI can meaningfully augment physician decision-making without introducing inequities in clinical decisions,” Centola says. “While our earlier results have shown that learning networks among clinicians can reduce implicit bias, these results suggest that there is hope that AI systems, when designed and deployed carefully, can improve medical outcomes without compromising fairness in patient care.”
AI and Medical Decisions
During the study, physicians answered four clinical questions regarding triage, risk assessment, and treatment for the patient they saw in the video, noting how they’d proceed if they saw the patient in person.
Doctors then received pre-generated suggestions from ChatGPT+ about two questions — one recommending the proper treatment among several options, and another estimating the diagnostic risk for the patient. Physicians could then change their decisions based on this feedback. Participants could even interact with the chatbot themselves for two questions regarding follow-up treatment (e.g., medication recommendations).
Is Generative AI the Future of Medicine?
The study also explored physicians’ perceptions of AI in clinical settings, as participating doctors were asked to rate the likelihood that large language model (LLM) tools like ChatGPT will have a role to play in healthcare for clinical decision-making and to recommend changes that could improve these tools.
After the trial, 90% of participating doctors indicated that they believe AI will become a significant component of future medical decision-making. Recommendations for improving the LLM tools focused on things like adding more evidence-based citations for recommendations and creating healthcare-specific tools.
While the study results are promising, there is much to learn about AI tools' general capability in medical decision-making, Centola says.
“Studying LLMs in general is still more nuanced than we would like it to be,” he says, “For instance, if initial clinical responses in this study had exhibited greater implicit bias, we don’t know how that would have affected the evolution of AI responses.”
Still, he believes future AI systems, in collaboration with humans, could eventually help diagnose patients accurately and fairly.
“This research is a step toward understanding how we can harness the power of AI responsibly in healthcare,” says Centola. “We already know that physician networks can be used to reduce medical errors and eliminate disparities in care; what we need to understand now is in which settings can AI reliably supplement human decision-making to enable more consistent clinical decisions across diverse patient populations.”
The study, “Physician clinical decision modification and bias assessment in a randomized controlled trial of AI assistance,” was authored by Ethan Goh, Bryan Bunning, Elaine C. Khoong, Robert J. Gallo, Arnold Milstein, Damon Centola, and Jonathan H. Chen and published in Communications Medicine.
