A new study published in the New England Journal of Medicine suggests that people often trust AI medical advice as much as, or more than, guidance written by human doctors, even when that advice is bogus or potentially harmful.
Researchers recruited 300 participants and asked them to evaluate medical responses that were either written by licensed physicians or generated by a large language model. The AI-generated responses had been pre-labeled by doctors as either high-accuracy or low-accuracy. Participants rated each response on validity, trustworthiness and completeness, and indicated how likely they would be to follow the advice.
The results show participants could not reliably distinguish between AI-generated responses and those written by doctors. They also show a preference for AI responses labeled as high-accuracy, rating them as significantly more valid, trustworthy and complete/satisfactory.
More strikingly, low-accuracy AI-generated responses performed very similarly to doctors’ responses across all measures.
“Participants not only found these low-accuracy AI-generated responses to be valid, trustworthy and complete/satisfactory, but also indicated a high tendency to follow the potentially harmful medical advice and incorrectly seek unnecessary medical attention as a result of the response provided.”
The study also finds that participants frequently perceived AI-generated responses as more thorough and accurate than doctors’ responses, while still expressing a preference for physician involvement in delivering medical advice.
Researchers warn that the growing trust in AI chatbots could lead to misdiagnosis and harmful consequences for people seeking medical advice.
“Ultimately, AI systems should be implemented in collaboration with medical professionals when used for the delivery of medical advice in order to prevent misinformation while reaping the benefits of such cutting-edge technology.”
Disclaimer: Opinions expressed at CapitalAI Daily are not investment advice. Investors should do their own due diligence before making any decisions involving securities, cryptocurrencies, or digital assets. Your transfers and trades are at your own risk, and any losses you may incur are your responsibility. CapitalAI Daily does not recommend the buying or selling of any assets, nor is CapitalAI Daily an investment advisor. See our Editorial Standards and Terms of Use.