Study Cautions Patients Against Relying on AI Chatbots for Drug Information

New Delhi: A study published on Friday has warned that patients should not rely on Artificial Intelligence (AI)-powered search engines and chatbots for accurate and safe drug information. The research, conducted by scientists from Belgium and Germany, revealed that many answers provided by these AI tools were incorrect or potentially harmful.

The study, published in BMJ Quality and Safety, highlighted concerns over the complexity of AI chatbot responses, which may be difficult for the average person to comprehend, potentially requiring a degree-level education to fully understand. The researchers pointed out that the introduction of AI-powered search engines in 2023 marked a significant change, offering more comprehensive search results and interactive experiences. However, these chatbots, despite being trained on vast datasets from the internet, can also generate misleading or harmful information.

The research team, from Friedrich-Alexander-Universität Erlangen-Nürnberg in Germany, found that while AI chatbots often produced accurate and complete responses, they also struggled with readability and frequently provided incomplete or inaccurate information, which could jeopardize patient safety.

For the study, the researchers assessed the readability, completeness, and accuracy of AI chatbot responses to queries about the top 50 most frequently prescribed drugs in the US in 2020, using Bing Copilot, an AI-powered search engine. The results showed that only half of the questions were answered with the highest level of completeness, and in 26 percent of cases, chatbot responses did not match the reference data. Moreover, over 3 percent of answers were entirely inconsistent with the reference information.

Notably, the study found that 42 percent of the chatbot responses could lead to moderate or mild harm, while 22 percent of them posed a risk of severe harm or death. A key issue identified was the chatbot's inability to grasp the underlying intent of patient questions.

Despite the advancements in AI technology, the researchers emphasized the continued importance of consulting healthcare professionals, as chatbots are not always reliable for providing accurate, error-free information.