The launch of ChatGPT Health in Australia sparks concern among experts, highlighting the potential risks of unregulated AI in healthcare.
A recent case involving a 60-year-old man who ingested sodium bromide, a dangerous substance, due to misinformation from ChatGPT, underscores the dangers of relying on AI for health advice without proper regulation. This incident, where the man was misled about the safety of sodium bromide, led to severe hallucinations and hospitalization, serves as a stark reminder of the potential consequences.
Alex Ruani, a doctoral researcher in health misinformation, warns about the lack of safety testing for ChatGPT Health. Without published studies evaluating its safety, it's challenging to determine which user prompts or data sources might lead to harmful misinformation. The absence of mandatory safety controls and risk reporting raises concerns about the platform's reliability.
ChatGPT Health, developed by OpenAI, is presented as a tool to provide health information and advice. However, the line between general information and medical advice is often blurred, and the platform's responses can be misleading. The lack of transparency regarding the methodology used by HealthBench and the absence of independent peer-reviewed studies further contribute to the uncertainty surrounding its safety.
The CEO of the Consumers Health Forum of Australia, Dr. Elizabeth Deveny, highlights the growing reliance on AI due to rising healthcare costs and long wait times. While ChatGPT Health could potentially help manage chronic conditions and provide research assistance, there are concerns about people taking AI advice at face value without critical evaluation. The rapid development and deployment of AI by large tech companies leave little room for government regulation, creating a power imbalance where the risks are disproportionately borne by those with fewer resources.
Deveny emphasizes the need for clear guidelines, transparency, and consumer education to ensure informed decision-making regarding AI in healthcare. The goal is not to halt AI's progress but to implement safeguards before misinformation and bias are amplified at an unprecedented scale, making corrections nearly impossible.
