A new study suggests hospitals may need more than just oversight, showing that comprehensive physician reviews can defuse public backlash when AI-assisted diagnosis causes harm.
Paper: Public reaction to hospitals after adverse events related to AI. Image credit: Antonio Marca / Shutterstock
In the medical field, artificial intelligence (AI) is increasingly being used for tasks such as diagnosis, treatment planning, patient communication, and clinical operations. However, according to a recent study published in the same journal, experimental results suggest that public reaction may be negative when errors in diagnosis or adverse events that harm patients are involved. npj digital public health. The authors also note that this trend is less pronounced if physicians continue to be substantively and interactively involved in AI-assisted decision-making.
background
In addition to the expected efficiency and performance gains, the use of AI in healthcare carries the risk of AI-related adverse events, or “unintended injuries caused by medical management,” especially when incorrect or incomplete AI-generated outputs are applied to patient care. These outcomes can result in decreased patient trust in the hospital, reputational damage, and legal action.
Little research has been done on how the public reacts to such events, particularly when it comes to attributing blame and responsibility to hospitals. Hospitals are obvious targets because they oversee the implementation and monitoring of AI systems through their organizational structures and are often considered primary healthcare providers, as in the UK.
Assessing public reaction to AI-related mistakes
The current report describes two studies. The first study investigated public reactions to hospitals in the context of a hypothetical adverse event involving a diagnostic error. This varied depending on the source of diagnostic interpretation and level of physician involvement.
An initial study of 299 online participants showed that even when endoscopists considered the results of the AI, AI-related adverse events were more likely to cause negative reactions to the hospital. However, in the latter case, the attribution of responsibility was significantly lower than in the AI-only condition, but still higher than in the human-only condition.
Substantial involvement of physicians reduces attribution of responsibility to hospitals
The second study involved 602 online participants and investigated whether collaboration between doctors and AI influences public reaction.
Unlike the first study, which investigated collaboration at a broader level, the second study investigated the structurally different ways in which collaboration occurs in real life. Participants were randomly assigned to one of the following conditions:
・For AI only (AI diagnosis)
• Human only (diagnosis by radiologist without AI assistance)
• Human-AI collaboration:
o Autonomous (AI-driven, less human involvement)
o Sequential (Radiologist reviews areas flagged by AI, moderate involvement)
o Interactive (radiologists review both AI-flagged regions and the entire image, integrating AI input into independent clinical evaluations and high levels of engagement)
The scenario was the same in all situations. This meant that the diagnosis of pneumonia was missed, to the detriment of the patient.
Across comparisons, public reactions to hospitals were worse when AI was used alone or with autonomous or sequential physician involvement. Participants attributed more responsibility to the hospital and were more likely to file a complaint or consider legal action when AI was involved without substantial physician involvement compared to meaningful human physician involvement. Sequential collaboration did not significantly change negative reactions compared to AI-only interpretation.
In the physician-only model or the conversational AI plus physician condition, physician involvement was higher and participants were less likely to attribute responsibility to the hospital or file a complaint against the hospital. Legal action was less likely in the interactive collaboration condition than in the AI-only condition.
These findings highlight the potential protective and reassurance value of ensuring high levels of human involvement and human oversight of AI integration tasks in healthcare. This suggests that visible and comprehensive physician oversight may be beneficial to reassure patients about AI-assisted medicine.
Restrictions
The researchers used a hypothetical situation because ethically it is not possible to induce real-world adverse scenarios. Although participants were primarily recruited from the United Kingdom, supplementary analyzes reported that the observed patterns did not differ systematically across countries.
conclusion
This is one of the earliest studies to examine public reactions to adverse events associated with AI use in hospitals.
This study shows that public reactions to hospitals associated with AI-involved adverse events are more negative than comparable human-only adverse events, especially when human physicians are not highly involved. Although participants believed such hospitals had more responsibility and reported being more likely to file complaints and take legal action, they reported that hospitals were still held accountable for AI-assisted decisions.
Although AI may be used as part of clinical workflows, hospitals were still perceived to be responsible for adopting, implementing, and monitoring AI use. The role of clear communication and workflow design in maintaining AI-related efficiency without risking loss of patient trust remains to be evaluated.
Future research should also examine how the public holds doctors, AI developers, and hospitals to varying degrees of responsibility for mistakes made during AI-assisted treatment. Another area of research involves differences in response to varying degrees of patient outcome.
Click here to download your PDF copy.
