A new commentary published in the Journal of the Royal Society of Medicine warns that the current risk-based regulatory approach to artificial intelligence (AI) in healthcare is insufficient to protect patients and could lead to over- or under-treatment and even discrimination against patient groups.
The authors found that although AI and machine learning systems can improve clinical accuracy, concerns about their inherent inaccuracy, opacity, and potential for bias remain, and are not adequately addressed by current regulatory efforts introduced by the European Union’s AI legislation.
The AI Act, passed in 2025, classifies medical AI as “high risk” and introduces strict controls for providers and adopters. However, the authors argue that this risk-based framework overlooks three important issues: individual patient preferences, the systemic and long-term effects of AI implementation, and patient disenfranchisement in the regulatory process.
Patients have different values when it comes to accuracy, bias, or the role AI plays in their care. Regulation must go beyond system-level safety and consider individual rights and participation. ”
Thomas Ploug, first author, Professor of Data and AI Ethics, Aalborg University, Denmark
The authors call for the introduction of patient rights regarding AI diagnosis and treatment planning. This includes the right to:
- Ask for explanation.
- Granting or withdrawing consent.
- Seek a second opinion. and
- Refusing to diagnose or screen based on publicly available data without consent.
They warn that without urgent engagement by healthcare stakeholders, including clinicians, regulators and patient groups, these rights risk being left behind by the rapid evolution of AI in healthcare.
“AI is transforming healthcare, but it must not come at the expense of patient autonomy and trust,” Professor Praug said. “The time is now to define rights that protect and empower patients in an AI-driven healthcare system.”
The need for patient rights in AI-driven healthcare – risk-based regulation alone is not enough (DOI: 10.1177/01410768251344707) by Dr. Thomas Ploug, Rikke Frank Jørgensen, Hanne Marie Motzfeldt, Naomi Ploug, and Søren Holm.
sauce:
Royal Society of Medicine
