Young-Tak Kim, PhD

One of the significant challenges associated with integrating artificial intelligence (AI) in clinical settings is understanding when it is safe to trust AI-based decisions.

Traditional metrics for evaluating AI performance, like accuracy, often do not address critical aspects of operational safety (i.e., meeting pre-specified reliability targets for rule-in and rule-out decisions), which leads to a hesitance in adopting such technologies.

To overcome this barrier, a new study led by Young-Tak Kim, PhD, and Synho Do, PhD, of the Department of Radiology at Mass General Brigham, introduces the Safety-Aware Receiver Operating Characteristic (SA-ROC) framework.

This tool indicates to providers when it is and is not appropriate to trust AI to help make clinical decisions, while also identifying a “Gray Zone” for judgments that require human review.

The researchers examined two FDA-cleared AI algorithms used for cancer screening. In a surprising turn, they found that the model with better performance metrics was less safe for clinical use under the most stringent safety requirements than the one with slightly poorer performance—revealing that looking solely at accuracy metrics can be misleading.

By offering a clearer understanding of how AI models operate in real-world scenarios, the SA-ROC framework could ultimately improve patient care and reduce physician workload with safer automation.

Published in npj Digital Medicine on February 20, 2026 | Read the paper: “Defining Operational Safety in Clinical Artificial Intelligence Systems”
Summary reviewed by: Synho Do, PhD, senior author