A new commentary in npj Digital Medicine, emerging from the Duke Health AI Evaluation & Governance Program’s strategic collaboration with Erasmus MC, argues that clinician involvement alone does not constitute meaningful oversight. Authors Davy van de Sande, Nicoleta Economou-Zavlanos, and Michel E. van Genderen identify four conditions needed to make human judgment an effective patient-safety mechanism: knowledge of the system, time to assess its outputs, authority to challenge its recommendations, and practical ability to intervene.
The authors offer a framework for embedding these conditions across the AI lifecycle, from procurement and deployment through monitoring and decommissioning. As predictive, generative, and agentic AI systems become more deeply integrated into care, oversight must be designed into the clinical, organizational, and technical environment, not left to individual vigilance.
This publication reflects the Duke–Erasmus MC collaboration in action: combining international expertise to develop practical, evidence-based approaches to trustworthy health AI.