Northwell ran an FDA-cleared AI across 32,000 real-world scans. The few times radiologists overruled it, they caught hundreds of blood clots it had missed.
The World Economic Forum used that study this week to make a point about AI governance. The model was rarely wrong. When it slipped, only a trained human caught the miss.
Models make mistakes. Someone has to catch them before they reach a patient.
Confident output can switch off an expert’s judgment. It looks right, so people wave it through. A human in the loop means nothing unless that person can overrule the model and knows the moment to do it.
The budget rarely reflects that. Companies spend heavily on the model and far less on the people who catch what it misses.
The model is the easy thing to fund. A person who can question a confident answer and be right takes years to develop. Most teams are not funding that.
Hundreds of those clots got caught because someone didn’t trust a clean scan.
Which decision on your team would you keep in human hands, no matter how clean the model’s output looks?
#CISO #AIGovernance #CyberSecurity #SecurityLeadership #AIRisk