The First Adversarial Audit of Criminal Justice AI System in Europe

Read our 2024 audit, where we partnered with Iridia, a human rights nonprofit organization, to conduct the first adversarial community-led audit of a criminal justice AI system in Europe: the RisCanvi predictive algorithm. Our audit tested a system that has shaped parole and sentencing decisions for Catalonia's prison population since 2009, directly deciding over 7,713 peoples' lives, and that had never been audited despite Spanish law requiring oversight since 2016. Our audit uncovered severe automation bias, how staff overrided the systems' decisions in less than 5% of cases; and risk scores that, when we reverse-engineered them, showed no significant relationship to the risk factors RisCanvi claims to weigh — getting it right just 17.9% of the time it flags someone as high risk.

Automating (In)Justice? gives lawyers, judges, and policymakers the evidence that a tool currently deciding who gets parole, furlough, or an open prison regime fails the basic tests of transparency, fairness, and reliability it was built to meet.