False Positives and the Law

When that spot on your mammogram turns out to be benign, that’s called a false positive. When a blood test shows your PSA level to be normal and you later learn that you had prostate cancer all along, that’s a false negative.

In the law, we rely on evidence to judge guilt or innocence. Was the fingerprint at the crime scene a false positive? Was the alibi a false negative? Doctors ultimately learn the truth about whether their tests were giving them correct diagnoses: their patients develop cancer…or do not. Legal guilt or innocence is almost never known with such certainty. Many a man has gone to the gallows proclaiming his innocence.

A guest lecturer at Stanford Law School recently suggested that the law should try to evaluate legal evidence the way medicine evaluates blood tests, using the theoretical framework of false positives and false negatives. When asked in the Q&A about how to go about that, he said his interest was in posing the questions, and that someone else would have to provide the answers. (Insert your favorite lawyer joke here.)

What if we, unlike the reticent professor, wanted to wade into the murky swamp of evidence? How might we test its reliability?

The clues are stored on ruled-and-numbered legal sheets in gray courthouse filing cabinets and in countless post-trial jury interviews. But they are so scattered and inaccessible that they have not been of much use. Enter the digital wizards of Silicon Valley. With today’s learning algorithms to sift and make sense of vast amounts of data, perhaps the secrets in those dusty tombs can now be unlocked. 

For instance: In how many cases did an eyewitness provide what was apparently the key to conviction? And in how many of those was the defendant later convincingly exonerated by DNA evidence or the confession of another?

What do juries say about what influences them? If we asked them in a systematic way over the course of thousands of trials, and made their answers searchable, what would our artificial intelligence programs tell us about what evidence is most important? And what would we say about that? Is the evidence that convinces juries of a kind we are comfortable being the basis for life or death judgments?

The law has a horror of false positives. A jury’s verdict is not a guide for treatment that can be given and withdrawn, it is a binary final determination: guilty or innocent. William Blackstone, the famous eighteenth-century English legal thinker, said: “It is better that ten guilty persons escape than that one innocent suffer.”

Our legal system relies on our humanity. Its foundation is our earnest desire to judge one another fairly. There is a mass of data on how well we do that to be gathered and scanned and looked at from all sides. On questions of the quality and reliability of legal evidence, artificial intelligence could help us get better at passing judgement on our fellow man. Far from being the threatening overlord of science fiction, it could strengthen an essential element of what makes us human.