The “Prosecutor’s Fallacy” is one example of why we think Statistics should be a required course in college. Let’s say the police have the DNA of a rapist. Only 1 in 3,000,000 people chosen at random will match that DNA sample. Your DNA matches. At your trial, the DNA expert testifies that you have only a 1 in 3,000,000 chance of being innocent. That is not correct, however. That’s an example of the Prosecutor’s Fallacy.
Yes, there is a very small chance that someone’s DNA would match if they were innocent. But that is not the same as saying there’s a very small chance that someone is innocent if their DNA matches.
This is basic conditional probability. And if you think about it, it’s just common sense. What you’re doing is switching the conditions around, and leaving the result unchanged. You can’t expect to change the conditions and not change the result.
To illustrate with an extreme example, we drew the picture you see above. A black circle indicates a DNA match. All guilty people are going to have a DNA match, obviously. And a tiny fraction of innocent people are going to have a DNA match, as well. But if the number of innocent people is large enough, then the number of innocent people whose DNA matches could actually be larger than the number of guilty people. Someone whose DNA matches is actually more likely to be innocent in that scenario.
Prosecutors and DNA experts aren’t the only ones who get this wrong. Courts do, too. The Ninth Circuit recently made a hash of it in their decision in McDaniel v. Brown, which will now be one of the first cases to be heard by the Supreme Court at the start of this year’s October term.
In McDaniel v. Brown, Troy Brown was prosecuted for the alleged rape of a little girl. The facts are pretty gruesome, but irrelevant here. What’s relevant is that, at his trial, the DNA expert testified that Brown’s DNA matched the DNA in the semen found on the girl, that there was a 1 in 3,000,000 chance that someone’s DNA would match, and that therefore there was a 1 in 3,000,000 chance that Brown was innocent.
Brown got convicted. He later brought a habeas petition to the District Court. He introduced a professor’s explanation of how the prosecution had screwed up. The District Court expanded the record to include the professor’s explanation, and found that the DNA expert had engaged in the Prosecutor’s Fallacy. In part because of that (there was also a chance it could have been his brother’s DNA), the District Court found there wasn’t sufficient evidence to convict.
The government appealed to the Ninth Circuit.
Now, the Ninth is known for being touchy-feely. It’s not known for its analytical prowess. Posner, they ain’t. But they bravely tackled this statistical conundrum. And they screwed up.
In trying to deal with the prosecution’s error, the Ninth swung too far in the other direction, finding that the DNA evidence at Brown’s trial couldn’t establish guilt, period. No jury could have found Brown guilty.
So the government took it to the Supreme Court, making two arguments. One is procedural — that the habeas court shouldn’t have been able to consider the professor’s explanations, but only the trial record, in determining the sufficiency of the evidence before the jury. The other argument is that even though the chances of Brown being innocent weren’t 1 in 3,000,000 they were still pretty damn low, and the DNA evidence is still plenty sufficient.
Brown’s lawyers, to their credit, don’t seem to be arguing that the Ninth Circuit did it right. Instead of characterizing the decision below as ruling on the sufficiency of the evidence, Brown’s attorneys argue that it was really a Due Process ruling. The testimony wasn’t so much insufficient as it was incorrect. It was unreliable. This is bolstered by the fact that the Ninth Circuit ordered a new trial (which Double Jeopardy would preclude after a finding of insufficient evidence, but which is standard after a Due Process finding of unreliable evidence.)
That’s not the way the Ninth characterized its ruling, however, so Brown wisely suggested that the Supreme Court might simply kick the case back for the Circuit to explain its ruling better.
Oral arguments are scheduled for October 13. We haven’t made any predictions yet about the upcoming term, so we’ll start here.
We think the state will convince Chief Justice Roberts and Justices Scalia, Kennedy, Thomas, Ginsberg and Alito of the following:
(1) The Ninth Circuit improperly remanded for a new trial, which is improper after a finding of insufficiency; and
(2) At any rate, the Circuit improperly found the evidence to be insufficient, when there was plenty of evidence of guilt.
We think that Justices Stevens and Breyer (we have no clue about Sotomayor) will dissent, arguing that the jury was totally thrown by the DNA expert’s mischaracterization, that this was a Due Process violation at the very least, and that the DNA evidence probably should have been thrown out entirely, so the Ninth Circuit should be reversed and the District Court’s original ruling should be reinstated.
What are the odds that we’re really right? Who wants to do the math?