Even Worse than Eyewitness IDs: The Police Sketch

Everyone knows that eyewitness identifications are completely reliable — that is, you can count on them to be wrong.  (Everyone does know this, right?)  If the person being identified is a stranger, the chances of a correct I.D. are slim to none.  There are lots of reasons for this.  Eyewitnesses rarely have or take the time to study and memorize a person’s features.  People of one race are awful at identifying people of another race, largely because the parts of the face which differ from person to person are different from race to race — which is why people of another race often “all look alike,” because you’re looking for cues in parts of the face that don’t vary much in that other race.  And people just generally suck at remembering details consistently and accurately.

Still, sometimes an eyewitness description is all you’ve got.  And so what if the eyewitness didn’t see every detail of the face — at least they can describe the parts they did see.  Trained sketch artists take the partial descriptions provided by eyewitnesses, and using sophisticated software they can put together composite sketches that show what the bad guy probably looks like.

We’ve all seen them on the TV news, and various crime dramas would lead one to believe that they’re pretty useful.  And now with IdentiKit software, the details can be adjusted here and there until the witness goes “that’s him!”

But we never hear, after the fact, whether the drawing wound up being all that accurate.  There’s a good reason for this.  The odds of the drawing being accurate are so low, they are below statistical significance.  You’ve probably noticed this yourself, on the rare occasion when a police sketch has later been released with a photo of the culprit — the resemblance even then is usually pretty slim.

A thorough study of composite sketches by Charlie Frowd, of the University of Stirling in Scotland, had participants study a photograph of an individual for a full minute, then describe the face for a trained police sketch artist.  How well could people then recognize the faces in these sketches?  The recognition rate was as low as 3%.

Three percent.

MIT scientists Pawan Sinha, Benjamin Balas, Yuri Ortrovsky and Richard Russell have a great article here that describes problems with composite sketches and ways to make the software better.

The image above was taken from that article.  A trained and experienced IdentiKit officer was given actual photographs of celebrities with distinctly recognizable faces.  He was given all the time in the world — no pressures — and worked directly from the photos themselves instead of having to rely on another person’s descriptions.  And those sketches you see up there are the best the software could do.

Well, maybe the problem is with what the IdentiKit tries to do.  After all, it just works on individual features one at a time.  The eyes, nose, mouth, etc. are worked on in isolation.  Humans don’t look at features in isolation, though.  So there’s another kit out there called EvoFit, that’s more like a photo array that gets to evolve.  The witness is shown 72 random faces.  She picks out the six that most resemble the culprit.  The facial features of those six are then scrambled and recombined to make 72 new pictures.  The witness then picks out, again, the six who most resemble the culprit.  The process is repeated once more to get an image that pretty much matches what the witness saw in her mind.

Now, there are tons of problems with this method.  The suggestivity of showing pictures is pronounced — when witnesses choose photos from an array, they often choose not the one that closest resembles the culprit, but instead pick the one that looks different from the rest — and when a picture has been chosen, that image often replaces the image in the witness’ memory.  She now remembers that face as being the face of her attacker, even though it wasn’t.  This method of scrambling digital faces poses the same problems.

Still, it is more reliable than the IdentiKit.  Instead of a 3% recognition rate, the EvoKit attains a whopping 25% recognition rate.

One in four.

-=-=-=-=-

People suck at identifying strangers.  Period.  And yet in-court eyewitness identifications are the nuclear bombs of trial.  The victim points at the defendant and says he’s the one what done it, and you can see the jurors’ minds turning off.  So far as they’re concerned, this trial’s over.  The defense lawyer’s got a lot of work to do, now, to overcome that.

What would be just and fair, of course, would be to allow some evidence of the unreliability of eyewitness identifications in general, and the reasons why IDs can be wrong, so that the defense can tie them to specific testimony by the eyewitness to show that she made the same mistakes.  Not asking the jury to make a logical fallacy that, because it happens a lot in general, it must have happened here as well.  But actually drawing the jury’s attention to specific reasons why this particular testimony is not trustworthy, supported by expert testimony on the unreliability of IDs.

Don’t hold your breath waiting for that to happen, though.