Facial recognition

Most of the drama shows on television build on some kind of facial recognition, a set of faces flickering rapidly in the background, getting a match just as the main characters rendezvous in front of the screen.

I looked into the performance of facial recognition for my book “Knowledge Discovery for Counterterrorism and Law Enforcement (Taylor and Francis, available from all good booksellers) but it’s been a while so I thought I would go back and look at the current state-of-the-art.

First, it probably doesn’t have to be said, but real systems don’t display all of the faces as they process them — if they did it would slow them up by a factor of more than a thousand.

Second, what is their performance? There are many variables: camera angle, lighting, amount of space (and so detail) available for image storage.

There are also different versions of the problem. One important one is deciding if this specific stored image matches this just-captured image or not. This is what is used with biometric data stored in passports; there’s a digitized version of the photo you submitted in the chip in your passport; when you cross a border, a photo is taken of you (again, under quite controlled conditions) and that new photo is matched to the old. Even for such a 1-to-1 match the error rate is not trivial — I’ve seen 25% quoted which seems high, but agrees with my own experience.

The more common problem (in tv shows) is that an image has been captured from, say, CCTV and the goal is to determine if the person with that image is in a large database of identified images. In the jargon, the database images are called enrolled, and the newly collected one is called a probe.

Performance is usually characterized by giving a False Match Rate (FMR), the rate of matching a probe to an enrolled image when they aren’t actually the same person. So, for an access control system, this is the rate at which the system would let an intruder in. At present, values of around 0.001 (1 in a thousand) are typical. For this value, then, the dependent variable is the False Non Match Rate (FNMR) which is the rate at which someone who does match gets missed. So, for an access control system, this means that a legitimate entrant gets locked out.  These are typically in the range 0.03 to about twice that (3 in a hundred).

You can see that these results are much, much weaker than those portrayed on tv. If the database contains a million images, then it’s not a case of exclaiming “We found a match” but “we found a thousand matches (and now we have to go through them and see if we think any of them is actually a match)”. Not finding a match would be much more surprising. Some systems seem to be much worse; you don’t have to look far to find stories of facial recognition systems that have never matched anyone, even when their images are known to have been captured as probes — part of the problem being that one person in a hoodie looks pretty much like any other person in a hoodie.

From an access control point of view, these rates mean that there’s a 1 in a thousand chance of an intruder getting is (which is probably acceptable for many situations), but 3% or more of the time, legitimate users will have to try again. I haven’t seen any data, but presumably the false non matches are not uniformly distributed, so that some people have to try again much more often than others (i.e. not all faces are equally recognisable).

Of course, these performance numbers are, if not in ideal conditions, then in reasonable conditions, whereas real images tend to be much more variable (weather, dust on the camera lens, shake on the mounting,…). And, of course, in real systems you can’t zoom in and miraculously produce more pixels as they seem to be able to do on tv. So there’s quite a long way to go. I think it’s fair to say that progress is being made — but facial recognition is a long way from production use.


0 Responses to “Facial recognition”

  1. Leave a Comment

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: