Facial recognition and automated license plate readers (ALPRs) are tools increasingly being used by law enforcement. They have become essential for crime analysts to help identify offenders and vehicles. Do you know how they work?

Basics of Image Recognition

ALPRs and facial recognition systems are more similar than you think. In fact, they are both variations of the same concept of image recognition using machine learning.

Data Collection

Every object has identifying characteristics related to edges, corners, textures, color, size, and others. When an image of an object is captured, these characteristics become translated into pixels. Pixels are very small squares of color, and these small squares are how an image can be displayed on a computer screen or monitor. The combination and arrangement of pixels are what makes the object in the image recognizable to the viewer.

The first step in the recognition process is to create a data library of objects within the images. This is where the system writes instructions to recognize the objects from the pixels in each image. If an image contains pixels in a specific order and arrangement, then the system identifies it as resembling a specified object. The name of the object is then assigned to the specified pattern of pixels and stored for later use. This process is replicated endlessly until the data reference library is complete.

Image Processing

After the reference library is complete, the next step is to introduce a new image containing an unspecified or unidentified object. When uploaded into a recognition software, the image is processed to extract the pixel pattern. The software then compares the extracted pixels to those in the library of pixel patterns corresponding to identified objects. If the pixel pattern from the image of the unidentified object is located within the existing pixel library, then the system recognizes them to be the same pattern. Therefore, the object in the new image is identified as being the same as an object within an image in the stored library. A match is reported.

Machine Learning

Modern recognition software programs work on the same simple concept of image comparison. The difference is they apply a more complex algorithm to distinguish details of objects. This is how ALPRs can recognize the color, make, and model of vehicles from an image.

Using machine learning (process where the system is able to learn by using algorithms and then draw inferences from patterns in data), pixel patterns are continuously added to the reference library. If the pixel pattern of a new image does not match one of the existing patterns, the pattern from the new image is added to the reference library. This increase the data library. With more patterns available to reference, there is an increased likelihood of the pixel patterns of two images matching in the future.

Each software company utilizes proprietary algorithms to improve the image recognition process. However, each algorithm is based on the simple concept of comparing unique elements within an image to characteristics within a series of existing images. 

Facial Recognition Systems

Based on the basics of image recognition, these concepts can be applied to one of the more controversial areas: facial recognition. Instead of recognizing objects within an image, facial recognition systems focus solely on the unique characteristics of faces present in each image.

Oftentimes, arrest booking photo collections are used as the reference library for law enforcement facial recognition systems. Booking photos often a good source of facial recognition images because the photos are often clear, taken with good lighting, and taken from a straight angle. These datasets are also often large, which increases the chance that a match will be made.

After a suspect image is uploaded into a facial recognition system, the pixels of the image are compared to pixel patterns from existing photo libraries. The best candidates are identified and provided as matches or possible matches within the system.

Do These Systems Make Errors?

Do facial recognition and ALPRs systems make errors? The answer is yes and no. When a ‘recognition’ or ‘match’ has been made, the system has done what it was programmed to do. Remember that is comparing a new image to images in the data library and provide results in the form of a match. Many times, this match is accurate. Other times, the match is not. While we might see this ‘wrong’ match as an error, the reason we are seeing this match as a possibility has to do with thresholds set within the system.

Thresholds are important because images are complicated. Thresholds allow for the system to identify possible matches, even if every aspect of the image does not match an image in the data library. Oftentimes, the image that we use for comparison to our data library is not the best. It may have been captured from a low quality camera with a bad angle, or the lighting at the time may not have been optimal. Thresholds allow the system to make comparisons by producing candidates where many of the comparison pixels are the same but some are not. While we definitely want to see matches where the system is 100% confident the image is the same as one in our database, we probably also want to see those where the system is 80% or 90% confident.

Candidates

Some facial recognition systems provide the strongest match candidates first or at the top. As the user scrolls through possible matches, they start with candidates with the best scores and work their way towards candidates with lower match scores. This process is similar to how an internet search engine works: best matches are at the top while candidates with fewer matching points receive a lower rank and are at the bottom or next page.

Conclusion

In conclusion, modern tools such automated license plate readers and facial recognition are nothing more than systems built to identify patterns. These systems identify the pixel patterns from an image, create a library of these patterns, and compare the pattern from new images to the existing library of patterns to produce a match. The more images in the library, the higher the chance of a match. With machine learning, we can expect these systems to improve in the future.