LPR Architecture

This is a high-level overview of the LPR system
An MJPEG stream is retrieved by a capture process which passes each frame to a background segmentation module. The capture process is currently implemented as a C++ class. It may make more sense for ESM to handle this; this requires additional discussion.
The background segmentation (BGSeg) module analyzes each frame for differences from a time-averaged reference image. Frames without change regions are discarded; frames with change regions are passed (along with the coordinates of the regions) to the localization and segmentation module. The BGSeg module is implemented as a platform independent C++ class.
The localization and segmentation (L&S) module analyzes the specified change region of each frame given to it. It performs non-linear enhancement and perspective correction, and then uses certain tunable parameters and heuristic measures to localize contours within the image which may be license plate characters. These contours are analyzed and grouped via k-means clustering. Clusters of characters which are believed to represent a single plate are then passed to the OCR process. The L&S module is implemented as a platform independent C++ class.
The OCR process is tesseract, an open-source OCR engine developed by HP. It can be trained to recognize specific fonts, and will be distributed with training data for license plate character sets. This is a separate, standalone process.
The recognized plate characters, along with other pertinent metadata (path to video frame, timestamp, location of plate within frame, possibly certain confidence measures, etc.) is stored in a database. The L&S module oversees the execution of the OCR process and can likely conduct the DB transactions to record this metadata once we decide on a schema for the metadata.
The search process must account for inaccuracies in plate recognition. Levenshtein distance is a method of fuzzy text matching wherein distance penalties are accrued for every character inserted or removed to make two strings match. This method can be tailored to OCR by assessing smaller distance penalties for characters which are frequently mixed-up (like I and J), and larger distance penalties for more dissimilar characters. This algorithm is currently implemented as a platform independent C++ function, although it could also be implemented as a stored procedure in the database.

LPR System Overview