Multinational License Plate Recognition using Generalized Character Sequence Detection

Automatic license plate recognition (ALPR) is generally considered a solved problem in the computer vision community. However, most of the current works on ALPR are designed to work on license plates (LP) from specific countries and use country-specific information which limits their practical applicability. Such ALPR systems require changes in the algorithm to work on other countries LPs. Previous works on multinational LP recognition are tested on datasets from various countries that share the same LP layout. To address this issue, this study presents a deep ALPR system designed to be applicable to multinational LPs. The proposed approach consists of three main steps LP detection, unified character recognition, and multinational LP layout detection. The system is mainly based on the you only look once (YOLO) networks. Particularly, tiny YOLOv3 was used for the first step whereas the second step uses YOLOv3-SPP for a version of YOLOv3 that consists of the spatial pyramid pooling (SPP) block. The localized LP is fed into YOLOv3-SPP for character recognition. The character recognition network returns the bounding boxes of the predicted characters and does not provide information about the sequence of the LP number. A LP number with an incorrect sequence is considered wrong. Thus, to extract the correct sequence, we propose a layout detection algorithm that can extract the correct sequence of LP numbers from multinational LPs. We collected our own Korean car plate (KarPlate) dataset and made it publicly available. The proposed system was evaluated on LP datasets from five countries which include South Korea, Taiwan, Greece, USA, and Croatia. In addition, a small dataset containing LPs from 17 countries was collected to evaluate the effectiveness of the multinational LP layout detection algorithm. The proposed ALPR system consumes about 42 ms per image on average for extracting LP number. Experimental results demonstrate the effectiveness of our ALPR system.