Multi-Task YOLO for Vehicle Colour Recognition and Automatic License Plate Recognition

Automatic License Plate Recognition (ALPR) systems are crucial in extracting vehicle information. However, ALPR alone is insufficient for robust vehicle owner identification, especially in the event of misidentification or covered license plates (LPs). Acknowledging the significance of vehicle colour in enhancing identification accuracy, this paper proposes a more secure and comprehensive approach by integrating car colour recognition (VCR) with LP detection and optical character recognition (OCR) tasks. Unlike the conventional two-stage ALPR systems, our solution introduces a novel one-stage YOLO-based multi-task model. It incorporates additional object detection heads onto the YOLO backbone, allowing for parallel processing and efficient real-time detection for all three tasks. The proposed model achieves spectacular results with mean Average Precision (mAP) scores of 0.778, 0.963, and 0.881 for OCR, LP detection, and VCR, respectively. Promisingly, our model is comparable to single-head, single-task models, which are trained solely for each task. We also demonstrate that our model outperforms a single-head multi-task model, which naively shares all tasks using one single head. Specifically, our model is 3.07x faster than the conventional approach, which involves inference of single-task models for OCR, LP, and VCR sequentially. Experimental results demonstrate that our solution is robust in simultaneously addressing OCR, LP detection, and VCR within a unified, single-stage framework.