Vehicle detection and classification using audio-visual cues

The road transport is one of the most common modes of transport. Road planning and traffic management is conducted based on survey of traffic volume. These surveys can be manual or automatic. Audio based survey suffers from low accuracy but has low computational cost. Video based survey has significantly higher accuracy but demands high computational resources and time. In this paper, we propose an approach which utilizes both audio and video of traffic data to perform automatic traffic survey. Vehicles are automatically detected by locating peaks in the smoothed short time energy of the captured audio signal. Video frames are extracted around the location of the detected peaks. Thus, the number of video frames to be processed is reduced considerably. Vehicle image from the extracted video frames are detected using background subtraction and three frame differencing. Noisy binary image thus obtained is transformed into single object using morphological processing. Features such as area, perimeter, maximum length, horizontal length and 32 features generated from the vehicle shape are used to characterize the image of vehicles. These feature vectors are used to train a multilayer feed-forward artificial neural network classifier for seven classes of vehicles. The effectiveness of the proposed algorithm is tested using a query audio to obtain an accuracy of 82%.