Towards Real-Time Detection, Tracking and Classification of Natural Video

We present a near real-time framework for detection, tracking, and unsupervised classification of entities in natural video streams based on similarity of motion attributes. The framework fuses motion detection and multi-object tracking, learning, detection (M-TLD) to extract motion states of video entities. Unsupervised clustering abstracts these motion states, with cluster centers associated with entities that exhibit motion similarity. The framework is evaluated on natural video from street corners in a downtown area comprised of pedestrians, vehicles, traffic and pedestrian lights as foreground objects, and other objects (trees, flags, buildings, sidewalks) that appear in natural scenes. Using motion attributes as features for classification reduces the computational burden of video processing and moves towards real-time performance for use on robotic platforms.