A considerable amount of research is undergoing in the field of computer vision is related to object tracking. However, object tracking, in general, is a challenging problem. The challenges in objects tracking arise due to abrupt object motion, changing appearance patterns of both the object and the scene, occlusions, camera motion etc. Most of the works are focused on a specific application, such as tracking human, car, or pre-learned objects. However, the research on “tracking an object” mostly outperforms using selective algorithms that are applicable for only fixed settings. To a great degree object detection and tracking methods are used for video coding applications. The enormous computational need of real time object tracking makes it hard to achieve. In this paper, we propose a new algorithm for object tracking in a live streaming video that requires lesser computational resources, as compared to the needs of such approach.