A novel integration of methods for person authentication and tracking is proposed for real time security systems. The implementation of the idea for this real time implementation follows a three step procedure-face detection, recognition and content-based tracking. Instead of analyzing continuous videos we sample the frame based on a method derived from Shannon's information theory model. The Face-detector detects multi-viewed faces in a video using feature-based kernel methods in a reduced feature space obtained using ICA. The identified "face regions" are then passed on to the face recognition system which is based on Active Appearance Models (AAM). Once the subject is recognized, it can be tracked in the video using kernel based object tracking method. Several space reduction techniques have been used like ICA, PCA and skin-color segmentation.