Sensor fusion for occupancy detection and activity recognition using time-of-flight sensors

New technologies in lighting enable the design of illumination systems that autonomously meet the needs of occupants. In recent years, lighting design for both commercial and residential spaces has advanced beyond task performance, to a broader range of occupant needs including economic, energy and environmental constraints, architectural integration, human health, productivity, interpersonal communication, and aesthetic quality. Such systems require distributed sensing and control systems with sensory feedback to detect the lighting conditions in the space. Occupancy sensing and activity recognition are core components of this distributed sensor-based control system, and multisensor fusion of integrated sensors is viewed as a key attribute. This paper describes an expanded view of multisensor technologies, signal processing and pattern recognition algorithms that are being developed and evaluated for occupancy detection and activity recognition. This paper presents a multisensor testbed system that incorporates a sparse array of time-of-flight range sensors with pattern recognition algorithms for geometric form and motion detection of human occupants. Three levels of analysis are implemented: (1) Occupant detection and tracking, (2) Occupant pose classification (sitting, standing, and walking), (3) Occupant activity sequence recognition. These algorithms are dependent on statistical training of human pose and motion and implemented with a Bayesian formulation for detection, classification, and recognition. Evaluation of resulting performance in the testbed conference room demonstrates pose and activity recognition accuracy of greater than 97% for single occupants.