A multimodal sensor system for automated marmoset behavioral analysis

The common marmoset is emerging as an important transgenic model for improving the understanding of the underlying neurological basis of many brain disorders. Automated systems for quantitative monitoring of marmoset behaviors in naturalist settings over long period of time are needed to facilitate this process. This paper presents the preliminary work toward building a novel multimodal acquisition system for the automated marmoset behavior analysis in home cage. In addition to integrating commercial available devices such as Microsoft Kinect sensors and microphones of different characteristics, we also developed a wireless flexible neck collar with acoustic and non-acoustic sensors onboard for marmoset vocalization recording and caller identification. Our initial effort has been focused on the real-time synchronization of multiple sensor outputs, the engineering design of the wireless collar, and algorithms for global 3D position and local head movement from a Microsoft Kinect sensor. With limited preliminary data, we are able to estimate 3D trajectories of two marmosets with a RMSE of ~3.2 mm and track colored ear tufts with an accuracy of RMSE ~1.8 mm. A larger dataset is needed for a complete assessment and validation. Our system architecture is modular and flexible, and can be extended to include more sensors and devices if needed.