Eye on You: Fusing Gesture Data from Depth Camera and Inertial Sensors for Person Identification

Person identification (PID) is a key issue in many IoT applications. It has long been studied and achieved by technologies such as RFID and face/fingerprint/iris recognition. These approaches, however, have their limitations due to environmental constraints (such as lighting and obstacles) or require close contact to specific devices. Therefore, their recognition rates highly depend on use scenarios. To enable reliable and remote PID, in this work, we present EOY (Eye On You)¹, a data fusion approach that combines two kinds of sensors, a 3D depth camera and wearable sensors embedded with inertial measurement unit (IMU). Since these two kinds of data share common features, we are able to fuse them to conduct PID. Further, the result can be transferred to a mobile platform (such as robot) since we have less constraints on devices. To realize EOY, we develop fusion algorithms to address practical challenges, such as asynchronous timing and coordinate calibration. The experimental evaluation shows that EOY can achieve the recognition rate of 95% and is very robust even in crowded areas.