Deep Neural Network–Based Double-Check Method for Fall Detection Using IMU-L Sensor and RGB Camera Data

Existing methods for fall detection may not detect a fall when it occurs or may generate a false alarm when a fall does not occur. In order to overcome these limitations and detect falls with 100% accuracy, a double-check method for fall detection in elderly people via an inertial measurement unit-location (IMU-L) sensor and a red-green-blue (RGB) camera is proposed. The IMU-L sensor is a combination of an IMU sensor (accelerometer and gyroscope) and an ultrawideband signal-based location sensor; the RGB sensor is mounted on a robot. The proposed method involves detecting and confirming the fall of an elderly individual via the IMU-L sensor and an RGB image, respectively. The IMU-L sensor is worn on the body to detect falls. When a potential fall occurs, the individual's location information is synchronized with the motion data. During detection, because of the sequential nature of IMU data, a deep learning technique called a recurrent neural network (RNN) is trained to classify falls. When the IMU indicates a suspected fall situation, the robot moves to the corresponding location and confirms whether a fall has occurred. During the confirmation stage, a convolutional neural network-based technique is applied to the RGB image data to recognize and confirm the fall. Repeated confirmed fall detections using this method classified falls more accurately than existing methods that use only an IMU sensor. We conducted a real-time experiment to validate our method using a dataset developed in a laboratory and achieved 100% accuracy in our experimental environment.