A Comprehensive Design of Hybrid R(2+1)D-CNN and Dense Network with Multi-Modal Sensor for Fish Appetite Detection

Fish is one of the most demanding protein sources in the food industry. However, the increasing demand must be followed by increasing production efficiency. One of the problems in fish production efficiency is an ineffective feeding method. In this paper, we address the problem of fish feeders using artificial intelligence in an aquarium. We propose fish appetite detection using multi-modal sensors resulting in the data for our AI system. Our AI system consists of R(2+1)D convolutional layers and dense networks to process video and accelerometer data. The video data is split into 20 frames and processed by an R(2+1)D convolutional neural network. The accelerometer data is used to train several networks such as 1-dimensional CNN, GRU, and dense (ANN). The system is implemented in an aquarium with two sensors i.e., a webcam camera and an accelerometer and a main board processing using Raspberry-Pi 4. Experimental results show that the proposed system outperforms other methods with validation accuracy up to 99.09% for the Zeromean dense model and up to 99.39% for the Filtered dense model. The work is useful for automation and efficiency in aquaculture.