FoRConvD: An approach for food recognition on mobile devices using convolutional neural networks and depth maps

Proper nutrition is fundamental for a healthy lifestyle, especially in our modern world where many diseases could be avoided with the right diet. The paper introduces an approach FoRConvD (Food Recognition using Convolutional neural networks and depth maps) for detecting types of food and their mass on mobile devices, by using only the phone camera. Our approach consists of two main components: food type detection and volume estimation. The detection of the food type is done with EfficientNet, a state-of-the-art convolutional neural network model suitable for mobile platforms. The method used to estimate the volume is called depth map fusion and involves taking different images from various angles, along with their depth maps and computing a 3D model of the object. The reliability and accuracy of the method proposed for food volume estimation is empirically proven by the experimental results, resulting in a slight volume overestimation of 0% -10%, depending on the shape of the object.