Simultaneous estimation of food categories and calories with multi-task CNN

In this paper, we propose simultaneous estimation of food categories and calories for food photos. Since there exists strong correlation between food categories and calories in general, we expect that simultaneous training of both brings performance boosting compared to independent single training. To this end, we use a multitask CNN. In the experiments, we collected calorie-annotated recipe data from the online cooking recipe sites, and trained multi-task and single-task CNNs. As results, the multi-task CNN achieved the better performance on both food category estimation and food calorie estimation than single-task CNNs.