A Framework to Identify Allergen and Nutrient Content in Fruits and Packaged Food using Deep Learning and OCR

Allergic reactions to food can depend on a wide range of factors and hence, the proportionate reactions of the same can vary. With such a wide range of unpredictability, classifying allergens, and the rate at which it would effect is what the scientists have been working on for years. To bring awareness of the food we consume and the potential threats it can cause us, in this paper we propose a 2-Tab Deep Learning based Application to provide the nutrient and allergen content in fruits and vegetables and, to display allergen information in packaged food using OCR. Through a novel Deep Learning Framework, the picture of the Fruit or Vegetable captured via an application is classified and recognized and the nutritional facts and allergen information is presented. The fine-tuned deep learning model which is deployed in cloud, obtained a good accuracy of 97.37 percentage on our dataset. For Packaged food, the picture of the Ingredient Index is captured via the application and the allergen information is presented after the text is recognized through Optical Character recognition which would be carried out in a remote server.