Deep Learning and Computer Vision for Estimating Date Fruits Type, Maturity Level, and Weight

According to the Food and Agriculture Organization, the world production of date fruits is 8,526,218 tons and around 1,302,859 tons in the Kingdom of Saudi Arabia (KSA) in 2018. There are several types of date fruits, and the most common in KSA are Barhi, Khalas, Meneifi, Naboot Saif, and Sullaj. Moreover, there are around five main maturity levels: Immature, Khalal, Khalal with Rutab, Pre-Tamar, and Tamar. Harvesting date fruits is performed according to its maturity level and type, which is a critical decision that significantly affects profit. In this paper, we propose a smart harvesting decision system to estimate date fruits type, maturity level, and weight using computer vision (CV) and deep learning (DL) techniques. The proposed system consists of three sub-systems: Dates maturity estimation system (DMES), type estimation system (DTES), and dates weight estimation system (DWES). We utilized four DL architectures, including ResNet, VGG-19, Inception-V3, and NASNet for both DMES and DTES and support vector machine (SVM) (regression and linear) for DWES. We evaluated the performance of the proposed system using the dataset collected by the Center of Smart Robotics Research. Using multiple performance metrics, DTES achieved maximum performance of 99.175% accuracy, an F1 score of 99.225%, 99.8% average precision, and 99.05% average recall. The maximum performance of DMES was 99.058% accuracy, F1 score of 99.34%, 99.64% average precision, and 99.08% average Recall. DWES achieved a maximum performance of 84.27% using SVM-Linear.