A Hybrid Deep Learning-Based Approach in Agricultural Sustainability: Apple Orchard Study

In the agriculture domain, the detection and classification of fruit images are critical tasks. This study introduces a novel hybrid model for the detection and classification of apples. For this reason, an on-site image-capturing system with RGB sensors was installed in an orchard to capture images from two different angles of apple trees at night, aiming to create a diverse database. For the apple detection task, three different YOLO versions (YOLOv5, YOLOv7, and YOLOv8) were used to identify the best detection model. Three sub-datasets( Sub-dataset 1, Sub-dataset 2, and Sub-dataset 3) of our main dataset were used to fine-tune the YOLO versions. Each sub-dataset has its labeling and sampling methods. Sub-dataset 3, with the fewest images, significantly improved the YOLO network performance in all versions. The combination of YOLOv7 with sub-dataset 3 achieved 82% accuracy, effectively detecting apples under various conditions. In classification, the detected apple images were classified based on their ripeness into three classes. The classes are determined by the time remaining until harvest. In the classification of apple images, three classification techniques were compared. First, fine-tuned models (VGG16, VGG19, Xception, ResNet50, ResNet50V2, ResNet152, and MobileNetV2) were tested that VGG16 achieved the highest accuracy of 0.924. Second, combinations of CNN models as a feature extractors ( VGG-16, RESNET-50) and machine learning algorithms as classifiers (SVM, Random Forrest, and XGBoost) that the VGG16 + XGBoost classifier achieved an accuracy of 0.923. Finally, a custom CNN model outperformed all others, achieving an accuracy of 0.983. Therefore, the combined YOLOv7-CNN hybrid model demonstrated efficient performance, making it a candidate for apple detection and classification in agricultural settings.