Method and Implementation of Rock Melon Detection and Localisation for Fast and Reliable Autonomous Harvesting

Robust detection and localisation is a crucial step in developing an autonomous rock melon harvester. Rock melons have a distinct texture, shape and colour that computer vision techniques can capitalise on. Any applied method must also consider performance that goes beyond computer vision such as hardware, speed, and suitability against the robotic arm that harvests the melon. This paper proposes a hybrid 2D with depth method, and compares its performance against traditional 2D image processing techniques. In addition, two types of texture descriptors were examined: (1) scaled Haralick Features (HF) and (2) uniform Multi-scale Local Binary Patterns (MSLBP). These methods were tested on a robotic system and compared using the following performance metrics (a) recall, (b) precision, (c) computational cost and (d) harvesting speed. Testing was performed for detection and localisation of rock melons under 4 levels of leaf occlusion: 0%, 25%, 50% and 75%. The proposed hybrid-MSLBP method is the best performing method for the robotic system used, achieving an average harvest time of 19.71s while a traditional image processing method lags behind at 102.32s.