HDTC: Hybrid Model of Dual-Transformer and Convolutional Neural Network from RGB-D for Detection of Lettuce Growth Traits

Automatic detection of lettuce growth traits is of great significance in modern greenhouse cultivation. Existing methods mainly focus on capturing coarse representations from RGB or RGB-D images with learnable convolutional neural networks. However, due to the significant appearance-varying discrepancies at different growth stages, coarse representations and inefficient depth fusion strategies limit the performance of automatic detection of lettuce growth traits. To alleviate the above problem, this paper proposes a novel detection method for lettuce growth traits based on transformer and convolutional neural network. In this method, we design a dual-transformer module and a residual module to effectively extract multi-scale representations and depth representations from appearance-varying lettuce images. In addition, a feature coupling bridge is proposed to fuse the multi-scale representations and depth representations. The experimental results show that our method outperforms the state-of-the-art methods.