Hidden Biases in Automated Image-Based Plant Identification

Plant identification is critical to support important biodiversity conservation actions such as biodiversity inventories, monitoring of populations of endangered organisms, and assessing climate change impact, among many others. Because deep learning has demonstrated impressive results in the field of computer vision in general, research on automatic plant identification has been shifting its attention towards deep learning approaches. However, some authors have noticed that an important methodological issue may have been overlooked in the design of many experiments, which may explain why, on one hand, some studies based on hand-crafted feature extraction approaches report very high accuracy levels, but, on the other hand, newer deep learning approaches used in events such as the PlantCLEF challenge report relatively lower accuracy levels. Because PlantCLEF uses same specimen photos exclusively in either the training dataset or the testing dataset, we postulate that this may explain the lower accuracies achieved. Specifically, we explore the following two questions: does using different images of the same specimen for training and testing introduce a significant bias in deep learning experiments as well as in those that use handcrafted features in classical computer vision techniques? Does it affect the accuracy of species identifications even in the more restricted domain of leaf-based automated species identifications? We also address the issue of scalability of accuracy results for both, a particular feature extraction approach and a deep learning approach. All experiments are conducted on a dataset of 7,262 photos of leaves of 255 species of plants from Costa Rica.