Hierarchical Deep Learning Models for Identification of Fish Species

Electronic monitoring is becoming an increasingly important tool for fisheries management and regulation since video footage can be used to identify endangered or vulnerable species and prevent illegal, unreported, and unregulated fishing. However, as the volume of video footage grows due to increased seafood consumption and fishing, manual review becomes near impossible. Therefore, computer vision algorithms can be leveraged to assist in the review process by automatically identifying fish species from images. A key challenge in this regard is distinguishing between nearly visually identical species, particularly given the small size of fish image datasets available that make achieving a high classification accuracy difficult. Therefore, we seek to address this problem by formulating a novel approach to fish species identification by constructing a hierarchy of deep learning models to improve overall species identification accuracy. We also study how model performance changes as the number of target categories increases. For evaluation, we utilize the Fishnet Open Images Database with over 86,000 images from electronic monitoring footage of fisheries in the western and central Pacific Ocean. We also focus on identification of the bigeye tuna (Thunnus obesus) species, which currently has a vulnerable IUCN conservation status and is often misidentified as yellowfin tuna due to visual similarities. Our result show that using a hierarchical approach with a smaller number of classification categories can provide a significantly higher identification accuracy.