Hierarchical Classification of Deep-Sea Videos

This research is based on previous research that proposes the use of AI algorithms to automatically annotate deep-sea videos [1]. Regardless of whether a deep-sea video is annotated automatically or by a human expert, some frames are difficult to annotate because one or more objects may be ambiguous to interpret. For example, the Funiculina and Umbellula Lindahli are two types of sea pens that are difficult to distinguish. Similarly, an object may be easily identified as a fish, but the type of fish may be difficult to determine. For these types of circumstances, we propose a hierarchical classification algorithm that will classify an object using a label that is higher in the taxonomy when the exact type of species cannot be determined with a high enough accuracy. For example, the algorithm may use the genus or family label when the algorithm cannot identify the species label. Not surprisingly, this leads to a higher recall and a lower precision because the hierarchical classification algorithm is able to identify objects that the previous algorithm could not, but the label that is predicted may be wrong or the algorithm may be penalized for not being precise enough (e.g., classifying a tilapia object as a fish rather than as a tilapia). Overall, we show that applying the new hierarchical classification algorithm over eight different experiments improves the F1 score by 7% and the count accuracy by 19%.