WildFishNet: Open Set Wild Fish Recognition Deep Neural Network With Fusion Activation Pattern

Wild fish recognition is a fundamental problem of ocean ecology research and contributes to the understanding of biodiversity. Given the huge number of wild fish species and unrecognized category, the essence of the problem is an open set fine-grained recognition. Moreover, the unrestricted marine environment makes the problem even more challenging. Deep learning has been demonstrated as a powerful paradigm in image classification tasks. In this article, the wild fish recognition deep neural network (termed WildFishNet) is proposed. Specifically, an open set fine-grained recognition neural network with a fused activation pattern is constructed to implement wild fish recognition. First, three different reciprocal inverted residual structural modules are combined by neural structure search to obtain the best feature extraction performance for fine-grained recognition; next, a new fusion activation pattern of softmax and openmax functions is designed to improve the recognition ability of open set. Then, the experiments are implemented on the WildFish dataset that consists of 54 459 unconstrained images, which includes 685 known classes and 1 open set unrecognized category. Finally, the experimental results are analyzed comprehensively to demonstrate the effectiveness of the proposed method. The in-depth study also shows that artificial intelligence can empower marine ecosystem research.