Pigmentation-based Visual Learning for Salvelinus fontinalis Individual Re-identification

Brook trout (Salvelinus fontinalis) is a freshwater fish species of ecological, economic, and cultural importance in eastern North America. Estimating the abundance, movement, and survival of brook trout in the wild is an important task for environmental management, and current methods often involve physical tagging or collection of DNA samples for each of the fish as their unique identifier. However, this process is expensive and inefficient. Meanwhile, although deep learning methods have proven effective for individual recognition of humans, it remains challenging to apply this system to wildlife biology due to fewer available images, different biometric patterns, and relatively poor image quality. In this paper, we develop a framework to automate the process of individual recognition of brook trout. Distinguished from simply adopting traditional feature descriptors (e.g., SIFT and HOG) or using deep neural networks on the raw images, our framework utilizes multiple modalities consisting of the region of interest and gray-scaled pigmentation patterns. We use these multiple modality features in a Convolutional Neural network to generate feature vectors as fish descriptors. These descriptors are then used to distinguish individual brook trout by ranking their relative distance in latent space. Our experimental framework demonstrates better results than baseline methods such as SIFT and HOG while being more robust to distortions characteristic of large imagery datasets collected through crowdsourcing and citizen science.

I. Introduction

The ability to identify individuals is a key component of natural resources management and the cultivation of fish species. Individual-based data have been used extensively in biological, ecological, and fisheries management science [1], [2]. Computer vision methods for individual recognition have been developed for several species based on their pigmentation patterns (e.g., giant panda, sharks) [3], [4]. However, considering the innate difference of patterns regarding different species, migrating such system into other species is a non-trivial task. As a widely distributed freshwater species, brook trout typically exhibits dark green or brown coloration with distinctive marbling of lighter shades on their back as well as yellow and red spots surrounded by blue halos on their sides (see Figure 1). To date, brook trout have not been the focus of efforts to use visual learning for individual identification. Here we aim to improve identification accuracy by developing a novel visual learning approach.