Recognizing Soft Biometric on Pedestrian Using Convolutional Neural Network

Video surveillance is very important in automatic surveillance. It is usually used to monitor any criminal activity and help to find the perpetrator. In the search for individual among pedestrians when the face image can no longer be recognized, then what can be recognized is the soft-biometric or attribute appears on the pedestrian. The attributes can be in the form of clothes or accessories worn by pedestrians. However, recognizing attributes is a more challenging task as the number of variations of the attribute is increasing. Therefore, there is a need of a system that can effectively recognize the soft-biometric of pedestrians that are expected to help in video surveillance. In this study, we propose a pedestrian soft-biometric recognition system that was built using the Convolutional Neural Network (CNN) method so that it can search for pedestrians wearing various attributes. In recognizing pedestrian soft-biometric, the authors conducted a study using transfer learning from the Resnet50 architecture. As this case belongs to the multi-label classification case, we also observed several scenarios for determining the threshold to better classify the appearance of attributes. Our system able to recognize 35 different attributes with the best results of this research indicate the average accuracy of 90.82% with 80.84% of precision and 81.08% of recall using test data.