Polygonized Silhouettes and Polygon Coding Based Feature Representation for Human Action Recognition

The characteristics of human silhouette shape can be used for action recognition and classification. In this paper, a novel feature extraction method for the silhouette-based classification of human actions in videos is proposed. The proposed method is based on polygonization of silhouette images and coding. Since conventional silhouette generation methods do not satisfy the integrity of silhouettes, Yolact++ is modified as a silhouette generator. Our innovative approach Yolact++ masks are used as silhouettes to overcome this problem. For this purpose, a new image form called Poly Silhouette (PoS), a new Polygonization (PoG) algorithm and a new Polygon Coding (PoC) algorithm have been developed. The polygonization step is based on, but is not similar to curve and image polygonization. It is fast, adaptable, and accurate on the contour coordinates of the PoS images. PoCs were generated by projecting each edge vector generated from the corner coordinates of the PoS onto the angular areas and codes for the PoS were formed. These codes are grouped into k-mers similar to genetic algorithms and are used as features. The proposed innovative feature extraction method guarantees that feature vectors of equal length are generated from any action video. Thus, no additional action is required to overcome the dimensionality problem. By using different k-mer lengths, the classification accuracy of the method versus computation time was analyzed and depicted in figures. The method developed was tested on HMDB51 & UCF101 datasets: for SVM 20.98%, 1.63% for k-NN 4.96%, 6.83%, respectively, and significant improvements were achieved.