HFR-video-based Image Pattern Recognition Using Pixel-level Temporal Frequency Response Matching

In this paper, we propose a novel method for dynamics-based image-pattern recognition that can be used to inspect a specified vibrating pattern by computing temporal frequency responses in time-varying brightness signals at all the pixels of a high-frame-rate video sequence. The temporal frequency responses, which are the histograms of temporal frequency components in time-varying brightness signals, can be calculated using the short-term Fourier transform, and they are matched pixelwise with those of a reference vibrating pattern to determine whether a specific vibrating pattern is recognized. The effectiveness of our dynamics-based image pattern-recognition method is demonstrated using several vibration-source localization results for flying multicopters and insects, the rotating or flapping frequencies of which were in the range of hundreds of Hertz.