VibCamera: mmWave and Camera Fusion for Multi-point Vibration Monitoring

As a diagnostic method of equipment operational status, vibration monitoring plays a significant role in industrial systems. It is necessary to monitor multiple equipment components simultaneously, due to their different vibration modes. Previous solutions either work in an invasive manner or face challenges in object localization and results correspondence. Therefore, we propose VibCamera, a vibration monitoring system that combines mmWave radar and computer vision technology. We propose an expand-shrink method to optimize object detection results of computer vision and combine camera localization results to extract mmWave signals. Additionally, we employ mmWave data recombination and respective fitting methods to calculate the vibration characteristics for each point accurately. The experiment shows that after fusing visual information, the target detection accuracy is improved to 94.8%, and the cluster point efficiency is improved by 23.3%. Furthermore, amplitude and frequency measurement errors are reduced to 29.1μm and 0.08Hz, respectively.