NoiseFlow: Learning Optical Flow from Low SNR Cryo-EM Movie

Cryo-EM movie in single particle analysis has extremely low SNR and requires aligning multiple frames to achieve signal enhancement. Currently, signal processing technique is adopted to estimate the motion vector between a pair of cryo-EM movie frames at patch-level, and the estimated motion vector is used as the reference for frame alignment, whose accuracy will determine the resolution of the reconstructed 3D structure of the particle. The patch-level motion may not well represent the beam-induced motion of particles since particles in a patch move towards different directions due to beam striking. However, the low SNR of cryo-EM movie makes it difficult to estimate the motion of particles at pixel-level. Meanwhile, existing optical flow estimation models only consider the ideal case where high-quality videos are provided, which fail to obtain optical flow from cryo-EM movie. In this paper, we diminish this limitation by proposing a model called NoiseFlow, a deep learning network for optical flow estimation from low SNR cryo-EM movie. NoiseFlow makes use of the multi-frame stacking module and the denoising module to extract noise-invariant features, and then computes the correlation volume from noise-invariant features to learn optical flow. For evaluation, we train our model on two synthetic cryo-EM movie datasets and infer on real cryo-EM data. The experimental results illustrate that NoiseFlow achieves state-of-the-art performance on both synthetic and real cryo-EM datasets.