A Cross-Modal Spatiotemporal Joint Predictive Network for Rainfall Nowcasting

Accurate rainfall nowcasting is crucial for protecting lives and property. Multisource meteorological data can reflect rainfall evolution from different perspectives. Making full use of multisource meteorological data is an effective method to improve the performance of rainfall nowcasting. For the rainfall nowcasting task, we propose a cross-modal spatiotemporal joint predictive network, referred to as CM-STJointNet. This is a nonlinear rainfall nowcasting model with radar and satellite data as inputs. A multitask learning (MTL) strategy is used to implement radar echo extrapolation and satellite imagery prediction, and this constraint realizes that the model forecasting follows the motion of clouds. We experiments conducted on the publicly available meteorological dataset [Storm EVent ImageRy (SEVIR)], CM-STJointNet uses radar and infrared bright temperature observations (IRBTs) as input data and realizes 2 h of rainfall nowcasting. Compared with four unimodal methods (ConvLSTM, PredRNN, Rainformer, and Earthformer), four multimodal methods (Rain-F, RN-Net, MetNet-2, and S2DIR) demonstrated superior prediction capabilities. The experimental results affirm the high efficiency of CM-STJointNet and provide more insights for future advances in rainfall nowcasting.