Cloud Detection from a Hyperspectral Infrared Atmospheric Sounder Using a Machine-Learning Model

Cloud detection is an essential step in the application of hyperspectral infrared (HIR) data. In this paper, a new cloud detection method using a LightGBM algorithm based on principal component (PC) space is proposed for HIR data from the High Spectral Infrared Atmospheric Sounder (HIRAS). Considering the difference of infrared radiation between ocean and land, day and night, this paper respectively builds the LightGBM day-land, day-sea, night-land and night-sea models. The truth cloud fraction of the HIRAS field of view (FOV) is determined by the collocated cloud masks of the Medium Resolution Spectral Imager-II (MERSI) onboard the FY-3D satellite. The HIRAS infrared channels were transformed into the leading K PCs as predictors through principal component analysis (PCA), with the advantages of reducing the correlation of infrared channels, accelerating model convergence and prediction. The cloudy FOVs in global datasets are randomly down-sampling to alleviate the impact of dataset imbalance. The validation experiments have shown that the LightGBM model has high accuracy for the completely cloudy and completely clear-sky FOVs. However, the partially cloudy FOVs are sometimes detected as the clear-sky. It may be because these partially cloudy FOVs have prominent clear-sky radiation properties. The cloud detection performance of the LightGBM model (accuracy=0.93, HSS=0.85) for land HIR data is better than that of sea HIR data (accuracy=0.89, HSS=0.65). It may be because there are more partially cloudy FOVs in the ocean. In addition, the imbalance of the ocean dataset significantly reduced its Heidke skill score (HSS=0.65). Compared with the HIRAS Lie cloud cover product, the accuracy of the LightGBM model for land (sea) HIR data is increased by about 0.22 (0.07). The time complexities of the algorithms have shown that the cloud detection speed of the LightGBM model is approximately 670 times that of the HIRAS/MERSI collocation cloud detection method. The higher cloud detection accuracy and faster efficiency are helpful to the operational application of the LightGBM cloud detection method.