Interpretability of Recurrent Neural Networks in Remote Sensing

In this work we propose the use of Long Short-Term Memory (LSTM) Recurrent Neural Networks for multivariate time series of satellite data for crop yield estimation. Recurrent nets allow exploiting the temporal dimension efficiently, but interpretability is hampered by the typically overparameterized models. The focus of the study is to understand LSTM models by looking at the hidden units distribution, the impact of increasing network complexity, and the relative importance of the input covariates. We extracted time series of three variables describing the soil-vegetation status in agroe-cosystems -soil moisture, VOD and EVI- from optical and microwave satellites, as well as available in situ surveys on crops across Continental U.S. to perform the experiments. Firstly, the models were validated in error terms. Secondly, the trained models were visualized and, thirdly, some useful statistics were extracted from the hidden unit activation heatmaps, accounting for redundancy and cluttering of activation responses. Results reveal how networks assign most of the relevance to soil moisture and focus on two phenological stages of crop growth.