Spatial Structured Prediction Models: Applications, Challenges, and Techniques

Spatial structure patterns are prevalent in many real-world data and applications. For example, in biochemistry, the geometric topology of a molecular surface indicates protein functions; in hydrology, irregular geographic terrains and topography on the Earth's surface control water flows and distributions; in civil engineering, wetland parcels in remote sensing imagery are often made up of contiguous patches. Spatial structured prediction aims to learn a prediction model whose input and output data contain a spatial structure. Modeling spatial structural information in prediction models is critical for interdisciplinary applications due to two reasons. First, explicit spatial structural information often indicates the underlying physical process, and thus enhances model interpretability. Second, spatial structural constraints also have positive side-effects of enhancing model robustness against noise and obstacles and regularizing model learning when training labels are limited. However, spatial structured prediction also poses several unique challenges, such as the existence of implicit spatial structure in continuous space, structural complexity in geometric topology, and high computational costs. Over the years, various techniques have been proposed for spatial structured prediction in different applications. This paper aims to provide an overview of the spatial structured prediction problem. We provide a taxonomy of techniques based on the underlying approaches. We also discuss several future research directions. The paper can potentially not only help interdisciplinary researchers find relevant techniques but also help machine learning researchers identify new research opportunities.