Mutual Information Analysis of Social Media Images and Building Functions

Understanding urban dynamics requires detailed insights into urban land use. On the most fine-grained level this classification is done on single building instance levels. This level of detail can hardly be solved using remote sensing only, but requires complementary data. Social media images are a promising additional image data source since they are captured on a global scale in vast volumes.In this study we investigate the relation between objects showing up in geotagged social media images and functions of buildings proximate to the image location. We propose a rasterization approach to embed features from images and labels from a target domain to calculate mutual information both domains share. In our study area of Los Angeles, USA, we show that using object detection is a valuable way of extracting features from social media images to predict building functions. Furthermore, we present the most significant object types for five types of buildings.