What You Can Learn by Staring at a Blank Wall

We present a passive non-line-of-sight method that infers the number of people or activity of a person from the observation of a blank wall in an unknown room. Our technique analyzes complex imperceptible changes in indirect illumination in a video of the wall to reveal a signal that is correlated with motion in the hidden part of a scene. We use this signal to classify between zero, one, or two moving people, or the activity of a person in the hidden scene. We train two convolutional neural networks using data collected from 20 different scenes, and achieve an accuracy of ≈ 94% for both tasks in unseen test environments and real-time online settings. Unlike other passive non-line-of-sight methods, the technique does not rely on known occluders or controllable light sources, and generalizes to unknown rooms with no recalibration. We analyze the generalization and robustness of our method with both real and synthetic data, and study the effect of the scene parameters on the signal quality.¹

1. Introduction

Consider a situation where one would like to recover information about a hidden scene in an unknown room without directly peeking inside. Staring at the blank wall of the room from outside may reveal nothing to the naked eye, yet the wall reflects extremely faint but meaningful patterns of light from the hidden scene. We show that by analyzing a video of the blank wall, we can infer information about a person’s activity or classify the number of people in a hidden region of the scene, with no prior calibration or knowledge of the environment. Real-time in-situ use of such a non-line-of-sight (NLOS) method can be critical for search and rescue operations, law enforcement, emergency response, fall detection for the elderly, and detection of hidden pedestrians for intelligent vehicles [6], [29].