PDAVIS: Bio-inspired Polarization Event Camera

The stomatopod (mantis shrimp) visual system has recently provided a blueprint for the design of paradigm-shifting polarization and multispectral imaging sensors, enabling solutions to challenging medical and remote sensing problems. However, these bioinspired frame-based cameras lack the high dynamic range and asynchronous polarization vision capabilities of the stomatopod visual system, limiting temporal resolution to ~12ms and dynamic range to ~72dB. Here we present a novel stomatopod-inspired polarization camera which mimics the sustained and transient biological visual pathways to save power and sample data beyond the maximum Nyquist frame rate. This bio-inspired sensor simultaneously captures both synchronous intensity frames and asynchronous polarization brightness change information with submillisecond latencies over a millionfold range of illumination. Our PDAVIS camera is comprised of 346x260 pixels, organized in 2-by-2 macropixels, which filter the incoming light with four linear polarization filters offset by 45°. Polarization information is reconstructed using both low-cost and low-latency event-based algorithms and more accurate but slower deep neural networks. Our sensor is used to image high dynamic range polarization scenes that vary at high speeds and to observe the dynamical properties of single collagen fibers in a bovine tendon under rapid cyclical loads.Video: https://youtu.be/mFuCeTMWEqY