Joint Demosaicing and Fusion of Multiresolution Coded Acquisitions: A Unified Image Formation and Reconstruction Method

Novel optical imaging devices allow for hybrid acquisition modalities such as compressed acquisitions with locally different spatial and spectral resolutions captured by a single focal plane array. In this work, we propose to model the imaging camera system for a multiresolution coded acquisition (MRCA) in a unified framework, which includes conventional devices such as those based on spectral/color filter arrays, compressed coded apertures, and multiresolution sensing. We also propose a model-based image reconstruction algorithm performing a joint demosaicing and fusion (JoDeFu) of any acquisition modeled in the MRCA framework. The JoDeFu reconstruction algorithm solves an inverse problem with a proximal splitting technique and is able to reconstruct an uncompressed image datacube at the highest available spatial and spectral resolution. An implementation of the code is available at
https://github.com/danaroth83/jodefu
.