Learning to Super-Resolve Blurry Images With Events

S
uper-
R
esolution from a single motion
B
lurred image (SRB) is a severely ill-posed problem due to the joint degradation of motion blurs and low spatial resolution. In this article, we employ events to alleviate the burden of SRB and propose an
E
vent-enhanced SRB (E-SRB) algorithm, which can generate a sequence of sharp and clear images with
H
igh
R
esolution (HR) from a single blurry image with
L
ow
R
esolution (LR). To achieve this end, we formulate an event-enhanced degeneration model to consider the low spatial resolution, motion blurs, and event noises simultaneously. We then build an
e
vent-enhanced
S
parse
L
earning
Net
work (
eSL-Net++
) upon a dual sparse learning scheme where both events and intensity frames are modeled with sparse representations. Furthermore, we propose an event shuffle-and-merge scheme to extend the single-frame SRB to the sequence-frame SRB without any additional training process. Experimental results on synthetic and real-world datasets show that the proposed eSL-Net++ outperforms state-of-the-art methods by a large margin. Datasets, codes, and more results are available at
https://github.com/ShinyWang33/eSL-Net-Plusplus
.