Document restoration using 3D shape: a general deskewing algorithm for arbitrarily warped documents

We present a framework for restoring arbitrarily warped and deformed documents to their original planar shape. The impetus for this work is the need for tools and techniques to help digitally preserve and restore fragile manuscripts. Current digitization is performed under the assumption that the documents are flat, with subsequent image-processing and restoration algorithms either relying on this assumption or attempting to overcome it without shape information. Although most manuscripts were originally flat, many become deformed from damage and deterioration. Physical flattening is not possible without risking further, possibly irreversible, damage. Our framework addresses this restoration problem with two primary contributions. First, we present a working 3D digitization setup that acquires a 3D model with accurate shape-to-texture registration under multiple lighting conditions. Second, we show how the 3D model and a mass-spring particle system can be used together as a framework for digital flattening. We show that this restoration process can correct document deformations and can significantly improve subsequent document analysis.