A multi-scale approach to extract meaningful annotations from document images

We propose a multi-scale approach to extract annotations from document images in a meaningful way. It compares the rectified captured image with original image, which can be obtained through image retrieval technology, at various resolutions, in order to remove the noise caused by nonuniform distortions, such as camera lens distortion and document surface curvature, while preserving the true annotations. It also provides users lots of flexibility with a voting scheme and potentially different weights at different resolution levels. In addition, we analyzed the final annotation image and found the meaningful pieces out of it, such as an image patch of a handwritten paragraph. This broadens the application of annotation extraction, and makes it easier to share the notes. It can be applied to various imaging systems, such as flatbed scanner, mobile phone camera, or fixed camera etc. Experimental results are presented and compared with previous approaches.