Stacked Hourglass CNN for Handwritten Character Location

The Marsden Collection is an important source of great interest to historians of New Zealand, but most of the pages are untranscribed and therefore keyword search is impossible. Stacked hourglass networks are a new architecture that has been shown to work on tasks that are relevantly similar to keyword search - they are capable of precisely locating semantic parts of objects (in the case of keyword search, the objects are words and the semantic parts are characters). We implemented a stacked hourglass network that we trained on simplified data generated out of individual characters drawn from the Marsden Collection. It learned to not just identify, but locate the characters in the image. Despite training on simplified data, our network can also correctly identify letters in unseen documents with 59% accuracy, which rises to 80% when considering the top 5 hypotheses.