Handwriting recognition using HMMs and a conservative level building algorithm

We describe a method for the recognition of cursively handwritten words using hidden Markov models (HMMs). The modelling methodology used has previously been successfully applied to the recognition of both degraded machine-printed text and hand-printed numerals. This paper addresses the additional problems that result from the greater degree of variability present in cursive script. The impact of this additional variability is reduced by applying various normalisation procedures to the scanned text. A novel conservative level building algorithm is proposed which maintains a list of several lexically plausible match sequences at each stage of the search, rather than decoding using only the most likely state sequence. This permits the use of a lexicon directly within the search procedure rather than as a post-processing step. Results are presented on a single-author database of scanned text. Depending on how many alternatives are maintained in the level building search, top-1 word recognition rates as high as 88% are achieved. These rise to 93% for the correct word being ranked in the top-3 candidates. In addition, the HMM methodology provides a likelihood measure that can be used to identify and thereby reject poorly recognised words. Using this mechanism nearly perfect recognition, (99.2%), can be achieved at the expense of rejecting or deferring a decision on 23% of the data.