Parsing and recognition of city, state, and ZIP codes in handwritten addresses

In this paper we present a solution to the general vision problem of parsing and recognizing a set of correlated entities in the presence of imperfect information. Our solution mechanism involves the generation of multiple hypotheses in the initial stages of the system, and the use of very-large vocabulary recognition, together with a database of all the valid combination of the correlated entities, to choose among the hypotheses. We have applied our ideas and techniques to the specific task of identifying the city, state and zipcode fields in handwritten addresses. Given the image of a handwritten address, our algorithm produces a ranking of the 76,121-entry database of valid (city, state, zip) triples in the U.S. and in nearly 75% of the cases, the correct entry for the input address is assigned a rank of at most 10.