Research and implementation of part-of-speech tagging based on Hidden Markov Model

As separate problems of English, MA, POS, PDR can be considered independent with each other. In a practical research system, they are dependent, solution of the prior one forms the base for processing the next one. We Consider different features of these problems, after a comprehensive study, a divide-and-conqueror strategy is proposed and resolves them separately. First, a knowledge-based method is put forward for the solution of MA. The whole MA processing is completed by many subordinate functions dealing with different particular marks of English words. A strategy of combining the word length with statistic enumeration is developed to distinguish between the periods and abbreviations. Then, an approach combining Rule-based method with Hidden Markov Model (HMM) is put forward for POS tagging. Rule is introduced prior to the HMM approach not only to lower the time cost, but also to resolve the problems that cannot be solved with HMM. Solution to the POS tagging with this approach reports an accuracy of 99.83%.