Visual voice lip-reading, so the computer can understand what the speakers want to express direction by looking at their lips. Lip reading is the easiest way to compare the early characters and templates from the frozen image is stored. It ignores the very nature and time changes. This method is very simple, but it's just simple elements can be classified, then it may not show significant speech recognition services. Behavior was characterized by more and more common. Because of the hidden Markov model is superior (HMM), which can be widely used in speech recognition. In recent years, is also used to lip reading identification. Classical HMM model, so that the two assumptions: hidden assumptions collected: in t+1 the state can only be in this country is not in the state before t; from the hidden visible state hypothesis: only by regulating the t hide the visible state, rather than the previous state. This hypothesis is not very useful in some applications (such as lip reading) is reasonable. Under certain conditions, in the t state not only limits the t-1, but also t-2. Therefore, this study modified the assumptions of the classical HMM to derive a new HMM model and algorithms, and applied to the lip-reading recognition is increasing discrimination.