Parallelisation of a statistics based dictionary generator

A statistics based dictionary generator was constructed based on a mathematical model proposed by the researchers from IBM T.J. Watson Research Centre. However, due to the large data site involved and the iterative nature of the algorithm, the execution of the generator is so time-consuming that its usefulness becomes a question. To improve the performance, parallelisation has been chosen as the avenue. In this paper, parallelisation of the generator will be described. The parallel algorithm will be analyzed. And the results of parallel execution time will also be discussed. In addition, some important conclusions will be drawn based on our experiments. The dictionary generator can be used in automatic machine translation. Thus, the benefit of the parallelisation can be considered, in general, in the context of natural language processing.