Application of machine speech recognition for real-time estimation of human cognitive process at man-machine interface

A real-time cognitive process estimation system has been developed, by employing machine speech recognition to estimate operators' thinking processes. The developed system consists of a user-interface, speech recognition subsystem, real-time cognitive process estimator, and nuclear power plant (NPP) simulator. The major point of the real-time estimator is that it equipped with the causal relation diagram (CRD) which presents the physical relations between the NPP parameters on man-machine interface (MMI). By using CRD and two types of information, one from data of operational sequence to MMI and the other from verbal information taken by speech recognition subsystem, this estimator computes degree of operators' confidence in judging primary cause of plant anomaly for each plant parameters of CRD. From a experiment, it was confirmed that the developed system would be usable for real-time monitoring operators' thinking process in diagnosing the plant anomaly at MMI.