Neural networks of formation and perception using motion via-points: an application to hand gestures

We have shown that a complex motion of the arm can be generated based on the optimization principle of smoothness in which two or more via-points are assumed to be a boundary condition. We have previously proposed a perception model for cursive-connected characters which has these via-points as features (Y. Wada and M. Kawato, 1995). Via-points are representative forms in the computational trajectory formation model of the human arm. The paper shows that a formation conversion from an intention to a set of via-points and a perception conversion from a set of via-points to an intention can be achieved using the same structural recurrent neural network based on bi-directional theory. As a concrete example, we demonstrate the formation and the perception of human gestures. In other words, the model is achieved by applying the motor theory of pattern perception, which is based on bi-directionals using neural networks. Finally, the paper shows that segmentation of a continuous motion is possible, a concept that can be useful to the field of engineering.