Cross-media translation between human motions and texts based on mental image directed semantic theory

The mental image directed semantic theory (MIDST) has proposed a formal language L/sub md/ employed for many-sorted predicate logic and whose semantics is given based on an omnisensual mental image model so called "Loci in Attribute Spaces". This language can be used for intermediate knowledge representation of multimedia contents and has been applied to an integrated multimedia understanding system IMAGES-M. This paper presents brief sketches of L/sub md/ and IMAGES-M and focuses on bi-directional cross-media translation between human motions and texts by IMAGES-M.