Color display system for connected speech to be used for the hearing impaired

A color display system for the hearing impaired which converts connected speech signals into color pictures on a TV screen has been developed. This paper describes the principle, the function and the performance of the system, and the visual images of the patterns displayed using the system. The system consists of a real-time formant tracker, a pitch detector, a memory system, a color coder, etc. In this system, the lowest three formant frequencies are extracted from voiced signals of connected speech by means of the formant tracker, and are converted to three primary color signals. The three primary color signals as a time pattern are represented as a spatial color pattern on the TV screen using the memory system and the color coder. In unvoiced portions, colorless and dapple patterns can be seen. The reproduced Pattern using this system is not only beautiful, but also easy to understand intuitively. Especially, the visual experiments show that simultaneous contrast effect of colors caused by the spatial representation visually compensates for coarticulation effect on connected vowels.