A study of feature extraction using supervised independent component analysis

Recently, independent component analysis (ICA) has been applied to not only problems of blind signal separation, but also feature extraction of images and sounds. In this paper, we study the effectiveness of Umeyama's (1999) supervised ICA (SICA) for feature extraction of handwritten characters. Two types of control vectors (supervisor) are proposed for SICA: 1) average patterns (Type-I); and 2) eigen-patterns (Type-II). To demonstrate the usefulness of SICA, recognition performance is evaluated for handwritten digits that are included in the MNIST database. From the results of recognition experiments, we certify that SICAs with both types of control vectors work effective for feature extraction. Actually, the within-class variance between-class variance ratio of SICA features with Type-I control vectors becomes slightly larger as compared with a conventional ICA.