Optimal feature selection using information maximisation: case of biomedical data

The hybrid information maximisation (HIM) algorithm is derived. This algorithm is based on maximising the mutual information (MI) between the input and output of a network using the infomax principle, and between outputs of different network modules using the Imax algorithm. These two folds enable reducing the redundancy in output units in addition to selecting higher order features from input units. We analyse the proposed algorithm and generalise the learning procedure of the Imax algorithm. We show that the proposed HIM algorithm provides a better representation of input compared to the original two algorithms when used separately. An example showing the power of the HIM algorithm in the analysis of EEG data is discussed.