Multi-Objective Optimization for Multimodal Visualization

Using data visualization techniques can be of significant assistance in exploring multimedia databases. Data visualization is typically addressed as a unimodal learning task, where data are described with only one feature set, or modality. However, using multiple data modalities has been proved to increase the performance of learning methods. In this paper a novel approach for exploiting the multiple available modalities for visualization is proposed, motivated by the field of multi-objective optimization. Initially, each modality is considered separately. A graph of the dissimilarities among the data and the corresponding minimum spanning tree are formed. The suitability of a particular data placement is quantified using multiple cost functions, one for each modality. The utilized cost functions are defined in terms of graph aesthetic measures, computed for the unimodal minimum spanning trees. The cost functions are then used as the multiple objectives of a multi-objective optimization problem. Solving the problem results in a set of Pareto optimal placements, which represent different trade-offs among the various objectives. Experimental evaluation shows that the proposed method outperforms current multimodal visualization methods both in discovering more visualizations and in producing ones which are more aesthetically pleasing and easily perceivable.