On the Optimization of Resource Utilization in Distributed Multimedia Applications

The application and research area of Multimedia Content Analysis (MMCA) considers all aspects of the automated extraction of new knowledge from large multimedia data streams and archives. In recent years, there has been a tremendous growth in the MMCA application domain (for real-time and off-line execution scenarios alike), and this growth is likely to continue in the near future. Multimedia applications operated in a real-time environment pose very strict requirements on the obtained processing times, while off-line applications have to perform within 'tolerable' time frames. To meet these requirements, large- scale multimedia applications typically are being executed on Grid systems consisting of large collections of compute clusters. For optimized use of resources, it is essential to determine the optimal number of compute nodes per cluster, properly dealing with the perceived computation versus communication ratio. This ratio generally depends on the characteristics of the application at hand, and on the software and hardware specifics of the computational environment. Motivated by these observations, in this paper we develop a simple and easy-to-implement method to determine the "optimal" number of parallel compute nodes. The method is based on the classical binary search method for non-linear optimization, and does not depend on the, usually unknown, specifics of the system. Extensive experimental validation on a real distributed system shows that our method is indeed highly effective.