We propose a new measure based on the informational entropy concept. It is designed to reflect the cognitive complexity, which users perceive when they work with alarm-processing systems. An experimental verification for the proposed measure is performed. The result shows that the proposed measure is superior to the other measures. We also propose a procedure for evaluating alarm-processing systems based on the analytic hierarchy process (AHP) with regard to integrating a series of deviations that must be considered. This procedure aims to perform effective simulator-based evaluations of alarm systems' design. An exemplary application is shown in this paper.