Comparisons of Heat Map and IFL Technique to Evaluate the Performance of Commercially Available Cloud Providers

Cloud service providers (CSPs) offer different Service Level Agreements (SLAs) to the cloud users. Cloud Service Brokers (CSBs) provide multiple sets of alternatives to the cloud users according to users requirements. Generally, a CSB considers the service commitments of CSPs rather than the actual quality of CSPs services. To overcome this issue, the broker should verify the service performances while recommending cloud services to the cloud users, using all available data. In this paper, we compare our two approaches to do so: a min-max-min decomposition based on Intuitionistic Fuzzy Logic (IFL) and a Performance Heat Map technique, to evaluate the performance of commercially available cloud providers. While the IFL technique provides simple, total order of the evaluated CSPs, Performance Heat Map provides transparent and explanatory, yet consistent evaluation of service performance of commercially available CSPs. The identified drawbacks of the IFL technique are: 1) It does not return the accurate performance evaluation over multiple decision alternatives due to highly influenced by critical feedback of the evaluators, 2) Overall ranking of the CSPs is not as expected according to the performance measurement. As a result, we recommend to use performance Heat Map for this problem.