Quality of Information Matters: Recommending Web Services for Performance and Utility

Widely used in modern software systems, web services have become a standard means of provisioning remote resources. As the number of available web services increases, multiple services that satisfy the same functional requirement can be used interchangeably. Given a set of interchangeable services, a software developer needs to find a web service that would provide the best performance and utility. However, web services are recommended based only on their system-related performance characteristics (so called QoS, whose properties include latency, reliability, availability, etc.), while their data-related performance characteristics (e.g., data freshness, correctness, coverage, etc.) are often overlooked. As a consequence, a recommended service may end up delivering information that is inaccurate or outdated, but with high performance. To address this problem, this paper introduces Quality of Information (QoI), a quality metric complementary to QoS, that measures to which degree a web service satisfies data-related non-functional requirements. To minimize the manual effort required to evaluate the results of invoking individual services, we introduce a comparative testing methodology based on the new concept of Objects of Interest (OI). By using OI, developers can normalize the relevant information obtained from dissimilar services, so it can be automatically compared. To concretely realize our ideas, we create QiSR, a system that recommends web services based on their QoI metrics. QiSR helps developers in determining how to match services’ input and output with application data requirements and how to measure the information quality of services. To evaluate the effectiveness of QiSR, we test it on representative manually selected web services. Our evaluation shows that services recommended based on both QoI and QoS exhibit better combined performance and utility than services recommend on QoS alone.