Policy-based Management of an E-commerce Business Simulation: An Experimental Study

Autonomic Computing lays out a vision of information technology in which systems manage themselves based on policies. As a result, policies are the new currency of interaction between people and computers, creating a new paradigm for interaction with autonomic systems. In this paradigm, interaction shifts (1) from low-level to high-level monitoring and control and (2) from manually performing actions to delegating tasks to automation. In this paper, we report an experimental study comparing and contrasting manual interaction to policy-based interaction to manage a simulated e-commerce website. In this study, we investigated issues related to human expertise and policy representation. Our results suggest that effective policy-based interaction depends both on the level of detail of the policies and on the experience of the system supervisor. Our results show an overall benefit of policy-based interaction, as measured by business and technology-oriented metrics. Performance was significantly better with policy-based interaction following expertise gained through manual interaction. Performance with manual interaction was marginally worse after policy-based interaction, suggesting the classic out-of-the-loop problem. In addition, highly detailed policy representations marginally improved performance for technology-oriented metrics but did not yield significant differences for business-oriented metrics.