Bootstrapping Automated Testing for RESTful Web Services

Modern RESTful services expose RESTful APIs to integrate with diversified applications. Most RESTful API parameters are weakly typed, which greatly increases the possible input value space. Weakly-typed parameters pose difficulties for automated testing tools to generate effective test cases to reveal web service defects related to parameter validation. We call this phenomenon the type collapse problem. To remedy this problem, we introduce FET (Format-encoded Type) techniques, including the FET, the FET lattice, and the FET inference to model fine-grained information for API parameters. Inferred FET can enhance parameter validation, such as generating a parameter validator for a certain RESTful server. Enhanced by FET techniques, automated testing tools can generate targeted test cases. We demonstrate Leif, a trace-driven fuzzing tool, as a proof-of-concept implementation of FET techniques. Experiment results on 27 commercial services show that FET inference precisely captures documented parameter definitions, which helps Leif discover 11 new bugs and reduce $72\% - 86\%$ fuzzing time compared to state-of-the-art fuzzers. Leveraged by the inter-parameter dependency inference, Leif saves $15\%$ fuzzing time.

1 Introduction

The REST (Representational State Transfer) architecture [1] nowadays has dominated the design of complex web services, such as public clouds (e.g., AWS and Azure), social networking (e.g., Facebook and Twitter), and code hosting (e.g., GitHub and GitLab). Typically, a RESTful web service exposes a set of RESTful APIs. A client requests an API providing parameter values, and the service responds with data represented in a certain common exchange format (e.g., JSON or XML). According to a recent survey of 40 real-world popular RESTful web services [2], modern services involve an average of 64 APIs and over 20 parameters per API. Testing such an input space of possible parameter value combinatorics is challenging, so automated testing is indispensable.