Hot-patching a web server: A case study of ASAP code repair

Software updates are the current standard to respond to software bugs. The software developer provides an update fix that is then applied by the administrator: the binary is modified and the service is restarted. Restarting a service inevitably leads to downtime and service unavailability; in the case of a multithreaded installation of Apache, restart takes several seconds and depending on the load of the web server, several hundred or even thousand client requests will be rejected with an error. Given the cost of restarts, system administrators attempt to minimize the frequency of service restarts or postpone a restart until the next maintenance window. However, to ensure the integrity of the system, code repair must happen as soon as possible (ASAP). We describe here the effectiveness of an on-the-fly update system that provides ASAP repair by integrating dynamic patches with a sandbox based on dynamic binary translation. To investigate the feasibility of ASAP code repair, we analyze the software updates released for Apache 2.2 between Dec 1st, 2005 and Feb 18, 2013. The study shows that such a system allows patching 45 of 49 bugs at runtime. Of the 4 unpatchable bugs, 1 bug is not applicable to dynamic update mechanisms, and 3 bugs require a restart. Furthermore, a performance evaluation of the prototype implementation shows that our approach adds low execution overhead (below 7% for different configurations that request a 287kB file).