Applying lessons from safety-critical systems to security-critical software

Researchers involved directly with the security of information-processing systems know that many such systems do not have the levels of integrity and sustainability that are much more prevalent for safety-critical systems. Safety-critical systems, many of which are industrial process control systems, are generally built and tested to much higher standards for handling system failure or aberrant behavior than is typical for even mission-critical information-processing systems. There is a long history of stringent standards for creating, running and sustaining safety-critical systems, particularly avionics, military systems, and the like. For example, international standard DO-178B, which was developed specifically for avionics but has been adopted by other fields, is acknowledged by the Federal Aviation Administration (FAA) and European Aviation Safety Agency (EASA) as a certification standard for avionics software. Also, NIST's Special Publication 800-82 “Guide to Industrial Control Systems (ICS) Security” provides guidance as to securing “Supervisory Control and Data Acquisition (SCADA) systems, Distributed Control Systems (DCS), and other control system configurations such as Programmable Logic Controllers (PLC).” In the U.S. government, the Federal Information Security Management Act of 2002 (FISMA) defines security objectives for information and information systems according to the traditional triad of confidentiality, integrity and availability. FISMA defines three levels of potential impact - low, moderate, high - on organizations or individuals were a security breach to occur. Many private-sector organizations do in fact categorize information processing systems according to business criticality. Certain key sectors, such as financial services, are obliged to classify systems as critical in order to comply with legal and regulatory requirements. For key critical operations, systems must incorporate sufficient resiliency so as to mitigate the risk of failure. Usually criticality falls into one of three categories, namely, based on regulatory requirements or guidelines, business continuity, or class of information held, such as nonpublic personal information, and processed within the system. In this paper we examine systems across the full spectrum of criticality, from non-critical, through security-critical and safety-critical systems, in terms of how they are engineered,. That is, we look at the processes by which they are designed, built, deployed, operated, modified and decommissioned. We also discuss how mission-criticality affects the requisite level of assurance. Finally, we describe how some of the demanding methods used to strengthen safety-critical systems, which are expected to exhibit high levels of assurance and integrity, might be adapted to the engineering of security-critical information systems.