Self-programming networks: Architecture and algorithms

The rapid growth and wide adoption of cloud computing platforms have made them very complex to operate and manage. The operators of cloud platforms and the underlying data center networks have desired to make them much more programmable. This has culminated in the development of the Software-Defined Networking paradigm. But you cannot program what you do not understand: the volume, velocity and richness of network applications seem beyond the ability of direct human comprehension. What is needed is a sensing, inference and learning system which can observe the data emitted by a network during the course of its operation, reconstruct the network's evolution, infer key performance metrics, continually learn the best responses to rapidly-changing load and operating conditions, and help the network adapt to them in real-time. This white paper describes Self-Programming Networks (SPNs), an ongoing research effort at Stanford for making networks autonomous; that is, to enable networks to sense and monitor themselves, and program and control themselves. We present a NIC-centric architecture for SPNs, describe systems and algorithms for (i) fine-grained network measurement using packet and probe timestamps taken at the edge, and (ii) nanosecond-level clock synchronization in real-time and at scale. We also describe how these enhancements can enable new applications on top of an SPN. While our initial work has been on developing the sensing and monitoring capabilities of SPNs, we describe future work on learning and control.