Netsimgym: the Gymnasium for Reinforcement Learning in Networking Research

Gymnasium (Gym) is a de facto framework for building a target environment of reinforcement learning (RL)- based control and optimization. Therefore, there are several efforts to interwork existing network simulators (e.g., ns-3) with Gym for RL-based networking research. In an effort to provide a new alternative RL playground for networking research, we developed NetSimGym, a wrapper framework that interconnects NetSim, which provides rich and convenient features for network simulations, and Gym. NetSimGym provides a set of core functions to exchange observation, reward, action, and related information between Gym and NetSim through the Protocol Buffers (protobuf)-based messaging interface. This enables NetSim users to easily integrate various RL algorithms with NetSim without needing to implement a data exchange and synchronization mechanism. We open the source code of NetSimGym under the Massachusetts Institute of Technology (MIT) license and expect it to be useful to NetSim users who want to adopt RL in research and development.