A Review of Physics Simulators for Robotic Applications

The use of simulators in robotics research is widespread, underpinning the majority of recent advances in the field. There are now more options available to researchers than ever before, however navigating through the plethora of choices in search of the right simulator is often non-trivial. Depending on the field of research and the scenario to be simulated there will often be a range of suitable physics simulators from which it is difficult to ascertain the most relevant one. We have compiled a broad review of physics simulators for use within the major fields of robotics research. More specifically, we navigate through key sub-domains and discuss the features, benefits, applications and use-cases of the different simulators categorised by the respective research communities. Our review provides an extensive index of the leading physics simulators applicable to robotics researchers and aims to assist them in choosing the best simulator for their use case.


I. INTRODUCTION
Physics simulators enable the vast majority of robotics research. It is commonplace to test and prove theoretical methods initially or solely in a simulator as robots themselves are oftentimes expensive, fragile and scarce. Physics simulators overcome these issues as they provide an environment that is cheap and allows users access to a variety of desired robots without the potential to degrade or break the physical platform. Simulation can run faster than real time (which is especially important for learning-based approaches), is parallelisable, and does not need to be physically tended for an environment to be reset.
The landscape of commercial and open source simulation options for researchers is in a state of perpetual flux, as new simulators are added to support the latest research trends, while others are deprecated. It is naturally difficult, therefore, to select an appropriate simulator for a given robotics project, and the literature to date reports an absence of any sort of comprehensive guide to help researchers find the right simulator for their specific purpose. Existing literature has a number of studies that compare the replication of The associate editor coordinating the review of this manuscript and approving it for publication was Guilin Yang . real-world physical accuracy and constraint satisfaction of physics engines, however, these studies often only compare a small subset of simulators and are usually focused on specialised tasks or sub-domains without consideration for robotics as a whole [1]. Many studies summarise information which can easily be gathered from simulator websites and forums without adding significant value from a research perspective [2]. The capabilities of these simulators in a variety of robotics-specific tasks is not explored or compared against one another.
Through this review we aim to fill the research gaps highlighted above. We focus on seven sub-domains which together capture the majority of robotics work involving simulators. In each sub-domain we discuss current challenges in the field, highlight the necessary capabilities of robotics simulators used in the field, and describe existing approaches to robot simulation in the literature of the sub-domain. Finally, we provide a summarising table in each section which details the capability of numerous robotics simulators in features we have identified to be relevant to robotics simulation in the subdomain.
We define a robotics simulator as an end-user software application that includes at least the following functionality: (i) physics engine for realistic modelling of physical FIGURE 1. Diversity of simulation scenes and environments throughout robotics (a) soft robotics in Simulation Open Framework Architecture [3], (b) medical robotics in Asynchronous Multi-Body Framework [4], (c) manipulation in PyBullet [5], (d) dexterous manipulation in MuJoCo [6], (e) legged locomotion in RaiSim [7] and (f) underwater vehicles in URSim [8].
phenomenon, (ii) collision detection and friction models, (iii) Graphical User Interface (GUI), (iv) import capability for scenes and meshes, (v) API especially for programming language used by the robotics community (c++/Python), and (vi) models for an array of joints, actuators and sensors readily available. This review covers only simulators which are actively being developed, used, or maintained. It is our understanding that anything otherwise would be of limited importance to the research community in the long-term. Additionally, this review focuses on robotics simulators and not physics engines. Physics engines are integral to every robotics simulator, but a physics engine alone does not constitute a robotics simulator unless it satisfies all criteria we use to define what a robotics simulator is.
This review acts as a guide to assist researchers in shortlisting the most relevant simulators for a given application to aid their decision making. We map out the landscape of current robotics simulators by categorising simulators that are actively maintained, and provide exemplar tasks and functionality that makes a simulator useful for a particular field or sub-domain. For each category we also provide a standardised summary table for concise communication of simulators and features. Figure 1 demonstrates the diversity of simulation environments required within robotics. The popularity of the robotics simulators discussed in sections II-VIII is portrayed visually in Figure 2.
We recommend researchers use this review as a guide for selecting a simulator for their particular research endeavour. We suggest first that a user has some idea of the particular robot platform(s) they wish to simulate (e.g. UR10, Husky, etc.), the method of actuation of the robotic platform(s), the sensors they intend to use and the physical operating environment (e.g. air, underwater, sand, city streets, etc.). The user then infers the relevant robotics research community from the denoted platform, so that, e.g., robot arms are found under manipulation. From here, we identify a subset of simulators that are likely to provide support for the required methods of actuation, sensors, and operating environment. If the planned endeavour is a crossover between fields, e.g., underwater manipulation, we suggest that all relevant sections be considered.

II. MOBILE GROUND ROBOTICS
Autonomous ground vehicle research -including legged, wheeled and tracked robots -is one of the largest studied domains in robotics. There are many fields which are incorporated in this sub-domain, including navigation, locomotion, cognition, control, perception, Simultaneous Localisation and Mapping (SLAM), and many others [9]. To motivate the use cases for simulators within mobile robotics we begin by investigating current challenges and competitions being run that represent some applications of mobile robotics research. The most prominent challenges are VOLUME 9, 2021 FIGURE 2. Citation count from 2016 to 2020 for reviewed simulators. Citations were gathered from Google Scholar using either one or more of a simulators' research paper, reference manual or other citation type and then filtered for robotics keyword. the Defense Advanced Research Projects Agency (DARPA) organised robotics challenges, starting in 2003 with an autonomous driving challenge, continuing in 2012 with a search and rescue challenge and now a subterranean competition running since 2018 [10]- [12]. The most current of these requires a team of robots to collaboratively navigate and map underground GPS denied environments to find human survivors, this event even hosts a virtual competition that is to be completed in the Gazebo simulation environment.
DJI has been running a robot competition featuring mobile robots since 2015. The task is to employ a team of robots in an arena to battle an opposing team using projectiles [13]. RoboCup is another prominent challenge that sees teams of mobile robots compete in games of football. Depending on the league, robots can either be legged or wheeled [14]. Each of these challenges require multiple robots to: coexist in the same environment and potentially interact with one another; navigate through and interact with terrain that may be geometrically uneven; and perceive the state of the robot in its environment with a suite of onboard sensors such as LiDAR, stereo-camera, GPS, or IMU.
Gazebo is a popular robotics simulator used in a wide range of mobile ground robot state-of-the-art (SOTA) research, for both legged [15], [16] and wheeled [17] robots. The Robot Operating System (ROS) interface provided by Gazebo contributes to the simulator's popularity, and simplifies the process of testing control software in simulation and transferring it onto the physical system. Gazebo also offers a model library for many commonly used sensors such as camera, GPS, and IMU. Gazebo provides capability to import environments from digital elevation models, SDF meshes, and OpenStreetMap. It is also possible to import robot models from the Universal Robot Description Format (URDF) files. Being a rigid body simulator, the simulator runs quickly and can simulate multiple robots in real-time. Although Gazebo itself doesn't provide motion planning functionality, its tight integration with ROS allows ROS path planners to be used.
CoppeliaSim (previously V-Rep) is another popular choice for simulating ground-based mobile robots. SOTA research uses CoppeliaSim for navigation planning of bipedal robots [18], differential-drive robots [19], and visual trajectory tracking of differential-drive robots [20]. Similar to Gazebo, CoppeliaSim is a rigid-body simulator which is able to simulate multiple robots in real-time. It also supplies a large model library of common mobile robot platforms and sensors including 2D/3D laser, accelerometer, stereocamera, camera, event camera, GPS, and gyro. CoppeliaSim offers path planning functionality through the commonly used OMPL library and supports height-fields for terrain specification.
Webots is another popular alternative, and is used in SOTA research to investigate, e.g., the performance of non-holonomic robot trajectory tracking [21], and to evolve bipedal robot gaits [22]. Webots has a large model database of mobile robots, environments and sensors. Sensors include accelerometer, camera, compass, GPS, inertial measurement unit, LIDAR, and radar. Webots also supports maps to be imported using the openDrive file format. Raisim [7] is a rigid-body physics simulator developed by ETH Zurich, used in research into learning dynamic policies for legged platforms [23]. Raisim allows uneven terrain to be imported using height-map images. Raisim is not as full featured as other reviewed mobile robot simulators but instead is developed to provide high fidelity contact dynamics models which are needed for transferring controllers from simulation to reality.
SOTA work into transferring locomotion policies for quadrupeds from simulation to real-world platforms was conducted using the PyBullet simulator [24] along with research into visual navigation on the turtlebot platform [25]. Pybullet is applicable to most mobile robotics applications as it supports rigid-body simulation with the possibility of faster than real-time simulation of multiple robots. It also supports the import of height-fields for terrain geometry specification. The sensors supported by PyBullet are quite minimal compared to other more specific mobile robot simulators with support for cameras and several less relevant sensors.
The recent large scale investment into self-driving cars has yielded a comparably large amount of research into the field of autonomous cars with the CARLA simulator a by-product. Several SOTA papers use CARLA for learning driving policies [26] and transferring policies trained in CARLA to the real world [27]. CARLA is a simulator targeted at self-driving car research and therefore has features aligned with this goal. It is scaleable to allow for large scenes through a distributed architecture and includes ROS integration. CARLA provides support to import environments through the openDRIVE file format. A large number of sensors are available for use including GPS, IMU, LIDAR, Radar and Camera. CARLA uses PhysX to compute the vehicle physics although the settings available to the user are restricted. Table 2 provides a comparison between the capabilities of the discussed robotics simulators in areas that are identified to be critical to the domain of mobile ground robots. Critical features identified include the ability to model sensors commonly used in the field, as well as common forms of locomotion, ability to import various environments, and inbuilt support for ROS. If the user wishes to realistically model complex environments, including sand, water, and gravel, Project Chrono provides this capability through an inbuilt Discrete Element Method (DEM) model [28].

III. MANIPULATION
The manipulation community within robotics is large and diverse, exploring everything from physical design of arms and grippers to algorithms for motion planning and control. Recent competitions and benchmarks provide insight into the current research avenues within the field and as an extension, the use cases for simulators within the manipulation research community.
One of the most prolific annual competitions is Robocup, which has a league called Robocup@home [29] for assistive robotics that compete in a domestic setting. Tasks that must be completed as part of the challenge rely upon manipulating rigid and deformable objects. The Tidy Up My Room Challenge from the International Conference on Robotics and Automation (ICRA) 2018 [30] and Fetch it! The Mobile Manipulation Challenge [31] are two additional challenges that are similar in that they all require multiple mobile manipulation platforms to complete multi-step tasks which include manipulation, often of rigid objects.
Other relevant challenges include the Amazon picking challenge, held between 2015 and 2017, which required robots to complete pick-and-place style tasks of seen and unseen objects, some of which were deformable and others which were transparent [32]. Most recently a challenge called the Real Robot Challenge looks at the dexterous manipulation of objects in both simulated and real-world environments. There are a number of benchmarks proposed within the manipulation community for benchmarking physical as well as algorithmic advances, relevant benchmarks to our review are task-based and include pick-and-place [33], assembly [34], peg-in-hole [35] and deformable objects [36].
From here we see that the field is at a state of simulating multi-task or multi-step scenarios, requiring the fine movement and contact modelling of rigid bodies. This, combined with the self-expletive need for stable physics that robustly handles contacts, reduces the field of viable simulators. To be VOLUME 9, 2021  useful for manipulation, research simulators must have actuator models for position control, velocity control, and torque control, as these are the most commonly used modes of control for physical arms. The simulator needs to support torque sensors as well as visual sensors, namely RGB and RGB-D. Finally, built-in features that are relevant specifically for manipulators are Inverse and Forward Kinematics solvers, and path planning. Less common -but becoming more relevant as computation becomes cheaper -is the need for modelling deformable objects as the underlying assumption that the robotics world is entirely rigid does not hold true.
Recent SOTA research has utilised a range of simulators for producing results. The capabilities of these simulators in areas relevant to the domain of robotic manipulation is summarised in Table 3. MuJoCo [37] is a simulator commonly employed within research, with notable contact stability being a reason for its popularity [38]. It was applied in an in-hand manipulation context for solving a Rubiks cube with a 24DOF robotic hand actuated with tendons [39]. SOTA research uses MuJoCO to train policies for robotic manipulators in simulation, whether just for proof of concept [40] or for later transferring onto real-world systems [41]- [43]. From the list of features a good manipulation simulation should have, MuJoCo supports most but lacks support for inverse kinematics and path planning.
Pybullet is used in studies with object collisions [44], pick and grasp dynamics [5] and for deformable object manipulation (i.e. cloth) [45]. Pybullet has a strong robotics target with functionality specifically implemented for those researching robotics. Functionality that may assist manipulation researchers includes: forward/inverse kinematics; Reinforcement Learning (RL) environments; Virtual Reality (VR) integration (for task demonstration); and deformable object and cloth simulation (Finite Element Method).
Gazebo [46] is used for robotic manipulation research [47], [48]. Although neither of these investigations rely on Gazebo to conduct dexterous manipulation, one of the studies did explicitly augment the simulation with external algorithms to deal with non-rigid bodies. Gazebo provides a simulation environment with the necessary actuators and sensors for robotic manipulation. It also provides support for ROS, which provides packages for forward and inverse kinematics, as well as path and motion planning.
CoppeliaSim is a robotics simulator with a range of user-centric features including sensor and actuator models, as well as motion planning, and forward and inverse kinematics support. PyRep was recently introduced as a python toolkit for robot learning built on CoppeliaSim, and has been shown to be capable of being used for manipulation, explicitly picking and placing cubes using a Kinova robot arm [49].
SimGrasp is used in a study for the design and simulation of tendon-driven underwater robotic hands [50]. It is a simulation package built on top of the Klampt't simulator, which markets itself as having better collision handling than the previously-mentioned manipulation simulators [51]. Klamp't lends itself to fast deployment of dexterous manipulation in robotics through the simulation of actuators and sensors with kinematics, dynamics and path planning.
Several works simulate deformable object manipulation using physics simulators that this review does not classify as robotics simulators. One such work uses Blender for cloth simulations [52], however this investigation only simulates cloth and the displacement of picked cloth coordinates. Another study uses Nvidia Flex to simulate fluids and deformable objects but abstracts away robot interactions [53]. Additionally, another work with Nvidia Flex simulates a robot completing a swinging peg-in-hole task with a 7-DoF Yumi robot. Flex is available through the ISAAC simulator [54].

IV. MEDICAL ROBOTICS
Medical robotics is a sub-domain of robotics research applying automation and robotics to e.g., surgery, therapy, rehabilitation and hospital automation [55]. Unlike competitions in other sub-fields of robotics, medical robotics does not have competitions focused on solving specific tasks, instead  [57]. The wide range of applications for medical robotics limits the number of medical robotics benchmarks to base the requirements of a medical robotics simulator on. We therefore base the requirements of medical robotics simulators on the needs of recent research being conducted in this field.
Due to the nature of therapeutic and rehabilitation interventions, studies are typically conducted in reality only. Instead, we focus on simulation for robotic surgery, including training and practice for real surgeries with a robotic surgical system, and training autonomous agents to attempt surgery in a safe environment.
The most prolific robotic surgical system is the da Vinci by Intuitive Inc., which consists of multiple arms with both rotational joints and tendon driven joints [58]. There are research robotic platforms that have been developed with similar hardware to the da Vinci, including the Raven II [59]. To realistically simulate these platforms, a simulator must be capable of simulating rotational as well as prismatic joints and should ideally simulate the tendons in both these robots for accuracy.
Surgeries performed with a robotic surgical system are often teleoperated by the surgeon. State of the art teleoperation controls have haptic feedback which gives the user force/torque feedback directly from the robot [60]. Robotics simulators for research into medical robotics benefit from having the tools to provide simulated force/torque feedback for haptic devices. Simulators which take in user input for teleoperation must run in real-time, otherwise user input would result in a delayed action in the environment. Robotic simulators for medical robotics also require deformable object simulation. As humans consist primarily of non-rigid tissue, realistic simulations require the ability to simulate deformable objects.
There are several simulators that offer some or all of the features required of a simulator in medical robotics. SOTA research in the domain of medical robotics uses robotics simulators such as the Simulation Open Framework Architecture (SOFA), CHAI3D [61], Asynchronous Multi-Body Framework (AMBF), CoppeliaSim, and Uni-tyFlexML. The capabilities of the identified simulators in areas relevant to the domain of surgical robotics is summarised in Table 4.
SOFA is a medical simulator used to study force and thermal feedback methods for minimally invasive surgery [62]. SOFA offers a plugin for teleoperation and haptic control, supports real-time deformable object simulation and through a robotics plugin supports tendon driven joints. These features paired with the strong focus on medical applications makes SOFA a good candidate for medical robotics research requiring simulation.
CHAI3D is another simulation framework used in medical robotics research to learn a neural network for autonomous tissue manipulation in simulation [63]. CHAI3D supports multi-DOF teleoperation controllers and haptic feedback systems, real-time simulation, and deformable object simulation.
AMBF is a simulator for medical applications developed at the Automation and Interventional Medicine Robotics Research Laboratory at Worcester Polytechnic Institute, built around CHAI3D and the bullet physics engine [4], [64]. AMBF enables fast-running simulation which uses CHAI3D for teleoperation support and haptic feedback, and bullet for soft body simulation and prismatic joints.
UnityFlexML is a simulator developed for use in machine learning applications [65]. It is based on Unity which has a large network of supported plugins including Nvidia flex for deformable object simulation and teleoperation control with haptics.
The remaining simulators used for research in medical robotics offer a limited subset of the necessary features identified. For example CoppeliaSim is used as a deep reinforcement learning environment to train both pick and reach policies on a surgical robot where deformable objects are not required [66]. CoppeliaSim does however support teleoperation and haptic feedback through a CHAI3D plugin, and supports prismatic joints for realistic simulation of surgical robot joints.
Another simulation environment that was used for medical research was a custom implementation using Open Dynamics Engine (ODE). ODE was used in the presentation of a framework for training users for robotics surgery [67]. Although ODE is a physics-engine and not a robotics simulator, the users added additional support for teleoperation and haptic feedback for use with the Raven II surgical robot.

V. MARINE ROBOTICS
Simulators for marine robotics can be divided into two categories, namely those which are designed for underwater vehicles (AUVs, ROVs, etc) and those which are suitable VOLUME 9, 2021 for surface vehicles (USVs, Ships, boats, etc). Competitions such as the Singapore AUV Challenge (SAUVC) [68], RoboSub [69], RobotX [70] and MATE ROV [71] focus on practical, challenging missions. The interested reader is directed towards [72] which provides an excellent review of such competitions in the field of marine robotics and the summary in Table 5 that summarises the capabilities of the reviewed simulators. The design of marine robots is greatly aided by high fidelity simulation in areas such as navigation, waypoint following, seabed mapping, and sensor-based control [73]. A good simulator should support different types of controllable vehicles, manipulators, sensors, and complex environments with accurate representation of hydrodynamic/hydrostatic forces. UWSim [74] and UUV Simulator [75] are the two most widely used options for underwater simulation.
Unmanned Underwater Vehicle (UUV) Simulator [75] is an extension for Gazebo which supports multiple underwater vehicles (ROVs and AUVs) and robotic manipulators with high fidelity representation of hydrostatic and hydrodynamic forces. A number of commonly used sensors are included eg. underwater camera, pressure sensor, IMU, Magnetometer, Doppler Velocity Log (DVL), etc. Models for fins and thrusters are also included for actuation. UUV allows researchers to create complex underwater environments with models already included for seabeds, lakes, ship-wrecks, etc. UUV simulator has been used for applications such as mapping [76] and path following [77].
Other notable gazebo extensions/packages include Rock-Gazebo [78] and freefloating − gazebo [73]. ROCK-Gazebo is the integration between Gazebo and the Robot Construction Kit (ROCK) framework to allow for real-time simulation. This involved extending the ROCK visualisation tool using OpenSceneGraph(OSG) for rendering underwater environments while Gazebo was used for physics simulation [75]. Rock-Gazebo has a number of limitations including not supporting multi-robot simulations [75]. freefloating − gazebo combines the dynamic simulation capabilities of Gazebo with the realistic underwater rendering of UWSim [74]. This allows it to model hydrodynamic forces. freefloating−gazebo lacks in that, due to stability concerns, it does not include the computation of added-mass forces [75]. Rock-Gazebo and freefloating − gazebo both have limited sensor support.
UWSim [74] is another open source option, which was developed at the Interactive and Robotic Systems Lab at the Jaume-I University. It utilises Bullet and OpenScene-Graph(OSG) for contact physics and supports a wide range of simulated sensors such as pressure sensor, force sensor, GPS, range camera/sensor, IMU, DVL, and much more. Multiple vehicles can be loaded and managed simultaneously while complex environments can be modelled using OSG and other 3rd party tools such as Blender. The underwater rendering is highly realistic and it already includes a default model for girona500 and ARM5E manipulator. UWSim has been used in a number of applications including controller design [79], path planning [80], 3D mapping [81], etc. It does, however, lag behind in terms of accuracy of simulation for dynamics and hydrodynamics of vehicles [82]. It also does not support simulation of manipulator dynamics (only kinematics) [82].
The most credible alternative to UWSim and UUV Simulator is the newly proposed StoneFish Library [82], a wrapper for bullet which supports standard sensors such as camera, pressure, DVL, multi-beam, etc. All hydrodynamic computations are based on the actual geometry of the body, which allows for better approximation of hydrodynamic forces. The simulation effects include added mass, buoyancy and drag. Underwater thrusters and vehicle manipulator systems are available for modelling more complex setups. The simulator supports advanced rendering of underwater scenes, including scattering and light absorption. The advanced rendering is computationally expensive though and requires a recent GPU.
Unity ROS Simulator (URSim) [8] uses ROS and the Unity 3D game engine. It has sensor models for camera, IMU and pressure together with noise models for sensor input. The simulator is capable of modelling environments used in competitions such as SAUVC and RoboSub. Unity allows for the modelling of hydrodyanmic forces such as buoyancy and drag. ROS provides functionality needed for control, communication, vision and sensing, with target applications in sensing, mapping, path planning, localization, obstacle avoidance and target acquisition. URSim is being actively developed, with new sensors (DVL, side scan SONAR, etc) and a robotic manipulator planned.
Surface vehicle simulation is relatively rare [83], primarily due to the complexity associated with modelling environmental factors such as waves, wind and water currents [83], [84]. Unmanned Surface Vehicle simulator (USVSim) [83] is a dedicated simulator for this application, which is an extension of Gazebo [46]. The freefloating plugin [73] supports USV simulations by improving the hydrodynamics and buoyancy effects. The lift-drag plugin was used for calculating foil dynamics. UWSim [74] offers accurate modelling of wave and water visual effects. Re-using and improving elements from the above tools allowed the authors to come up with a robust simulator, which has been used for path planning [85].

VI. AERIAL ROBOTICS
In this section we focus on unmanned aerial vehicles (UAVs) which are the most popular field of research within aerial robotics.
Competitions such as the UAV challenge [86] and the International Aerial Robotics Competition (IARC) [87] are open competitions in the field. In the UAV challenge, the goal is to demonstrate the utility of UAVs on real-world missions such as medical rescue or delivery of essential items. IARC is the longest running collegiate competition for aerial robots, focusing on missions relating to human-robot interactions, robot-robot interactions and interactions of robots with complex environments. The NASA SAND (Safeguard with Autonomous Navigation Demonstration) [88] competition aims to address safety-critical risks associated with flying UAVs in US airspace.
Modern UAV simulators allow researchers to replicate complex real-world environments by modelling turbulence, air density, wind shear, clouds, precipitation and other fluid mechanics constraints [89]. They also support various sensors -eg. Lidars, GPS, camera, etc. Digital elevation models or height maps are also used to simulate the terrain underneath the UAV. Aerial robotics simulators include Gazebo, AirSim, Flightmare, jMAVSim, and Webots all of which are included in Table 6 along with a comparison of important features required for aerial robotic research.
Gazebo [46] is a popular simulator for both indoor and outdoor applications [90]. Gazebo relies on the LiftDrag Plugin to simulate aerodynamic properties, and supports many common sensors such as stereo-cameras and LIDAR. The Hector plugin [91] adds UAV-specific sensors such as barometers, GPS receivers and sonar rangers. Gazebo supports a comprehensive list of UAV models [46], and open-source hardware controllers such as Ardupilot and PX4 which can be integrated for hardware-in-the-loop simulations. Gazebo, however, features limited rendering capability compared to Unity and Unreal Engine [92]. Gazebo has found application in, e.g., autonomous navigation [93], landing on moving platforms [94], multi-UAV simulation [95], and visual servoing [96].
Microsoft's AirSim [92] is based on Unreal engine, and supports IMU, magnetometer, GPS, barometer, and camera sensors. AirSim provides a built-in controller called sim-ple_flight, and also supports open-source controllers such as PX4. AirSim is resource-intensive and hence requires large computing power to run when compared with other simulators [89]. It has been used in drone racing [97], wildlife conservation [98], and depth perception from visual images [99].
Flightmare [100] combines a flexible physics engine with the Unity rendering engine into a powerful simulator. Flightmare simulates high-fidelity environments including warehouses and forests. Sensor models are available for IMU and RGB cameras with ground-truth depth and semantic segmentation. The simulator is well suited for applications in deep/reinforcement learning.
jMAVSim [101] is another widely used simulator, mainly due to its tight coupling with the open-source PX4 controller owing to the initial goal of testing PX4 firmware and devices [92]. jMavSim supports basic sensing and rendering [92].
Webots [102] is an open source simulator with an extensive set of supported sensors, including cameras, LIDARs, GPS, etc. Users can add custom physics to simulate things such as wind and integrate data from OpenStreetMap to create more realistic environments. Integration with the Adrupilot flight controller is supported. Webots has been used in multi-agent simulations [103], mitigation of bird strikes [104] and landing applications [105].

VII. SOFT ROBOTICS
Soft Robotics is generally a harder simulation problem than other robotics domains which often assume that the robot and world it operates in are mechanically rigid. Soft robotics requires simulating deformable objects and support for unconventional modes of actuation, including tendon or cable, pneumatic, and heat transference. Simulators must also support contact dynamics between the soft robot and soft/solid materials or fluids. As an emerging research field, competitions are relatively recent in their inception. The 2016 Robosoft Grand Challenge consisted of three team challenges: manipulation, terrestrial locomotion, and underwater locomotion [106]. The Annual Soft Robotics Competition ran annually between 2015 and 2018. It consisted of several categories, with a panel of judges awarding prizes based on contribution and design [107]. Table 7 provides a comparison between the capabilities of different robotics simulators used to simulate soft robots in areas relevant to the domain.
Soft robotics typically employs Multiphysics packages such as COMSOL, ANSYS, and Abaqus, which solve through Finite Element Method (FEM), as well as simulating aspects including heat transfer, electric conduction, magnetism, and fluid flow. They typically lack sensing, however they are fully capable of modelling soft actuation mechanisms and are used for more fundamental studies (e.g., not including environmental modelling). They have a range of modules available to support different physics, however tend to be expensive to purchase.
Abaqus, for example, is used to model laminar jamming structures [108], 3D locomotion of soft robots with electrostatic actuators [109], deflection of a soft robot produced by thermal conduction [110] and a soft robotic grippers [111]. Abaqus models non-linear behaviour well, and supports a large range of material properties with a material model library. The Abaqus FEM simulation is considered to be the industry standard. ANSYS is another modelling package used in soft robotics research, which simulates electrical, thermal and structural properties in simulation [112]. ANSYS Fluent is a well-developed package for fluid simulation, which is popular for e.g., underwater soft robotic modelling [113]. COMSOL has more user-definable material properties than Abaqus, lending itself to research methods which use materials with unique properties. It is also considered more user-friendly than ANSYS. In SOTA research, Comsol is used to simulate a flexible inchworm with actuation through magnetic fields [114], a caterpillar-like robot actuated by light [115].
SOFA is a popular open-source simulator that has been used to simulate cable driven soft robots [3], and for FEM simulation of four-legged soft robots [116]. SOFA has several useful features for robotics, including a ROS bridge and a Soft Robot plugin for modelling and actuation. Actuators from the soft robot plugin include tendon-driven and pneumatic actuators. SOFA is supported by an active open source community that regularly adds new modules and features alongside its internal development.
Evosoro is a soft robot simulator based on the Voxelyze physics engine. It uses Spring-Mass modelling for simulation of voxel-based soft robots and includes variable-volume actuation but no sensing. Evosoro is a comparatively fast soft robot simulator, which has been coupled with evolutionary algorithms to design robot morphologies [117], and as a design tool for real deployments of soft robots [118]. The simulator was found to have a significant gap when solutions were transferred to reality however, owing to the (fast, relatively inaccurate) Spring-Mass modelling.

VIII. LEARNING FOR ROBOTICS
Learning for robotics has been an important topic of research over the last decade. Due to the sample inefficiency of current Reinforcement Learning (RL) algorithms as well as the need to explore the state-action space that may lead to robot failure or damage during training, the majority of works on deep RL are first learned in simulation before being deployed on hardware. Due to the relative recency of robotic deep learning, there are relatively few competitions in the domain of learning for robotics. The Real Robot Challenge [119] is a notable exception in which participants learn dexterous manipulation of objects with a parallel manipulator, and their learned policies are compared both in simulation -in phase 1 of the competition -and hardware in a later phase. Tasks that must be completed as part of this challenge include pushing an object to a target location, lifting it to a specified height, and moving it to a target position and orientation.
Though there are few challenges targeted towards learning for robotics, learning methods have been applied in a number of other robotics challenges. A learned locomotion controller for a quadrupedal robot was recently deployed in the DARPA Subterranean challenge [123], for example.
Learning for robotics differs from the other sections covered in this work because it is concerned with implementation on a robot rather than the type of robot or the environment that a robot is deployed in. Due to the emerging popularity of robotic learning, a guide for simulator selection is included in this work. Learning methods can be applied in a wide range of robotics fields, and so the features pertinent to those fields should be considered alongside the features required for learning itself. For instance, applying learning methods to soft robotics requires support for soft contacts and materials as well as the ability to perform rapid iterative policy learning.
One common application of deep learning is to learn manipulation and grasping policies. For these tasks, the fidelity of rigid or soft body contact dynamics is important, as well as having sensors to support policies for such tasks. Another application is in path planning or locomotion over rough terrain with a mobile robot, and so researchers may require complex terrains to be modelled. Many simulators such as Gazebo, Raisim, MuJoco, and PyBullet allow non-flat rigid terrains to be imported from heightmap images or mesh files, but do not model soft or granular materials at a large enough scale to mimic soils, gravels, or fluid terrains. Project Chrono is one alternative with in-built support for deformable terrain, granular terrain, and fluid simulations, as well as being parallelisable. Locomotion and path-planning policies for aerial robots such as quad-rotors has been achieved in Raisim [136], Flightmare [100], and Gazebo [137], [138].
In each of the described applications, sensor support is an important consideration for researchers to consider when selecting a suitable simulator. Force-torque sensors and vision sensors are common requirements and are supported in simulators such as Gazebo, PyBullet and CoppeliaSim. Gazebo provides support for noise models which can be applied to sensor outputs. Because simulation is an abstraction of real-world conditions, policies learned in simulation typically degrade when transferred onto hardware. Overcoming this reality gap is one of the most important considerations for researchers in the learning community to address when selecting a simulator. It is also important that simulation environments vary between episodes, commonly using a technique called domain randomisation [42], to diversify the training data and allow the robot to properly explore the shared state-action space. Many simulators have in-built support, e.g., the ability to reset a simulation environment without shutting down the entire simulator-and to vary initial positions and orientations of robots, cameras, and objects within the simulation-is common to many robotic simulators. The ability to randomise the textures of rendered objects in simulation and characteristics of the camera used to render them is built into MuJoCo and demonstrated in [42]. This functionality is not innately supported in Gazebo, though an external plugin has been created to do so [139]. Randomising object mass and inertia, as well as friction coefficients, is another method of performing domain randomisation common to many of the simulators considered. Applying small random forces to robots also aids in overcoming the reality gap but is not possible in all simulators. Supporting multiple physics engines is another domain randomisation technique that prevents learned policies over-fitting to the simulation environment [140]. Gazebo, V-Rep, and Pybullet support multiple back-end physics engines, whereas some other simulators such as Raisim and MuJoCo do not. The quality of rendering is also an important factor for learned policies that rely on visual data.
Due to the large amount of data needed to train neural network parameters and properly explore the state-action space it is also important that the chosen simulator facilitates collecting this data in a timely manner. There are a number of simulator features that can facilitate deep learning in a timely fashion, including: supporting parallel simulation -either through simulating multiple robots in one environment or running multiple simulations in parallel with multi-threading or multiprocessing -the ability to run in headless mode, and rapid dynamics solvers that allow simulations to run faster than real-time. Due to the GPU-based physics engine of Nvidia Flex -which is available to use as a physics engine in the NVidia Isaac robotic simulator -a walking policy for humanoid robots could be learned in 16 minutes on a single CPU and GPU. Flex also supports distributed GPU simulations which can further reduce training times by up to eight times on some tasks [130]. Flightmare is able to maintain 200,000 steps per second while simulating 150 quadrotors in parallel, allowing it to train locomotion policies for the quadrotors much faster than in real-time [100]. Running similar simulations of a humanoid robot in Gazebo, V-Rep, and Webots [141] showed that the Gazebo was more CPU-intensive than the other two simulators and Webots was the least intensive of the three. Computational load is relevant to researchers considering simulators for learning because it is important to perform either as many simulations in parallel as possible, or to run a simulation as quickly as possible so that training time can be reduced.
Evolutionary robotics is a subset of learning that is distinct from the majority of deep RL methods, though many of the challenges with simulating environments for deep RL are shared in the field of evolutionary robotics. Improving the reality gap is just as important for locomotion policies and part designs developed through evolutionary techniques as for those developed with deep RL, and evolutionary methods VOLUME 9, 2021  The future of robotic simulators is predicted to see advancements with A) widespread use of differentiable physics [151], B) increased stability and speed of simulation [155], and C) increased rendering capabilities within simulation [156].
also require a large number of time steps or simulations to be run. Simulators that have been used for evolutionary methods in the literature include ODE [142], Nvidia Physx [143], Bullet [144], [145], V-Rep [146], Gazebo [147], and Webots [148], voxcad [117], and Project Chrono [149]. Table 8 compares relevant features of common simulators used for learning in robotics. Features important to this field include: those that enable domain randomization, such as the ability to apply random external forces to the robot and employ multiple back-end physics engines; common sensors required by learned policies such as RGBD, LiDAR, and force sensors; and realistic rendering capabilities for learned policies that rely on visual data.

IX. FUTURE
Physical simulation is tightly intertwined with continued advances in robotics research. It is increasingly important, especially in fields such as robotic deep learning.
In a recent debate style workshop for sim-to-real, debaters proposed the progression of simulator accuracy as an important step in progressing simulator technology (Fig. 4) [150]. Improved accuracy can be attempted in a multitude of approaches as simulators abstract away real phenomenon, making a coarser representation of the world. The most prolific phenomenon to model well is contacts with large improvements likely to be seen with improved methods for collision detection and resolution. Collision detection is very resource intensive and is often a source of instability within simulators. One option is to replace phenomena that are difficult to model in simulation with a neural network that can be trained to replicate the properties of that phenomena with high accuracy, and integrate into the simulator [151].
Differentiable simulators are a fast growing area of research which is tightly coupled with robotics. The availability of automatic differentiation libraries contributes to the large number of new publications in this domain. The primary benefit of differentiable simulation -the ability to use gradient based rather than black-box optimisation approaches -promises a leap in efficiency and opens up previously intractable problems to learning-based optimisation. Several papers have proven examples which show the applicability of such simulators for system ID [152], policy creation [153] and embedding physics in neural networks (Fig. 4) [154].
Plugins and tools which are currently supported in some simulators will likely become even more prolific and ubiquitous. Features that are most likely to be adopted by a wider range of simulators include support for the ROS middleware, and integration with external renderers such as Unity or UnrealEngine for more realistic camera streams (Fig. 4). It is likely we will see more robotic simulators also integrate baseline tools for domain randomisation, system ID, and black-box optimisation.
We are also likely to see further integration in benchmarking and algorithmic frameworks. Examples of this are RL frameworks like OpenAI Gym [157], spinningup [158], and robosuite [159]. Benchmarks and algorithmic implementations will likely become embedded within the simulator framework much like path planners and kinematic solvers already are. This will make it easier to benchmark algorithms against the SOTA.
Finally, we predict that we will see further research into estimating and modelling uncertainty of simulators. Having a metric that encapsulates when a simulator is accurately projecting the real world is immensely advantageous. It provides researchers with an estimation of how likely a solution created in simulation will transfer to the real world, and where additional modelling is required [160].

X. CONCLUSION
Simulators aid robotics research in a multitude of ways. The benefits include reduction in cost, better management of time, and an added level of safety when dealing with complex environments. This review article provides a detailed summary on the type of simulators available for researchers in seven different, prominent, domains of robotics research. Each section covers a range of aspects including competitions, simulator support for features needed in each domain -sensors, actuators, environments -and the current SOTA. Section IX also provides a discussion on developments that we can expect to see in the not so distant future.
To the best of our knowledge, this is the first review article on robotic simulators covering such a diverse range in domains of robotics research. It is an excellent starting point for new researchers and a useful reference guide for experienced researchers. Hence, we hope that more studies like these are published over the coming years as new simulators enter the field and as some seasoned ones become obsolete.

ACKNOWLEDGMENT
(Jack Collins and Shelvin Chand contributed equally to the work.)