A Deep Reinforcement Learning Framework for Control of Robotic Manipulators in Simulated Environments

Industrial robots play a crucial role in a wide range of industrial processes. Because of the complexity of the work environment in which these systems are deployed, more robust and accurate control methods are required. Deep reinforcement learning (DRL) is a comprehensive approach that does not require an initial source of structured data for its learning process. Instead, DRL generates its own data based on its experiences within a work environment. To generate its own data, DRL requires integration with virtualized environments provided by simulators. These tools must include scenarios in industrial contexts and allow integration with machine learning tools, among other capabilities. Currently, several platforms support the simulation of various scenarios and generation of synthetic data, thus facilitating the development of end-to-end systems based on artificial intelligence, such as DRL. This article presents an extensive review of the software tools applied to DRL-based control systems for robotic manipulators. The selection of these tools is based on their efficiency, scalability, and compatibility with contemporary industrial standards and offers insights into their practical application in real-world scenarios. This study established a complete framework for designing and developing control systems for robotic manipulators using end-to-end DRL. This framework outlines the tools in detail, including simulators, APIs, libraries, and methods, and their interactions with each other. Additionally, it discusses the practical implications of this framework, highlighting its potential applications in industry and addressing some of the challenges and limitations encountered in applying DRL to complex robotic systems.