Dynamic Obstacle Avoidance for Magnetic Helical Microrobots Based on Deep Reinforcement Learning

Magnetic helical microrobots hold immense promise in biomedical domains owing to their compact size and efficient propulsion capabilities. However, navigating these microrobots through dynamic and unstructured environments, particularly when encountering numerous dynamic obstacles, remains a formidable challenge. In the study, a control framework based on deep reinforcement learning (DRL) with the objective of guiding a microrobot through dynamic obstacles towards specified target goals is introduced. Initially, we design and fabricate a microdrill capable of propulsion via external magnetic rotating fields produced by our magnetic actuation system. Subsequently, we construct a custom training environment, adhering to the OpenAI gym interface, to serve as the simulator for training purposes. Utilizing the proximal policy optimization algorithm, we conduct training of the navigation policy within this simulator. Simulations and experimental validations conducted in dynamic environments affirms the efficacy of the proposed method.