Mixed-Reality Based Multi-Agent Robotics Framework for Artificial Swarm Intelligence Experiments

The term “Swarm intelligence” outlines a broad scope, which is generally defined as the collective behaviour of many individuals towards a certain task, each operating autonomously in a decentralised manner. Swarms are inspired by the type of biological behaviours of animals and this technology involves decentralised local control and communication, which ensures the problem-solving process is more efficient through modification and infusion into swarm robotic technology. Swarm robotics related experiments and applications are relatively expensive compared to other types of robotics experiments. The most common solution is to use computer based virtual simulations and accordingly, this study introduces a reliable and cost-effective hybrid solution using Mixed Reality (MR) technologies to perform swarm experiments on real robots and virtually deployed robots simultaneously. MR bridges the gap between the virtual and physical realms through Augmented Reality (AR) and Augmented Virtuality (AV) and addressing the weaknesses of simulators and providing a more accurate representation of real-world performance. This approach provides a platform for researchers to explore swarm robotics with enhanced realism and interactivity and conduct experiments that mimic real-world conditions. To validate this method we conducted experiments with both physical and virtual robots in near real-time in a mixed reality environment. In addition, an open-source, distributed, and modular mixed reality simulation framework was implemented with support libraries and applications. Our work not only demonstrates the potential of MR in swarm robotics but also emphasizes its value in advancing the understanding and application of swarm intelligence principles in practical contexts.