Swarm Robots Search for Multiple Targets

This paper addresses the challenge of swarm robots search for multiple targets simultaneously. Techniques are investigated gradually and a systematic scheme which is based on mechanical particle swarm optimization and artiﬁcial potential ﬁeld is eventually developed. The innovative extension makes the bio-inspired particle swarm optimization ﬁrst endowed with the robots’ mechanical properties which reduces the control expense and is already beyond the conventional application scope of this algorithm. The scheme closely considers the practical applications of real robots thus uses the differences, for example, signal frequencies, between the targets for organizing corresponding sub-robot groups aiming at different targets. Those robot groups which move towards non-aimed targets are applied with penalties thus an unimodal objective function for each robot group is built. Meanwhile, the developed method contains the ability for obstacle avoidance based on the module-switching strategy according to their priorities. The methods for controlling the group size and make balance of the search convergence and diversity are investigated, too. Rich simulations and experiments with real robots have been performed to verify this work. Promising results show the effectiveness and robustness of the proposed search method.


I. INTRODUCTION
Researchers have proposed many methods which are inspired from biology and nature. The traditional methods are often pale in comparison with bio-inspired ones since the nature brings us highly adaptive and optimized systems. These methods are usually categorized as 'bio-inspired' and have been used in many areas including the robotics field, e.g., Sedlaczek and Eberhard used an augmented Lagrangian particle swarm optimization (ALPSO) for the structural optimization of a hexapod robot [1], Arena et al. introduced the locomotion generation for a drosophila inspired legged robot [2], Boyer et al. reported a robotic navigation sensor inspired by an electric fish [3], a crawling robot is designed inspired from inchworms [4], and a biomimetic wet adhesion pad for a wall-climbing robot was developed inspired from arthropods like stick insects [5], a self-organized swarm robot for target search and trapping was proposed inspired by bacterial The associate editor coordinating the review of this manuscript and approving it for publication was Ehsan Asadi . chemotaxis [6], whereas collective behavior is designed for a termite inspired autonomous robots team for construction task [7].
Among those many bio-inspired methods, mimicking the behaviors of swarms of eusocial animals for robots to perform target search has received extreme attentions. For examples, Doctor et al. discussed using a bio-inspired algorithm of particle swarm optimization (PSO) for multi-robot search tasks with focus on optimizing PSO parameters [8], Tan et al. proposed an ant colony algorithm for mobile robots robots realtime optimal path planning [9], Kisdi and Tatnall simulated the case of robots search for caves on mars using honey bees search strategy [10], Varga et al. presented a method based on ant colony and honey bees behaviors for target localization and decision making [11], respectively. In the work of [12] the glowworm swarm optimization (GSO) is used for real robots multiple radiation sources search which is very interesting. The authors of this study also have done many done many investigations on target search performed by robots, where methods have been developed for searching a single target cooperatively [13], [14]. However, there might be several targets simultaneously in the searched environment. Handling multiple targets search based on our previous investigations and performed by swarm robots becomes a natural research continuation. It mainly focuses on investigating strategies and methods for swarm robots multiple targets search in an unknown environment. To give the readers an impression in advance, one experimental scenario is already shown in Figure 1.
The rest is organized as follows. It begins with an overview of the most relevant related work in Section II. Section III presents the strategies for single target search by swarm robots and related simulations are immediately shown. Extensions and corresponding simulations for multiple targets search are gradually unfolded in Section IV. The experiments with real robots and results are presented in Section V while conclusions are given in Section VI to close this paper.

II. RELATED WORK
Several survey articles in the area of swarm robotics, especially the papers on swarm robots search, have stated related topics from their individual perspectives. For examples, Barca and Sekercioglu presents a general review on swarm robotics [15], which indicates that there are still a series of issues should be addressed for swarm robotics. Most of the mentioned issues are also involved in swarm robotic search. A thorough review is given by McGill and Taylor, which indicates the common challenges for multiple sources localization [16], i.e., multiple targets search. In that work, GSO is identified as a relatively good choice for multiple targets search. Couceiro et al. presents a benchmark work for swarm robots distributed techniques in a search task, where five algorithms are compared and robotic Darwinian particle swarm optimization (RDPSO) is the winner [17]. In the following study, we investigate robotic search with highlights on swarm robots search for multiple targets concerning real robots, and their potential for practical applications.
Communication strategy and task allocation, especially the latter, are two important techniques for multi-robot search. In the search tasks, robots have to form coalitions to accomplish the assigned tasks coordinately since the separate individuals are usually unable or insufficient. A formal analysis and taxonomy of the task allocation for multi-robot systems is presented in [18], in which the relationships between task allocation and optimization subjects have been shown. Reinoso et al. give a coordination method for multirobot exploration while the cost of reaching a target and its utility are taken into account [19]. A global map is required for this method to be implemented. Panchu et al. propose a Genetic Algorithm based methodology which aims at multi objective optimization of integrated route planning and multirobot task allocation for reconfigurable robot teams [20]. A mathematical tool to analyze the dynamic task allocation for a multi-robot system is used [21]. That is, somehow, very similar to the dynamic neighborhood strategy of our study in response to the environmental changes or effects from other robot actions. A hybrid reactive or deliberative approach to the multi-robot integrated exploration problem has been presented in [22]. A solution for multiple heterogeneous mobile robots task allocation under resource constraints based on leader-follower coalitions is presented by Chen and Sun [23]. The detailed discussions cover the leader selection, coalition forming, evaluation, and submission procedures. Only one task is assigned at each time. The task allocation problem for swarm robots based on a bee algorithm is investigated, highlighting the performance influenced by the number of robots and the number of targets [24].
Swarm robotic search has many potential applications. Among them, surveillance and tracking targets are probably used most. For examples, Jung and Sukhatme use a mobile sensor network to track multiple targets [25], Kolling and Carpin simulate the case of multiple robots surveil multiple moving targets in behavioral cooperation [26], Hollinger et al. consider the problem of multi-robot search for a moving target as NP-hard search path planning in which an algorithm is provided achieving linear scalability in the number of searchers [27], Undeger and Polat present a real-time solution for pursuing a maneuver target [28], Tsokas and Kyriakopoulos have investigated tracking walking people by mobile robots [29], Oh et al. use unmanned aerial vehicles (UAVs) to search and track targets [30], whereas Wang et al. use UAVs formation flight platforms to detect the target [31]. Innovative applications include, e.g., disabled robots considered as targets searched by other robots which mimics the behavior of removing corpses of necrophoric bees by other alive bees [32], treating the human microvasculature tumor targeting and drug delivery problem by those controlled nanorobots or microrobots with magnetotactic bacteria as propulsion and steering system [33].
Robotic search can also be classified as single robot searches for a single target, single robot searches for multiple targets, multiple robots search for a single target, as well as swarm robots search for multiple targets which is the main focus of this study and will be introduced in the following.
Parker is a pioneer in the area of multi-robot coordination and task allocation [34], as well as on observation of multiple moving targets [35]. These swarm robotics topics are mainly investigated based on non-bioinspired robotic means. A modified bees algorithm is used for guiding robots to search two targets [36]. The target selection is based on the probability of the target to be found due to its distance to the robot, and fitness value. In this manner, the system has to consist of a large number of robots. The multiple targets enclosure issue is also addressed [37], where the target selection is based on robot density. This entire method is validated in simulation. Prasetya and Yasuno try to handle multiple mobile robots tracking two known targets based on PSO [38]. However, no specific strategy is presented in the simulation for selecting a target, whereas only the proximity, which of course may change during the search, is used to select the target. Furthermore, convergence is not guaranteed thus one or several of the targets may not be found. In many investigations, it is claimed that the used robots are searching unknown targets. However, during the search process they indiscriminately and extensively use the target position for calculating the distance to the robot and further use this for evaluating performance functions. It is very clear, that those researches remain only in the simulation.
The work of Derr and Manic [39] is close to ours in the view of the technique we both based on for distinguishing different targets. However, the essential solutions distinguish a lot. Moreover, in their study those environmental obstacles are not considered although they mentioned that each robot becomes a moving obstacle to others. One has to complain that its limited documentation makes it hardly convincible. Nevertheless, its weighted wireless coefficient applied to PSO's velocity update for avoiding overshooting to the target shows its effectiveness. The glowworm swarm optimization (GSO) method, as the name suggests, is inspired from glowworms and first proposed by Krishnanand and Ghose [40]. It is actually a variant of the ant colony algorithm and similar to PSO [41]. However, it is defined with luciferin which is the luminescence quantity. Each agent is taken as one glowworm who is emitting light and its intensity is proportional to the associated luciferin. Up to now, this might be one of the most reasonable methods for swarm robots to search multiple targets, and it will be compared with the method proposed in this study. However, GSO is ineffective within zero gradient 'dead spaces', i.e, local traps. An approach based on particle swarm optimization and Darwinian particle swarm optimization for multi-robot exploration was presented, where obstacle avoidance is included as a penalty generated from a potential field like concept and affects the robot velocity update [42]. However, it seems they require only one robot to reach the target finally. Moreover, it emphasizes a unique global optimum search, i.e., single target search. Later on this research group investigated communication constraints [43], in which the focus is how to deploy an initial pose of the robots. Unfortunately, robotic Darwinian particle swarm optimization (RDPSO) can not guarantee a successful convergence, and the fault-tolerance is semi-finished, i.e., it only detects the failures but can not handle the self-repairing [43].
Most currently existing methods, on the one hand, have no 'one agent-to-one robot' mapping where an agent can be an individual ant, bee, bacterium, or particle. Thus, bioinspired methods are just used as algorithms for purposes such as optimization. On the other hand, in some investigations, although one agent corresponds to one robot, they have no detailed considerations of endowing the real robots mechanical properties to the agents in the search planner. Thus, inevitably they leave heavy burden to robot control afterwards. Therefore, usually only very simple robot motions can be performed via those methods.
Up to now there appears to have been relatively limited investigation on swarm robots search for multiple targets concerning the effectiveness for practical implementation. The reasons are multifaceted. First of all, this task is mathematically NP-hard in the number of targets and number of robots, which makes many methods useless for many robots. Second, it lacks reasonable theoretical foundations and question mapping. Third, too many arbitrary setups in simulation validations make the real implementation daunting. Most of the investigations in literature are based on their ad-hoc experimental setups which omits a lot of information from the natural environment and unfortunately makes the generality questionable.
Therefore, unlike those methods in the existing literature, the proposed approach in this study distinguishes itself by respecting the goals of mapping the task of swarm robots search for multiple targets to multimodal function optimization, incorporating the robots mechanical properties into the search planner itself, as well as increasing the generality for real robots practical implementation.

III. SWARM ROBOT SEARCH OF A SINGLE TARGET
A swarm of robots is able to achieve many complex tasks which an individual robot is sometimes not able to do. This section provides a method for a mobile robot swarm to search a single target.

A. METHOD PROPOSED FOR SINGLE TARGET SEARCH
Particle swarm optimization is a population-based stochastic optimization algorithm inspired by the swarm behavior of birds flocks and fish schools [44]. Traditionally, PSO is used as an optimization tool while in this study it is extended and used as the cooperative search planner for the swarm robots target search.
A target usually can be a source which radiates a signal, e.g., odor, light, heat, or sound. It can be used mathematically as an optimum in the signals field. Thus, we have developed earlier the method of mechanical PSO which is summarized as for guiding the robot swarm to the searched target. Here, x contains the position and orientation information of all the robots, defined in the plane by where N p is the number of robots (swarm size). Correspondingly,ẋ indicates the velocity of robots. Here M is a generalized mass matrix. The upper row in (1) describes the robots' position update, and the second is the velocity update. They calculate the position and velocity of each robot at the next step k + 1 by using the current and previous information. Here x best,k self are the best positions of robots' own experiences, and the best positions of the robots in certain neighborhoods are described by x best,k nhood (or in the entire swarm by x best,k swarm ). The velocity behavior of the robot swarm is summarized into three terms which correspond to inertia, cognitive and social effects. The inertia term describes the influence of the current velocityẋ k to the velocity at the next state. The time step is indicated by t while This term attracts the robots to the best position of their own experiences. The social term describes a common recognition that the robots learn from those robots who experienced the best positions up to now in their specific neighborhoods (or in the entire swarm when taking the entire swarm as a neighborhood). Three scaling factors h k f i (i = 1, 2, 3) are referred to the inertia, cognitive and social terms, respectively. Here M possesses both the information of masses and rotational inertias of the entire swarm. Different to the basic PSO algorithm, in mechanical PSO the robots mechanical properties are included which is very useful for guiding swarm robots. After each step, the robots are evaluating their fitness values, in practical applications there are usually the signal strengths detected by robots.
Although PSO based methods are often successful for practical problems in engineering, the basic PSO can not be directly applied for constrained optimization problems. However, most practical problems such as, e.g., the swarm robots target search, have to obey constraints. This study uses the augmented Lagrangian multiplier method to treat constraints. Together with PSO, the variant named augmented Lagrangian particle swarm optimization (ALPSO) was developed and successfully used for engineering constrained problems [45]. By such a method, the PSO algorithm is greatly widening its applicability. For this study, the eventually proposed mechanical PSO inherits this method for dealing with constraints during robot search. The basic ideas of ALPSO are stated in the following.
Since a robot swarm searching a target in the environment can be mapped to the particles in the search domain to find an optimum, mathematically the problem including constraints can be formulated as where x is a state point containing the generalized variables and of course can also represent the position and orientation of robots. Using the augmented Lagrangian multiplier method each constraint violation is penalized with a finite and adaptable penalty factor r p,i . In this method, a criterion that optimizes a single objective function f s (x) has the form with In (7), f s (x) is the basic objective function, and each P i (x) specifies one constraint function. Hence, the constrained problem can be converted to an unconstrained problem as a sum of original objective function and penalized constraint functions. Here λ is the Lagrange multiplier vector and r p is the penalty factor with the initial values λ 0 i = 0 and r 0 p,i = 1. Note that λ i and r p,i are unknown in advance and are adjusted during every iteration due to the changing constraint condition. The mechanical PSO is developed for swarm mobile robots to search a target cooperatively. It is easy to extend the method to 3D and also include the torques. The force definitions can also be extended, e.g., to define an external force to steer the robots manually. Different to the basic PSO algorithm, in mechanical PSO the robots mechanical properties are included which are very useful for guiding swarm robots, besides the nice features inherited from PSO variants. From another point of view, this method considers each robot as one body in a multibody system. Thus, the robot swarm's motion can be described within the view of a virtually linked multibody system. The robots influence each other by forces and torques but without direct physical connections. Therefore, the robots are joined together as a whole which yields our systematic robot swarm. Mechanical PSO uses rich mechanical and mechanical physical properties to replace complex control strategies, and solves some control problems from the mechanical layer. It is verified that mechanical PSO is capable to guide the swarm robots to search for a target cooperatively obeying the PSO rules and meanwhile obeying the mechanical features of the involved robots.

B. SIMULATION OF SINGLE TARGET SEARCH
One simulation case is set up to verify mechanical PSO for a single target search. A quadratic function is defined to describe the signal field, and the minimizer (x m 1 , x m 2 ), i.e., the target, in this simulation is set to (0, 5) which is marked by an ' * ' in Figure 2. The inequality constraints are set in the domain −10 ≤ x 1 , x 2 ≤ 10 to which describe the two shaded prohibited areas in Figure 2(a). The seven blue areas in the figure indicate the obstacles. In this case 100 volume and mass contained robots represented by red circles join the search group in the simulation. The initial distribution of robots is shown in Figure 2(a). An independent obstacle avoidance module is used locally for the collision avoidance. Finally all robots have successfully sieged the target under the guidance of mechanical PSO, see results in Figure 2 During the search, those robots don't know where the target is located. They update their positions according to mechanical PSO and after each step they report their current positions to mechanical PSO for evaluation. The closer to the target, the better fitness value is obtained for a specific robot. For real robots practical application, these fitness values are replaced by corresponding source signals detected by robots. Extensive target search simulations have been performed by using mechanical PSO, and more examples can be found in our previous work [14], [46].

IV. SWARM ROBOTS SEARCH FOR MULTIPLE TARGETS
This section describes multiple targets search performed by a robot swarm. It is an arduous task since difficulties include, e.g., in some rescue scenarios the robots have different duties and one robot finds a target but it might be not the purpose of this specific robot. Strategies are thus investigated in conjunction with mechanical PSO to enrich the ability of our method.

A. PROBLEM MAPPING
Constraint handling, robotic search planner design, task allocation and so on were often seen as distinct. However, in the investigation of swarm robots cooperatively search, one observes that these topics can be unified within a reasonable optimization framework based on mechanical PSO. Mechanical PSO is first of all capable of driving robots to search a single target in the environment by defining the search task as a single modal optimization problem and by endowing PSO with the robots mechanical properties. Correspondingly, the mapping origin in basic PSO is a set of particles moving towards a promising area to locate a global optimum. The PSO features in this case dominate the target search and tasks like obstacle avoidance and robots mutual avoidance are handled by robots locally. By this structure, the task allocation is partially avoided based on different modules and the rest mainly relies on the PSO characteristics.
However, many cases require to find all the equal quality solutions in optimizing objective functions. For real robot applications, the signal distribution may be multi-peaked, multimodal, due to several signal sources or the obstructive environment. Therefore, very naturally, we map the tasks of swarm robots search for multiple targets as optimizing multimodal objective functions with swarms of volume and mass contained particles. Up to now, only very few work in swarm robotics are dedicated to this chanllenge to the best of our knowledge. Notice that this is optimizing a multimodal objective function, not multi-objective functions [47], [48]. Exact equal quality global optima do physically hardly exist, and usually every local minimum is an useful solution. Nevertheless, it is still very necessary to investigate equal quality global optima seeking as the theoretical foundation. This is based on the fact that robot sensors have their measurement tolerances which may treat two highly similar targets as equal quality targets.
The main difficulty in multimodal optimization is the trapping in local optima. Therefore, if one can create strategies based on the obtained results and let the swarm avoid local optima while finding global optima in the multimodal objective functions, mechanical PSO can be enriched for searching multiple targets. The method of glowworm swarm optimization is emphasizing on finding multiple local optima. This usually can be handled by equipping simple robot sensors for distinguishing. However, what we want to investigate is a more special and more difficult situation which has several equal quality global optima. These equal quality global optima in practice can originate from different sources. Strategies should ensure robots to avoid local traps, and find global optima (targets). In the following, different strategies are investigated gradually to assist mechanical PSO for searching multiple targets simultaneously.

B. FURTHER ENRICHED MECHANICAL PSO
Particle swarm optimization can be capable of handling multimodal optimization with reasonable variations. Among  these varieties, it is the general idea to separate the swarm to 'species' (sub-swarms) and each specie is responsible to converge to one of the global optima. Strategies involved are mainly based on balancing exploration and exploitation.

1) ATTEMPTS BASED ON CURRENT STATIC NEIGHBORHOODS
Firstly, attempts have been performed based on the current mechanical PSO to see which functionalities are missing. For mechanical PSO, the neighborhood best mode has been introduced. However, its neighborhood size is static, although the members in one specific region may change during the search. This helps to reduce the observation requirements for real robots. For verification, the objective function is used where x, y ∈ [−10, 10]. The contour map in Figure 3 illustrates the distribution of f 1 (x, y) in the search landscape. In this study, the minimum is taken as the optimum. In Figure 3, three local minima are illustrated by blue rhombus. They locate at (−10, −5), (−2.5, 10) and (7.5, 10), respectively. Their objective function values are all equal to −1. The other two red pentagrams are global optima at (−2.5, −5) and (7.5, −5) both with objective function value of −2. Figure 4 illustrates the result of one typical run based on pure mechanical PSO, in which robots are represented by black mass points and the robot volume is temporary neglected so as to avoid the situation of missing the global optimum due to robots mutual avoidance.
From Figure 4 one can see that the two global targets are found by robots and one robot near one of the global optima doesn't converge in the desired search time. However, there are many robots trapped in the local optima which shows the limitation of mechanical PSO. This happens because the selected neighborhood size limits the social recognition of the swarm members. For swarm robots to search multiple targets, all the robots are expected to aggregate at global optima (true targets). However, the current too simple strategy doesn't work well for this purpose.

2) SYNTHESIZED DYNAMIC NEIGHBORHOOD METHOD
A stationary neighborhood gives the swarm members a stationary social recognition. The possibility of propagating information about the global optimum to every robot is significantly reduced when local clusters show up during multiple targets search. This is why many members are trapped in certain local optima. In the neighborhood best mechanical PSO, the size n (number of members), and sensing range r (radius of the neighborhood), play significant roles during the robot search. A stationary neighborhood gives the swarm members only a stationary social recognition. This is why many members are trapped in certain local optima. In order to solve this problem a dynamic neighborhood method is constructed with its size n and range r updated by with α, β > 0, δ, η > 1, α, β ∈ Z, δ, η ∈ R, 0 < ε < 1, (14) where k is the index of the update step. In (12) the number meets 0 ≤ n k i ≤ N p and in (13) the radius meets 0 ≤ r k i ≤ r sense in which r sense is the robot sensing distance. Indeed, the updates of (12) and (13) are correlated. They reach a consensus by adjusting the parameters simultaneously in (14). The idea of changing the exploration range also appears in other algorithms, e.g., the swallow swarm optimization [49]. However, the proposed method in this study distinguishes from them since it considers both exploration range and number of neighborhood members. Moreover, the parameter tuning procedures for the synthesis in (12) and (13) are programmed to be autonomous. Therefore, balancing of exploration and exploitation becomes adaptive according to the current search situation. Specifically, the update of n k i and r k i is based on the idea that robots can virtually assume two different kinds of roles in the entire swarm, i.e., leaders and followers. A member that holds the best fitness value L in its neighborhood is a leader, otherwise it is a follower in a certain region. A leader and its followers can be seen as a sub-swarm. VOLUME 8, 2020 Each sub-swarm is capable to discover an optimum. If a follower (robot i) follows a leader who has a smaller objective function value L k leader, i compared to the mean value of all other leaders L mean, k leader within a small tolerance ε L mean, k leader , then the neighborhood of robot i will be reduced, i.e., n k+1 In opposite, its neighborhood will be enlarged. Otherwise, its neighborhood remains unchanged. Therefore, with an increased neighborhood, this follower is reassigned to follow a new leader with lower (better) fitness value. This manner only broadcasts the best fitness value of each subswarm, rather than the global optimum or the robot position directly, and arrives only at the sub-swarm level. This is also realistic and more convenient in real robots implementation comparing to other grouping and splitting strategies, for example, the strategy addressed by Zheng and Tan et al. [50] which is more from the simulation perspective and based on some artificial rules.
When the swarm size is big enough, the proposed synthesized dynamic neighborhood mechanical PSO is able to find all the global optima and avoid local optima. For verification, simulations are done using the function f 1 , see results of one of the runs in Figure 5.
At the beginning of the search, 40 swarm members are distributed randomly in the environment, see Figure 5(a). By applying the proposed synthesized dynamic neighborhood version of mechanical PSO, all the swarm members converge to the two global optima and overlap, see Figure 5(b). The problem of trapping in local optima is solved using this method.
The synthesized dynamic neighborhood method works well when the swarm size is big enough. However, it is not a method that can mathematically guarantee 100% success during multiple targets searches. In addition, the number of members converging to the different global optima can be very different, see Figure 6, which is another simulation for the three targets example   The corresponding contour map of function f 2 is shown in Figure 7. The four blue rhombi in Figure 6 are the obstacles. In addition, the robots volume in this simulation is considered. Therefore, the robots are not allowed to overlap and it is reasonable to perform statistical investigations, e.g., how many robots converge to one specific global optimum. From Figure 6(d) one can see that the number of robots finally arrive at different targets are very different. Furthermore, in both simulations in Figure 5 and Figure 6, a large population of robots is required. However, it is sometimes not necessary or not possible to have a huge robot swarm.

3) INCLUDING ARTIFICIAL POTENTIAL FIELDS AND PRE-ASSOCIATION OF DIFFERENT FREQUENCIES
In order to solve the remaining issue of balancing sub-swarm size, further strategies are investigated. Based on the synthesized dynamic neighborhood mechanical PSO, artificial potential fields are introduced to combine with pre-association of different signal information, e.g., different frequencies, to assist mechanical PSO. The pre-association of different frequencies is just an exemplary utilization for forming corresponding since later in the experiments two light sources with different frequencies are searched. For other scenarios, different signals or signal intensities can intensities can be used. With such additional information, it brings the possibility to balance the sub-swarm size during the multiple targets search.
The method of artificial potential fields (APF) was first introduced by Khatib [51]. It is used for the obstacle avoidance for manipulators and mobile robots. It is a method that generates forces on the robot by applying potential functions p(x). Thus, the robot follows the enforcement of APFs and moves from a higher potential position to a lower potential position, and finally reaches the minimum of the potential field. In a potential field p(x), the robot motion is driven by the gradient ∇p(x).
The robot stops if the gradient reaches zero, i.e., if ∇p(x * ) = 0. Here, x * is the critical position and sometimes may also be designed as the goal position which can also be local minima. The APF method can be efficient but the robot may be stuck at those zero gradient positions. One solution for solving this problem is applying an additional perturbation to destroy the deadlock which is the basis of many APF variants. Another solution is to augment the potential field with a 'search-based planner' on which this study bases. In this study, the robots are mainly guided by mechanical PSO which serves as the 'search-based planner'. The APFs are used to assist the grouping of sub-swarms and balancing the size of these sub-swarms for different targets.
The general concept for the application of APF can be shown in the following example. In Figure 8(a) a robot needs to move to the red star without collision to the blue obstacle. The red star specifies the goal position which is (0, 0) in the map, and the blue obstacle occupies a rectangular area. By including two APFs, the robot can be driven to the target. One APF is defined by function which constructs an attractive potential field that is able to generate forces to drive the robot to the optimum (0, 0). In our robotic search scenarios, this target potential field is usually also the performance function. The obstacle APF is for example described by function It has a repulsive function around its boundary and can generate much greater forces than the one from the target. If the robot moves close to this obstacle, the repulsive force will enforce the robot to move away from the obstacle. The overall potential field formed by functions p 1 and p 2 is demonstrated by the surf map in Figure 8(b). A target can radiate signals to the environment with a certain frequency. Practically, if different targets are searched by real robots, different sensors can be applied to distinguish  the targets through, e.g., colors, odors or frequencies. In this study it assumes the targets are emitting signals with different frequencies. Under this assumption, we try to combine the synthesized dynamic neighborhood mechanical PSO with artificial potential fields and pre-association of different frequencies for multiple targets search.
A sub-swarm can be trained for searching a certain target that radiates a signal with a specific frequency. Figure 9 illustrates this idea. The circles represent robots and the stars indicate the two targets in the environment. Target locations are determined by function f 1 , see (11). The whole robot swarm is split into two sub-swarms which are marked with different colors to distinguish them. In this case, the two sub-swarms have the same size. Two targets radiate signals with different frequencies. Therefore, e.g., sub-swarm 1 is grouped for searching target 1, while sub-swarm 2 is grouped for searching target 2. Each sub-swarm can identify both frequencies, but is aiming for only one target.
The pre-associated sub-swarms can also be attracted by other undesired targets. For sub-swarm 1, in order to avoid the attraction from target 2, a repulsive APF is introduced. Combining f 1 and s 1 , the objective function for sub-swarm 1 is finally VOLUME 8, 2020 Its corresponding potential is demonstrated in Figure 10. From Figure 10 one can see that only one minimum (optimum) at (−2.5, −5) is recognized in the environment by sub-swarm 1. If robots move close to target (7.5, −5), due to the influence from s 1 , the robots will be forced to move out of this area since a repulsive force is applied. If all the sub-swarms are affected by appropriate APFs to avoid attractions from other targets, the aimed target is the only global optimum in the searched environment for each specific sub-swarm. In real robots implementation, these APFs are defined according to the signal intensities detected by robots since the target positions are unknown to the robots. Thus, the APF function is usually constructed as a function of signal intensity φ. One commonly used form is where α is a constant scaling factor which needs to be identified during experiments. Similarly, one can also apply an attractive APF, which can be taken as performance function as well, to drive the sub-swarms to move to their aimed targets. In this method, if we increase the APF intensity then the modified objective functionf 1 can usually approximately show the performance as a unimodal function. This method has the hope of not requiring to use a large robot swarm during multiple targets search since each target requires only 3 robots to form the sub-swarm for a success search based on mechanical PSO. In those real searched environments, this operation can be done by strongly amplifying the intensity of one certain frequency signal. In addition, repulsive and attractive coupled functions can be used.
Simulations using the methods of artificial potential fields and pre-association of different frequencies in conjunction with the enriched mechanical PSO from Section IV are also implemented. It is reasonable to include APFs to orient each sub-swarm aiming only at one specific target. The used APFs for each sub-swarm are applied to restrain the attraction from other targets and enhance the attraction from the aimed aimed target.
The function  is used in this simulation. This function has two global optima. Two repulsive artificial functions are used in this case to avoid the attraction from another non-aimed target. They are By adding them to the objective function f 3 , the new objective functions for the sub-swarms are wiht their contour maps shown in Figure 11. Because of adding an artificial function, the non-aimed target position can even become the maximizer. A strong repulsive force is thus generated to push away the robot from the non-aimed target. This force varies depending on the distance from the current robot to the repulsive source.
Simulation results with obstacles in the environment for functions f 1 3 and f 2 3 are illustrated in Figure 12. The robot volume and mass are considered, too. Both of the targets are found. In Figure 12(a), ten robots are randomly distributed in the environment. They have difficulties to detect all the optima because the swarm size is relatively small. However, after about 100 s, both targets are found, see Figure 12(b).
A simulation case with 9 targets is also done for further verifying of our work. Figure 13 is its contour map and Figure 14 shows the simulation results. Comparing the results in Figure 14(a) and 14(b) one can see that the obstacles may affect the grouping outcome. Figure 14(a) is the result from  the case without obstacles, and robots are evenly distributed, i.e., each of the sub-swarms contains five members as desired. Figure 14(b) has obstacles in the environment, and two subswarms have four members while other two sub-swarms have six members.
There are two challenges, namely finding a suitable assignment of robots to target locations and generating collision-free trajectories for each of the robots, which may have conflicts. The method proposed by this study for this issue is to perform module switching between obstacle avoidance and search. Mechanical PSO is the main search planner and obstacle avoidance is performed by each robot locally. Once conflicts happen, the algorithm immediately performs an emergent coordination and the obstacle avoidance usually has the superiority.

A. EXPERIMENTAL SETUP
Experiments are performed in a 3 m × 3 m area at the Laboratory of Robotics and Multibody System in Tongji University, Shanghai, China, see example in Figure 1. The real robot is an omnidirectional mobile robot with 4 omni-wheels developed by our own. This robot has a circular chassis (see Figure 15) with radius of 0.2 m. All the robots are driven by the proposed algorithm. Each of them can communicate with others, thus establishing a robot swarm. An indoor positioning system for swarm robots is developed. Each robot is equipped with a pattern for positioning, such as the letter ''D'' shown in

B. VERIFICATION OF THE METHODS FOR A SINGLE TARGET SEARCH
The single target search experiment is done based on the mechanical PSO method purely. Figure 16 shows the four critical statuses of one run of searching for a single target by a robot swarm with four real robots. From Figure 16 one can see the mechanical PSO qualifies for the target search via a robot swarm while obstacles are included in the searched environment.

C. VERIFICATION OF THE METHODS FOR THE MULTIPLE TARGET SEARCH
In this part, experiments are performed under the strategy of APFs and pre-association of different frequencies in conjunction with mechanical PSO. A swarm with eight robots is employed. It is split to two sub-swarm and for each has 2 real robots and 2 simulated robots. To verify the robustness of the developed strategies, one more complicated objective function is employed which is f 4 = sin(π cos π 4 (x + 1) − π sin π 4 (y + 2) + 4) + sin(π sin π 4 (x + 2) + π cos π 4 (y + 2) + 2), (25) where In Figure 17(a), all the robots are initially distributed in the environment. The red circles in the simulation side represent the simulated robots, while the grey circles represent the real VOLUME 8, 2020 robots, that also shown on the experiment side. The four robots marked by yellow square belong to the sub-swarm 1 while the robots marked by green square belong to the sub-swarm 2. Practically, they are formed automatically during the search by distinguishing the detected signals.
In Figure 17(b), the experiment status at time t = 30 s is illustrated. We notice during the experiment that two sub-swarms are generated to their own aimed targets. The robots who are near the non-aimed target will be pushed away by the penalty forces.
Finally, each robot sub-swarm has aggregated at its aimed target, see Figure 17(c). Due to the localization error, the real robots in the real scenario have deviations comparing to their representatives in the simulation side. By applying this strategy, the number of robots which surround a target can be guaranteed or can be determined due to the detected signals.

VI. CONCLUSION
This study first provides the mechanical particle swarm optimization (mechanical PSO) method for single target search performed by swarm robots. Different strategies have been investigated such as static and dynamic neighborhood methods, strategy of including artificial potential fields and pre-association of different frequencies, based on mechanical PSO for seeking solutions of multiple targets search. The dynamic neighborhood method is an advance of the basic (static) neighborhood best version. It uses dynamic size and range to describe the neighborhood. In this strategy, mean Lagrangian function value of all the leaders has been applied as reference to vary the neighborhood. Under dynamic neighborhood method, the swarm robots are capable to avoid local traps and to find the targets. However, it cannot guarantee that the robot swarm can find all the targets. When the robot swarm size is big enough, all the targets can usually be discovered.
The strategy of combining artificial potential fields with mechanical PSO is based on the fact that different targets may radiate signals with different frequencies. This strategy is capable to guide a robot swarm with several sub-swarms for multiple targets search. This method can ensure a successful search of all the targets, even only very few robots are arranged to search for each target. This is a benefit from the nice feature of mechanical PSO and the modified objective function for each pre-associated sub-swarm is able to approximate a unimodal function.
Simulations and experiments have been done to verify the investigated work. The robot mechanical properties are directly in mechanical PSO in this study, which is extremely convenient for robot control. Mapping the question of 'swarm robots search for multiple targets' to the 'multimodal objective function optimization' is an important reason to make this systematic method successful. The simulated robots and real robots mixed search provides a new way for the research of robot swarm target search which closely bridges the simulation side and experiment side.
As natural continuations, searching some other specific radiation sources is worth to be investigated for further verify the robustness and generality of the proposed methods. Besides these, improving the real robots localization accuracy or even liberating localization necessity during the search is also our ongoing consideration. a Senior Research Associate. He is currently a Full Professor and the Founding Director of the Laboratory of Robotics and Multibody System, Tongji University, Shanghai, China. He also serves as the Robot Group Leader with the Institute of Engineering and Computational Mechanics, University of Stuttgart. His research interests include swarm robotics and robotic control, as well as interdisciplinary collaborative research on robotics-multibody system dynamics-optimization. He is active in this area and plays as IROS AE and many robotics journals reviewer. He has served before in many national and international organizations, e.g., as the Chairman of the International Association for Multibody System Dynamics (IMSD) or DEKOMECH (German Committee for Mechanics).