A High-Precision Collaborative Control Algorithm for Multi-Agent System Based on Enhanced Depth Image Fusion Positioning

The collaborative control of the multi-agent system (MAS) marks the trend of intelligent transportation system (ITS). However, the collaborative control of MAS with flexible sampling periods remains a challenge, because under-driven systems are prone to random delays, data loss and sensor failures in semi-unstructured environment. Against the background of the semi-unstructured environment in a Dutch greenhouse, this paper puts forward a universal collaborative motion control algorithm for the MAS of automated guided vehicles (AGVs), in the light of the first-order dynamics of the system. The proposed algorithm is called continuous-step-rotate-run (CSRR). Besides, the enhanced depth image fusion positioning (EDIFP) scheme was designed to mitigate the disturbances on the control algorithm, arising from flexible sampling periods and data loss. To verify its effectiveness, the CSRR control algorithm was tested on an MAS of three under-driven BigPan AGVs. The results demonstrate that our algorithm can collaboratively control the AGVs in an effective and stable manner. The simple algorithm offers a desirable solution to the collaborative control of various MASs.


I. INTRODUCTION
Population aging and urbanization are two demographic trends shaping today's world. Under these trends, the percentage of the labor force engaged in agriculture is falling, and the labor cost of agricultural workers is on the rise. The soaring labor cost, coupled with the high labor intensity in unhealthy operations like pest control [1], [2], directly pushes up the cost of agricultural production: harvesting alone contributes to 25% of the total cost of agricultural production [3], [4]. This contradicts the growing demand for agricultural products, posing a major challenge to the sustainable development of agriculture [5].
The associate editor coordinating the review of this manuscript and approving it for publication was Dalin Zhang. From the engineering perspective, the above challenge can be tackled by developing a quality and low-cost intelligent mechatronic system [6] that can replace human operations in an efficient manner [7]. Essential to intelligent transportation system (ITS), the automated guided vehicle (AGV)can effectively mitigate the risk of workers involved in dangerous operations [7], enhance the production efficiency, and ease the labor shortage induced by population aging [8]. Multi-pleAGVs are often grouped into a multi-agent system (MAS) to perform such operations as grazing [9], farming [10], fruit picking [11], and sorting [12].
Despite its various advantages, the MAS faces several limitations in agricultural applications, owing to the complexity, flexibility and cost of the specific problem. The environment of many applications, unlike those of factories and warehouses, is highly unstructured, or semi-structured at the best [13]. In China alone, the semi-structured agricultural acreage reaches a staggering area of 981 Khm 2 . Take a 6000m 2 Dutch greenhouse ( Figure 1) for example. 500 L medicinal solution must be sprayed within 1.5 h. The spraying operation could be implemented by an MAS of five small AGVs with a capacity of 100 L each, or a large, complex and costly AGV with a capacity of 500 L. This calls for collaborative control of the MAS under semi-structured environment. The key aspects of the control strategy include roads, lighting and communication [1].
Over the years, the MAS collaborative control has attracted much attention in the academia. Some scholars evaluated the MAS performance by the speed of state convergence. For example, [14] provided the necessary conditions for state convergence, using frequency domain techniques like the Nyquist criterion. References [15], [16] adopted a classic method, Lyapunov stability analysis, to prove the state convergence of the MAS. Reference [14] introduced the inputto-state stability (ISS) to evaluate the stability of the control strategy in nonlinear dynamic environment.
The MAS performance has also been evaluated by another index: the quadratic cost function of state convergence and control cost. For example, [17] proposed an optimal consensus control protocol based on the linear quadratic regulator (LQR), and established a closed-loop control gain matrix.
The model predictive control (MPC) has also been employed to achieve the collaborative control of the MAS, thanks to its ability to convert realistic constraints (e.g. operation speed and collision warning interval) into control parameters. For instance, [18]designed a discrete-time MPC consensus control scheme with discrete-time first-order and second-order dynamics MPC consensus control scheme with time-varying interactive topology. References [19], [20] explored deep into the MAS collaborative control based on MPC solutions.
The MAS collaborative control can be simplified by fixing the sampling period, a common approach to facilitate sampling control. However, the period should be fixed with a reasonable length. Otherwise, the excessively rapid sampling will bring too much redundant information, increasing the overheads of information transmission and processing. Then, the control strategy will be less effective. Moreover, it is difficult to implement fixed-period sampling of the MAS state in real-world scenarios, due to the existence of multiple constraints on communication (e.g. random delays, noise interference, and data loss). Adding to the difficulty of fixed-period sampling, the clocks of the agents cannot be synced easily in a semi-structured environment without GPS support [21].
The event-triggered control (ETC) can effectively overcome the defects of fixed-period sampling. This energyefficient control strategy only executes sampling, calculation or update under the trigger conditions [22]. Under observerbased mode or self-triggering mode, the ETC technique has been applied to the first-and second-order collaborative control of linear and nonlinear MASs [23]- [28].
The sliding mode control (SMC) provides a desirable solution to the deteriorating performance of the MAS induced by uncertainty and interference, laying the basis for collaborative control of the MAS. Under control inputs of the SMC, the system state slides along the given sliding surface function towards the desired state. Based on Laplacian matrix, [29] presented a sliding surface function for MAS collaborative control. Reference [30] preserved and coordinated the finitetime connectivity in a second-order MAS with limited sensing range, designed a distributed controller based on integral SMC and artificial potential field (APF), and proved that the controller achieves robust finite-time collaborative control, without sacrificing the connectivity of the communication network.
Furthermore, many other control strategies have also been successfully adopted to realize the collaborative control of the MAS, including but not limited to the adaptive control [31], linear matrix inequality (LMI) [19], and game theory [32].
The main objectives of the MAS research are identifying the stable conditions for collaborative control, and then formulating suitable control methods for existing systems. Nevertheless, most of the existing studies only verify the theoretical results through simulation, rather than actual application or field testing. Many of their control strategies only apply to a specific MAS, failing to support general applications.
Drawing on graph theory and nonlinear adaptive control theory, [33] examined the leader-follower group consensus control protocol, in which the leader moves at a constant speed and makes the decisions, while each follower only knows the relative position between him/her and another agent. By the APF method, [30] controlled a set of nonholonomic MAS robots to marshal into different shapes without collision. Considering the interactive topology, [31] explored the directional control of a mobile robot with monocular vision, and introduced image processing algorithm into the consensus protocol. Reference [32] investigated a set of collaborative controltasks with high reference value, and deployed image sensors on a Pioneer D3-PX robot to collaboratively clean up color boxes scattered in a room. Reference [34] combined the APF and robust control technique into a formation control model, and estimated the unknown parameters in the model by an adaptive fuzzy logic algorithm.

A. RESEARCH CONTRIBUTIONS
This research mainly makes the following contributions: • A continuous-step-rotate-run (CSRR) control algorithmwas designed for the collaborative control of the under-driven MAS of a set of AGVswith flexible sampling periods. The physical constraints of the drive system, namely, the output speed of the motor, were considered in the algorithm design, making the MAS smooth and efficient. By contrast, the output speed is allowed to change randomly in existing simulation research.The CSRR control algorithm was verified through an example analysis on BigPan, a self-designed two-wheel differential drive AGV.
• The proposed collaborative control strategyis simpler and more versatile than the existing strategies. Our strategy applies to the MASs in varied topologies, provided that the AGVs and the ITS satisfy the basic constraints.
• The enhanced depth image fusionpositioning (EDIFP) scheme was developed to enhance the accuracy of visual positioning under semi-structured environment. In the proposed scheme, the depth data and RGB images are fused and filtered to reduce the effects of nonuniform illumination and heat sources. The accurate positioningis conducive to the precise collaborative control of the MAS. In addition, the algorithm proposed in this paper has been effectively verified in BigPan's actual MAS system, which provides a strong reference and reference value for subsequent research.

B. ORGANIZATION
The remainder of this paper is organized as follows: Section II introduces the self-designed BigPan, and formulates the CSRR control algorithm for the MAS; Section III puts forward the EDIFP scheme, which achieves accurate positioning for MAS collaborative control; Section 4 verifies the proposed control algorithm through system tests; Section 5 puts forward the research conclusions.

II. MAS COOPERATIVE CONTROL UNDER NON-UNIFORM SAMPLING CONDITIONS A. ''BIGPAN'' AGV SYSTEM
Under the semi-structured environment of the Dutch greenhouse [35], the authors designed a two-wheel differential drive AGV called the BigPan [36]. The architecture of the self-designed AGVfor pesticide spraying is illustrated in Figure. 2 and Table. 1. It can be seen that the prototype was extended from the full-featured version of a basic AGV in Figure 1(d). Specifically, the motor and two drive wheels are located on the central axis, forming a differential drive system. Thus, the AGVcan move along the axis of the drive wheels, as an under-drive system. The AGV is controlled by an artificial intelligence (AI) controller (Nvidia Jetson TX1/TX2), a vehicle control unit (Freescale MC9S12X), and an electronic control unit (ECU) controller. Among them, the AI controller is provided with two webcams (Logitech HD Pro Webcam C920) to realize visual navigation control from first-person perspective. All the controllers are connected wirelessly to the host.The subsequent versions of the AGVwere exhibited on the 15th China International Agricultural Trade Fair in 2017.

B. BASICS OF MAS COLLABORATIVE CONTROL
The MAS and its workspace can be transformed into adirected graph and several stochasticnonnegative matrices. The transform is a necessary step for setting up a first-order control model.
For the MAS, the communication network can be expressed as a directed graph G = (V, E, A), where V is the set of n network nodes (AGVs), E ⊆ V × V is the set of edges (between the AGVs), and A = [a ij ] ∈ R N ×N , (a ij ≥ 0, ∀i, j = 1, 2, · · · , N ) is the adjacency matrix. Each edge e ij = (v j , v i ) describes the information flow from AGV j to AGV i . If e ij ∈ E, then a ij=0 , i.e. there is no information flow between AGV i and AGV j . It is assumed that noinformation flows within any AGV, i.e. a ij=0 . Then, the set of neighbors of AGV i can be expressed as From AGV i to AGV j , the path is an ordered sequence of edges , all of which are elements in E. If an AGV i has an edge to any other AGV in the directed graph, this AGV is the root of the directed spanning tree of the MAS. The graph Laplacian matrix L = [l ij ] ∈ R N ×N can be defined as: If the sum of each row equals (1), then a nonnegative square matrix is a stochastic matrix. If lim k→∞ A k = 1 n c T (c is an n × 1 vector of constants), then the matrix is stochastic indecomposable and aperiodic (SIA).

C. FIRST-ORDER DYNAMICS MODEL OF MAS
The top view of the global coordinate system is presented in Figure 3, because this paper only considers the AGV control on the approximately 2D plane. Note that the azimuth angle θ ∈ [−180 • to 180 • ] lies between the heading of the AGV and the positive direction of the x-axis The clockwise direction is defined as the positive rotation direction. Assuming that the motions along the x and y-axes are decoupled, the MAS dynamics on the x and y-axes can be respectively expressed as: where,x i (t),ŷ i (t) ∈ R are the coordinates of each AGV; uˆv i (t) and uˆy i (t) ∈ R are the control inputs along the x and y-axes, respectively.
Here, the AGVs are sampled at flexible periods to determine their positions. Let a1 = {t 0 , t 1 , · · · , t k }, k ∈ N be the sequence of sampling periods. It is assumed that each sampling period is an integer multiple of the minimum sampling period h, and is selected from a finite set m k h = t k+1 − t k ∈ = {n 1 , n 2 , · · · , n τ h}, ∀k ≥ 0 (n i is an integer). Through flexible-period sampling, the position and control input of AGV i at time k can be respectively derived from the zero-order hold as the following discrete equations: Equation (3) can be rewritten as: where, The decoupling model (2) is controlled by the following protocol: The state transition of the AGV can be described as: Under the control protocol (6), the heading of the AGV can be adjusted by changing the speeds of the driving wheels, bringing changes to the AGV trajectory.
Then, the MAS coordinate system was defined. As shown in Figure 3, the MAS coordinate system has two axes: x and y -axis. The x -axis points to the heading of the AGV, and the y -axis is coaxial with the two driving wheels. The two axes are perpendicular to each other.
Let v l and v r be speeds of the left and right wheels of the AGV, respectively. Being a limited differential underdriven system, the AGV cannot move in the axial direction. Hence, more constraints should be added to equation (6). On this basis, a CSRR control algorithm (Algorithm (1)) was developed an alternative to the decoupling model (2).
For simplicity, the planar section of the MAS was divided into four regions ( Figure. 3), according to the MAS coordinate system. Then, the state of each AGV can be updated by Algorithm (1). At time k, the azimuth angle θ cannot be calculated unless the rotation direction is fixed. If the steering VOLUME 8, 2020 angle surpasses the response ability of the drive system, the corresponding AGV needs to decelerate, adjust its angle and accelerate again. This process inevitably breaks the continuity of motion, suppressing the operating efficiency of the AGV. To solve the problem, a limit on steering angle θ was set based on the acceleration capacity of the motor. In other words, the adjustment in each sampling period was broken down into several parts to ensure the continuity of AGV motions.Once the angle is adjusted, the AGV will move along until the next sampling periodor reaching the target position.
If v < v max , the movingspeed will be limited to v = v max .

D. FIRST-ORDER CSRR CONTROL
Through the above analysis, the CSRR control algorithm was developed for the first-order control of the MAS. The specific steps of the algorithm are presented in Algorithm (1).
Direction adjustment and forward movement are two key actions in AGV control. Thus, each sampling period contains a rotation time [t k , t k + T ) and a forward movement time [t k , t k+1 ). According to the constraints on output speed of the two driving wheels ( v l ≤ v max and v r ≤ v max ), the desired azimuth adjustment can be obtained as θ max ∈ [−90 • , 90 • ]. The head or tail of the AGV should point away from the current target as much as possible, unless it is required to rotate by 180 • in special operations (e.g. replenishment) [36].
If T ≤ t k+1 − t k , ∀k = 0, · · · , N , the position of AGV i (x i (t k+1 ), y i (t k+1 )) will fallon the line between (x i (t k ), y i (t k )) and (x i (t k+1 ),ŷ i (t k+1 )) at time t k+1 . Thus, we have , · · · , x N (t k )] T and y(t k ) = [y 1 (t k ), y 2 (t k ), · · · , y N (t k )] T , then: where The next step is to discuss the collaborative control of multiple AGVs. The β k , k = 1, 2, · · · satisfying 0 < β k < min i (1/λ i (t k )m k h j∈N i a ij (t k )) was selected, which makes k in equation (10) a stochastic nonnegative matrix with positive diagonal elements.The stochasticmatrix M ∈ R N ×N is SIA, if it has diagonal elements and itscorresponding directed graph has a spanning tree [17]. The same serve as necessary but insufficient conditions for k .
Let S = S 1 , S 2 , · · · , S k be a set of finite SIA matrices with the same size m×n Then, any sequence S im , S im−1 , · · · , S 1 of matrix products of any length is SIA. If the product sequence S im , S im−1 , · · · is infinitely long, the existence of a column vector c makes: if t = t k then 2: Choose the value of β k that satisfies 0 < β k < ) represent the current position of AGV i , then 10: if P k ∈ Region 1 then R DIR = 1, M DIR = 1 and θ id (t k ) = θ iT ; 12: else if P k ∈ Region 2 then R DIR = −1, M DIR = 1 and θ id (t k ) = θ iT ; 14: else if P k ∈ Region 3 then R DIR 1, M DIR = −1 and 16: If the switching directed graph (SDG) G(t k ), k = 1, 2, · · · has a spanning tree at each time, and there exist β k and k satisfying the above conditions, then the AGVs with firstorder dynamics can be coordinated collaboratively with flexible sampling periods. In our model, the length of rotation time T = (|θ i (t k ) − θ id (t k )|πd)/(360 ×v) can be changed within [t k , t k+1 ) by adding or reducing the rotation speed of wheelsv (v ∈ [0, v max ])), such that λ i (t k ) ∈ (0, 1]. Note that θ i (t k ) is the azimuth angle of AGV i at time t k , θ id (t k ) is the target azimuth of AGV i , and d is the diameter of the driving wheels. In this way, the k can be confirmed as SIA: Hence, it is possible to realize the collaborative control of the AGVs in the MAS.

III. AGVPOSITIONING UNDER SEMI-STRUCTURED SCENARIO
In MAS collaborative control, the position of each AGV is an important input parameter. Based on RGB images, the visual positioning is generally effective. However, this positioning approach relies on camera calibration, and its accuracy is easily affected by external factors like light. The positioning accuracy can be improved by binocular visual positioning to a certain extent, especially when the AGV is blocked. Compared with visual methods, infrared depth sensing is an institutive and accurate means to identify the spatial position and distance of AGVs. Unlike indoor environment, the Dutch greenhouse is a semi-structured environment with various disturbances. Among them, sunlight and other heat sources severely disturb the sensing operations, especially infrared depth sensing. For the accuracy of MAS collaborative control in semi-structured environment, this paper puts forward the EDIFP scheme ( Figure 4), which covers the following three parts: classification of features, i.e. regions of interest (ROIs), based on RGB image, modelling spatial coordinates through camera calibration, and 3D spatial reconstruction and AGV positioning based on depth image.
In our scheme, an Intel RealSense D415 depthcamera (effective range: 10 m; field of view: 69.4 • × 42.5 • ) is equipped with a visible light sensor and an infrared depth sensor. In this way, the camera can simultaneously capture RGB images (1920 × 1080) and depth images (1080 × 720) at 30 and 90 frames per second (FPS), respectively.

A. CAMERA CALIBRATION
Before positioning, the main objects in the monitoring space ( Figure 5 (a)) must be fully calibrated, including the ground, the walls and the top surfaces of theAGVs. In this paper, the calibration is performed using a GP290 12 × 9 calibration board. Through calibration, the internal and external parameters of the camera were optimized, and the feature plane of the main objects was generated ( Figure 5 (c)).

B. FEATURE IMAGE SEGMENTATION
Currently, images are often segmented by frame-difference method, CamShift method, etc. For images containing AGVs, these methods are outshined by deep learning-based semantic segmentation in accuracy and adaptability. Here, the RGB image is segmented by the semantic segmentation method proposed by Chen et al. [40]. The feature labels were trained under the semi-structured environment, and used to segment the main objects more accurately as shown in Figure 5(d).

C. THE EDIFP PROCESS
(1) 3D point cloud transform. To reconstruct the 3D monitoring space, it is necessary to transform the data of the depth image into 3D point cloud. In our scheme, the depth image has the same size and resolution as the RGB image. The size of the depth image can be expressed as Size img = (w, h), where w and h are width and height, respectively. The coordinates of depth image data D ij can be described as ( Then, the global coordinates (X ij , Y ij ) of the data point can be computed by triangular similarity: where, f is the focal length of the camera. Then, the transformed dataset was added the data of the corresponding RGB image, forming a standard 3D point cloud {P PCD (x, y, z, R, G, B)}.
The interferences and measurement errors should be eliminated to obtain the ideal distribution of the point cloud. Therefore, a grid model was established for the point cloud before data segmentation, and the vector of each data point VOLUME 8, 2020 was calculated. As shown in Figure 5(b), the sampling plane is smooth locally, although the point cloud is not uniform. For a point p in the cloud, an approximate plane Plane approx can be created based on its k neighboring points: (12) where, n vct represents the normal vector of Plane approx ; d O is the distance between the origin of the coordinates and Plane approx .
Through iterative computing of the set of neighboring points {q j (x j , y j )}, j ∈ [1, k] of p i , the approximate tangent plane of p i and the approximate normal vector can be obtained. In this way, the error function f err can be minimized: The value of f err is negatively correlated with the distance from the fitted plane to the ideal plane. Thus, the point cloud was divided into several approximate planes, each containing a set of neighboring points. (

3) Plane fitting and positioning.
Under the semi-structured environment, the plane fitting should consider the ground, the walls and the AGVs. The normal vector of the fitted plane should not deviate far from that of the feature plane. Otherwise, the fitting result should be discarded. In this paper, the distribution of each approximate plane is clustered by Gaussian mixture Model (GMM) [37] and neural network (NN) [38]. Then, the remaining planes were merged with theknown feature planes. Let PF and PV be the feature plane and the plane to be verified, P PF and p PV be the points on PF and PV , and n PF and n PV be the normal vectors of PF and PV , respectively. Then, the planes can be filtered by: where, θ, d and dv are the angle, distance and distance vector between PF and PV , respectively; THR and d THR are the error thresholds for the angle and distance, respectively; {P PV } is the set of points on the PV ; {P ROI } is the set of points in the ROI, i.e. only the data of the ROI are verified.
Finally, the AGV position {P 1 RGB , P 2 RGB , · · · , P n RGB } from the RGB image and that {P 1 depth , P 2 depth , · · · , P m depth } from depth image need to be fused together. Let {w 1 RGB , w 2 RGB , · · · , w n RGB } and {w 1 depth , w 2 depth , · · · , w m depth } be the fusion weights of the AGV positions from RGB and depth images, respectively. Then, the position fusion formula can be expressed as: Figure 5(e) provides the fusion results on AGV positions. The proposed EDIFP schemeachieveda greater than 0.03m accuracy in actual tests.

IV. EXAMPLE ANALYSIS A. OVERVIEW OF TEST PLAN
Based on the BigPan, the authors constructed a test platform forMAScollaborativecontrol. As shown in Figure 6, the test platform is deployed in a 4×4 m room with south-facing windows. The boundaries of the test site were marked in yellow to control the AGV motions. In daytime, strong sunlight can enter the room, and change rapidly with the elapse of time. The indoor and outdoor temperatures are 36 • C and 38 • C an enter the room, and chrelative humidity is about 85%. These conditions are similar to those of the Dutch greenhouse. The high temperature and humidity have a great impact on the data collection by the infrared depth sensor.
Three BigPan AGVs were adopted for our tests. Among them, AGV2 (diameter: 55 cm; height: 77 cm) is the fourth generation robot, carrying a 140 L medicine tank and a complete spray system, while AGV1 and AGV3 (diameter: 55 cm; height: 23 cm) are the fifth generation robots. The latter two AGVs do not have any spray system. The maximum speed of the three AGVs stands at 0.8 m/s. To improve the dynamic performance, the upper limit of speed is generally set to 0.65 ∼ 0.7 m/s. In our tests, the maximum speed is set to 0.6 m/s, and the maximum acceleration, to 0.2 m/s 2 .
To enhance the positioning accuracy, the two webcams (image sensors) on the Intel RealSense D415 depth camera were calibrated by a calibration plate, and a spatial coordinate model was established before our tests. The two sensors can capture RGB image and depth image, and support the proposed EDIFP scheme. During the tests, a position calibration plate was placed on each AGV, with its center directly above that of the AGV's upper surface.
The host is a laptop (CPU: Intel Core i7-4700MQ, 2.4GHz; RMB: 8GB; GPU: GeForce GTX 765M) that connects multiple external devices (e.g. a wireless router). The host and the multiple AGVs are linked up wirelessly, forming an MAS communication network.

B. MAS COMMUNICATION NETWORK
The communication between AGVs is the key to the effective control of the MAS. Before the tests, a wireless communication network [39] was implemented. As shown in Figure 7, the network encompasses three main parts: a Host, a wireless router and a set of AGVs. The workflow of the communication network is as follows: Step 1. Set up the IP address of the router (e.g. 192.168.1.1), and expose it to all authorized AGVs.
Step 2. Enable the dynamic host configuration protocol (DHCP) of the router, and assign a new IP address to newly entered AGV. Rather than preset an IP address to each AGV, each AGV is identifiedby its name and hardware ID. Similar to media access control (MAC) address, this identification strategy prevents the waste of address resources.
Step 3. Fix the IP address of the host and keep it in the same network segment as the router (e. g. 192.168.1.100). In this way, the other AGVs can send a registration request to the host and joint the network. Then, the system will enter the ready state. The host manages the list of AGVs via the agent manager. All AGVs in the online state are included in the list. Their state is maintained periodically by a thread.
Step 4. Power on the AGVs in turn and connect them to the wireless route. Each AGV will send a registration request to the host, after obtaining the automatically assigned IP address. Once the request is accepted, the AGV will be connected to the MAS. For the three AGVs, their IP addresses are 192.168.1.101 -103. The login/logout of the MAS is of critical importance. Once an AGV accesses the network, Algorithm (1) will be implemented to execute the control. Considering their similarity, the login and logout were merged into one module ( Figure 8). The login/logout process can be split into 7 steps: (1) The client and server establish a connection through socket and OSLIP protocol; (2) The MAS user interface (UI) sends a registration notification to the agent manager, once the client is successfully connected; (3) The agent manager creates aproxy object; once authenticated, the object will be added tothe list of AGVs; (4)Once created and activated, the AGV sends a verification message to the agent entity; (5) When the entity verifies the message, the online state will be replied; (6) The AGV will notify the agent manager to update the entity state and update the UI; (7) The logout process is basically the same as the login process, except for the message update. The offline state is divided into normal and abnormal situations. Normal offline is a request from the agent entity (the server may also require the entity to logout). Abnormal offline often occurs at network failure. There are two solutions to network failure: (1) Socket exception will be detected, triggering the offline event; (2) A daemon thread is set in agent manger, which periodically requests all AGVs in the list to confirm if the agent entity is still online, and removes the offline entities. More details of MAS communication are available in Zhang's research [39].

C. SIMULATION TEST
Considering the complexity of MAS collaborative control, our CSRR control algorithm was simulated on Simulink before offline deployment. Specifically, the data obtained by each sensor were visualized on a control program written on Simulink. The control commands were issued by the OSLIP protocol, with the M language. The Simulink execution supports both single or multiple AGVs, and allows real-time adjustment of controller parameters and measured values, providing the user an intuitive view without complex coding. As shown in Figure 9, the simulation framework mainly consists of stream server block, stream client block, and agent controller. The first two blocks form a communication flow in a client/server (C/S) architecture, under which the sensors in each AGV exchange information. To monitor the MAS state,  the AGV communication is relayed by the stream server, although the AGVs are capable of direct communication.
Implemented in the host, the CSRR control algorithm computes the control outputs based on the state of each AGV, i.e. the speeds of the left and right driving wheels. The micro control unit (MCU) of BigPan will respond to each received control command.

D. RESULT ANALYSIS
Under the conditions in Section 4, the control effect of the CSRR control algorithm on the MAS of AGVs with firstorder dynamics was verified through tests. The trajectories of the three AGVs are displayed in Figure 10.
To prevent the AGVs from collision, a virtual OBB (diameter: 65 cm) was designed around each AGV (diameter: 55 cm). When the three OBBs are in contact with each other near the origin (target point), the three AGVs are considered to have the same system state.
The OBBs of different AGVs might overlap each other within the allowable range, due to errors in measurement and control. Because the OBB has a larger diameter than the AGV, the overlapping distance d is smaller than d < 0.05 m, within the range of allowable error. In actual application of the BigPan, the sensor diameter is larger than the OBB diameter. Thus, it is unlikely for AGVs to collide with each other in actual operations. Figure 11 and Figure 12 show the time variation of the speeds and azimuth angles of the three AGVs, respectively. It can be seen that the three AGVs were adjusted simultaneously, and moved very close to the target point.

V. CONCLUSION
This paper attempts to collaboratively control an MAS of multiple under-driven AGVs withflexible sampling periods and changing communication topology. A Dutch greenhouse with semi-structured environment was taken as the research background. Based on the directed graph, a first-order collaborative control algorithm called the CSRR was developed for the under-driven AGVs, which guarantees the convergence of system states. In addition, the EDIFP schemewas designed to mitigate the disturbances on the control algorithm, arising from flexiblesampling periods and data loss. This scheme combines RGB and depth images, and thus enhances the positioning accuracy of AGVs, despite the light and heat sources in the semi-structured environment. To verify its performance, the CSRR control algorithm was applied to a simulation on an MAS of three BigPan AGVs. The results on AGV trajectories, speeds and azimuth angles demonstrate the effectiveness of our algorithm. Coupled with the test data [36], the simulation results provide a valuable reference for the largescale application of MASs in semi-structured environments. Besides high accuracy and smoothness, the CSRR control algorithm reduces the number of interventions in the drive system and suppresses the loss of drive components, improving the system durability. The preliminary work on the reliability and durability of the drive system can be found in Zhang's research [35].
TIANFAN ZHANG is currently pursuing the Ph.D. degree with the School of Automation, Northwestern Polytechnical University (NWPU). He is currently working as a Lecturer with Hubei Engineering University. His research areas are in consistency control of systems engineering, multiagent in complex environment, self-driving vehicle, distributing control systems, embedded systems, pattern recognition and deep learning, public safety analysis, and management.
DAN LIU is currently an Assistant Lecturer with the School of Economics and Management, Chang'an University, Xi'an, Shaanxi, China. She is also a Visiting Scholar with the Department of Civil, Environmental, and Geomatics Engineering, Freight Mobility Research Institute, Florida Atlantic University, USA. Her research areas are in intelligent transportation systems, logistics and transportation management, and big data analysis.
XIAO JING is currently pursuing the Ph.D. degree with the School of Cyberspace Security, Northwestern Polytechnical University (NWPU). His research areas are in big data, crime prediction, machine vision, and machine learning.
ZHE LI received the Ph.D. degree from the School of Automation, Northwestern Polytechnical University (NWPU). He is currently an Associate Professor with Hubei Engineering University. His research areas are in consistency control of multiagent in complex environment, multirate with multiagent systems and distributing control systems, big data, and crime prediction. VOLUME 8, 2020