Ant Colony Optimization With an Improved Pheromone Model for Solving MTSP With Capacity and Time Window Constraint

The optimization of logistics distribution can be defined as the multiple traveling salesman problem (MTSP). The purpose of existing heuristic algorithms, such as Genetic Algorithm (GA), Ant Colony Algorithm (ACO), etc., is to find the optimal path in a short time. However, two important factors of logistics distribution optimization, including work time window and the carrying capacity of the vehicle in distribution system, have been ignored. In this paper, we consider the influences of time limitation of modern commercial logistics and carrying capacity of the vehicle on the logistics optimization, and then propose a MTSP with constraints of time window and capacity of each salesman. We design a novel hybrid algorithm by combining the minimum spanning 1-tree with ACO to find the optimal solution. In addition, we improve the pheromone update rules to increase the search efficiency of ACO algorithm. The experiments show that the novel hybrid algorithm achieves a shorter path than the other algorithms.


I. INTRODUCTION
Logistics distribution industry becomes more and more important due to the rapid development of the e-business. Nowadays, there are many challenges in logistics distribution industry. For example, Nov. 11 th is a peak date for online shopping that all the products online have a big discount and people go on a shopping spree, so there are billions of express items needed to be delivered during that time. Therefore, it is important to deliver the products to customer within a short time. Service composition [1]- [3] is a way which can combine all the logistics services processes efficiently. Another challenge is the balance between paths and items and it is more difficult.The logistics services process in China is to deliver all items from the head office to each brand. At the same time, the optimization of time cost and road distance, the amount of items, the number of transport vehicles as well as their carrying capacity are taken into account.
The associate editor coordinating the review of this manuscript and approving it for publication was Shouguang Wang . MTSP is a generalization of the well-known traveling salesman problem (TSP), where at least two salesmen could be used to deliver items in the solution. Obviously, the logistics distribution question is the same as a multiple traveling salesman problem with constraints. Moreover, MTSP could be extended to a wide variety of vehicle routing problems (VRP) according to incorporating a number of additional side-constraints. In addition, finding a quick and efficient way to transport goods and materials in earthquake relief work or military operation is significant.
There are many solutions to solve TSP like Tabu search, simulated annealing, genetic algorithms. The ant colony system (ACS) is improved by this paper which is also a well-behaved solution. It is a kind of the group of metaheuristic methods. This idea was firstly proposed in the early 90s by Dorigo [4]. ACO is generated by simulating the real behavior of ants in nature and it is widely used to solve practical problems [5]- [7]. Many different transforms of ACO have been proposed. On the basic of these, a new optimal ACO with some improvements is proposed in the paper.
First of all, the minimum spanning 1-tree [8] can be applied to construct the path in MTSP. Secondly, the improved model and pheromone update rules make the algorithm more effective.
Our main contributions are as followings: (1) Redefining multiple traveling salesman problem with time window and capacity of each salesman, which comes from real life problem such as logistics industry; (2) Building an improved pheromone model with new pheromone update rules; (3) Combining the minimum spanning 1-tree with ACO based on the relationship between the optimal TSP tours and spanning tree.
The rest of the paper is organized as follows. Section II shows the related work about TSP and ACO on solving TSP. In the section III, the definition of MTSP and possible solutions for MTSP are discussed. Section IV discusses the algorithm of ant colony system. Section V proposes an optimal ACO algorithm on the basic of previous section. Section VI shows the evaluation and experiment of the improved algorithm. Section VII concludes the paper and outlines the future work.

II. RELATED WORK A. RELATED WORK ON TSP
TSP is a typical branch of metaheuristic algorithm and there is a lot of research on it. We list a number of representative works recently, and there is an overview article by Panwar and Gupta [9], where earlier works are discussed in detail. They conclude some TSP for solving soft computing techniques.
(1) Ant Colony Optimization has been widely used in proving the complexity of combinatorial problem. The movement of each ant is effected by the intensity of pheromones contained on the path. The ACO algorithm can calculate good results in a short time after a few iterations [10].
(2) In [11], Ding et al. try to look for the shortest Hamiltonian cycle in all clusters, then remove a selected edge in every cycle to build an intra-cluster path. At last, attach the whole intra-cluster paths in a particular sequence to organize a complete tour. A novel method called Two-level Genetic Algorithm for the Cluster TSP is proposed, which is a good choice to alter an extensive TSP into a CTSP that can be settled by the TLGA. This method is well applied to large-scale TSPs.
(3) In [12], the conventional particle swarm optimization (PSO) is improved. This method takes the heuristic factor, crossover operator and adaptive disturbance factor into account. Then, the authors propose a novel hybrid discrete PSO algorithm which can improve the search performance in convergent speed and precision. This method can be applied to solve the problem of path optimization in TSP.
(4) In [13], random keys are introduced to solve the coding problem of genetic algorithm. Random keys play an important role in the problems where permutations of the integers are required as well as feasibility problems caused by traditional one-or two-point crossover.
Also, there are some other algorithms [14]- [18] related to TSP. For example, in [18], Xu et al. redefine a general colored traveling salesman problem and propose a Delaunay-triangulation-based Variable Neighborhood Search algorithm. The new algorithm performs well in large-scale problems.

B. RELATED WORK APPLYING ACO TO TSP AND MTSP
In this section, we review the literatures that ant colony optimization is used to settle traveling salesman problem as well as multiple traveling salesman problem and figure out their pros and cons and clarify the remaining gaps and challenges for further investigations.
In [19], Mueller and Kiehne present a hybrid way by combining artificial ants with neurons intelligently. In the paper, they test diverse parameter combinations with the purpose of getting the best performing parameter settings. They subsume the obtained insights into an intelligent architecture including Ant Colony Optimization and Self Organizing Map. In [20], a resource allocation method (LET-ACO) is proposed and about 40% energy consumption can be avoided in LET-ACO.
In [21], Olivas et al. propose a method in which a fuzzy system is used to improve the parameter adaptation in ACO. The dynamical adaption of the parameter is responsible for the evaporation of the pheromone trails in the algorithm. Basically, the fuzzy system is responsible for the abilities of the algorithm for exploration and exploitation in the search space and the goal is to get better results. They compare their method with the original algorithm and the result shows that their method performs better than the original algorithm. But they do not have enough evidence to reject the null hypothesis in Burma14 problem. They also propose a fuzzy system for parameter adaptation in the Ant Colony Optimization (ACO meta-heuristic) in [22]. In this case, they use the dynamic adaptation parameters of Alpha and Rho to decide the performance of ACO whether in a global or in a local search.
In [23], Neyoy et al. explore a novel method of diversity control in ACO. To avoid or slow down full convergence, the authors use the dynamically varying alpha parameter. According to the performance of different variants of the ACO algorithm, they choose the best one as the basis of the proposed approach. Moreover, a convergence fuzzy logic controller with the purpose of maintaining diversity to avoid premature convergence is created.
In [24], Wang et al. propose an ant colony algorithm with neighborhood search called NS-ACO to solve the problem of dynamic TSP which is composed by random traffic factors. They take the advantage of the short-term memory to enhance the kinds of solutions. Also, to find the optimized solutions, they used the operations of swap, insertion and 2-opt moving. As a consequence, the experiments show that the method of NS-ACO performs better than conventional ACS and the ACO with random immigrants. VOLUME 8, 2020 In [25], Elloumi et al. propose an improved PSO algorithm combined by ACO and apply it to TSP. According to the comparison experiment, the performance of improved PSO algorithm is perfect. Generally, ACO and PSO have some advantages in solving TSP. On the one hand, the original PSO algorithm can be applied well to solve the problem of moderate dimensions and search space. However, the original PSO algorithm cannot be applied to solve the problem when it is in a more complex search space. They improve the ACO algorithm and enhance the performance to solve complex problems.
As shown above, ACO and its improved algorithms perform well in TSP and MTSP. And there are many other nature-inspired methods to solve this kind of problem. Because of the lack of some constraints such as time and load for each salesman, these existing methods cannot be applied well to the actual industry. In this paper, we consider these constraints when apply MTSP into courier industry or other realistic fields and find the optimal path quickly to reduce the total cost.
Besides, there are also some other algorithms that can inspire some ideas on solving MTSP [26], [27]. In [28], Huang et al. propose a niching memetic algorithm for Multi-solution Traveling Salesman Problem. In order to enable the parallel search of multiple optimal solutions, this algorithm is characterized by a niche preservation technique. They also use an adaptive neighborhood strategy to improve the search efficiency. In [29], Jiang et al. propose a new efficient hybrid algorithm for large scale multiple traveling salesman problems. They require the minimum and maximum number of cities that each salesman should visit. Then they combine partheno genetic algorithms and ant colony algorithms which is sufficiently effective. In [30], Vandermeulen et al. want to make the workload balanced so that they use average Hamiltonian partition problem. By using AHP, they improve the partitions of a graph and guarantee to improve the solution?s quality. This algorithm is good at solving large problems within a better run-time. In [31], Venkatesh et al. propose a multi-start iterated local search algorithm for the maximum scatter traveling salesman problem. They develop insertion and modified 2-opt moves on two local search algorithms.

III. PROBLEM DEFINITION AND FORMULATION
TSP is a typical combinational optimization problem that belongs to NP complete problem [10]. The aim is to find the shortest path by visiting every node once and then returning to the start node in a complete weighted graph G with n nodes and n(n-1) edges. Furthermore, MTSP could use more than one salesman which is different from the TSP with only one salesman. When considering the carrying capacity and time windows, this problem becomes more meaningful in real life.
The MTSP with constraints can be defined as follows. First, given a set of nodes, arrange m salesmen at a single depot node. Then, finding tours for all m salesmen, and they all need to start and end at the depot. Meanwhile, the other nodes can be visited exactly once and the cost of visiting all nodes will be least. The two constraints are: every node is visited in a fixed time window and every salesman should take items as many as possible [32]. Specifically, for a city i, the salesman should visit it in a time window E i , L i , where E i is the earliest time that the salesman could visit and the salesman should visit city i before L i . And for every salesman, the maximum items they can carry is M .
Comparing the TSP with the MTSP, it is obvious that MTSP is more adequate to simulate real life situations, and it is capable of handling plenty of salesmen problem.
MTSP can be applied in the following context: print scheduling, workforce planning, transportation planning, mission planning, production planning, satellite systems, etc. Each context includes numerous types of real applications.
Consequently, there are various solutions proposed for MTSP. At first, some accurate solutions like cutting plane, branch and bound, and Lagrange relaxation were proposed. Then, heuristics are applied to this problem. Moreover, evolutionary algorithm, simulated annealing, tabu search, genetic algorithms and neural networks also have a good ability on working out the MTSP.

IV. ANT COLONY OPTIMIZATION A. INTRODUCTION OF ACO
Ant colony optimization is a heuristic algorithm which is introduced into many combinatorial optimization problems due to it is one of the highest performance computing methods for MTSP [33]. Many efforts have been made on ant colony optimization techniques in different areas. It takes inspiration from ants' social behaviors of finding the best way between food and the ant nest. Ants can communicate with each other about amount of food and the distance of route. The process can be divided into two parts and a positive feedback mechanism is applied by the ants [34]. As shown in Figure 1, (a) and (b) are the first part, while the second part includes (c) and (d). At the first part, each ant chooses routes randomly and releases pheromone in the routes. A global pheromone updating rule is applied when all ants have finished their tours. Pheromone accumulates at a higher rate on the shorter path, so that more ants begin to choose the shorter path which has a higher intensity of pheromone. At the second part, ants affect each other by pheromone which is the main factor to lead the ants to find shorter route. Then the process is iterated to find the global optimal solution. In ACO, artificial ant replaces the real ant. They both can affect surrounding environment such as the intensity of pheromone in the path [35].

B. ORIGIN ANT COLONY ALGORITHM
Some definitions are introduced in Table 1, in order to describe the problem clearly. At first, there is no pheromone on every path and τ ij (0) = const. Ant k(k = 1, 2, . . . , m) chooses path by the intensity of pheromone on its way of moving. Set tabu k (k = 1, 2, . . . , m) to mark the visited cities and it will be updated according to the transfer of ants state. During the iterative process, p k ij (t)(the state transition probability) relies on both the amount of information and the existing heuristic information on the path [36].
In the formula 1, J k (i) = {C − tabu k } is a collection of cities that ant k can visit in next step. α denotes the extent of the group cooperation of ants. That is to say, The bigger α is, the more likely an ant chooses a path passed away by its companion and the better group cooperation of the whole ant has. β is expected heuristic factor which shows the relative importance of visibility and the value of choosing a path. The bigger β is, the higher state transition probability closes to greedy criterion. η ij is a heuristic function, which corresponds to β.
For each ant, a smaller d ij can decide a bigger P k j (t) that is the expected value of moving from i to j.
Considering that too much accumulation of pheromone will cover up the heuristic information, so that after ants finish their tours, pheromone needs to be weakened. At t +n, set the pheromone on path (i, j) as follows: ρ is pheromone volatilization coefficients, and 1 − ρ is pheromone remaining factor. Avoiding too much pheromone left, set ρ ⊂ [0, 1). τ k ij (t) is pheromone on path (i, j) which released by ant k every time it finished one route. Ant-Cycle Model applied in this method is as follows: Q represents the concentration of pheromone, which can decide the rate of algorithm. L k is the distance between cities where ant k passed.
Steps of Origin Ant colony algorithm: (1) Initialization Parameters Divide m ants into n cities, initialize pheromone τ ij (t) = const on every path (i, j) and it is constant. Set τ k ij (0) = 0, and other parameters like N c max , α, β, ρ, Q and so on.
(2) Iteration process While (N c < N c max ) do for i = 1 to n − 1 do Traverse n cities for k = 1 to m do For m ants for j = 1 to n do For n cities ant k chooses next city j according to formula 1 then k moves to city j and add j into tabu k ; Calculate tour length of every ant, and update the pheromone by formula 3, 4 and 5 on every path; N c = N c + 1; end while.
(3) End and output the result.

V. NEW ANT COLONY OPTIMIZATION A. PROBLEM DESCRIPTION BY GRAPH THEORY
In order to improve the search efficiency and obtain the shortest path. Thus, this part, we propose a new algorithm which uses the minimum 1 tree (1-MST) to construct the path. According to the relationship between 1-MST and optimal route, we establish the optimal search library by proximity is established and uses simplified pheromone diffusion model in this paper. In addition, time window and capacity constraints make it more practical significance.
The problem is described by using graph theory as followings. There is a weighted connected graph G = (V , E, D). The set of vertex in the tree T. V = {v 0 , v 1 , . . . , v n−1 } that represents n cities. Every point has two pairs of weights, and these are carrying capacity weight pair w 1i , w 2i and time window weight pair E i , L i . In this problem, w 1 and w 2 denote the number of carrying items that needs to take away or bring; E i is the earliest time the salesman can arrive and L i is the latest time salesman must arrive. E is a edge set that connects different vertexes. D is nonnegative weights matrix that shows the distance between two cities. The capacity of a salesman is W and the number of original salesman is M . By the total carrying items M total , W = M total /M is obvious.
Tree T is the 1-MST of Graph G =< V , E >, the total weights of which is L(T ). T + (i, j) denotes the 1-MST that includes the edge e(V i , V j ). So the proximity of edge e(V i , V j ) VOLUME 8, 2020 is α(i, j) = L(T + (i, j)) − L(T ). We can conclude that α(i, j) ≥ 0 only when e(V i , V j ) is an edge of 1-MST. The smaller α(i, j) is, the more likely it exists in the shortest route. As a result, we need to build a collection of best choice for each city. N cities of smallest proximity should be added into the original collection. When initializing pheromone matrix, we can set it in terms of the collection. Finally, it can weaken the bad influence of first random path that easily leads into a local optimal solution. Now, we have to divide the MTSP into M TSP by means of copying start vertexes V k (1 ≤ k ≤ M − 1). And the distance M between start vertexes is infinite and the distance between duplicate vertexes and other vertexes is the same as the first start vertex. It is easy to solve the problem by using the methods on TSP.
Next, we use the Kruskal method to get a minimum spanning tree. At the same time, we add the start vertex and two shortest edges connected with it into the tree. The new tree is called 1-MST. Then the T + (i, j) and the proximity between edges are easily to get. When initializing the pheromone, each edge will be given the different original value by the proximity α(i, j). In this way, the edges in the 1-MST are more easily to show up in the best solution, and it can accelerate convergence. Now, MTSP can be split into n TSP.

B. PATH SELECTION
Every ant starts from an empty solution and chooses cities judging by pheromone and heuristic information. The possibility of the next city chosen by ant k is where is the average degree of capacity, Furthermore, whether a city can be selected should depend on two constraints. First, the carrying items cannot be more than the capacity, at the same time, try to put more items.
Secondly, every ant has to pass a city at a special time window [E i , L i ]. In order to make it easier, set penalty value D 1 for early arrival and D 2 for late arrival D 1 < D 2 . Obviously, we can get the penalty value d and total penalty value D. For every city, if the value is too big that is d i > d or D > D max (constraint 3), this city cannot be the next city.

C. MODEL AND UPDATE RULES
When an ant searches the path, it releases pheromones that directly have influence on two cities along the path. At the same time, the pheromones spread out from the edge, which also have impact on the behavior of the other ants in the vicinity, in order to reduce the interference when other ants elect the next city. So it can improve the convergence rate [37]. In Figure 2, X represents the distance between the source of pheromones; Y represents the amount of information; Point O shows the source location. The pheromone of point O is D max ; θ < 90 • which is fixed; The highness of cone: h; Outward diffusion radius: r, and r = h tan θ ; σ is the distance between point C and O; So the pheromone in C is When an ant passes an edge (i, j), it will releases the pheromone which will affect the proximal cities. The city m is within the range of the diffusion, and the formula of pheromone on path (i, j), and path (i, m), are as followings: The pheromone in city j spreads to city k is the same as formula for city i.
, D max = γ τ k ij and σ = d im (City i is the source) or σ = d jm (City j is the source). Avoiding falling into the local optimum, replace Q by Q(t): And set volatile coefficient as follows: And now we can get the new rule for pheromone changes:

D. IMPROVED ALGORITHM
The proposed algorithm is combined with 1-MST, improved pheromone model and new update rules. More realistic conditions are taken into account. First of all, set up parameters. Maximum iterations N max , ant number is Ant = (n + M − 1)/1.5, capacity of ant W = 40, solution of ant is S k , best denotes the shortest route in one iteration, and set α = 1, β = 2, ρ = 0.9. The algorithm is described as follows: (1)Read the data and process it. Prepare the required data.
(2)Build 1-MST and get the proximity, then initialize pheromone matrix PM , the original solution S k and the best solution best.
(3)Initialize the ant colony, and calculate the heuristic information of every city.
(4) 1 while cycle times are less than N max 2 { 3 for for k = 0, 1, · · · , Ant − 1//every ant 4 for k = 0, 1, · · · , city do//every city 5 In order to evaluate the method we proposed, we took several MTSP problems from the TSPLIB website [38] (the best results are shown in Table 4) and did the test. Capacity information and time windows are added into them. The experiments results on modified figures are in Table 2. As we can see from Table 2, the best solution of Berlin52+, Eil51+ and Eil76+ are 7679, 437 and 548 respectively, the optimum solution of Berlin52, Eil51 and Eil76 with no constraints are 7542, 426, 538 respectively which are shown in Table 3. From this experiment we can see that three instances have been solved and the results are closer to best results without constraints, so our algorithm performs well when constraints are taken into consideration.
Due to other algorithms of solving MTSP with no constraints, we set all constraints invalid in our algorithm. Then we compare our proposed algorithm results with ACO in the algorithm convergence and experiment results are output as shown in Table 3. The relative error (R i − (1)) 2 /n where R i is the length of every route.
As Table 3 shown, new ACO can get better results than normal ACO. Next, we compare our method with other algrithems such as improved genetic algorithms (IGA) [39], invasive weed optimization algorithm (IWO) [40], improved partheno genetic algorithms (IPGA) [40] and the instance in [29] and [41]. The results of this comparison are shown in the Table 4. As Table 4 shown, the results demonstrate that the New ACO can be well applied to find high quality solutions than the other algorithms.

VII. CONCLUSION
In this paper, we have presented a method for solving the multiple traveling salesman problem based on the improved ant colony optimization. The improved ACO has used the relationship between MTSP and 1-MST, and the simplified pheromone diffusion. The new pheromone update rules helped a lot to achieve a better solution. The experiment results have shown that the new method has quick convergence speed and can be well applied to find best solutions.
In future, we decide to combine multi-objective TSP with parallel processing [42], [43]. Parallel processing can improve algorithm speed and reduce algorithm execution time. This method may be greatly improve the speed of finding best solutions. Furthermore, we will implement an application that cannot only help the logistics distribution industry but also make the transportation in earthquake relief work or military operation easier.