Agent-Based Simulation of Autonomous Vehicles: A Systematic Literature Review

Autonomous vehicles (AVs) may have a transformative influence on transportation systems. AV use is expected to reduce crash, transport costs, and land use. However, the net effects of the autonomous vehicle are unknown because there are no fully-autonomous vehicles in reality. The simulation systems are indispensable to be built up not only for cost but also for obtaining massive data. An agent-based simulation is capable of simulating the entities in a complex system, such as the AV system. Considering the potential of agent-based models to complement existing models and the difficulty to understand, to duplicate and to compare different agent-based models, this paper review existing agent-based modeling of autonomous vehicles with the help of ODD protocol, which has been applied in the review of agent-based land-use change models and agricultural policy evaluation. The significant variables that researchers have taken into consideration in sensitivity analysis and in the different scenarios are explored in the paper. Fleet size is the primary variable researchers considered, and we also discussed the probable causes of different AV replacement rates. We have assessed the quality of reviewed articles to gain more acceptance from the agent-based modeling community. Most of the reviewed papers follow the ODD protocol of agent decision making. However, in order to improve the level of modeling transparency, the executable and related data shall be available, at least the data source and the programming language/simulation platform should be provided. Several recommendations for further research are also presented.


I. INTRODUCTION
Autonomous vehicles (AVs) have attracted people's eyes for the decrease of fleet size [1], more accessible to the park [2], and self-driving. Drivers can enjoy in-car time instead of paying attention to the driving environment, especially for disabled individuals [3], elderly [4], and unlicensed people [5]. AVs may also have an impact on travel behavior, congestion [6], but the net effects of autonomous vehicles are certainly unknown. A large-scale of autonomous vehicles applied in our daily life has still been a long time, and it also costs tremendous to operate autonomous vehicles fleet in reality. The simulation systems are indispensable to be built up not for reduced cost but also for obtaining massive data. At the same time, taking safety concerns and legal restrictions of field test into consideration, the simulation may be the best way to explore what the potential influence would be.
The associate editor coordinating the review of this manuscript and approving it for publication was Atif Iqbal .
Agent-based models (ABMs) use self-governing agents with a bottom-up approach [7] to simulate the entities in a complex system. It is better than the conventional model and simulation method in flexibility, hierarchy, and intuition, it researches the complex system from individual to the entirety, from microcosmic to macroscopic view. The transportation systems involving autonomous vehicles are complex, and we need to consider the interaction between humans, vehicles, the road network, and the environment scenario. The components in the system are interrelated and interact with each other. Additionally, compared with traditional simulation models, agent-based modeling can also take the bounded rationality and behavioral heterogeneity of agents (people, vehicles) [8] into account. So large-scale of agents need to be built up in the simulation system of autonomous vehicles.
With the development of computational power, a more sophisticated model can be built up, researchers have set up many agent-based models with more details and real-world scenarios. However, researchers would develop the models for their purpose, the major variable that they focus VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ on and the simplicity of the model would be different, and different simulation platform would lead to different ideas of dealing with real-world problems. The travel and environmental implications of autonomous vehicles were explored in [9]- [14], some researchers focused on the parking requirements with the advent of autonomous vehicles [15]- [18]. The traffic congestion caused by AVs were also examined [19], many researchers also interested in the performance of the autonomous vehicle systems [3], [13], [18], [20]- [24]. Then some papers may pay attention to the modal share of the autonomous vehicle and the existing travel modes [22], [25]- [30].
Although there are many types of research on agent-based simulation of autonomous vehicles with various research purposes, few reviews are focusing on this field. Berrada and Leurent reviewed the models of the transportation systems involving autonomous vehicles, and the modeling reviewed in this paper contains spatial modeling and economic modeling. Agent-based modeling, about 11 of all 21 articles considered, was one of the spatial modelings and foud to have the potential to explore the effects of the autonomous vehicles. However, the reviewed papers are published up to the end of 2016 [31]. In addition, research on agent-based simulation of autonomous vehicles has witnessed a significant increase both qualitatively and quantitatively. There are more than 40 papers of this filed have been published, it is necessary to systematically review the up-to-date articles to have a detailed analysis of the evolutions of this domain. It is difficult to understand, to duplicate, and to compare different agent-based models, so a standard protocol for describing such models is needed.
The 'ODD' (Overview, Design concepts, and Details) protocol was designed in 2006 to facilitate the communication and replication of agent-based models [32]. Kremmydas et al. explored the ABM literature on agricultural policy evaluation from 2000 to 2016, and they evaluated the model transparency and the modeling of the agents' decision processes with the ODD or the ODD+D [33] documentation protocol [34]. Groeneveld et al. systematically reviewed the characteristics (e.g., uncertainty, adaptation, learning, interactions, and heterogeneities of agents) of describing human decision-making in agent-based land-use change models (LUCC ABMs) with the help of the ODD + D model description protocol [35]. The result shows that the ODD protocol facilitates review ABM in the field of agriculture and land use. However, the application of the ODD protocol to have a systematical review on the field of agentbased simulation of autonomous vehicles is missing.
Thus, considering the potential of agent-based models to complement existing models of the transportation systems involving autonomous vehicles and the task of reviewing several ABMs that deal with an autonomous vehicle system, the purpose of this paper is to systematically review existing agent-based modeling of autonomous vehicles with the help of ODD protocol, then to gain more acceptance from the agent-based modeling community. The critical variables considered in the simulation and the simulation output are reviewed in this paper, the suitable fleet size and the determinant variables are also examined. We also conducted a quality assessment of the collected literature. Several recommendations for further research are also provided.
The remainder of this paper is organized as follows. Section 2 provides the methodological approach used for data collection and analysis. Section 3 presents the systematic review process. In section 4, the quality of the reviewed studies is described. The limitations and strengths of this paper are proposed in section 5. The discussions are presented in section 6. Section 7 provides conclusions.

A. SEARCH STRATEGY AND DATABASES SEARCHED
Following the PRISMA (preferred reporting items for systematic reviews and meta-analyses) guidelines [36], five databases were searched in May 2019 for peer-reviewed papers by keywords contained in the title, abstract, and the main body of the papers included Web of Science, Sci-enceDirect, SPRINGER LINK, IEEE Xplore, TRID. The first two of the databases are comprehensive, and the other databases include journals for various disciplines, such as computer science, and transportation. These are the databases involving agent-based modeling of AV and containing the quality academic journals that are available for the authors. There are three categories of search terms and at least one term from each group must be used in combination: (1) agent, agent-based modeling and agent-based simulation; (2) autonomous vehicles (AV), autonomous taxi (aTaxi), autonomous mobility on demand (AMOD), shared autonomous vehicles (SAV) and shared autonomous electric vehicle (SAEV); (3) impact, implication, and effect. The search terms were adjusted to match the specific structure and requirement of each database. Duplicate and irrelevant papers were eliminated, and references within identified papers were reviewed for further studies. Despite not being peerreviewed, some of the papers are still important for the broad understanding and the interest of the filed of agent-based simulation of autonomous vehicles.

B. INCLUSION CRITERIA
Articles must satisfy the following criteria to ensure the legality of paper included in this review: (1) be written in English; (2) use agent-based modeling or agent-based simulation as tools; (3) contain autonomous vehicle research; (4) focus on the transportation system.

C. DATA EXTRACTION
A standardized data extraction table was extracted from the reviewed papers with the matrix method. Information extracted from each eligible paper contained some general information (author, year, journal), some information extracted from the ODD + D protocol (case study, the purpose of the study, object of decision making, adaptation, learning, sensing, prediction, interaction, heterogeneity, stochasticity). Moreover, the others are extracted due to the field of agent-based simulation of autonomous vehicle (demand, human agent, ride-sharing, relocation, network, speed, data source), the extracted data can be found in detail in the supplementary material. We adopted parts of the ODD + D model description protocol to have an explicit comparison of decision making and the agent behavior [33], then we adopted some information that is important for understanding the performance of the autonomous vehicle system.
Forty-four papers were chosen from the selected papers and were extracted by the author independently to ensure the reliability and validity of data extraction. Divergence among the authors was discussed until a consensus was reached. Ultimately, the authors approved of 80% of the extracted data, indicating high reliability and validity.

D. QUALITY ASSESSMENT
A modified checklist was adopted to assess the quality of the eligible and valid papers. Papers were evaluated with 15 different criteria divided into three parts: the quality of the model, the quality of the agent behavior, the description of the simulation, and the simulation results quality.
The modified checklist was separated into five aspects to explore the key factors which affect the quality of methodology: (1) data source; (2) transport network; (3) speed of AV; (4) ride-sharing; (5) relocation. If the data source of the trip is given, readers can obtain more information about the simulation. The transport network is divided into two aspects: the node-link network and the gridded network. The node-link network provides more details than the gridded network, such as the link-level travel time, especially the network is from high-fidelity maps. To some extent, the network represents the reality of simulation. In the grid-based simulation, the agents may aggregate in the grid, and the demand and supply will be handled within the grid, and the movements of the agents would be limited to fixed directions. In the network-based simulation, the agents may be generated at the exact coordinates, and the demand and supply will be handled in disaggregate ways.
The setting of the speed of AV in the reviewed papers are usually divided into three categories, fixed speed, considered congestion but set fixed peak and off-peak speed, and speed varies over time, whether the speed is fixed or vary over time would affect the system performance. Ride-sharing and relocation also reflect the authenticity of the simulation, and they are both critical for the future operation of the AV system.
If agents adapt their behavior to other agents or their environment change of state, we consider the agents are adaptive. If agents change their decision rules over time to improve their performance, learning takes place. Sensing means that agents may perceive characteristics, behavior, and actions of other agents or they perceive the environment. Prediction refers to agents foresee future conditions or consequences of their decisions. Interaction refers to the interaction among agents and the interaction between agents and entities. Heterogeneity, agents may differ in parameters, and exchanging an agent at the beginning of the simulation would change the outputs of the simulation due to the heterogeneity of agents. Stochasticity reflects that some process (including initialization) in the simulation is modeled by assuming they are random or partly random.
In order to assess the description of the simulation and the simulation results quality, the modified checklist includes (1) the simulation implementation details, which indicate that whether programming language/simulation platform is presented in the simulation; (2) the average waiting time which shows the performance of the system, we adopt 5 minutes as a threshold since the max waiting time and the time step are set as 5 minutes in some of the reviewed papers; and (3) the service or reject rate, whether all the requests are served or not would affect the system efficiency.
The criteria used for evaluating the selected paper are listed in Table 1. The possible range of assessment scores is also shown in Table 1. The results of the quality assessment would be discussed in section 4. QUALITY AND REVIEWED STUDIES. Figure 1 shows the search and retrieval process. The number of literature retrieved from each database mentioned above was 994 (Web of Science), 3490 (ScienceDirect), 5534 (SPRINGER LINK), 583 (IEEE Xplore), and 168 (TRID). After getting rid of duplicates, a total of 3147 different records were extracted from five databases, of which 364 were identified following the screening of titles and abstracts. There were three considerations for eliminating irrelevant and ineligible papers: not about agent-based simulation, not in an autonomous vehicle environment, and the full text was not available. Therefore, the full text of 43 publications was retrieved. The reference lists of included papers were reviewed, and potential papers were gathered. Finally, 44 published papers matching all of the criteria were retrieved in this review. Figure 2 illustrated the geographic distributions of case study areas of the reviewed papers. The number of papers focuses on the US area is the highest number. Then Europe is also studied by some articles, but there is no paper involving the region of China. The probable reason may be the availability of the data required. Another cause maybe all the reviewed papers are published in English, and in China, researchers might focus on the technique to achieve fully autonomous driving. Figure 3 plots the number of papers per year since 2013, and the number of articles is increasing. Although all the papers were searched in May 2019, there are seven papers included, which is more than the obtained papers in 2015.

A. AGENT-BASED SIMULATION
In this section, the authors emphasize the agent-based simulation of autonomous vehicles. Readers would obtain more understanding of the field of agent-based simulation of autonomous vehicles, we paid attention to the purpose of the simulation, the significant variables considered in the sensitivity analysis and different scenarios, which would affect the output of the simulation, to some extent indicated the performance of the model, then we would illustrate the simulation platforms. Those are significant for readers to have a better understanding of the simulation, and to compare the different papers on the models and the system performance.

1) PURPOSE OF THE STUDY
The significant purpose of the reviewed studies is the prediction (N = 33), which inflects that in the field of autonomous vehicles systems, ABMs are usually used to assess the impacts of autonomous vehicles. The other purposes are system understanding (N = 22) and management or decision support (N = 22). Furthermore, ABMs are hardly used for hypothesis testing.

2) THE KEY VARIABLES CONSIDERED IN THE SIMULATION
As shown in Fig. 5, the critical variables taken into consideration in the sensitivity analysis and different scenario of the simulation that would affect the system performance are fleet size, demand, strategy, ridesharing, pricing schemes, configurations of stations, trave mode, vehicle capacity, service area, refuel/recharge time, maximum waiting time and cruising time. The variables extracted mean that they vary in the paper, but the variables not included do not mean they are not necessary.
The prior factor researchers considered is fleet size (N = 27), whenever the conventional vehicle was replaced by AV [1], aTaxi [37], AMOD [26], ATOD [23], SAV [9] or SAEV [38]. Different fleet sizes would inevitably affect the performance of the autonomous vehicle system, which often serves customers with a vehicle fleet. The papers relating to fleet size and the vehicle replacement rate are reviewed in detail in the next section.
Then the primary factor considered in the reviewed papers is demand (N = 12), which containing two situations, the trip demand and the AV penetration. Burns et al. did a sensitivity analysis on average daily trip demand rate [39], Fagnant and Kockelman [9] set the different trip generation rate as double trips, half trips and quarter trips, the trip generation rate varied by different region: downtown, urban, suburban and exurban [25], [40], [41], Hyland and Mahmassani [42] used different taxi demand to explore the AV system performance. Martinez and Crist [43] tested the shared-mobility scenarios with a penetration rate of either 100% or only 50% of all trips. Bösch et al. investigated the needed fleet size to serve various highly detailed travel demand (1% to 10%) served by AVs [1], different proportions of passengers (100%, 20%) were able to drive themselves were examined in [44]. The replacement rate of car trips by AT mode varied from 0% to 100% with corresponding fleet size variations was examined in [19]. Harper et al. [11] examined different AV penetration ranging from 5% to 100% in downtown Seattle to explore the various outputs of the simulation. Javanshour et al. [24] investigated the performance of the AMOD system under demand uncertainty (2%-20% market penetration).
The researcher may also take strategy (N = 11) into consideration when exploring the performance of the system, including scheduling strategies [20], assignment  strategies [42], deployment strategies [23], operation strategies [12], [45], hailing strategies [29], tolling strategies [46], relocation strategies [9], [22], [24], [44]. Different relocation strategies (N = 4) were tested by researchers, with or without, alone or in combination, to examine each strategy's effectiveness. Fagnant and Kockelman [9] evaluated four vehicle-relocation strategies, Scheltes and de Almeida Correia [44] adopted relatively simple relocations strategy (during the morning peak; during morning peak and evening peak) to have a preliminary study of the potential of relocation. Heilig et al. [22] examined the effectiveness of AMOD services with relocations trips during nighttime compared with no relocations. Javanshour et al. [24] explored the performance of the system with and without rebalancing.
Ride-sharing (N = 7) is another variable that researchers have taken into consideration, for ride-sharing may reduce the fleet size. There is also the situation of ridesharing, and one is the users' willingness to share, and the other is whether the AV is ridesharing or not. The percentage of Clients' willingness to share was examined in [15], varying from 0% to 100%. Shen et al. evaluated the performance of integrated AV-PT services with two extreme settings of the percentage of people's willingness to share (0%,100%) [47]. The performance of different proportions of rider groups' willingness to share varying from 0% to 100% was evaluated in [14]. The performance of two different system configurations (ridesharing system, carsharing system) of AV fleet was assessed in [16], [28], [43], [48].  Vehicle range (N = 5) is usually taken into consideration when the research object is related to the electric vehicle. Different scenarios with various vehicle ranges were introduced in [38], [40], [41] to explore the impact of vehicle range on SAEV system performance. Jäger et al. investigated the effects of battery capacities from 20 to 40 kWh [49]. Loeb et al. examined the simulation outputs with different vehicle range varying from 100 km to 325 km, increased by 25 km [2].
Researchers usually use pricing schemes (N = 4) to explore the market potential of AVs. Chen and Kockelman [25] adopted four pricing strategies, Hörl [27] explore the AV mode share with different pricing schemes, Liu et al. [50] also examined the mode-split with four possible fare rates, Wen et al. [29] explored the modal shift to AV+PT service under different fare schemes. Different travel modes (N = 5) would affect the performance of the system since the heterogeneity of different types of autonomous vehicles. Martinez and Crist [43] explored the performance of the urban mobility system with or without high-capacity public transport. Then they evaluated the system performance with or without Taxi-Bus [17], Chen and Kockelman [25] explored the results with or without SAEV, Lu et al. [12] estimated the environmental impacts of different aTaxis: electric aTaxis, gasoline aTaxis, and conventional gasoline cars, Simoni et al. [46] compared the modal split and traffic conditions of AV-oriented scenario compared with the SAV-oriented scenario.
The configuration of stations (N = 3) usually divided into two categories: the parking stations configurations and the charging stations configurations. The effects of the number of parking stations and parking station locations were explored in [3], [26]. Loeb et al. [2] examined the impacts of the number of charging stations.
The service area (N = 3) would affect the performance of the system. Usually, researchers fixed the researcher area. However, Fagnant and Kockelman [9] examined the results of the different service area (greater or smaller), Jäger et al. [49]  explored the different outcomes of the exclusive service area, mobility hubs, whole city, Shen et al. [47] tested the simulation outputs of three area sizes.
The refuel/recharge time (N = 3) was seldom taken into consideration, but it would reinforce the realism of the simulation. In Chen's research [40], each SAV would need 15 min to refuel, each SAEV would need 240 min to recharge, and in fast charge scenario, the recharge time would be 30 min, which is similar to Loeb and Kockelman' research [38], except that the refueling time of gasoline hybrid-electric SAV is 2 min. Then Loeb et al. [2] set the charge time varies from 100 to 325 min.
Vehicle capacity (N = 2), to some extent, means the capability that passengers one AV can carry at one time. The impact of different vehicle capacities varying from 1 to 4 was examined in [29], [41]. Dia and Javanshour [18] assumed two different AMOD scenarios, and in one scenario, the maximum waiting time is zero and up to 5 min in another scenario. Liu et al. [13] also set the different maximum waiting time from 5 min to 20 min to explore the minimum taxi fleet size required in each scenario. Zhang et al. [15], [16] examined the impact of cruising with the empty cruising time varying from 0 to 30 min.

4) THE SIMULATION PLATFORMS
Because of the immaturity of the fully autonomous technology of vehicles and the numerous cost of the field test, many simulation platforms are employed to explore the implication of the advent of autonomous vehicles.  Table 2 presents a list of these platforms. Most of the cited papers employ the MATsim (multi-agent-based simulation) to model the operation of the autonomous vehicle system.

B. THE VEHICLE REPLACEMENT RATE AND THE PROBABLE CAUSES
According to Fig. 8, the fleet size or replacement rate is one of the main outputs of simulation. Moreover, as shown in Fig. 5, the fleet size is the significant variable considered in the simulations. Furthermore, the vehicle replacement rate indicates the efficiency of the autonomous vehicle system. This section would discuss the probable reasons for different AV replacement rates. Fagnant et al. [10] assumed that the travel demand, average speeds, and average trip distances would affect the system performance. However, most of the reviewed papers did not provide accurate information   on average trip distances. According to Marczuk et al.' research [3], fleet size mainly depends on service area, the average demand, the level of service (in this paper, we assume that is related to the average waiting time, service and reject rate), the routing strategy, the relocation strategy, and the configuration of the facility. Additionally, the required fleet size would potentially be reduced by ride-sharing [14].
To some extent, the travel distance would be primarily affected by the research area, whether it is an urban area, city center, or metropolitan area. The travel speeds during the simulation basically depend on the network, in a grid network, the speed is often set as a constant, and the only difference is whether it is in peak time or not. Furthermore, in the network-based simulation, the speed is usually set as link-based, sometimes it is constant, and at the other time, it is changed by the network-loading. We set the ridesharing as the indicator of the routing strategy, for ridesharing or not would affect the vehicle's ability to shuttle passengers at a time, so that affects the routing strategy significantly.
In conclusion, the significant variables considered in this review that would affect the fleet size or replacement rate are: (1) service area, (2) the average demand, (3) average speed, (4) average waiting time, (5) service and reject rate, (6) ridesharing, (7) the relocation strategy, and (8) the configuration of the facility. The replacement rate of AV is shown in Table 3.
All of the factors mentioned above that would affect the replacement rate have been taken into researchers' consideration more or less in recent researches. Fagnant and Kockelman have explored the replacement rate of the autonomous vehicle with a case study in Austin, with their teammates many times. In their previous research, one autonomous vehicle would replace about ten traditional human driving vehicles, the replacement rate was at 1:11 [9] with link-level travel times and at 1:9 [10] with constant speed, respectively. Additionally, approximately 8% more vehicle miles traveled (VMT) may be generated in [10]. The network-based simulation assumed a larger service area and lower trip density, the average waiting time in the network-based simulation is about 1 minute through the day compared to 0.3 minutes in the grid-based simulation. The grid-based simulation may have a better performance than the network-based simulation. However, it limited the actuality of the realistic scenario. The replacement rate of [9] and [10] was similar to Bischoff and Maciejewski's research [37], in which one autonomous taxi could replace ten conventional vehicles, this paper did not consider relocation, and the travel demand is a million levels, so the average waiting time is longer, about 2.28 minutes. Bösch et al. also assumed that autonomous vehicle would lead to a reduction of up to 90% of the total vehicle fleet with max waiting time of up to 10 minutes, compared to [37], not all of the request would be served by AV, the speed in this research was fixed and the average waiting time is about 2.67 minutes [1]. Martinez and Crist [43] explored that 90% of vehicles could be removed under a ride-sharing TaxiBot configuration supported by high-capacity public transport. Burns et al. [39] found that the request could be server by AV in Ann Arbor with only 15% of the vehicles, and there is no ride-sharing taken into consideration, and the waiting time is less than one minute. In Zhang's research [15], [16], the vehicle replacement rose to 1:14 with a case study of a hypothetical city, compared to [9], the research area was same, Zhang considered ride-sharing which may result in smaller fleet size and the travel demand was also lower. However, ride-sharing may lead to more average waiting time. Then in Fagnant and Kockelman's later research, the autonomous vehicles were also operated with dynamic ride-sharing and no speed limitations in the simulation. The replacement rate can achieve at ten approximately [48]. They had also examined the vehicle replacement with various fare levels per mile. The replacement ranges from 5.6 to 7.7 without ride-sharing [50]. Bienzeisler [53]  discussed that an autonomous vehicle would replace private cars between 7 and 8 in Cottbus without ride-sharing and relocation. In [12], the optimized autonomous taxi fleet size would reduce the current fleet size by 80% with the system performance reach the average waiting time of fewer than 3 minutes, and the service area was small about 44 square miles. Dia and Javanshour explored that the AMOD would result in fleet size reduction of 43% with zero passenger waiting times, and 88% with maximum passenger waiting times of 5 minutes, but lead to vehicle-kilometers traveled (VKT) increase of 29% and 10%, respectively [18]. Then Javanshour et al. [24] also explored the AMOD system performance in 2019, and they assumed that an AMOD system could reduce 84% of the conventional personal car, but would lead to an increase of VKT by 77% with car-sharing and by 29% with ride-sharing, the ride-sharing in this paper was not allowed for larger than two passengers. There was no dynamic ride-sharing, and the service area was one part of the whole urban environment, about 88.75 square kilometers. Heilig et al. [22] also assumed that the AMOD would reduce the fleet size by 85% with ride-sharing and relocation. However, it saved 20% of all vehicle kilometers. The results from Lokhandwala and Cai's research [14] showed that shared autonomous taxis could potentially reduce the fleet size by 59% and decreased total travel distance (up to 55%) with 100% ride-sharing. Liu et al. [13] examined that autonomous taxi would reduce the current fleet size by 19% with nondetour ride-sharing and by 27% with detour ride-sharing.
The researches mentioned above have not considered electric vehicles. When the private vehicles were replaced by autonomous electric vehicles or shared autonomous electric vehicles, the distribution of charge infrastructure, the charge time, the vehicle range, and the charge strategy may significantly affect the replacement rate. In [40], the replacement rate ranged from 3.7 to 6.8 for different battery recharge time and vehicle range. When ridesharing is also under consideration, the SAEV replacement rate ranged from 3.7-13.4 in [41], ranged from 3.75 to 11.5 in [38] due to different recharge time, vehicle range.

IV. QUALITY AND REVIEWED STUDIES
The scores of the quality of each eligible paper range from 0 to 16, as shown in Table 1. According to the table, 81.8% of the papers described the source of the data of travel demand, and the percentage of researches that employed the simulation in the node-link network is 79.5%. In the selected papers, 73.9% illustrated the simulation platform. There are 26 pieces of research (59.1%) in which the speed of AV was varied, and 20 papers (43.5%) considered ride-sharing, and the same number of articles focused on relocation problems.
88.6% of the reviewed papers showed that agents would be adaptive in decision making. The passenger agent would wait for more time for the request, and the vehicle agent would like relocation to the low supply area to transport passengers. 47.7% indicates that the agent would learn to have a better performance, especially in the MATsim platform, agents learn to have a better utility. The agent behavior of sensing, prediction, and interaction are explored in all the reviewed papers. Usually, there are three types of agents, the human agent, the AV agent, and the fleet operator agent.
Fleet operator agents and the human agents would sense the location of the vehicle, and the AV agent would predict the time to pick up a customer. Moreover, the interaction between AV agents and the human agent is always taking place when the human request an AV, and AV transport the passenger, and sometimes, the ride-sharing is also taken place.
Fourteen papers showed the heterogeneity of agent, that would usually be the preference of different people, the different types of the autonomous vehicle. Stochasticity is also considered in the researches (N = 42) for the random distribution of the agent in the simulation Initialization. 81.8% of the studies provided the programming language/simulation platform, as for the waiting time, 31.8% did not show information of it, 13.6% displayed that the waiting time would be more than 5 minutes, and 54.6% suggested that the waiting time can achieve less than 5 minutes. Not all of the trips were served in the simulation, and only 28 papers (63.6%) served all the requests.

V. LIMITATIONS AND STRENGTHS
The following limitations of this review should be taken into consideration when interpreting the existing results. First, we limited our search to papers published in English. Thus, relevant literature published in other languages was excluded. Second, five databases were used to help searching articles, and the reference lists of the included papers were also reviewed. However, some papers may be ignored, for they are not included in the databases. Then, all included studies were agent-based simulations, and the control method of the autonomous vehicle system instead of a single autonomous vehicle was discussed. This study had several strengths. First, it used an extensive search strategy to locate papers and rigorously screened papers through well-defined inclusion/exclusion criteria. Second, the quality of the included papers was assessed in a standardized and reproducible way.

VI. DISCUSSION
The advantages of simulation techniques are that simulation techniques can reduce capitalized cost but obtain massive data, the procedure can be accelerated in the simulation and simulation is the safest choice to research, in the simulation, the scenario can be controlled. The disadvantage of the simulation techniques maybe that how to make sure the simulation techniques reflect the real situation. Researchers should simplify the problem and, at the same time, maintain realistic scenarios. Usually, data is required for calibration. At one time, different methods may get the same output for the same problem, while at the other time, the effects of different methods may differ.
Agent-based simulation of the autonomous vehicle system is a relatively emerging research field. When conventional vehicles are replaced by self-driving vehicles, the consideration is how large a fleet of self-driving cars can meet people's travel needs without reducing system efficiency. Then, when designing such a system, it is necessary to consider travel demand of autonomous vehicles, whether it is fixed or changing, whether the demand comes from previous travel survey, and whether the preferences of passengers are considered, including the preference for autonomous driving, the preference for ride-sharing. Corresponding to travel demand is the size of the service area. It is necessary to understand the various strategies of the system operation, including scheduling strategies, assignment strategies, operation strategies, tolling strategies, and relocation strategies to have a better understanding of the operation of an autonomous driving system, In order to ensure the efficiency of the system, the maximum waiting time needs to be set, which usually is set as 5 minutes or 10 minutes or 30minutes.
Early researches showed that one autonomous vehicle could replace about ten traditional cars. Under more ideal conditions, the replacement rate of autonomous driving would reach 1 to 14. With further research, the simulation scenarios were more realistic, and the replacement rate for autonomous driving declined, which was approximately 1 to 5. Recent researches indicated that the replacement rate is no longer a definite value, but there will be different suitable replacement rates in different scenarios. Maximum waiting time, ride-sharing, and detour are the factors that affect the fleet size. For electric vehicles, vehicle mileage, charging time, and configuration of charging stations would greatly affect vehicle replacement rates.
According to the ODD protocol, the agent behavior mainly includes adaptation in decision making, learning, sensing, prediction, interaction, heterogeneity, and stochasticity. The human agent chooses to continue to wait for the arrival of the autonomous vehicle or chooses other travel modes after waiting for a simulation period. Such behavior is adaptive. The agent perceives vehicles on the road and predicts the arrival time of vehicles to express its sensing and prediction behavior. Ride-sharing and changes in congestion-based travel time in mind reflect interaction. The heterogeneity of agents is mainly withdrawn through different vehicle types, different willingness to ride. Stochasticity mainly occurs during the initialization of the simulation. Agents can improve their performance by learning to change decision rules. Most of the cited papers employ the MATsim (multi-agent-based simulation) to model the operation of the autonomous vehicle system. At first, MATsim was not able to simulate autonomous vehicles, so researchers often use this tool to combine with other simulation tools or with their coded program. Until [19] and [21] add the autonomous vehicle toolkit into MATsim, it has been a most useful platform to model the operation of the autonomous vehicle system.
MATsim extends the genetic algorithm, and the agent performs collective learning through co-evolution. MATsim can try different plan modifications (for example, rerouting and rescheduling) in multiple iterations until the system reaches dynamic user balance.
Some factors are also crucial for the design of an agentbased simulation system of the autonomous vehicle, such as the spatial resolutions, temporal resolutions, booking or not, average trip length, and average travel time. However, for some papers did not provide such information, these factors are not discussed in this paper.
We have not discussed the ways AVs could be used in the future, AVs were simulated similarly in the models, the difference may be the data source, as for taxi, authors used taxi data, and as for other car-sharing services, they might use travel survey data. The reviewed papers seldom focus on the private autonomous vehicles, and as for the other ways, AVs are usually car-sharing, the difference is whether they are ride-sharing or not, which has been explored in the simulation.
Although there has been a fruitful development of models and solution techniques to research the operation of the system in an autonomous vehicle environment, there are still many research questions, such as the following: In order to facilitate the validation of the models, many researchers employed a simplified input for simulation. Some of the travel demand is static and may not be a real-world demand, and the applied network in the agent-based model is also not real, it sometimes uses the grid network, and some of the reviewed papers use the homogeneous characteristics.
There were some other simplifications of the network and the vehicle parameter, those methods were necessary but would make the simulation away from the real world, such as the fixed travel speed, fixed travel time.
The authors decided some simulation settings instead of having a survey, such as the heterogeneity of the passengers.
The proposed models used to the operation of the autonomous vehicle system might be complicated and computationally expensive, sensitive to modeling errors. It is necessary to profoundly improve the robustness, versatility, and precision of the proposed model. It would confuse the readers that some of the papers do not have a clear description of the data and the simulation implementation details, and some information on initialization.
Based on the literature review, a thorough analysis of agent-based simulation of the operation of the autonomous vehicle system has been made. There are significant opportunities for innovation in operation methods within this domain. These include: With the advent of autonomous vehicles, researchers often explore the performance of the traffic system. If all the existing travel modes are replaced by autonomous vehicles, in their precious researches, they often simplified the road network due to the computational power. However, the real network would provide more detailed information, which is essential for deep examination for more real travel time and travel flow distribution. In order to make the simulation close to the real-world circumstances, the traffic flow models can also be integrated into the simulation.
The travel demand is another factor that should be taken into consideration, the fixed travel demand contributes to the initial research for the data can be obtained more easily, but it must be noted that the travel demand may be changed since autonomous vehicle provide chances for some disabled and elderly to travel with much more convenience. Demand and supply are changed dynamically, so the interaction between them should also be a dynamic process.
As the autonomous vehicles can park themselves when autonomous vehicles finished their trips, they can return home or find a cheaper area to park, the parking demand would also be changed, so it is also needed to considerate parking demand variation.
Some of the researchers show that shared autonomous vehicle would be one of the major travel modes in the future, what is more, it may replace the exiting travel modes, so there are some researches in which all the travel demand are severed by SAV. When taking SAV into consideration, the operation strategy is the essential factor that should be paid attention to, and different strategies may lead to various efficiencies of the system.
Ride-sharing is another factor that results in different performances. With [17], without [26], [42], [49] and dynamic [14], [41] ride-sharing have been explored in previous researches. With ride-sharing, the passenger numbers that one SAV may serve are different. In some researches, the SAV would not serve another passenger if there are already two customers in the car, and in other researches, SAV would like to serve another passenger until there are four passengers in the car.
If the autonomous vehicle is an electric vehicle, some other factors needed to be taken into consideration, the vehicle range, the recharge time, and the distribution of the charge station will affect the performance of the system, which is similar to the distribution of parking stations of SAV.
Rebalancing or redistribution strategies would also be taken into consideration, as the empty trip of the autonomous vehicle, and due to the empty trip and the increased travel demand, congestion should also be paid attention to, though the autonomous vehicle may increase the road capacity.
The output of the system may be the optimum fleet size, the replacement rate, the empty VMT, the empty VKT, the response time, and the serving rate. The researchers may considerate some of them due to their purpose, if researchers focus on the performance of the system, they may try to explore the minimized empty VMT or the empty VKT, and if the researchers focus on the experience of the passengers, the response time should be minimized, and the serving rate might need to be higher.
Although all the existing travel modes might be replaced by an autonomous vehicle in the future, the traffic modal split rate is not sure when the fully autonomous vehicle has not been in reality for several years. The autonomous vehicle and the existing travel modes such as the conventional passenger car, and the transit would all be available in the long term. The ascertainment of the mode split rate should be taken into consideration when researchers explored the implications that autonomous bring to us, and the relationship between the autonomous vehicle and the existing travel modes should be focused on, whether autonomous vehicle supports the current travel modes especially public travel modes or competes with them deserve much more consideration.
The preference of the passengers and the heterogeneity of the travel demand, the travel modes should also be taken into consideration, such as people's willingness to travel alone, and people's preference of the fellow travelers, the preference of the travel modes, and preference of the vehicle type. Some users may prefer electric vehicles for their awareness of environmental protection, and others may prefer conventional vehicles than autonomous vehicles for their pleasure in driving a car. The previous researches often based on the existing travel demand. It is necessary to have a stated preference survey to obtain more accurate information, such as people's willingness to travel with autonomous vehicles.
The existing researches have achieved the purpose of the researchers, different operation strategies such as the assignment strategies, the operational policies, the relocation strategies, and the tolling strategies would have significant effects on the performance of the system. A robust algorithm would be able to deal with different scenarios. The simulation methodology should be applied in different countries and areas to obtain more case studies evidence to obtain more meaningful results.
Several recommendations made in this review present a foundation for the future study of the agent-based simulation of the AV system. Recent breakthroughs in AV technology have enabled researchers to explore the effects of AV on the transportation system. However, the application of AV technology in the operation of the vehicle system field is still in its early stages. There are still considerable opportunities to develop the operation method of the AV system.

VII. CONCLUSION
In this paper, we present a thorough and systematic review of agent-based simulation of autonomous vehicles with the help of the ODD protocol. In order to have a strict evaluation process, this review has provided a detailed discussion and analysis of modeling methods of the autonomous vehicle system, such as the purpose of the simulation, the major variables considered in the sensitivity analysis and different scenario, the output of the simulation, the simulation platforms and the cause of varying AV replacement rates. The fleet size is the primary variable researchers take into consideration, and researchers need to have a careful consideration of all the causes to find out the most probable replacement rate.
The review has also carefully compared the papers from the methodology quality, the agent behavior, and the simulation outputs. Most of the reviewed papers follow the ODD protocol of agent-decision making behavior. However, in order to improve the level of modeling transparency, the executable and related data shall be available, at least the data source and the programming language/simulation platform should be provided.
The AVs simulation procedures are not discussed in this review since it is hard to generalize the simulation procedures, different reviewed papers show different permutation and combination of simulation processes, but we have examined the agent decision-making process. The AVs simulation procedures may be discussed in detail with further research. Furthermore, the mode choice behavior is an interesting topic that we would consider. We may also pay attention to the review of agent-based simulation for the diffusion of AVs.
Bibliometric and content analysis might be a suitable methodology to obtain the information of research hotspots of this domain when the literature quantity of this field gets a significant increase. Google Scholar may be a suitable data source to get more information about the related papers of this domain. We did not choose the Google Scholar as a data source since in this review since we could not access Google Scholar. We could only visit Google Scholar Mirror, but we could not download some of the papers searched in Google Scholar Mirror.
The present systematic review shows that the operation of the autonomous vehicle system is in its infancy. Limited by the development of autonomous vehicle technology and hardware support, only simualtion experiments could verify the proposed methods. Future work examining their adaptability and validity based on field testing is warranted. Finally, further research is needed to develop efficient and generic operation method of the autonomous vehicle system.