Framework for Vehicle Dynamics Model Validation

Vehicle dynamics models are widely used in many areas of the automotive industry. The usability of each model depends on how well it is able to mimic the behavior of the real vehicle. Each simulation model must go through a thorough investigation process first, which is called model validation. Although, vehicle dynamics simulation models and methodology for computational model validation are well established fields, a general framework for vehicle dynamics model validation is still lacking. This project aims to develop a comprehensive methodological framework for vehicle dynamics model validation. In this paper the aim is to present the high level layout of the proposed framework, introducing the main blocks and the tasks related, also addressing some critical issues regarding vehicle dynamics model validation such as validation metrics and vehicle parameter measurement and estimation. An important part of the proposed methodology is a sophisticated vehicle dynamics measurement system, which gives the opportunity to estimate a bunch of vehicle parameters during dynamic testing, which can be useful for several reasons, e.g. fine-tuning the parameters of the Pacejka Magic formula. As a case study some vehicle dynamics test based parameter estimations are shown to justify the raison d’être and investigate possible applications.


I. INTRODUCTION
Vehicle dynamics simulations are widely used and become more and more important in the modern era of vehicle development (e.g. active and conventional suspensions design, vehicle controller design, advanced driver assistance systems, development of driving simulators, autonomous driver algorithm development), as applying simulation is generally more cost-efficient, safer, and faster than real-world vehicle tests. The more, in simulation a wide range of parameters can be easily modified and tested in a short time in a very flexible fashion.
Having a reliable and validated vehicle model is not only important in the development phase but for the operation of different vehicle software components, as they are responsible for safety-critical control processes such as braking and cornering stability [1].
Model validation is the review and evaluation process of how a model works, usually done by the model developer and engineers being familiar with the given real system. "It refers to the processes and techniques that the model developer, model customer and decision-makers jointly use to assure that the model represents the real system to a sufficient level of accuracy." [2] However, the above-mentioned advantages are only present, when the model and its parameters are appropriately accurate, and the simulation results well reflect the real-world phenomenon. Sophisticated models for vehicle dynamics have lots of parameters, some of them being difficult to measure. The necessary validation process, therefore, requires a plethora of testing as well as measurements which are expensive and time-consuming.
Each model's usefulness is measured by how well it can mimic the behavior of the real vehicle. Model validation is a process that each simulation model must go through before it can be used. Despite the fact that vehicle dynamics simulation models and methodology for computational model VOLUME 4, 2016 validation are well-established topics, there is still a need for a generic framework for vehicle dynamics model validation [3].
The purpose of the project is to create a complete methodological framework for validating vehicle dynamics models. The goal of this paper is to describe the proposed framework's high-level layout, presenting the primary blocks and activities associated with them, as well as certain critical concerns linked to vehicle dynamics model validation, such as validation metrics and vehicle parameter measurement and estimation. A sophisticated vehicle dynamics measurement system is an important component of the proposed system, as validation metrics -which are intended to quantify the model's credibility -should account for the uncertainty of real-world measurements. As a result, precise and accurate measurements of all relevant system response quantities (hereinafter referred to as SRQ) are required for validation. Furthermore, using such a measurement system allows for the estimation of a number of vehicle parameters during dynamic testing, which can be useful for a variety of reasons, such as fine-tuning the Pacejka Magic Formula parameters, tracking the change of: weight distribution, center of gravity height (h CG ), yaw inertia (θ z ), etc. during long dynamic tests. To illustrate the possible applications, some vehicle dynamics test-based parameter estimations are provided as a case study.
Vehicle simulation models become rather complex asbesides the increasing complexity of the subsystems -they need to reflect the entire transportation system more and more [1]. According to this trend, the model validation process also becomes more detailed and intricate.

A. THE STRUCTURE OF THE PAPER
The paper is organized as follows. In section II, important works on computational model validation, vehicle dynamics, and vehicle dynamics model validation are discussed, the emphasis is on discovering the key elements of validation methodology -such as validation metrics. Then, in section III the high-level layout of our proposed vehicle dynamics model validation framework is introduced with more detail discussion on parameter estimation and validation metrics. Finally, in section IV a case study is presented, where dynamic vehicle test-based parameter estimation is discussed and at the end a result of a validated vehicle model is shown.

II. PRELIMINARIES AND MOTIVATION
The research domain of vehicle dynamics model validation has two main components: computational model verification and validation (V&V) and vehicle dynamics (vehicle models, simulation and vehicle parameter measurements, estimation).
Considering general model validation, the field is well established. Carson states in [2]: "The goal of verification and validation is a model that is accurate when used to predict the performance of the real-world system that it represents, or to predict the difference in performance between two scenarios or two model configurations. The process of verifying and validating a model should also lead to improving a model's credibility with decision-makers." This idea is also supported in [4] and [5]. According to many experts there is no absolute valid model [4], [5], [6], [7]. Therefore, there will always be discrepancies between the measured physical phenomenon and the simulation results. The purpose of the simulation is to give an answer to a specific question, or give information for engineers in a decision making process, therefore as defined in [3] the model only needs to be validated in the domain which is suitable for the application. Any further work on the validation can improve the predictability of the model, but if this is not required for the purpose, then it unnecessarily increases the cost and time of the process.
Vehicle dynamics models have well established mathematical background, one of the earliest works on this topic is [8] from Olley (1946), and one of the first vehicle dynamics models was proposed by Segel (1956) [9]. Since then several works have been presented in the field of vehicle dynamics including Millikens' Race Car Vehicle Dynamics [10], Vehicle Dynamics by Zomotor [11], Pacejka's tire and Vehicle Dynamics [12], Vehicle dynamics: Theory and Application from Jazar [13], and plenty of more.
There is a broad range of publications and books regarding vehicle dynamics and vehicle dynamic simulations. Many papers can be found on the topic of vehicle model validation as well, but according to Kutluay -who did a comprehensive literature review in this field in 2014: "Many of the publications which claim to present a validation methodology or technique tend to only offer the application of a methodology to an individual case." [3].
The full methodological connection between simulation and real measured data comparison, and the method of error detection and parameter identification -in other words, a general framework for vehicle dynamics model validationis still lacking.
"Existent works on validation methodologies for vehicle dynamics simulations focus on different aspects of the question. Neither there is a standard in experimentation and data handling processes in vehicle dynamics modeling, nor is there a standard reasoning process in the vehicle dynamics modeling application in validation analysis. Most of the applications rely only on visual comparison and subjective judgment. Diagrams types used in visual comparison also do not follow any recognizable pattern and their contents and structures are determined at will by the research team. Most of the time, the team which developed the model also decides if the simulation is valid. This whole process chain diminishes the credibility of these models." [3] One of the first studies regarding vehicle dynamics simulation model validation methodology is [14] from 1990. The validity of the model is defined as the agreement between the simulation's predictions of a vehicle's responses and the actual measured vehicle's responses within a previously specified level of accuracy. It is stated that the simulation results may only be valid within a range of the operating conditions. A model is probably only valid for a specific task, for example, a validated lateral dynamics model with suspension degree of freedom is not necessarily valid for ride quality analysis [3]. The methodology presented in [14] is based on statistical analysis, therefore several experimental test runs need to be carried out for each test case to gather sufficient data and to reduce the influence of random error. The data should be investigated in time and frequency domains as well.
Since then several other authors -including [15], [14], [16], [17] -also support the idea that the validation of a vehicle dynamics model should include steady-state and transient tests and consider analysis in time and frequency domain as well.
In [18] Bernard and Clover defined three topics to investigate during a model validation: • Conceptual validity -Is the model appropriate for the vehicle and maneuver to be tested? • Verification -Are the equations in the simulation fully replicating the model? • Data validity -Are the input parameters reasonable? Running the simulation claims to be the only way for verification due to the high complexity of the simulation models, as it is not possible to check each of the equations [18].
The methodology presented in [19] defines four possible areas causing discrepancies between simulation model results and real world data: • Mathematical model. • Computational model programming.
• Vehicle or environment input data.
• Numerical accuracy and stability. The model validation should consider the domain of application of the model, since the degree of validation always has a limit and the model should be useful for a specific question (in a specific range). Care should be taken if the previously validated model would be used for a different task, especially when the behavior of a subsystem needs to be investigated in more depth, as a valid model according to the analysis of general system response does not guarantee sufficiently valid subsystem models [19].
Although in [2] Carson stated that "a model is not valid until all of its subsystems are valid". We can see that in the case of highly complex systems -such as vehicles -a model that is considered valid for a general analysis, is not necessarily valid for subsystem analysis. For example, a model that is accurate enough and useful for general longitudinal, lateral, and combined (braking and cornering or acceleration and cornering) behavior analysis, does not need a sophisticated brake model, that considers energy dissipation, brake pad and disc temperatures, and the variation of the coefficient of friction between the two due to temperature and sliding velocity. But for example, if the braking performance needs to be analyzed in a long run, the above-mentioned model is probably not valid for the task.
The validation method presented in [19] is summarized in four steps: • The mathematical model's conceptual validity. • Face validity (reasonableness) of the simulation model response. • Input, intermediate, and output variable consistency. • Agreement between the simulated and the reference system responses. The works presented in [14] and [15] are about vehicle parameter measurement and vehicle model validation. The validation process consists of three main phases: • Collection of experimental test data (Steady-state, transient, and frequency domain responses). • Independent vehicle parameter measurement • Comparison of simulation results with test data using the same driver inputs.

A. PARAMETER MEASUREMENTS
The model can only be validated under a limited number of operational conditions, as previously stated. In their 2002 paper, Wade-Allen et al. [17] discussed the importance of parameter measurement according to the operating conditions. If the vehicle model is used for analyzing the vehicle behavior on the grip limit of the tires, then the parameter measurements must be carried out taking into consideration the operating conditions that can occur in these situations. Therefore in this case, for example, the tire characteristics must be measured in a higher slip angle range -in the tire saturation region -and at a wider normal load range (because of the higher normal load variation due to load transfer). The importance of independent parameter measurement is emphasized in [15] and [14]. Complex vehicle simulation models can only provide adequate results if the subsystem parameters are set to an appropriate level of accuracy.
According to [18] faulty data entry is an important risk factor -even if the parameter measurements are reliable. The most endangered part is tire and suspension data, as both have many parameters and this step is tend to have human error. Extra care should be taken during the parametrization of the model.

B. VEHICLE DYNAMICS TESTS
In this section, various viewpoints on the type of vehicle test required for model validation are investigated.
In [16] the following five test cases are defined as primary validation maneuvers: • steady-state lateral dynamics (low-frequency cornering) • Transient lateral dynamics (wide frequency range steering input) • Longitudinal acceleration (throttle inputs) • Longitudinal deceleration (braking inputs) • Road disruption input (suspension kinematics and ride dynamics) A sixth group is defined as well, called 'other maneuvers'which are imitations of a real-life situation such as double lane change or a fishhook -but are not considered as a primary validation maneuver.   [20] The tests performed in [14] and [15] include steady-state, transient and frequency domain responses. The following maneuver sequence was followed: • Quasi-steady-state • Step response • Pulse response (evaluated in frequency domain) • Real-world like maneuver: lane change Regarding data comparison time domain was used for steadystate, low-frequency, and nonlinear effects, then frequency domain was used for high-frequency transient maneuvers.
Wade-Allen et. al. [17] used both steady-state and transient maneuvers: "In most of the papers no validation metrics or confidence intervals are used and no statistical analysis is performed in relation with validation. Instead, a subjective and qualitative judgment is reached through visual graphical comparison of overlaid time histories of test and simulation results usually." [3] A proper model validation framework should contain a method for comparing time histories in order to quantify the discrepancies and get a picture of the degree of validity. Oberkampf and Barone in [20] discussed various features that should be incorporated or excluded in validation metrics and developed validation metrics that are based on the statistical concept of confidence intervals.
Oberkampf [21] and Trucano [22] argued that "both uncertainties and errors should be quantified in the comparison of computational and experimental results." Accordingly, Fig. 1 illustrates the flowchart of comparing system response quantity (SRQ) obtained from both the simulation and real-life experiment. As stated in [20] "The SRQ can be any type of physically measurable quantity, or it can be a quantity that is based on, or inferred from, measurements. For example, the SRQ can involve derivatives, integrals, or more complex data processing of computed or measured quantities such as the maximum or minimum of functionals over a domain. When significant data processing is required to obtain an SRQ, it is important to process both the computational results and the experimentally measured quantities in the same manner." They refer to validation metric, as the "mathematical procedure that operates on the computational and experimental SRQs." The main task to validation is comparing the validation metrics' results with the accuracy requirements (which depends on many factors) for the intended use of the model. In [20] three validation metrics approaches were reviewed: parameter estimation and system identification, hypothesis or significance testing, Bayesian analysis, or Bayesian statistical inference. They proposed six properties that validation metrics should include to be useful in the engineering decisionmaking process. The six attribution are presented here in a manner that only includes the information relevant to our current vehicle simulation domain: • Validation metrics should either: -explicitly include an estimation of numerical error in the SRQ resulting from the computational simulation -such as including the upper and lower estimated bound on the error in the SRQ, but it would add significant complexity to the calculation and interpretation of the metric. --Or exclude the numerical error in the SRQ, only if the numerical error was previously estimated to be small compared to the experimental uncertainty.
• The metrics should be a quantitative evaluation of the aggregate accuracy for a specific SRQ, including: -the combined modeling assumptions, -the physics approximations, -the physical parameters of the model.
• A metric should include, an estimate of the error resulting from post-processing of the experimental data that is compared to the simulation result. • A metrics should explicitly incorporate an estimate of the measurement errors in the experimental data for the SRQ that are the basis of comparison with the computational model. There are two types of measurement errors: bias (systematic) errors and precision (random) errors. The minimum requirement for validation metrics is to include an estimation of precision errors. As much as possible, the metrics should include an estimate of bias errors as well. • A metric should take into account the number of experimental measurements. The authors emphasize the importance of multiple measurements and estimating the accuracy of the experimental result. • A metric should not include any indications of the level of agreement between computational and experimental results (e.g. characterizing them as "good" or "excellent"). Validation metrics should be used to assess the degree of agreement between computational models and experimental measurements. The metric should be kept separate from how satisfying the results are. In [23] common measures are evaluated used to quantify the discrepancy between time histories in fields such as statistics, computational mechanics, signal processing, and data mining. The following existing metrics were evaluated: • Vector norms • Average residual and its standard deviation • Coefficient of correlation and cross-correlation • Sprague and Geers (S&G) metric Their proposal is a structured combination of some of these measures. The new metrics classify error components associated with three physically meaningful characteristics (phase, magnitude, and topology), and utilizes norms, crosscorrelation measures, and algorithms such as dynamic time warping to quantify discrepancies. [23] For phase error cross-correlation method is used with a tunable penalty function to have a linear penalty for small time-step (local errors) and larger time-step (global errors) differences. Magnitude error is analyzed after minimizing the global, local phase difference between the data sets and also the slope differences -because slope difference is a topological error, not a magnitude error. Dynamic Time Warping is used to reduce local phase and slope differences. After that L1 vector norm is used to measure the relative magnitude differences. The topology error -the measure of discrepancies in the slope -is calculated on the derivative of the time-shifted, warped channels. Then L1 norm is used to quantify the topology error. [23]

D. CONCLUSION OF THE LITERATURE REVIEW
The validation of computational models is well established. Several papers have been published in this field in the last decades. Recently, the quantification of discrepancies between simulated and experimental measurements' time histories improved a lot. Also, significant efforts have been undertaken to develop the so-called validation metrics. However, only a fragment of these methodologies has been implemented in the practice of vehicle dynamics.
The mathematical background of vehicle dynamics is also well established and simulation models in the field are used in a wide range of applications. Several methods and ideas can be found regarding parameter measurements and vehicle dynamics tests for model validation. The problem is that usually, they are only applicable for a narrow range of scenarios -for example only for lateral dynamics in the linear range of the tire. However, there is no general framework that gives a concise guideline for the wide range of vehicle dynamics model validation -that includes the determination of validity criteria, vehicle parameter measurements, vehicle dynamics tests, validation metrics for a given simulation model domain.
According to the above, the main research gap targeted in this paper is the general idea for a comprehensive vehicle dynamics model validation framework, which is based on a sophisticated vehicle dynamics measurement system. In this paper, besides the framework, some automated parameter estimation methods are also presented as practical use cases to demonstrate the idea. Finally, the result of model validation is shown, where the previously mentioned estimated parameters were used.

III. FRAMEWORK PROPOSED FOR VEHICLE DYNAMICS MODEL V&V
In this chapter, the proposed V&V framework for vehicle dynamics models is presented with a high-level system layout and some general explanation. Our goal is to create a sophisticated system that gives a guideline for the majority of vehicle model validation.
Our goal is to develop a system where the majority of the iterative tasks are swapped by automated systems. The general idea is shown by Fig. 2. Following the definition of the validity criteria, individual parameter estimations for each important subsystem are performed to the most sufficient level, implying that the subsystems with a significant impact on vehicle behavior in the investigation domain should be thoroughly measured. In addition to the individual parameter measurements, the measurement system constants -such as the position of the acceleration sensor in the chassis coordinate system -must be measured since they are critical for proper post-processing of the vehicle dynamics measurements. In the next block vehicle dynamics measurements are carried out with a sophisticated measurement systemwhich is capable of measuring all the important movements, phenomenons such as vehicle body movement, suspension movement, tire forces and moments, side-slip angle, camber, etc. Then based on the measurements -in the previous two blocks -the vehicle model parameter identification is solved by an automated system (denoted by the dashed line on Fig. 2), as an output, it gives the base parameter set for the simulation environment, also after the post-processing the driver inputs (steering wheel angle, throttle position, brake pedal force/position, gear) and the measured system outputs (vary for each model, but usually vehicle speed, longitudinal and lateral acceleration, yaw velocity, etc.) are present for the comparison of simulation and real-world response quantities. Finally, the recursive process of validation (denoted by doted line on Fig. 2) is to be fulfilled by appropriate machine learning algorithm [24], the main steps of the iterative validation process are presented with the blue and red shapes on Fig. 2. As an output of the parameter estimation, the mean value of the parameters and the uncertainty of the estimation are given. This will define the base parameter set for the simulation. The iterative loop is the following: • Running the simulation using the driver inputs from vehicle tests. • Compare the two SRQ set by computing the previously defined validation metrics. • Compare each metric with the belonging validity criteria. VOLUME 4, 2016

FIGURE 2. Proposed framework for vehicle dynamics model validation
• If the criteria is not met, then fine-tune the vehicle parameters, in the range given by the uncertainty of the estimation. • The process ends when the validity criteria is met. In the following subsection, the validation metrics and validity criteria for vehicle dynamics models are discussed in more detail.

A. SPECIFICATION
In Fig. 2 the steps of the validation process are defined, but in order to have a complete framework, some preparatory tasks need to be included which are not in the figure. The first step -following the nomenclature of the widely used V-model -is the specification, which contains all the initial information gathering regarding the real-world system, the simulation environment, and the purpose of the simulation. This information all necessary to define the following: • initial vehicle parameter set, • the required vehicle dynamics measurement system (sensors, accuracies), • the list of individual parameter measurements (if necessary), • the vehicle dynamics test cases, • the validation metrics and validity criteria.
An important and one of the initial steps is to define the validity criteria. Prior in this paper, a literature review regarding validation metrics are presented. The suggestions presented in section II-C should be implemented when these metrics are defined for vehicle dynamics models. As discussed previously in the above-mentioned section, a validation metric is the quantification of the discrepancies between the measured and simulated system responses.
Each validity criteria defines the desired maximum value of the belonging validation metric. It is easy to see, that the defined metrics and criteria can be different for each vehicle dynamic model and application. For example, different SRQ is important for ride-comfort analysis, than lateral dynamics, also different accuracy is required for a high-level analysis of the vehicle dynamics, than for a deeper investigation of a subsystem.
Each computational model has several input and output quantities, but depending on the use case the importanceand required accuracy -for each SRQ can be different. Also, the input parameters and the model operating conditions (e.g. lateral and longitudinal acceleration, speed) can be in different ranges. Answering the following questions: "What question needs to be answered using the simulation-based investigation?" and "What engineering decision needs to be supported with its data?" will give information about: • Which system response needs to be investigated during the validation process, • and what the required accuracy is.
The validity criteria is a list of the specified maximum values of the different validation metrics for the given application. These values need to be defined based on the previously gathered information, basically, the validation process will end if these criteria are met -meaning the model will mimic the real system with the desired accuracy.
When defining the required accuracy of the validation metrics the following factors should be considered according to [20]: • complexity of the model, and the engineering system; • difference in hardware and environmental conditions between the engineering system and the validation experiment; • increase in uncertainty due to extrapolation of the model from the validation conditions to the conditions of the intended use; • the risk tolerance of the decision-makers involved; • the consequence of failure or underperformance of the system. The validity criteria for a vehicle dynamics simulation model in our system is the set of SRQs and the acceptable discrepancy in the given input range and the weight of the SRQ. See an example in table 1. Where V M x is the validation metric containing the three quantities presented in [23]: phase, magnitude, and topology, V C x is the validity criteria for that metric -the maximum of the validation metric for a given SRQ, the weight represents how important the given criteria is. For example, the second row of table1 is meaning that in the −5..5m/s 2 lateral acceleration range the discrepancy of chassis roll angle (V M 2 ) should be below the validity criteria (V C 2 ), and the weight of this indicator is 0.7. As there are no standard metrics and criteria for vehicle dynamics model validation, determining the required accuracy is part of the validation work.
In order to have one specific value that represents how accurate the model is, we introduce the "degree of validity" that is defined as the weighted sum of the discrepancies of each validation metric from the validity criteria.
Where • DoV is the degree of validity, • V M n is validation metric n, • V C n is the validity criteria n, • w n is the weight for validation metric n. This is important for the conventional model validation methods where parameters are modified manually in an iterative way, as well as for a machine-learning based validation systems. In the latter case, the DoV can be used for the reward function/ cost function.
If the SRQs necessary for the validation and the SRQs for the parameter estimation are defined, then besides the vehicle dynamics tests, the list of sensors can be defined for the vehicle dynamics measurement system. As mentioned in section II, the validation metric should consider the uncertainty of the measurement system. Therefore if the validity criteria is defined, the required accuracy of each sensor in the vehicle dynamics measurement system can be defined.
At the end of the specification step, the below-listed elements are at our disposal: • Validity criteria. • Requirements for the vehicle dynamics measurement system. • Requirements for the vehicle dynamics tests.
• List of simulation model parameters.
• List of vehicle parameters that need to be measured.

B. VALIDATION CONCEPT
The following step -also based on the V-model -is the validation concept. This step is the comprehensive planning of the measurement and validation processes. Here basic decisions need to be taken, that have an effect on the whole following processes, for example: defining the layout of the measurement system, deciding the estimation/measuring method for each parameter, detailed planning of the vehicle dynamics tests.
Based on the requirements from the previous step, the vehicle dynamics measurement system can be specified and composed in more detail. Also, parameter measurement/estimation can be detailed. Our approach for parameter determination is the following. For sufficient data from vehicle dynamics measurement -which is also important for the accurate validation metrics -a sophisticated measurement system is essential. This allows measuring, estimate vehicle parameters during the dynamic tests, which are required anyway for the validation. With well-planned vehicle test scenarios, we can gather sufficient information for each vehicle parameter estimation. Therefore, the separate measurements such as mass, weight distribution, inertial parameter measurements, suspension K&C, engine dynamometer etc. can be augmented with the a merged dynamic test based estimation or in some cases the dynamic estimation may replace some individual measurements. It is important to note that each subsystem that has a significant effect on vehicle behavior must be measured individually with the greatest accuracy possible. For example, if the suspension elasto-kinematic behavior is to be investigated using vehicle dynamics simulation, then the dynamic test based method is probably insufficient.
Shortly the approach is that for example, during a straightline test (acceleration, constant speed, coast-down, braking) sufficient data can be gathered from the following vehicle subsystems. By measuring the tire forces and moments, wheel angular velocities, and vehicle speed, we can estimate parameters for the powertrain (engine torque characteristics, gear ratios, efficiencies), tire (longitudinal slip characteristics, rolling radius, rolling resistance, loaded radius), aerodynamic parameters (drag and lift forces), brake system parameters (brake force distribution, brake pressure -braking torque characteristics). During J-turn maneuver, tire lateral characteristics can be measured, also vehicle transient turning behavior. With a step steer maneuver the vehicle yaw inertia and also some tire parameter (relaxation length) can be estimated.
The estimations discussed in the above two paragraphs can create additional requirements for the measurement system.

VOLUME 4, 2016
Based on the requirements for test cases given by the previous step (III-A) and the test requirements for parameter estimation, the detailed vehicle dynamic test plan can be created, including all the test cases that are necessary for the parameter estimation, for the validation process and system warm-up, initialization as well.
Also based on this information the necessity of the individual parameter measurements can be decided.
At the end of this block the following information is at our disposal: • Specification and detailed composition of the vehicle dynamics measurement system. • Specification and detailed plan for the vehicle dynamics tests. • List of vehicle parameters with the measurement or estimation method.

C. VEHICLE PARAMETER ESTIMATION
The idea is that utilizing the information gathered during dynamic test events some of the necessary vehicle subsystem parameters can be estimated with sufficient accuracy. Furthermore, by knowing the uncertainty of each estimation during the iterative model validation part we can further improve the accuracy of these parameters, by fitting the vehicle model SRQs. In the proposed system we would mostly rely on the dynamic test-based parameter estimations, but 'workshop measurements' are inevitable. Besides the case discussed in the previous section, some measurements also necessary for the vehicle dynamics measurement system as well, for example defining damper potentiometer motion ratios, positions, and orientation of the GPS and IMU (Inertial Measurement Unit) systems in the vehicle coordinate system. Also, some parameters are hard to estimate but very easy to measure instead, such as wheelbase and wheel track.
The different subsystem parameter measurements and estimations are discussed in the following subsections, beginning with the tire -as it has a significant impact on vehicle behavior -and then moving on to all of the vehicle subsystems.

1) Tire
Tire characteristics have a major effect on vehicle behavior, it influences grip, balance, control, and stability, hence the tire model is a crucial part of each vehicle model [25], [12].
The goal here is to gather information about the force and moment characteristics to fit the Pacejka Magic Formula. Basically, there are two options for tire characterization measurements: indoor and outdoor testing.
Indoor testing is carried out on a "flat-trac" tire force and moment measurement system such as [26]. In this case, the test conditions are well-controlled in large ranges. This means that the tire can be tested under a wide range of normal load, slip ratio, slip angle, inclination angle, pressure, etc. Also, the temperature of the tire can be well-monitored and controlled. For example between two side slip angle sweep tests, a condition can be set that the next measurement only starts if the tire surface cooled down to a specific temperature. But there is also a downside to this method, although the surface of the rotating belt is normally covered with an abrasive coating to better represent a real road surface, it is still different from it and this has a significant effect on the measurement results [27], [12].
Outdoor testing is executed with a real vehicle equipped with a measurement system (acquiring information of the following: tire forces and moments, side-slip angles, dynamic camber angles, tire pressures, tire temperatures, etc.) usually on a proving ground. The advantage is that it is on a real road surface, therefore the tire operating conditions are realistic. The downside is that the parameters that have a big influence on the grip are hardly controlled, also the ranges are usually smaller. For example, the normal load is limited by the vehicle static corner weights and the load transfer, the inclination angle is limited by the static camber, chassis roll, and suspension/steering kinematics, and compliance. The maneuvers need to be chosen with care to have desired operating conditions for the tires.
Obtaining data from only outdoor testing should consider that, the side-slip angle sensor is mounted to the wheel, therefore it measures the wheel's side slip, not the tire's. If lateral force is acting on the tire, the contact patch moves sideways relative to the wheel, therefore during this movement, there will be a discrepancy between the tire and wheel slip angles. This phenomenon can be modeled and taken into account if there is information regarding tire lateral stiffness. This may need to be measured separately before the dynamic tests.
The most accurate option is to obtain information from both measurements, create a tire model parameterization based on flat belt measurements and use the vehicle-based tire test data to scale this model to the real road surface [27].

2) Steering, Suspension Kinematics and Compliance
Suspension subsystem -similarly to the tire -has significant effect on vehicle behavior as it directly influences the tire operating conditions (normal load, camber angle, etc.). The most complete and most accurate method for suspension kinematics and compliance measurements is using K&C test machines such as [28].
However by measuring the • chassis movement (yaw, pitch, roll angles, ride height), • each wheel position relative to the chassis (wheel travel, steering angle), • the wheel orientation relative to the ground (slip angle, inclination angle loaded radius), • and the tire forces and moments, • and steering wheel angle, during vehicle dynamic testing, the steering and suspension kinematics and compliance characterization can be estimated, although the measurement precision falls behind the previously mentioned K&C testing.
Also some basic measurements need to be taken in the workshop regarding the the starting suspension geometry (camber, toe, ride height, etc.).

3) Aerodynamics
Wind tunnel testing is the ultimate solution for aerodynamics because it provides complete information about aerodynamic drag, lift, center of pressure, and so on. However, basic aerodynamic parameters can be estimated by measuring the tire normal forces during dynamic tests. The normal forces on the two axles plotted against vehicle speed can give information about the downforce and downforce distribution (in other words center of pressure) of the vehicle. Also aerodynamic drag can be estimated during coast down tests. Speed must be measured with caution since, in most cases, the speed of the vehicle is measured, but in this case, the speed of the airflow must be measured. The effect of wind can be reduced by conducting testing in both directions and having accurate data regarding wind conditions, but it is preferable if the system is equipped with pitot tubes that can measure the speed of the airflow.

4) Powertrain
Engine-powertrain dynamometers provide the most accurate solution for the powertrain measurements. If the vehicle dynamics measurement system is utilized for this operation, additional data from the vehicle on-board diagnostics (OBD) such as engine rpm, throttle position, clutch application, selected gear, and so on are required. By measuring throttle position, engine rpm and driving torques on the wheel -by a wheel force transducer -the engine torque map can be obtained during a straight run acceleration test. This has also the drivetrain efficiency included, as measured on the wheel. The gear ratios can be defined by seeing the relation between the engine and wheel rpm.

5) Brake system
In most of the vehicle dynamics simulation software usually a "brake factor" is defined, which is the amount of brake torque produced per unit brake pressure. This can be estimated by measuring the brake torque on the wheels, brake pressures at each brake line. Measuring brake torques and pressures gives information also about the brake distribution of the vehicle. Adding pedal travel and force the "brake interface" can be characterized. Because brake disc temperature has a substantial effect on braking performance, disc temperature sensors should be installed in order to explore the temperature dependency of the disc-pad coefficient of friction.

D. VALIDATION PROCESS
Final step is the iterative validation (see on Fig2 in dashed frame). The base parameter set for the vehicle containing most of the crucial parameters such as tire characteristics, inertial parameters, suspension geometry and kinematics, powertrain parameters are given. Also, the test track needs to be parametrized, this information usually can be obtained from the track operating company.
In this automated step, the previously estimated parameter set is filled in the vehicle model. A test run is carried out using the driver inputs from the real test. Then by comparing the two time-histories an algorithm will calculate the degree of validity then, modify the parameters to achieve the validity criteria. For sure, some constraints need to be defined, to avoid unreal values during this process. As mentioned in a previous chapter, this validation algorithm will modify the parameters within the measurement/estimation accuracy range. For example, if the estimated vehicle yaw inertia is 3200kgm 2 and the uncertainty is +/-200kgm 2 , then the algorithm can modify this parameter in that range.
In the beginning of section III the block diagram of the proposed framework is presented augmented with the preparatory steps. In sub-section III-A and III-B these preparatory steps are defined. Than in sub-sectionIII-C the parameter estimation is discussed. Finally, the iterative steps of the validation is presented in this section.
In the following, a case study is presented for parameter estimation utilizing the above-described principles.

IV. CASE STUDY: PARAMETER ESTIMATION
As a case study, a model validation was carried out using some of the above-mentioned principles. In this section, some examples are presented from the automated parameter estimation part of our model validation system (see the block denoted by the dashed line in Fig. 2). During the validation process some parameters were automatically estimated from the vehicle dynamics test data, but some of them were measured manually in a conventional way. Most of the estimation methods are based on statistical estimation and carried out in Motec i2 data analysis program [29].
The vehicle used for the validation is a Mercedes-Benz Cla 250 7G-DCT, which was equipped with the vehicle dynamics measurement system of John von Neumann University. The test cases were carried out on the Dynamic Platform of ZalaZONE Proving Ground.

A. MEASUREMENT SYSTEM INTRODUCTION
As mentioned before, to execute proper validation of a sophisticated vehicle model, an advanced vehicle dynamics measurement system is crucial. The measurement system contains several sensors, the following is used for the presented estimation methods: • Kistler RoaDyn S625 -wheel force transducer • Race Technology Speedbox INS -GPS and IMU • Motec 58043 -brake pressure sensor • Motec 59006 -steering angle sensor With these sensors, the parameters presented in table 2 can be measured during the tests. The data logging system is based on Motec products. Each channel is logged by an ACL (Advanced Central Logger). The post-processing and parameter estimation is done in Motec i2 software. Math channels are defined to calculate vehicle inertial parameters, powertrain and brake parameters, and visualize tire force and moment characteristics, etc. In the followings, some of the methods are presented. VOLUME

B. MASS AND CENTER OF GRAVITY (CG) POSITION ESTIMATION
The inertial parameters (mass, weight distribution, CG position, etc.) are generally easy to measure with weight scales. Our approach -as previously described -is to estimate these parameters during dynamics tests by utilizing the data from the advanced measurement system -which is necessary for proper model validation anyway -to track the changes during long test events. In order to have a baseline value for comparing the dynamic estimation method, a conventional mass and CG position measurement was carried out. During this measurement, the vehicle was already equipped with the measurement system. Therefore, the corner weights were measured with a Breda Racing weight scale set and the Kistler wheel force transducers as well. See the measurement layout in Fig. 3. As it can be seen, the weight scales on the front axle are equipped on a vehicle lift, this way the height of the CG can be measured using the method described from pages 27 to 33 in [30].

1) Estimation of mass and axle load
First, the mass and CG longitudinal position estimation is presented. The conventional method is presented first as a baseline. For this measurement, the weight scales were on a  It is important to note that the corner weights measured by Kistler need to be compensated -by adding the mass of the wheel's outer part. This is necessary, because the structure of the measuring wheel, it is not capable of measuring its own weight. All measurements were repeated three times. See the results in table 3. For corner weights and total mass, the differences of the two methods were below 0.6%. In the following, the dynamic test-based method is presented. The basic principle of the parameter estimation system is that as an initial step, the calculation channel filters out situations -with an "if" function -that are not suitable for determining the given parameter. Then, using the measured channels the math channel calculates the parameter in each time step, then a separate math channel determines the statistical mean value and standard deviation of the channel.
In case of mass and weight distribution, the wheel forces are close enough to the static load if the vehicle is moving straight on level ground without any acceleration and with a low speed -to avoid aerodynamic lift effects, or standing on a level surface and the wheels are not tensioned (such as after braking while still applying brake pressure). The corner weight channels give the calculated value if the following criteria are met: • The absolute value of the longitudinal acceleration is less than 0.5m/s 2 . • The absolute value of the lateral acceleration is less than 0.5m/s 2 . • The absolute value of the steering angle is less than 10°. • The vehicle speed is less than 50km/h. The first two conditions ensure a steady-state, the third filters out the effect of the steering geometry (normal force variation due to king-pin inclination), the fourth condition filters out the effect of aerodynamic forces. Then, using the filtered data, the parameters can be calculated with the appropriate equation.
Total mass of the vehicle: The mass on the front axle: The mass on the rear axle: where F z-s are the wheel normal forces and m wf t (20kg) is the mass of the outer part of the wheel force transducer, G is the gravitational acceleration. In the lower indexes FL is front left, FR is front right, RL is rear left, and RR is rear right.
The front weight distribution is: Where m F is the mass on the front axle. For each parameter two channels calculate the statistical mean and standard deviation for the entire data set. This gives a specific value and the standard deviation provides information about the accuracy of the estimate. After the weight distribution is given the longitudinal position of the CG can be calculated as follows: and the rear Where • a is the longitudinal distance of the front axle from the CG. • b is the longitudinal distance of the rear axle from the CG. • W B is the wheelbase. The dynamic mass and weight distribution estimation method allows taking into account the changing conditions during a longer test case such as changes in mass and weight distribution due to fuel consumption or passenger changes.
Based on the results it can be concluded that the dynamic estimation method can measure the mass and CG x and y position of the vehicle in motion with sufficient level of accuracy.

2) Estimation of CG Height
Center of gravity height affects the dynamic load transfer, hence the load transfer can be measured -by measuring all the tire normal forces -the (h CG ) can be estimated. For this, any steady-state behavior is suitable where the vehicle has sufficient acceleration. Here data from a steady-state braking maneuver was used. In steady-state the longitudinal load transfer can be calculated with the following equation: Where a y is the lateral acceleration and W B is the wheelbase. By rearranging the equation we get the following: This equation only true in steady-state, therefore all transient data needs to be filtered. This was done with the following conditions:  Fig. 4 shows the results of the estimation during a braking maneuver. The estimated and filtered calculation channel can be seen at the top, which displays value only when the previously described requirements are met.
In a previous work [31], this method was compared to the widely used "Lifted axle" method, which is based on the static load transfer due to axle lifting. The difference between the two measurements was 9,2mm (1,7%).

C. YAW INERTIA ESTIMATION
This parameter was estimated using data from a step-steer maneuver. If we consider the car as one rigid body, then the yaw acceleration is proportional to the yaw moment and VOLUME 4, 2016 • F y is the lateral force in the wheel coordinate system for each wheel; • F x is the longitudinal force in the wheel coordinate system for each wheel; • δ is the steering angle of each wheel including steering, static toe, kinematic toe, elastic toe; • t is the lateral position of each wheel from the CG. The lateral and longitudinal forces given by the wheel force transducers are converted into the chassis coordinate system to correct the effect of wheel steering angles. Also, there is a lag in the signal due to the wireless communication between the measuring wheels and the on-board electronics. Therefore all the channels from the Kistler system were adjusted with a "time shift" function in Motec i2 by -63ms. As the yaw velocity change occurs rapidly during a step-steer maneuver all the channels used for the estimation must be synchronized.
For the proper estimation, the data set needed to be filtered so that the math channel only uses data when there are sufficient yaw moment, and yaw acceleration.
• Absolute value of the yaw moment is above 4000 N m • Absolute value of the yaw acceleration is above 50 • /s 2 If the above-mentioned conditions are true then the channel gives the calculated yaw inertia, if not true, then gives no value. Finally, the mean value and standard deviation of the channel are calculated. In

D. ENGINE TORQUE CHARACTERISTICS ESTIMATION
In concerns of the powertrain, some values are available from any vehicle database, such as gear and final drive ratios, as well as engine torque characteristics at full throttle. Although engine max load characteristics are insufficient for a vehicle model, also real values frequently differ from catalog values.
We have information about the propulsion torque on the wheels due to the measurement system, therefore we cannot measure the pure engine characteristics but rather with the overall drivetrain efficiency.
The propulsion torque was calculated with the following equation: where M y-s are the driving torques on the wheels. Then this channel was filtered by the following conditions to only have pure acceleration without brake application and to only investigate the required conditions: throttle position and gear: • Throttle position (from OBD) is more than 95%. • Brake pressure (front and rear) is below 1bar. • Gear is 2 (in the example). From the filtered wheel torque channel then the engine estimated torque was calculated with the following equation: The results of full-throttle characteristics in second gear can be seen in Fig. 5.
The test scenario was an acceleration from standstill with different throttle positions. This way sufficient information can be gathered to have full and part load characteristics in all gears. Measuring in all gears is important because we can get a picture of the efficiency of each gear.

E. ESTIMATION OF BRAKE PARAMETERS
In the simulation software, one of the main brake parameters is the so-called 'brake factor', which gives the braking torque exerted by unit brake pressure. Its unit is N m/bar. In addition, we need to specify the brake torque distribution, which is a percentage of brake torque on the front axle divided by the total braking torque. Finally, the value of the maximum braking torque that can be applied by the braking system shall be defined. To determine these, the following math channels were created.
Braking torque on each wheel is the M y channel filtered by the following conditions: • Brake pressure at the given wheel is more than 10bar. • Throttle pedal position is below 5%. The brake factor is calculated with the following equation: The brake torque distribution is calculated with the following equation: The results of the estimation can be seen in Fig. 6

F. MANUAL VALIDATION OF THE VEHICLE DYNAMICS BASED ON THE ESTIMATED PARAMETERS
In this section, we present the results of the vehicle model validation referring to Fig 2 denoted by dotted line.
For the validation, conventional manual method was used. Our goal was to test the output correlation with the previously estimated parameter set. Generally, it is an iteration process, where the main steps are the following: run the simulation, compare the outputs with the real logged data, based on the differences look for the reasons, then change the parameter FIGURE 6. Estimation of brake parameters that affect that particular phenomenon. For the simulation, a high fidelity vehicle dynamics simulation software -AVL VSM [32] -was used which is frequently applied by the automotive industry. We considered that the software has well established computational base and is used by several industrial parties. Therefore during the validation process, we focused on the fine-tuning of the vehicle parameters and not on the equations behind the model.
The following test cases in the simulation model was carried out: First test case is stand-still equilibrium, the vehicle stands on a level road, then steering is turned both sides. This test is for checking the standstill equilibrium: ride heights, static normal forces, etc. and to check the suspension geometry and steering characteristics: steering angle vs wheel angle, camber variation due to steering, etc.
Then a straight run test (acceleration, constant speed, coast-down, braking) for checking the rolling resistance, aerodynamic drag, propulsion characteristics, braking performance, longitudinal characteristics of the tire.
The next test was a steady-state cornering test to investigate the over-, understeer characteristics of the vehicle, tire lateral performance, roll gradient, and roll stiffness distribution.
Then transient steering input (sinusoidal, step steer) to investigate the transient response, tire lateral performance, etc. and combined test cases (acceleration or braking in turn).
Finally combined test case with the driver input from real logged data that can be seen in Fig. 7. The graph shows the vehicle speed and the tire longitudinal forces, red is measured, green is the simulated result.

V. CONCLUSION AND FUTURE WORK
First a literature review was presented which concluded that there is no standard general framework for vehicle dynamics model validation. The topics of validation metrics and vehicle parameter measurement and estimation was investigated more deeply. VOLUME 4, 2016

FIGURE 7. Comparison of experimantal measurements with simulation results
A high level layout for a vehicle dynamics model validation framework was presented. Also insights regarding parameter estimation and validation metrics, validity criteria were discussed. Then, a part of the proposed framework was demonstrated via a case study using real-world vehicle measurements and high-fidelity automotive simulation software.
The goal is to create a framework with predominantly automated processes. For this as a base, the validation metric and validity criteria need to be further developed. The task of determining which parameters should be examined and what level of accuracy is appropriate for the validation of a given simulation model is not trivial. Investigating this topic and developing a comprehensive metric for vehicle dynamics model validation metric is part of the future work. Also, the automated parameter estimation needs to be further developed and augmented for more parameters. Finally, the iterative process of parameter fine-tuning and data comparison is to be automated.

VI. ACKNOWLEDGEMENT
ATTILA WIDNER received the M.Sc.degree in vehicle engineering in 2020 at Széchenyi István University. He started his doctoral studies at the Budapest University of Technology and Economics in 2022. In parallel, he is working both at John von Neumann University and HUMDA Lab (which is the research institution of the Hungarian Motorsport Development Agency) as a researcher in the field of vehicle dynamics simulation and model validation. TAMÁS TETTAMANTI received the M.Sc. and Ph.D. degrees in traffic engineering in 2007 and 2013. He acts as an associate professor and also participates in research and industrial projects as a researcher as well as project coordinator. His main interests include road traffic modeling, estimation, and control with applications in intelligent, and autonomous transportation systems. He is co-author of over 140 scientific papers, two patents, and several books. He is member of the Public Body of Hungarian Academy of Sciences (Committee on Transport Engineering). He is a Management Committee member at European Cooperation in Science and Technology COST Action CA162222 (Wider Impacts and Scenario Evaluation of Autonomous and Connected Transport). VOLUME 4, 2016