DNN-Based Surrogate Modeling-Based Feasible Performance Reliability Design Methodology for Aircraft Engine

The risks and costs of developing a new aeroengine are fundamentally depending on the performance design final proposal. Thus, this paper presents a novel aeroengine performance design methodology that is committed to managing the effectiveness and economic availability of the design proposal. To reach such a target, the presented methodology formulates the traditional thermal cycle design problem as a reliability-based fuzzy optimization. The performance reliability is predicted by the deep neural network (DNN)-based surrogate models while a hyperparameter tuning technique is proposed to find the optimal DNN topology for better implementing a particular deep learning task. The testing results imply the DNN models with optimized topology possess remarkable function approximation capability in global so that achieves significantly higher prediction accuracy. Moreover, the DNN-based surrogate models only cost nearly 0.003% as much computing time as MC simulation (2.3591 sec vs 64746 sec, for 20 samples). Such kind of remarkably higher computational efficiency facilitates the optimization for reliability-based fitness calculation. The efficiency of the presented methodology can be further verified by abundant feasible cycle proposals. The obtained cycle solutions can achieve expected reliability (>95%) in all reference states without unnecessary performance redundancy. Besides, the diversity of feasible cycle solutions contributes to the selection of best proposal associated with engineering situation. The presented effort is favorable to acquire a more cost-efficient design proposal and enrich thermodynamic system design theory.


I. INTRODUCTION
It was inevitable: the desire for flight security always reminded the commercial aeroengine designers of the negative effects of turbomachinery degradation. Specifically, turbomachinery degradation is caused by several mechanisms including erosion, fouling and foreign objects hitting [1]. Long-term effects of these mechanisms induce uncertain changes in turbomachinery blade geometry, including blade surface damage, increased clearances, etc. These geometric changes result in the actual performance of the turbomachinery to uncertainly deviate from its designed performance, which will lead to potential overall performance loss. As an example, the potential overall performance loss would bring The associate editor coordinating the review of this manuscript and approving it for publication was Cristian Zambelli . the following risks to the commercial aeroengine, such as insufficient net thrust and worsening fuel economy. Specifically, the former one threatens the flight performance and safety of the jetliner while the latter one increases the total fuel consumption with a drop in airline profits. Therefore, the potential overall performance loss, which is caused by degradation, remains an ongoing challenge for aeroengine.
To address this crucial issue, the practical approach is to compensate for the potential overall performance loss over the aeroengine life-cycle by maintaining overall performance redundancy. The performance redundancy comes from the further promotion of aeroengine design performance. Essentially, the modern aeroengine is a type of gas turbine engine that operates according to the Breton cycle [2]. The useful cyclic work is the source of aeroengine performance and heavily depends on two key factors. One corresponds to unit VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ cyclic work while the other is the mass flow of working substance. For the first factor, the unit cyclic work is strongly associated with total pressurization rate and adding heat [3]. These two items are respectively defined by some cycle parameters, including the pressurization rate of each compression component and combustor outlet temperature (or called turbine entry temperature, T ET ). Obviously, increasing these cycle parameters can effectively improve the unit cyclic work. For the second factor, there is an approximate linear growth correlation between the cyclic work and the mass flow of working substance. Such relationship points out that more air inflow can directly improve the aeroengine overall performance. However, the reality is beyond these expectations. For instance, based on a given level of cooling technology and material type, turbine blade life is empirically halved for each 10 K rise in temperature of the incoming gas flow [4]. Undoubtedly, the higher TET cycle design scheme would bring more inconveniences for turbine blade design, such as the super-alloy selection, cooling system options and blade structure strength. Moreover, more air inflow means that the aeroengine must therefore be oversized with heavier quality. As a result, hanging the comparatively massive engines would increase the wing root bending moment and even make a possible violation in the aircraft structural strength design. Thus, the extra enhancement of cyclic work highlights the risks, since both technology backwardness and design conflict might undermine this will.
In the final analysis, the commercial aircraft engine is still a for-profit industrial product and must compete in terms of both effectiveness and economic availability. The risk of developing an entirely new aeroengine is so high that manufacturers prefer more conservative candidate proposals with mature technology. Besides, selecting the final design proposal is an extremely complex issue of cost-efficient due to the large capital expenditure and the long payback period. Thus, a variety of feasible design proposals is beneficial for the ultimate decision under an overall consideration of manufacture technique and available materials, etc.
As mentioned above, it is crucial to avoid unnecessary performance redundancy whenever possible through precisely boosting the useful cyclic work. To achieve this goal, thermodynamic cycle design needs to be formulated as an uncertainty-based design problem [5], which quantitatively considers the effect of uncertainty factors and then designs the key parameters to ensure the reliability for working conditions of interest.
Probabilistic design method is proved to be an effective method to solve the uncertainty-based design problem. Lots of relevant research is devoted to probabilistic design methods in the robustness and reliability of aerospace engineering and civil engineering [6]- [9]. These research works wisely utilized the probabilistic design method to settle the quantitative analysis of uncertainty. Then the proper key design parameters were selected to reach the reliability targets. The benefits of these studies encourage further application of probabilistic-based methodology in aeroengine performance reliability design.
Despite the above encouraging progress, a critical issue ought to be noted that hampers the further application of probabilistic design method. To be specific, the key to implementing probabilistic-based analysis and design is to acquire the probability distributions of the concerned parameters. In the pursuit of this target, the essential process is predefined the uncertainty factors and then performing the randomized trials based on Monte Carol (MC) simulation [10]. However, MC simulation is an extremely expensive computational process that requires a huge investment in time. The disadvantageous effect of this characteristic is further amplified in the optimization problem, which requires calculating the reliability-based fitness by MC simulation for all particles (or individuals) in each iteration (or generation). Thus, it would cost unaffordable time to carry out the reliability-based optimization design if hundreds or even thousands of MC simulations are needed.
In order to avoid the time-consuming MC simulation, considerable attention has been currently paid to the surrogate model technique. The surrogate model is explained as a black box program which only imitates the mapping relationship of the real physical models based on some particular mathematical models, such as polynomials [11], support vector machine [12] and Machine Learning (ML) [13]- [15], etc. One branch of ML is deep learning, which has lately attracted extensive interest. Deep learning is usually implemented using a deep neural network (DNN) architecture. The deep is interpreted as the number of layers in the network, which endows the DNN with impressive capability for learning representations of data with multiple levels of abstraction [16]. This remarkable ability makes the DNN become the ideal candidate for the surrogate models.
In this study, the DNN is utilized to develop surrogate models for predicting aeroengine performance reliability in each concerned state. Moreover, to further release the potentials of DNN, MINLP-based hyperparameter tuning technique is proposed to search for the optimal DNN topological structure. Employing the DNN-based surrogate models with optimum topology, the reliability-based fuzzy optimization problem is formulated to solve the feasible thermodynamic cycle proposals. Then the proposed approach is applied to solve the commercial aeroengine conceptual design problem with uncertainty effects. This paper is structured as follows: Section 2 illustrates the modeling technique for the commercial aeroengine. In Section 3, implementation of MINLP-based hyperparameter tuning technique is introduced in detail with its application in DNN training procedure. Section 4 investigates the utilization of DNN-based surrogate models in order to formulate fuzzy optimization for commercial aeroengine performance reliability design. Section 5 quantitatively discusses the optimization results to validate the proposed methodology. Conclusions and perspectives are given in Section 6.

II. COMMERCIAL AEROENGINE MODELING
The aim of aeroengine modeling is to investigate the probability distributions of overall performance parameters associated with the specified cycle parameters and predefined uncertainty. It is necessary to yield the training sets, which are the combinations of thermal cycle and reliability output responses.

A. OVERVIEW OF COMMERCIAL AEROENGINE
This work studies the high bypass ratio dual-spool turbofan engine, which is an extensively-used type to power commercial jetliners. A simplified cross-section of this engine is displayed in Fig. 1, with its major components marked.
The operation mechanism and layout could be briefly described as follows. When the airflow accesses the engine inlet, it is divided into bypass stream and core stream. The bypass stream is compressed and then directly exhausted to produce the main thrust by increasing the inflow momentum through the engine. Besides, a small amount of thrust is generated by the core stream, which is also employed as the working medium for multistage compression, heating, and expansion processes. The compression and expansion are split into parts by LP spool and HP spool. Mounted on the LP spool, fan and booster stage are driven by LP turbine.
Connecting through the HP spool, the HP turbine supplies power to actuate the HP compressor. The kinetic energy source for turbo machines operating comes from burning the fuel in the combustor.

B. MODELING PRICIPLE
Based on the principles of aerodynamics and thermodynamics, this study developed the component-level steady-state performance simulation model. This type of model is widelyused in the conceptual design phase of aviation propulsion system. Moreover, this model is constructed as a modular architecture to improve the modeling efficiency, as shown in Fig. 2.
For evaluating the uncertainty performance behavior in concerned conditions, the established model is composed of two principal parts, including uncertainty turbomachinery performance module and turbofan engine prototype model.

C. UNCERTAINTY TURBOMACHINERY PERFORMANCE MODULE
This module is embedded into the turbofan engine prototype model to simulate the uncertainty turbomachinery performance owing to degradation. The effects of above uncertain changes are represented by changes in turbomachinery VOLUME 8, 2020  performance parameters, including isentropic efficiency (η i ) and mass flow capacity (W cor ). To achieve above uncertain changes of parameters, the relevant measure is to cause random numerical oscillation of theoretical design performance. The theoretical design performance is depicted by the standard maps. These maps represent the technical characteristics of a real turbo machine on the test bench [17]. With the instance of fan, Fig. 3 displays the possible change ranges of efficiency and mass flow capacity based on its standard map.
According to the references [18], [19], these uncertainty parameters could be numerically described by the stochastic uncertainty and obey the normal distribution. Thus, the numerical model of efficiency and mass flow capacity are where, S is the scaling coefficient later in Section 2.4.1; r is the random number predefined to obey normal distribution. Due to the absence of turbomachinery monitoring data from airline company, this investigation bounds the numerical range of uncertainty performance parameters based on the 3-σ principle [20], as shown in Table 1.

D. TURBOFAN ENGINE PROTOTYPE MODEL
As the principle part of the developed simulation model, the prototype model is jointly organized by in-design point simulation and off-design point simulation.

1) IN-DESIGN POINT SIMULATION
The in-design point simulation investigates a certain configurational aeroengine of many possible thermal cycle schemes. Multiple thermodynamic cycle alternatives are studied to comparatively evaluate the overall performances in one reference operating state. Once a thermal cycle candidate is elected, the technical parameters of each turbomachinery component are designed. To be specific, the turbomachinery standard maps are scaled to associate the designed component performances in the design point. In addition, the chosen cycle also decides the feature sizes of aeroengine, such as the throat area of exhaust. In summary, in-design point simulation practically designs the geometry of the aircraft engine, which is prior to the off-design point simulation.

2) OFF-DESIGN POINT SIMULATION
Off-design point simulation calculates the performance of the designed aeroengine under the numerous operating states that described by the regulation schedule, altitude and Mach number in flight. In a particular operating state, the critical issue for off-design point simulation is to determine the collaborative operating condition.
For the studied commercial aeroengine, its collaborative operating condition needs seven decision elements to determine whose vector format could be given as where, ψ i (i = 1, . . . , 5) are the auxiliary coordinates of each turbomachinery component in its scaled maps, which depict the locations of operating point representing the in-service condition; ψ i (i = 6, 7) are two of the N HP , N LP and N t4 , which are chosen according to the user-defined regulation schedule.
To ensure that there is only one solution of decision elements to match one unique operating state, the collaborative operating condition is represented by seven nonlinear implicit equations. These equations supervise the mass flow compatibility among all parts and the power balance for each spool. According to the sections in Figure 2, the equations are formulated as the residual function in the programming and presented as (1) The mass flow residual function of internal duct of fan and booster (2) The mass flow residual function of booster and compressor The mass flow residual function of the compressor and HP turbine The mass flow residual function of bypass and external duct of nozzle The mass flow residual function of LP turbine and internal duct of nozzle The power residual function of the HP spool want f 6 (ψ) = η HP P HPT − P Compr η HP P HPT → 0 (8) The power residual function of the LP spool A nonlinear implicit equation group is composed of all the residual equations and formulated as Thus, solving this equation group is the key to determining the collaborative operating condition, and is presented as where, ε is precision factor and its value is set as 10 −5 . Then the Newton-Raphson algorithm is adopted to solve the ψ, which is updated in each iteration [21].
where, J is the Jacobi matrix that contains first derivatives of the residual errors with respect to the decision elements. Moreover, J is formulated as More details of modeling techniques of each component can be found in these references [22]- [24].

III. DNN-BASED SURROGATE MODEL FOR RELIABILITY PREDICTION
In the reliability-based optimization, it would cause the expensive time cost to calculate each individual fitness by performing the MC simulation. To replace the timeconsuming MC simulation, DNN is exploited to construct surrogate models for predicting the aeroengine performance reliability under multiple flight conditions. Meanwhile, MINLP-based hyperparameter tuning technique is presented to determine the optimal DNN topological structure for enhancing network performance.

A. DEEP NEURAL NETWORK
By abstracting the mechanism of human brain functioning, Artificial neural network (ANN) construct artificial neurons and establish the connections between artificial neurons according to a certain topology structure. Through network topology structure, ANN is intended for the approximation of the implicit non-linear relationship between input and output variables through extracting the features of the training scenarios.
The ANN formulation between output responseŷ and input variables x can be structured as where, f ANN (·) is the ANN model; D train is a training set; is the vector of hyperparameters that will be further explored in the following text. Fig. 4 displays the topology structure of shallow neural network, which is a classical type of ANN. It is composed of an input layer, a hidden layer as well as an output layer. Each layer is connected to the front and back layers by neurons, using the output of the front layer as its input.
DNN is a special form of ANN. Compared with the shallow neural networks, the most significant characteristic of DNN is that it contains at least two or even hundreds hidden layers, as shown in Fig. 5. In this deep-layered, hierarchical structure, multiple nonlinear transformations are implemented to fully extract the features of training data set [25]. The more accurate presentation of data characterizations makes the DNN more capable at expressing the highly complex nonlinear relationship between input and output values.
Despite the remarkable advantage mentioned above, the unprecedented complexity of the DNN topology structure makes it more difficult for hyperparameters tuning. The definition of hyperparameter is the predefined network parameters that programmers ought to specify before ANN training [26]. These presetting design the topology structure, and have profoundly influences on training converge and generalization ability of a neural network [27]. Therefore, the hyperparameters must be appropriately adjusted for training a high-precision DNN.
At present, the leading approach for hyperparameters searching is still the manually adjusting referred to the prior VOLUME 8, 2020  knowledge from previous tuning observations [28]. Frankly speaking, this is already an extremely time-consuming and painful work for even an experienced programmer to obtain a well-tuned shallow neural network. However, a well-tuned DNN is almost impossible to acquire by manual adjustment due to the increase of the number of hidden layers. Taking the 6-hidden-layers DNN as an example, there are 64,000,000 (=10 6 × 2 6 ) practicable combinations to design the topological structure, if the N ne ranged from 2 to 11 and chosen one of the two activation functions for each hidden layer. Hence the previous manual hyperparameters tuning method is facing challenge and it is worthwhile devoting much effort to this.

B. MINLP-BASED HYPERPARAMETER TUNING TECHNIQUE
In order to acquire a well-tuned DNN automatically and efficiently, a new hyperparameter tuning technique is proposed based on mixed integer nonlinear program using Mesh Adaptive Direct Search algorithm.

1) HYPERPARAMETERS OF DEEP NEURAL NETWORK
Three kinds of key hyperparameters, including network topological parameters, learning rate and activation function of each layer are focused in the presented technique as follows.

a: NETWORK TOPOLOGICAL PARAMETERS
The feedforward DNN is investigated and its structure is graphically described in Fig. 4. The structure of DNN is defined by network topological parameters, including the number of layers, the number of neurons in each layer.

b: LEARNING RATE
Based on the gradient descent algorithm, learning rate determines the size of search step for updating connection weights in network training process. If the learning rate is too small, the network training process will converge very slowly. On the contrary, it will cause oscillations near the optimum convergence point, which is difficult to eliminate or even results in training failure. Empirically, its value setting ranges from 0.0001 to 1 that depends on network topology and training data.

c: ACTIVATION FUNCTION
The activation function performs the nonlinear transformation that represents the mapping relationship between input and output response of neurons. Thus, the selection of activation function in output layer and each hidden layer deeply affects the network performance of training and generalization. Three types of classical activation functions, including sigmiod, tanh and purelin [29], are of our major concern. The images and formulations of the mentioned activation functions are displayed in Fig. 6.

2) PROBLEM DEFINITION OF HYPERPARAMETER TUNING
In essence, the aim of hyperparameter tuning is to find a proper hyperparameter combination at a finite admissible set. Therefore, the hyperparameter tuning procedure could be defined as a combinatorial optimization problem. Its optimization objective can be set to minimize the training error of loss function by searching for an optimal hyperparameter solution at a finite admissible set. Therefore, the optimization formulation for hyperparameter tuning can be presented as where, L(·) is the loss function that measures the fitting ability; Z represents the positive integer number set. This optimization problem is illustrated as follows. The DNN is a highly nonlinear model and its loss function is a non-convex function, so loss minimization is a non-convex nonlinear optimization problem. Obviously, the learning rate is a continuous variable bounded in a specified numerical range. However, it is so noticed that N ne of each hidden layer must be set to positive integer. Moreover, the activation function selection of each layer is a categorical variable for which the measurement scale is a set of activation function categories.
Above discussions indicate that hyperparameter optimization ought to be formulated as a Mixed Integer Nonlinear Program (MINLP), which contains the nonlinear objective function and combines continuous variables with categorical variables or even ordinary discrete variables such as positive integer. Owing to different types of decision variables, the MINLP for hyperparameter tuning fails to be solved within the some frequently-used optimizers, such as gradient descent method and various types of evolutionary algorithms.

3) MESH ADAPTIVE DIRECT SEARCH ALGORITHM
To address the above problem, the Mesh Adaptive Direct Search algorithm (MADS) is exploited as the underlying algorithm to perform the MINLP-based hyperparameter tuning [30], [31].
The implementation of MADS algorithm basically depends on the generation of mesh, which realizes the discretization of the design space. The discrete structure of mesh at k th iteration is defined as where, V k is the set of points where the objective function and constraints have been evaluated by the start of iteration k; m k is the size parameter of mesh; z is a n-dimensional integer vector; D is the set of mesh directions and is constructed as a n × n D matrix representing a fixed finite set of n D directions. an integer vector. In a series of varying-size meshes, MADS algorithm implements an adaptive seek iteratively. Each iteration contains two steps, including the search and the poll. Through the user-defined search strategy, such as LHS [32] and VNS [33], the search step generates multiple testing points anywhere on the mesh to find a better solution. If the current search step fails to achieve this goal, the poll step is performed to explore the mesh near the x k . The following set of poll trial points is strictly defined as where, D k is the set of poll directions; p k is the poll size parameter; To be specific, the poll trial points are generated at the place that is in the vicinity of the best current solution x k . The distances between x k and poll trial points are limited by the size parameter of poll. Within two-dimension mesh, Fig. 7 displays the locations of poll trial points in three iterations and graphically compares the size parameters of the mesh and poll. For the next iteration, the mesh size parameter is also updated with m k+1 = κ ω k m k (18) where, κ < 1 is a fixed positive. ω k is strictly negative if the iteration k failed to find a better solution, or positive if iteration k success. In other words, when an iteration fails to improve the current best solution by search or poll, the next iteration is initiated on a finer mesh. On the contrary, a coarser mesh is desirable.
In Table 2, the implementation of MADS is briefly illustrated within the pseudo code.

C. DNN TRAINING PROCEDURE
Based on the LM back-propagation algorithm [13], network training iteratively adjusts the network parameters (mainly connecting weights) to approximate the mapping relationship of input and output response from the feeding training sets. To acquire qualified training sets, this subsection investigates the necessary mathematical skills to preprocess the original data, including feature scaling and statistical analysis. Nevertheless, this subsection also studies the overview of DNN training with the application of MINLP-based hyperparameter tuning.

1) FEATURE SCALING FOR INPUT OF TRAINING SET
The numerical range differences of input elements cause the gradient direction more easily deviating from the optimal search direction, which is unfavorable to the efficiency of network training. To address this issue, feature scaling is exploited to eliminate this effect of scale by normalizing the dimension of each decision variable to the same numerical range [34]. Using the Z-score normalization, the normalized value of samples {x (i) , i = 1 . . . , n} in j th dimension iŝ where,x j is the mean; σ j is the standard deviation. Through the above numerical transformation, each dimension of x (i) is centered to mean 0 and scaled to standard deviation 1.

2) STATISTICAL ANALYSIS FOR OUTPUT OF TRAINING SET
Based on the turbofan model mentioned in Section 2, the MC simulation is undertaken to acquire the data of the concerned performance parameters. Referring to the obtained data with specified performance benchmarks, statistical analysis is performed to calculate the performance reliability of the predefined key design parameters.
The reliability value R of the i th scenario in the k th state is presented as (20) where, (·) is the standard normal distribution function; ξ and are the functions to calculate mean and standard deviation, and respectively formulated as where, r is the random series characterized by a defined probability distribution; y(·) is the performance output response of k state; y * is the performance benchmark.
To sum up, the preprocessed training set D train , which associates normalized input with the corresponding reliability outputs in all states, is

3) OVERVIEW OF ESSENTIAL PROCESS
As shown in Fig. 8, the DNN training procedure integrates three principal phases steps, including the training sets preprocessing, network training and MINLP-based hyperparameter optimization.  The first stage is to preprocess the data of MC using feature scaling and statistical analysis. In the second stage, MIMLP-based hyperparameter tuning initializes or updates the hyperparameters to refresh the topology of DNN. As for the third stage, LM back-propagation algorithm is adopted to update the network parameters (connection weights and threshold value) [35]. Both the second stage and third stage will iteratively carry out until meet the user-defined stopping criteria, including the error convergence or reach the maximum number of iterations.

IV. SURROGATE MODELING-BASED FUZZY OPTIMIZATION
Based on the already trained DNN-based surrogate models, this section studies the implementation of fuzzy optimization. The aim of optimization is to acquire the multiple feasible cycle solutions of the commercial aeroengine. Fig. 9 displays the typical commercial flight mission profile. It graphically describes the basic capabilities of a jetliner by performing the following multiple specific missions. Some critical missions are conducive to explore the leading performance requirements of installed engine.

A. FLIGHT MISSION ANALYSIS
First of all, it is the top priority for flight security to assure that the aircraft could accelerate to take-off speed in the limited distance. To achieve this goal, the designed aeroengine is requested to produce sufficient thrust in airplane ground rolling, even under some adverse weather conditions. Secondly, it spends the majority of flight time in the cruise phase for the passage plane. Thus, the aeroengine thrust in cruise occupy an essential position of the flight performance and safety. Thirdly, the specific fuel consumption in cruise is directly related to the economy of the entire jetliner flight. Fourthly, flight at the top of climb is approaching the performance envelope of the equipped engine so that it is necessary to be considered. However, there is no need to overemphasize the flight at the top of climb because it is not always an indispensable flight mission. In the end, the rest of specific missions usually are transient states that are completed in a short time.
As mentioned above, this study mainly concentrates on three operating conditions, including take off, cruise and top of climb. Table 3 lists the specifications of the concerned operating conditions, which are depicted by flight condition, regulation schedule and corresponding overall performance requirements. The International Standard Atmosphere (ISA) defines that T amb = 288.15K, P amb = 101.3kPa, which is the setting of the standard sea-level atmospheric condition.

B. OPTIMIZATION PROBLEM STATEMENT
The implementation of optimization is to solve the feasible cycle solutions for reaching the requested overall performance requirements under the consideration of uncertainty.

1) OBJECTIVE
The primary purpose of optimization is to maximize the overall performance reliability of each operating condition as much as possible. The objective functions are presented as

2) DECISION VARIABLES
Thermodynamic cycle defines the overall performance at all operating conditions, which is one of the most important attributes for a gas turbine engine [36]. Therefore, the decision variables are the cycle parameters of on-design point (in SLS condition), including the pressurization rate of fan (π F ), booster (π B ) and compressor (π C ), turbine entry temperature (T ET ), bypass rate (R BP ) and air inflow of inlet (W in ). The vector format of decision variables could be presented as

3) CONSTRAINTS
The parameter setting of the developed turbofan model refers to the technical characteristics data of CFM56-7B [37]. According to the original cycle of CFM56-7B, the ranges of design variables are bounded, as shown in Table 4. Except for decision variable bounds, engine weight (W E ) is regarded as another constraint. W E is about a 10% reduction, compared with W E,Origin . Then this constraint is presented as the weight ratio.
For high bypass ratio turbofan engine, the weight can be estimated by following polynomial [29]: (27) where, π total is the total pressurization ratio that is equal to the product of π F , π B and π C .

C. FUZZY OPTIMIZATION MODEL USING DNN-BASED SURROGATE MODELS
The conventional deterministic optimization is engaged to extract a best solution associated with a rigorous mathematical system criterion. By loosening this criterion to varying degrees, the fuzzy optimization expresses a compromise for yielding multiple feasible solutions. On this account, fuzzy optimization is also called flexible programming [39]. Therefore, fuzzy optimization is undertaken to obtain multiple acceptable cycle solutions for reliability-based preliminary design.

1) NORMATIVE FORMULATION OF FUZZY OPTIMIZATION
The normative fuzzy optimization model is composed of three principle elements: A finite set of feasible alternatives; a set of fuzzy constraints which softly restraint the selection of alternatives; the fuzzy objective functions that quantify the satisfaction of each alternative. Referred to the above presentation, the normative formulation of this study is presented as where, f DNN is the DNN-based surrogate model; subscript l and u represent lower and upper limits for design variables, respectively.

2) IMPROVED Z-SHAPED MEMBERSHIP FUNCTION
In fuzzy programming, the feasible solutions are solved with imprecise objectives and constraints, which are not measured by specific value but in terms of satisfaction (or called desirability). The degree of satisfaction needs to be quantified at the level of mathematical scale and then transferred into an equivalent real-valued model. This process is called defuzzification. One practicable defuzzification method is associating satisfaction with membership function. Thus, the membership function quantifies corresponding satisfactory of objectives and constraints. Based on Z-shaped membership function [40], the improved Z-shaped membership function is defined by the following formulas where, µ is the defuzzified satisfaction degree. τ is the objective value; τ u is the completely satisfactory objective value while τ l is the completely unsatisfactory objective value. Fig. 10 presents the images of Z-shaped membership function and improved Z-shaped membership function. Obviously, the notable feature of improved Z-shaped membership function that is introducing q to reshape the partially satisfied region.
The value setting of q is regarded as the weight to quantify the importance of each objective. If an objective has a greater significance, the value of q will be set larger. Otherwise, q will be equal to a relatively smaller value.

3) REAL-VALUED FORMULATION OF FUZZY OPTIMIZATION PROBLEM
With different parameter settings, the presented membership function is exploited to defuzzify the desirability of objectives and constraint. Furthermore, the defuzzified constraint is transformed as a pseudo objective function. This transformation assists the programmer to weigh the feasibility and desirability of a solution. Therefore, these two objective functions make the studied problem become a multi-objective problem, which can be reformulated as where, λ is the first objective value that represents the satisfactory value of each objective; β is the second objective value that represents the satisfactory value of each constraint.
According to the discussion of section 4.1, the value setting of q (j) refers to the order of importance, which is, For comparison, the studied problem is also formulated as a mirrored deterministic optimization. Its objective function is developed by integrating the reliability prediction responses of DNN models. Therefore, the deterministic optimization is formulated as where, w is the weight of each objective and has a similar value setting with q.

A. VALIDATION OF COMMERCIAL AEROENGINE MODEL
Referring to the performance characteristics of CFM56-7B [41], the overall performances of concerned states are calculated by both the established turbofan model and Gasturb R [42], as a comparison. The comparative results are listed in Table 5.
The results imply that the errors between both models are within the acceptable range of engineering calculation. Thus, it verifies the simulation accuracy of the developed aeroengine model. Besides, the results of the established model are used as the performance benchmarks of each working condition.

B. PERFORMANCE ASSESSMENT OF DNN-BASED SURROGATE MODELS
In this section, the prediction accuracy of the obtained DNNbased surrogate models is evaluated to validate the effectiveness of MINLP-based hyperparameter tuning technique. In order to effectively cover the entire design space, the number of training scenarios shall be 30 times the dimension of the input [3]. Thus, this work generates 180 training scenarios and another 20 testing scenarios. Feeding these samples, DNN-based surrogate models are training with the  employment of MINLP-based hyperparameter optimization method. The fitness curves of hyperparameter optimization are presented in Fig. 11.
Selecting the best hyperparameter scheme of R Fn,Tko is presented for the sake of illustration. In this case, MADS approach the optimal hyperparameter combinations, which are specifying the N HL ranged from 3 to 6. Then, the network performances of each combination are evaluated through 20 testing samples, which are measured by the 2-norm and infinite-norm of prediction errors. The following results, including the optimum hyperparameter schemes and accuracy observations, are shown in Table 6.
Obviously, the best accuracy is observed from the optimal hyperparameter combination of N HL = 3. Thus, this hyperparameter scheme is selected to design the topological structure of R Fn,Tko DNN-based surrogate model. Likewise, the optimum topology of the rest DNN-based surrogate models are also determined, as shown in Table 7.
Surveying the selected hyperparameter schemes for each DNN model, it is found that they are very different in all respects. This diversity indicates that different deep learning tasks always require different hyperparameter combination to achieve the best effects.
A further test is implemented to compare the network performances of the well-tuned DNN models and the nontuned DNN models with the same N HL . For the non-tuned DNN models, the default topological structure is that each hidden layer has six neurons and uses the sigmiod, as well as its output layer exploits the purelin. The results of this comparison are graphically illustrated in Figure 12. For better demonstration effect, the testing samples are orderly arranged according to their reliability values from 0 to 1.
As shown in Fig. 12, the non-tuned DNN models reach basically acceptable accuracy to predict some testing samples whose target value is equal to 0 or 1. However, there are significant inaccuracies when the non-tuned DNN models predict the testing samples whose target value range from 0.1 to 0.9. Different from the non-tuned DNN models, the predictions of the tuned DNN models are highly consistent with the target values of all testing samples. It indicates the well-tuned DNN models have accessed to excellent function approximation capability in global. While the non-tuned DNN models are missing this ability, and only barely qualify for pattern classification if all the samples are labeled 0 or 1.
In this work, all the simulating calculations are performed by one computer equipped with an Intel CPU (E5 2680v2) and 16-GB RAM. Based on this machine, the computing times are measured to compare the computational efficiency of MC simulation and DNN-based surrogate models. The Table 8 demonstrates the total time cost for evaluating 20 testing samples, as well as the accuracy of DNN-based surrogate models.
The results show that the maximal absolute error is not beyond 0.042. Thus, the DNN-based surrogate models reach acceptable accuracy. In looking at the specifications for the computing time, it is worth noting that MC simulation takes 64746 sec to complete the comprehensive reliability prediction for 20 cycle samples. However, the DNN-based surrogate models only spend 2.359 sec to finish the same task.
Therefore, the DNN-based surrogate models are capable of replacing the MC simulations for probabilistic analysis, with satisfied prediction precision and significant computational efficiency improvement. VOLUME 8, 2020

C. SOLUTIONS OF THE OPTIMIZATION MODEL
Applying the DNN-based surrogate models, the optimization process is carried out by Elitist Non-dominated Sorting Genetic Algorithm (NSGA2) [44]. The termination condition of NSGA2 is set to reach the maximum iteration number. The final iteration respectively captures final Pareto fronts of two optimization models, as shown in Fig. 13. The following images indicate that it is an approximately inverse proportional relationship to describe the trade-off between λ and β. According to the computing time in Table 7, performing MC simulation would cost 3237 sec for one individual to calculate the reliability-based fitness. Thus, it is easy to estimate the total time cost would be approximately 187.3 days to complete the optimization under the given parameter setting (population size = 25, maximum iteration number = 200). However, these two optimization procedures respectively cost 4.38 min and 4.45 min using the DNN-based surrogate models.
Uniformly along the two Pareto fronts, eight Nondominated cycle solutions are respectively picked for assessment. These cycle solutions are respectively input to the thermodynamic-based aeroengine model to perform the MC simulations. Then, the simulation data of overall performances is utilized to calculate the reliability of each operating state. Table 9 shows the comprehensive assessment of all cycle solutions, including weight ratio and reliability results. The results demonstrate that the original cycle scheme only achieves approximately 50% reliability for all operating states. While all optimized solutions make the thrust reliability surpassing 98% and SFC reliability reaching to a minimum of 92.87%, which shows satisfactory reliability. Observed the R W of all solutions, it is found that the deterministic optimization strictly adheres to the constraint and rejects the relevant unfeasible solutions. While the fuzzy optimization mildly violates the constraint and generates a solution whose R W = 0.916 > 0.9.
Another investigation is implemented to analyze the detailed information of all optimized Pareto solutions, and the corresponding results are shown in Table 10. Compared to the original cycle, many cycle parameters of optimized cycles are improved to create more performance redundancy, which are in line with our expectations. However, it is disappointing to find that the cycle solutions of deterministic optimization are extremely similar. This similarity is reflected in the very narrow numerical ranges of each cycle parameter. For instance, the ranges of T ET and R BP are respectively [1662.4, 1665.2] and [5.321, 5.332]. These two parameters are fixed so that the quantity of available choices has been substantially diminished. Thus, it means that the deterministic optimization makes a limited contribution to the flexibility in the design. However, a wider flexible design universe is provided by all the cycle alternatives from the fuzzy optimization. Obviously, the numerical ranges of these cycle parameters are much larger. Comparatively, the ranges of T ET and R BP are respectively [1639.8, 1654.2] and [5.239, 5.409]. The same situation also goes for other cycle parameters. The above situation indicates that the fuzzy optimization is more capable of generating multiple feasible cycle alternatives with diversity.  A case study is carried out to further compare the two minimum T ET proposals presented in Table 9. According to the results of MC simulation in Table 8, both two solutions are verified to achieve the expected reliability for all operating conditions. Besides, the most striking difference between these two solutions is in T ET . In particular, the T ET of deterministic optimum solutions is 1662.4K while the T ET of fuzzy optimum solution is 1639.8K. The difference of T ET is 22.6 K, which is a large gap referring to the presentation in Introduction.
For each operating condition, the performance original data is presented in the form of empirical cumulative distribution function (CDF), as shown in Fig. 14.
The observation is that all the empirical CDF curves of optimized solutions are in the place that reach higher reliability value, referring to the benchmark lines. Nevertheless, all the thrust CDF curves of deterministic optimum solution are moved further than that of fuzzy optimum solution. This means that deterministic optimum solution causes excessive thrust redundancy to reach the expected reliability at the price of higher T ET , which also reduces SFC redundancy.
For fuzzy optimization, the reason for the preference for moderate solutions is related to its objective functions. These objective functions are formulated by improved Z-shaped membership function specified by different q (j) . As presented in Eq. (31), each q (j) is set to be less than 1. This value setting is in accordance with the law of diminishing marginal utility [45]. To be specified, this law makes the increment of satisfaction decay as each unit increase of objective function fitness. It effectively suppressed NSGA2's strong willingness of minimizing all fitness value by radically increasing cycle parameter values. However, this willingness is fulfilled in deterministic optimization and causes the above excessive increases and the tendency that all solutions converge to be the same. While fuzzy optimization maintains the diversity in solutions and makes the cycle parameters are rationally increased to avoid unnecessary performance waste.

VI. CONCLUSION
This study proposes the feasible performance reliability design methodology using the DNN-based surrogate models. For enhancing the DNN performance, MINLP-based hyperparameter tuning technique is presented to obtain the optimum hyperparameter combination. With the application of well-tuned DNN model, the presented fuzzy optimization is implemented to design the commercial aeroengine thermal cycle under the consideration of uncertainty. The main conclusions of this study are presented as follows: 1. The proposed MINLP-based hyperparameter tuning technique can acquire the optimal hyperparameter combination of the specified DNN automatically and efficiently. This technique frees the programmer from the heavy work of hyperparameter manual adjustment, and lays the solid foundation for the DNN model to effectively execute a particular deep learning task.
2. Based on well-tuned hyperparameters, the trained DNNbased surrogate models could achieve high accuracy for comprehensive reliability prediction with negligible computing time cost (about 0.003% as that of MC simulation). These two characteristics enable DNN-based surrogate models to replace the time-consuming probabilistic analysis based on MC simulation so that facilitate the reliability-based optimization 3. The optimization results indicated that deterministic optimization is easy to result in homogeneity of solutions. While the presented fuzzy optimization can acquire a variety of feasible solutions, which could reach the satisfied reliability (>92.87%) with diversity. Providing a variety of feasible candidate proposals could also assist the decisionmaker to select wisely according to practical engineering situations.
4. The application of improved Z-shaped membership function could efficiently regulate the interrelations of each feasible objective. That regulation contributes to producing moderate solutions, which are rationally increasing the cycle parameters for creating reasonable performance redundancy. Thus, it reduces the potential overall performance waste, and conduces to deal with risks related to technical issues and aeroengine comprehensive cost.

CONFLICT OF INTEREST STATEMENT
The authors declare that there is no conflict of interests regarding the publication of this article.