Robust H∞ Deep Neural Network-based Filter Design of Nonlinear Stochastic Signal Systems

Recently, deep neural network (DNN) schemes based on big data-driven methods have been successfully applied to image classification, communication, translation of language, speech recognition, etc. However, more efforts are still needed to apply them to complex robust nonlinear filter design in signal processing, especially for the robust nonlinear H∞ filter design for robust state estimation of nonlinear stochastic signal system under uncertain external disturbance and output measurement noise. In general, the design problem of robust nonlinear H∞ filter needs to solve a complex Hamilton-Jacobi-Isaacs equation (HJIE), which is not easily solved analytically or numerically. Further, robust nonlinear H∞ filter is not easily designed by training DNN directly via conventional big data schemes. In this paper, a novel robust H∞ HJIE-embedded DNN-based filter design is proposed as a co-design of H∞ filtering algorithm and DNN learning algorithm for the robust state estimation of nonlinear stochastic signal systems with external disturbance and output measurement noise. In the proposed robust H∞ DNN-based filter design, we have proven that when the approximation error of HJIE by the trained DNN through Adam learning algorithm approaches to 0, the HJIE-embedded DNN-based filter will approach the robust nonlinear H∞ filter of nonlinear stochastic signal system with uncertain external disturbance and output measurement noise. Finally, a trajectory estimation problem of 3-D geometry incoming nonlinear stochastic missile system by the proposed robust H∞ HJIE-embedded DNN-based filter scheme through the measurement by the sensor of radar system with external disturbance and measurement noise is given to illustrate the design procedure and validate its robust H∞ filtering performance when compared with the extended Kalman filter and particle filter.


I. INTRODUCTION
I N the last decades, the robust H ∞ filter has been widely applied to the field of signal processing when the statistical information of external disturbance and measurement noise are unavailable and their worst-case effects on the filtering error need to be efficiently attenuated. Therefore the robust H ∞ filter is always used to estimate the state or detect the signal in the signal transmission systems corrupted with uncertain external disturbance and random measurement noise [1], [2]. In the linear stochastic signal systems, the H ∞ filter design needs to solve a Riccati-like equation or a corresponding linear matrix inequality (LMI) for filter gain [3]. In many practical signal processings, stochastic signal transmission systems always have nonlinear dynamic behavior with uncertain external disturbance and output random measurement noise [4]- [6]. However, nonlinear H ∞ filter design problem for the state estimation of nonlinear stochastic signal system needs to solve a highly nonlinear filter-system state-coupled partial differential Hamilton-Jacobi-Isaacs equation (HJIE), which can not be efficiently solved at present [3], [7].
The conventional extended Kalman filter [8] and uncented Kalman filter [9], which are based on the linearization at the predicted state of previous stage, are not easily employed to treat this nonlinear optimal filter design problem of nonlinear stochastic signal system with uncertain external disturbance and output measurement noise [10]. The reason is that their filtering performance will be degraded by the error propagation due to iterative linearization and unknown statistics of external disturbance and output measurement noise. The particle filter (PF) [11], [12] is suitable for the filtering design of nonlinear stochastic signal systems with given noise statistics but unsuitable for filtering design problem under uncertain external disturbances and measurement noise with unknown or uncertain statistics. In order to avoid solving a complex nonlinear partial differential HJIE for the minmax robust H ∞ filter design of nonlinear stochastic signal systems, several interpolation methods such as fuzzy interpolation method [13]- [15], global linearization method [16] and gain scheduling method [17] have been employed to interpolate several local linearized systems to approximate the nonlinear stochastic signal system with external disturbance and measurement noise so that HJIE of robust nonlinear H ∞ filter design could be interpolated by a set of local Riccati-like equations or a set of equivalent local linear matrix inequalities (LMIs).
For example, the T-S fuzzy interpolation method is used to interpolate several local linearized systems by some fuzzy bases to approximate nonlinear stochastic signal system [13]- [15], [18]- [20]. Then a T-S fuzzy filter is employed to interpolate these local linear filters by some fuzzy bases to achieve robust nonlinear H ∞ filtering performance by solving the corresponding LMIs. Besides, the global linearization scheme employs some smooth interpolation functions to interpolate several local linearized systems at the vertices of the convex hull of polytope Ω of all linearized systems of a nonlinear stochastic signal systems to approach the nonlinear stochastic signal system [16], [21]. Then, a global linearization filter can be designed by interpolating local linear filters by some smooth functions through solving a set of LMIs to achieve the robust H ∞ filtering scheme [16], [20]- [21]. However, these interpolation schemes need to solve a large number of LMIs. For example, if n local linearized systems are interpolated to approximate a nonlinear stochastic signal system, it needs to solve n 2 LMIs for nonlinear robust H ∞ filtering design [13]- [15], [18]- [19]. Therefore, for a highly nonlinear stochastic signal system, it will increase the design complexity of nonlinear robust H ∞ filter in the design procedure and the computational complexity of robust H ∞ interpolation filter at every time step. Furthermore, these interpolation filter design methods of local linearized systems are based on the fact that the solution of HJIE is limited to the form of the quadratic Lyapunov function V (x(t)) =x T (t)Px(t) for some P T = P > 0, wherẽ x(t) denotes the filtering (estimating) error. This will lead to a conservative solution of nonlinear HJIE. Since the state x(t) of signal system is unavailable and is to be estimated in the filter design problem, the interpolation functions need to base on the estimated statex(t), it will increase the design difficulty and filtering accuracy, especially at the begining of estimation when a large initial estimation error.
In this study, as shown in Fig. 1, a robust H ∞ deep neural network (DNN)-based filter scheme is proposed as a codesign of H ∞ filter scheme and DNN learning algorithm for the robust state estimation of nonlinear stochastic signal system by solving ∂V (x(t)) ∂x(t) via the output of DNN for the H ∞ filter gain k * (x(t)) from HJIE directly. The design procedure of H ∞ DNN-based filter is divided into two phases, i.e., (i) off-line training phase and (ii) on-line operation phase. In the off-line training phase, the proposed HJIE-embedded DNN in Fig. 1 is to be trained by Adam learning algorithm to output ∂V (x(t)) at time t via solving HJIE of the nonlinear robust H ∞ filtering design problem, and ∂V (x(t)) can be used to produce H ∞ filter gain k * (x(t)), the worst-case external disturbance v * (t) and output measurement noise n * (t) simultaneously for the next step off-line training as shown in Fig. 1. However, the most difficulty to solve HJIE by training DNN to output ∂V (x(t))

∂x(t)
is that HJIE is the function ofx(t),x(t) and x(t), where x(t) is unavailable and to be estimated. Therefore, the H ∞ filter gain k * (x(t)), the worst-case external disturbance v * (t) and output measurement noise n * (t) will be sent to nonlinear stochastic signal system, Luenberger-type filter and estimation error system to replace unavailable v(t) and n(t) to generate y(t), x(t),x(t) and x(t) =x(t) +x(t), respectively, for the next training step at time t in the off-line training phase. The replacement of unavailable external disturbance v(t) and measurement noise n(t) by the worst-case v * (t) and n * (t) in the H ∞ filtering scheme does not influence on the H ∞ filtering performance because the robust H ∞ filtering scheme is designed based on the worst-case v * (t) and n * (t) instead of v(t) and n(t). After off-line training phase of robust H ∞ DNN-based filtering scheme, it will be shifted to on-line operation phase of H ∞ DNN-based filter scheme. In the on-line operation phase, y(t) is generated by real stochastic signal system with real v(t) and n(t). We have proven that as the error (θ(t)) of solving HJIE approaches to zero by training DNN through Adam learning algorithm, the output of HJIE-embedded DNN will approach to ∂V (x(t)) ∂x(t) of HJIE of the robust H ∞ filtering strategy and the proposed HJIEembedded DNN-based filter will approach to the real robust H ∞ filter of nonlinear stochastic signal system. However, in practical applications, we will stop the Adam learning algorithm in the training phase when the absolute HJIE error | (θ(t))| ≤ δ for some precribed small positive value δ.
The contributions of this paper are described as follows: 1) A novel HJIE-embedded DNN-based filter scheme is proposed for the co-design of H ∞ filtering scheme with DNN learning algorithm for the robust state estimation of nonlinear stochastic signal system with uncertain external disturbance and output measurement noise. The worst-case external disturbance v * (t) and output measurement noise n * (t) are employed in the off-line training phase to replace v(t) and n(t) to generate output y(t) by nonlinear signal model,x(t) by the filter model, andx(t) by filter error model and x(t) =x(t) +x(t) so that the x(t)-x(t)-x(t)-coupled nonlinear HJIE could be solved by HJIE-embedded DNN for H ∞ DNN-based filter design. Adam learning algorithm is employed to efficiently train the weighting parameters of DNN to output ∂V (x(t)) ∂x(t) to solve nonlinear partial differential HJIE directly for the robust H ∞ filtering design of nonlinear stochastic signal system. We also prove that when the error of solving ∂V (x(t)) from HJIE approaches to 0 in the DNN training process, the proposed HJIE-embedded DNN-based filter scheme can approach to the robust H ∞ filter design of nonlinear stochastic signal systems. 2) Instead of solving V (x(t)) from HJIE of the robust H ∞ filter design of nonlinear stochastic signal system in the conventional design methods, the proposed HJIE-embedded DNN directly solves ∂V (x(t)) ∂x(t) from HJIE to obtain the robust H ∞ filter gain k * (x(t)) directly, which can avoid the very difficult calculation work of ∂V (x(t)) ∂x(t) from the numerical values of V (x(t)) andx(t) in the conventional design method.
3) The proposed robust H ∞ DNN-based filtering scheme can provide a co-design of H ∞ filtering scheme and deep learning algorithm to bridge the gap between the traditional theoretical nonlinear robust H ∞ filter theory and recent pure big data-driven deep learning schemes to facilitate the application of DNN schemes to treat the complex nonlinear H ∞ filter design problem in nonlinear stochastic signal systems to save much training data and computation time of the conventional big data-driven deep learning filter design method.
The remainder of this study is organized as follows: In Section II, the problem description is given. The theoretical robust H ∞ filter gain and the worst-case external disturbance and measurement noise are solved based on ∂V (x(t)) ∂x(t) of HJIE. The robust H ∞ HJIE-embedded DNN-based filter scheme is proposed through the Adam learning algorithm in Section III. It is also shown that the proposed robust H ∞ DNN-based filter scheme will approach to theoretical robust H ∞ filter if the approximation error of HJIE approximated by DNN via the off-line training through Adam learning algorithm converges to 0. A simulation example of the state estimation of incoming ballistic missile by the sensor of radar is given to illustrate the design procedure and validate the performance of proposed H ∞ DNN-based filter is given in Section IV. Finally, a conclusion is given in Section V.
Notation: R n : The set of n-tuple real vectors. R n×m : The set of real n × m matrices. x(t) ∈ R n , then where t f denotes the terminal time. The Jacobian vector and Hessian matrix of V (x(t)) are denoted as ∂V (x(t)) notes the identity matrix with dimension a × a; 0 a×b denotes the zero matrix with dimension a × b.

II. PROBLEM DESCRIPTION
Consider the following nonlinear stochastic signal systeṁ where x(t) ∈ R n is the state vector, y(t) ∈ R m is the measurement output, v(t) ∈ R nv denotes the stochastic external disturbance, and n(t) ∈ R nn denotes the stochastic measurement noise. The system functions A(x(t)), B(x(t)), and C(x(t)) are nonlinear function of state vector x(t) which satisfy Lipschitz condition. Before the discussion of filter design of nonlinear stochastic signal system in (1), the observability of nonlinear stochastic signal system (i.e., the ability of observing x(t) from output measurement y(t)) must be guaranteed at first. In general, the observability criterion of nonlinear stochastic signal system in (1) can not be expressed in a simple explicity condition (e.g., rank condition or matrix equality in the linear system). The following lemma is provided from [36] to test the observability of nonlinear stochastic signal system in (1). Lemma 1. (see [36]) Define the augmented vector Z(t) = [y T (t),ẏ T (t),ÿ T (t), . . . , y (n) T (t)] and the corresponding Hessian matrix H z (x(t)) = ∂ 2 z(t) ∂ 2 x(t) . Then the nonlinear stochastic signal system in (1) is observable at the equilibrium point x(t) = 0 if there exists a constant ε > 0 and a constant matrix T such that the absolute values of the leading principal minors ∆ 1 (x(t)), . . . , ∆ n (x(t)) of T H z (x(t)) satisfy the following conditions: where n is the dimension of x(t) and the principal minor ∆ i (x(t)) is the determinant of the matrix by deleting the last n − i columns and rows of Hessian matrix T H z (x(t)).
For the simplicity of design, we assume the nonlinear stochastic signal system in (1) is observable (i.e., the observable conditions in Lemma 1 are satisfied) and therefore the following conventional Luenberger-type filter can be employed to estimate the state of nonlinear stochastic signal VOLUME 4, 2016 FIGURE 1: The flow chart of robust H∞ HJIE-embedded DNN-based filter scheme of nonlinear stochastic signal system in off-line training phase. It is a co-design of the conventional H∞ Luenberger-type filtering scheme and an HJIE-embedded DNN learning scheme to be trained via Adam learning algorithm for the robust state estimation of nonlinear stochastic signal system in (1) with uncertain external disturbance and output measurement noise. Since x(t) is unavailable in the off-line training phase, we need to embed an estimation error system in (4) to generatex(t) so that we could obtain x(t) =x(t) +x(t) for calculating A(x(t)), B(x(t)) and C(x(t)) in HJIE. Furtherx(t) is inputed into HJIE-embedded DNN to output ∂V (x(t)) ∂x(t) to generate the H∞ observer gain k * (t), the worst-case external disturbance v * (t) and measurement noise n * (t) as well as to solve HJIE = 0. If HJIE = (θ k (t)) = 0, (θ k (t)) is fedback to train DNN by Adam learning algorithm. In the off-line operation phase, if (θ k (t)) → 0, we show that the output of DNN approaches to ∂V (x(t)) ∂x(t) and we will shift to on-line operation phase. In the on-line operation phase, y(t) is generated by real v(t) and n(t). In general, we do not need to train DNN in on-line operation phase. However, if | (θ k (t))| > δ for some prescribed small value δ > 0. We could feedback (θ k (t)) to train DNN in the on-line operation phase without influence on the H∞ filtering performance.
wherex(t) is the estimated state andŷ(t) is estimated measurement output.
In the above Luenberger-type filter, we will design nonlinear filter gain k(x(t)) so that the estimate statex(t) could approach to x(t) as possible despite the uncertain external disturbance v(t) and measurement noise n(t) in (1).
Let us denote the estimation error as Then the estimation error dynamic system is given as follows: We assume the statistics of external disturbance v(t) and measurement noise n(t) are unavailable and therefore nonlinear Luenberger-type filter design in (2) is employed to treat the robust H ∞ state estimation design problem of nonlinear stochastic signal system in (1) to efficiently eliminate the effect of uncertain v(t) and n(t) on the state estimation.
The robust H ∞ filter design of (2) for the nonlinear stochastic signal system (1) is formulated as the following minmax Nash stochastic game problem [ where the symmetric weighting matrix Q = Q T ≥ 0 is on the estimation errorx(t), the Lyapunov function V (x(0)) ∈ R 1 denotes the energy of the initial estimation error in (4), t f denotes the final time of state estimation and ρ denotes the desired upper bound of filtering ability and E(·) denotes the expectation operation. The physical meaning of robust H ∞ filter design strategy in (5) is that the worst-case effect of external disturbance v(t) ∈ L 2 [0, t f ] and measurement noise n(t) ∈ L 2 [0, t f ] on the estimation errorx(t) must be minimized by the Luenberger-type filter in (2) from the perspective of mean energy. In general, the initial Lyapunov energy function V (x(0)) ≥ 0 in the numerator of (5) due to the initial errorx(0) must be extracted from the estimation error energy within 0 ∼ t f because it is independent on the design of H ∞ filter. Further, the worst-case effect of external disturbance v(t) and measurement noise n(t) on the estimation error of robust H ∞ filter must be equal to or less than a prescribed attenuation level ρ from the perspective of mean energy. Theorem 1. The minmax robust H ∞ filter design strategy in (5) for nonlinear stochastic signal system in (1) and Luenberger-type filter in (2) is solved by the following worstcase v * (t) and n * (t) and T is the solution of following nonlinear partial differential HJIE, is defined as follows: Proof. See Appendix A.
Remark 1. From HJIE in (9), it needs to solve a complex partial differential ∂V (x(t)) of DNN into HJIE in (9) can let HJIE approach to zero, then DNN can solve HJIE and we can use the ∂V (x(t))

∂x(t)
to generate the H ∞ filter gain k * (x(t)) and the worst-case external disturbance v * (t) and measurement noise n * (t). In the conventional interpolation methods in [13]- [22], they assume the solution V (x(t)) of HJIE in (9) is of the quadratic Lyapunov form V (x(t)) = x T (t)Px(t) and nonlinear stochastic signal system can be interpolated by a set of local linearized systems through fuzzy interpolation method [13]- [15], global linearization method [16] or gain scheduling method [17] so that we can solve a set of Riccati-like algebraic equations for the filter gain k * (x(t)).
In general, it is very difficult at present to solve the nonlinear partial differential HJIE in (9) analytically or numerically for robust H ∞ filter design in (8) for nonlinear stochastic signal system in (1) because HJIE in (9) is a nonlinear differential function of x(t),x(t) andx(t). Especially, while the state vector x(t) is unavailable and only output measurement from HJIE due to the unavailable terms A(x(t)), B(x(t)) and C(x(t)) of unavailable x(t) in (9) for H ∞ filter design in (8) by the conventional methods.

Remark 2.
In the robust control design of nonlinear stochastic system [33], the HJIE of H ∞ control design problem is only the functional of x(t), which is assumed available and can be measured directly. However, in the nonlinear H ∞ filter design problem, x(t) is unavailable and needs to be estimated from output measurement y(t). Further, the observer gain k * (t) in (8) also contains the term C(x(t)), which is unavailable too. Therefore, more efforts are needed to directly solve HJIE in (9) for the nonlinear H ∞ filter design problem because A(x(t)), B(x(t)) and C(x(t)) in HJIE are unavailable. In this study, to overcome this difficult, the estimation error dynamic model in (4) is needed to be embedded with Luenberger-type filter in H ∞ HJIEembedded DNN-based filter scheme as shown in Fig. 1, but with unavailable v(t) and n(t) being replaced by v * (t) and n * (t) in the off-line training phase, respectively.
In the last decades, several approximation methods such as gain schedule method [17], [21], global linearization method [16] and fuzzy method [20] have been applied to interpolate some local linear stochastic signal systems to approximate nonlinear stochastic signal system in (1). In this situation, the nonlinear stochastic signal system is approximated by the following L local linear stochastic signal systemṡ where h i (x(t)) is the ith local interpolation function such as fuzzy basis in fuzzy interpolation method [20], smooth function in global linearization method [22] or smoothing function in gain schedule method [17], A i , B i and C i are local linearized system parameters of the ith local linear system.
In this situation, the Luenberger-type filter in (2) is approximated by the following form (12) and the estimation error dynamic equation in (4) can be approximated by In this situation, the robust H ∞ filter design strategy in (5) for a local linear interpolation estimation error system in (13) needs to solve a set of L 2 Riccati-like algebraic equations [15]- [17], [20]- [22] for the design of filter gains k If a system is very highly nonlinear, then a large number of L local linear syatems in (11) are needed to be interpolated to approximate the nonlinear stochastic signal system in (1). Consequently, a much larger number of L 2 Riccati-like algebraic equations are also needed to be solved for k * i , i = 1, · · · , L of H ∞ filter gains in (12), which will lead to a design complexity. Further, a complex Luenberger-type filter in (12) is needed to be computed at very time instant for the estimation ofx(t) if L becomes large. Further, the limit selection of quadratic Lyapunov function V (x(t)) =x T (t)Px(t) as the solution of HJIE in (9) can make the design of robust H ∞ filter more conservative and the approximation error between (1) and (11) may also deteriorate the H ∞ filtering performance.
Recently, some deep learning algorithms have been widely used to learn some system behaviors by big data-driven methods with successful application to image pattern classification [24], language translation [25] and speech recognition [26] etc. Though these deep learning schemes may have led to a very good result, they have to employ a very large amount of experimental data for training deep neural network. Further, the conventional DNN approaches can not be applied to some specific filter design performance such as the optimal H 2 filtering performance and robust H ∞ filtering performance in nonlinear stochastic signal systems, which are very important in signal processing designs. In the nonlinear H 2 and H ∞ filter design, these exist a large number of nonlinear stochastic system models in practical applications. Further, we have accumulated many powerful theoretical nonlinear filter results in the last decades. Even these nonlinear models and theoretical results of nonlinear filter designs are complicated and difficult to solve, they can be employed to achieve as experts to train DNN to achieve nonlinear H ∞ filter design with a large amount of training data and time saving than the conventional big data-driven DNN learning algorithm.
In this study, a novel method of DNN will be proposed to directly solve ∂V (x(t)) ∂x(t) from the complex partial differential HJIE in (9) for the H ∞ filter gain k * (x(t)) in (8). Since the HJIE in (9) could guarantee the H ∞ filtering performance in (5), the proposed H ∞ DNN-based filter could not only efficiently attenuate the effect of external disturbance and measurement noise on the state estimation error but also significantly save a large amount of training data which are needed in the conventional big data-driven DNN approaches.
In the off-line training phase of the proposed H ∞ DNNbased filter scheme in Fig. 1, since v(t) and n(t) are unavailable, y(t) is generated by nonlinear stochastic signal system model with v(t) and n(t) being replaced by the worst-case v * (t) in (6) and n * (t) in (7). This does not influence on the H ∞ filtering performance because the H ∞ filter gain k * (x(t)) in (8) is designed based on the worstcast disturbance v * (t) and noise n * (t) as shown in Theorem 1. In the on-line operation phase of H ∞ DNN-based filter scheme, y(t) is generated by the real nonlinear stochastic signal system in (1) through the true value v(t) and n(t).
Remark 3. In the previous works, in order to avoid solving a complex nonlinear partial differential HJIE of the robust H ∞ filter of nonlinear stochastic signal systems, the T-S fuzzy interpolation method in [19], [20], [22], the global linearization method [16] and gain scheduling method [17] have been employed to interpolate several local linearized systems to approximate the nonlinear stochastic signal system so that HJIE of robust nonlinear H ∞ filter design could be interpolated by a set of local Riccati-like equations or a set of equivalent local LMIs under the assumption of quadratic solution V (x(t)) =x T (t)Px(t) of HJIE, i.e., with different interpolation methods to interpolate some local linearized systems to approximate a nonlinear stochastic system so that the nonlinear partial HJIE can be transformed to a set of LMIs under the quadratic solution V (x(t)) =x T (t)Px(t), which could be easily solved by LMI Toolbox in Matlab. In this study, we focus on using DNN to directly sovle ∂V (x(t)) ∂x(t) from the complex nonlinear partial differential HJIE in (9) of H ∞ filter design of nonlinear stochastic signal systems so that DNN can output ∂V (x(t)) ∂x(t) to generate the H ∞ filter gain k * (x(t)) directly instead of solving V (x(t)) =x T (t)Px(t) by the interpolation schemes in the conventional design method. In this case, we can not only avoid the difficult calculation in solving HJIE but also save more training and computation time. to solve HJIE in (9) after the training of Adam learning algorithm in (14) and (15) based on the error of HJIE = (θ k (t)).

VIA A CO-DESIGN OF H∞ FILTER SCHEME AND DEEP LEARNING ALGORITHM
Due to the difficulty to directly solve ∂V (x(t)) ∂x(t) of HJIE in (9) for the filter gain k * (x(t)) in (8) by the conventional methods for the design of Luenberger-type filter in (2) to achieve minmax robust H ∞ filter strategy in (5), DNN is proposed to be trained to solve ∂V (x(t)) ∂x(t) of the HJIE in (9) for k * (x(t)), v * (t) and n * (t) in (6)-(8). Since A(x(t)), B(x(t)) and C(x(t)) in HJIE need the information of x(t), which is unavailable and to be estimated, to overcome this difficulty, we need to synthesize x(t) by x(t) =x(t) +x(t). Therefore, both Luenberger-type filter model in (2) and the estimation error dynamic in (4) are needed to be embedded as shown in Fig. 1 to obtainx(t) andx(t), respectively. Therefore, the proposed H ∞ DNN-based filter can synthesize x(t) =x(t) +x(t) for the calculation of A(x(t)), B(x(t)) and C(x(t)) for solving HJIE in (9) as shown in the HJIE block of HJIE-embedded DNN in Fig. 1. As shown in Fig. 2, the proposed HJIEembedded DNN architecture with inputx(t) is trained by the error (θ k (t)) of solving HJIE = 0 to output ∂V (x(t))

∂x(t)
for H ∞ filter gain k * (x(t)). HJIE-embedded DNN in Fig. 2 consists of an input layer, several hidden layers, an output layer and an HJIE block. The neurons in hidden layers of DNN employ LeakyReLU as the activation function [27], which is the same as ReLU when inputx(t) is positive and is not equal to zero but a constant gradient whenx(t) is negative. Therefore, we have the advantage of ReLU and avoid the disadvantage of dead ReLU when inputx(t) is negative, i.e., LeakyReLU operates as follows [37]: where α 1 and α 2 are some constant with α 1 , α 2 ∈ (0, 1).
In the off-line training phase, as shown in Fig. 1, we need the nonlinear stochastic signal system model in (1) to generate y(t) for Luenberger-type filter in (2) and the estimation error dynamic system in (4). However external disturbance v(t) and measurement noise n(t) are unavailable in real nonlinear stochastic signal system in (1). In the robust H ∞ Luenberger-type filter design, the real external disturbance v(t) and measurement noise n(t) are not considered directly but replaced by their worst-case v * (t) and n * (t) in the design procedure from the minmax robust H ∞ filtering perspective. Therefore, we use v * (t) in (6) and n * (t) in (7) to replace real v(t) and n(t), respectively, without the influence on the estimation performance of robust H ∞ filter. From the flow chart of robust H ∞ HJIE-embedded DNN-based filter design in Fig. 1,x(t) from the filter estimation error system in (4) is inputed to DNN to expect generating the desired output ∂V (x(t)) ∂x(t) , which is to be used to produce v * (t), n * (t) and k * (x(t)) according to (6), (7) and (8) in Theorem 1, respectively. Therefore, the nonlinear signal system model in (1) with v * (t), n * (t) and u * (t) = k * (x(t)) is employed to produce y(t) in (1). Luenberger-type filter in (2) can producê x(t), and estimation error system in (4) can generatex(t) in the off-line training process of DNN simultaneously. The DNN output ∂V (x(t)) is also sent to HJIE to check whether the output ∂V (x(t)) ∂x(t) approaches to its true value by checking HJIE = 0 or not in (9). If HJIE = (θ k (t)) = 0, the error of HJIE will be fedback to train the weighting parameters in the hidden layers of DNN in Fig. 2 via Adam learning algorithm to minimize the objective funtion 2 (θ k (t)) as follows [28]: where θ k (t) is the weighting parameter vector in the hidden layers of DNN which is to be trained for DNN to generate the output ∂V (x(t)) at time t. l is the learning rate and K is the number of training steps at time t. The bias-corrected estimatorm k (t) andv k (t) are given as [28] where is the gradient vector of partial derivatives of objective function 2 (θ k (t)) with respect to θ k (t) at the training step k at time t. M denotes the batch size. λ 1 , λ 2 ∈ [0, 1] in (15) are the degree of previous impact influence on the current direction and can be specified by the designer with the concept of momentum to avoid being trapped in a local minimum and speed up the learning rate [29]. λ k 1 and λ k 2 denote the kth power of λ 1 and λ 2 , respectively. τ in (14) is a small number to be used to prevent the denominator from being zero. m k (t) and v k (t) in (15) are the moving average of gradient and squared gradient at time t. Withv k (t) in (15), we can take the advantage of the idea of adaptive learning rate, which is always large at the beginning and small near the minimum. In (14) and (15), the Adam learning algorithm can take both the advantages VOLUME 4, 2016 of momentum and RMSProp [30] as an efficient parameterspecific adaptive learning method with easy implementation and great performance, one of the most popular learning algorithm of neural networks of some optimization problems recently.
Remark 4. Unlike the conventional big data-driven DNN, with the help of nonlinear system model in (1), Luenbergertype filter in (2), estimation error dynamic system in (4) and HJIE of theoretical H ∞ filter of nonlinear stochastic system in (1), the proposed DNN-based filter scheme can converge quickly by Adam learning algorithm scheme in (14) and (15). The convergence of weighting parameter vector θ k (t) of Adam learning algorithm in (14) and (15) has been proven in [31]. If the number of hidden neurons of DNN and the number K of training steps is large enough, the updating weighting parameter vector θ k (t) in (14) and (15) can converge to a globally optimal parameter vector at a linear convergent rate.
In the off-line training process of HJIE-embedded DNN in Fig. 1, the output ∂V (x(t))

∂x(t)
of DNN is fed to HJIE to calculate the error of HJIE as follows:
The error of HJIE in (16) will be fedback to train DNN by the Adam learning algorithm in (14) and (15). It is expected to produce the precise ∂V (x(t))

∂x(t)
of DNN of the HJIE-embedded DNNbased filter in Fig. 1 will approach to the H ∞ filter gain k * (x(t)) in (8) of the Theorem 1 for the Luenberger-type H ∞ filter in (2) for the nonlinear stochastic signal system in (1).

Proof. See Appendix B.
Remark 5. (i) From Theorem 2 and Theorem 1, we could see that the proposed HJIE-embedded DNN-based filter in Fig. 1 will approach to the robust H ∞ filter in (8) as (θ k (t)) of HJIE in (16) approaches to zero via Adam learning algorithm in (14) and (15). However, in the practical filtering applications, we will stop the training phase and transfer into the operation phase if | (θ k (t))| ≤ δ for a small prescribed δ or the training steps achieve a given number K in (14). In the simulation example in the sequel, the number of training step at each time t is given by K = 30. (ii) In this study, unlike the conventional big data-driven DNN schemes, we could significantly save a much amount of training data and training time of HJIE-embedded DNN because the nonliner stochastic signal system model in (1), Luenberger-type H ∞ filter in (2), the estimation error dynamic system in (4) and theoretical H ∞ filtering result HJIE in (9) are employed to generate the necessary training datax(t),x(t) and x(t) = x(t) +x(t) in the off-line training phase of H ∞ HJIEembedded DNN-based filter in Fig. 1. Based on these training data generated by the corresponding system models and the theoretical HJIE result for the robust H ∞ filtering strategy, the H ∞ HJIE-embedded DNN-based filter design becomes possible for solving the very complex robust H ∞ filter design problem of nonlinear stochastic dynamic signal system in (1), unlike the most conventional big data-driven DNN schemes to be trained only for deciding yes or no (i.e., 0 or 1) in the conventional imaging classification and speech recognition problems [23]- [27].
For the convenience of training and practical design, the continuous nonlinear stochastic signal system in (1) can be represented by the following nonlinear stochastic samplingdata signal system as follows: or where ∆t > 0 denotes the sampling time.
Similarly, the Luenberger-type filter in (2) and the estimation error system in (4) are also represented by the following nonlinear sample-data stochastic signal systems, respectively.
At t = ∆t, we randomly select N sample vectors near the errorx(∆t) and then send them into DNN to output , which is used to generate v * (∆t), n * (∆t) and k * (x(∆t)) in (6)-(8) and sent to calculate HJIE = (θ k (∆t)) to feedback to train the weighting parameters of DNN by Adam learning algorithm. After K training steps, the output ∂V (x(∆t))
After we have finished the above HJIE-embedded DNN VOLUME 4, 2016 training process of H ∞ HJIE-embedded filter, we begin the operation phase of DNN-based H ∞ filter. In the on-line operation phase as shown in Fig. 3, we do not need to use the information of v * (t) and n * (t) generated by DNN but the v(t) and n(t), which will occur themself in the real nonlinear stochastic signal system in (1). Therefore, every time step, we only need to inputx(t) to DNN to output ∂V (x(t)) ∂x(t) to compute filter gain k * (x(t)) = C(x(t))) T for Luenberger-type H ∞ filter and estimation error system. However, the training of DNN is necessary if the absolute error | (θ k (t))| ≥ δ for a prescribed small value δ.
Remark 6. For the off-line training processes, there are some data processings usually being used before training DNN such as normalization or standardization in the deep learning [30]. If the errorx(t) is so large that all the random sample data will influence the training speed, we can standardize all the sample data before sending them into DNN. Hence, we can not only speed up the training speed but also improve the estimation accuracy in the off-line training phase.

IV. SIMULATION EXAMPLE
After the robust H ∞ HJIE-embedded DNN-based filter design in Fig. 1 is proposed in the above sections for the robust state estimation of the nonlinear stochastic signal system in (1), in this section, a simulation example is given to illustrate the design procedure based on Fig. 3 and to validate the filtering performance of the proposed H ∞ DNN-based filter of nonlinear stochastic signal system. In this study, an H ∞ robust state estimation problem of incoming ballistic missile by the measurement of radar system in Fig. 4 is given to confirm the robust state estimation performance of the proposed H ∞ HJIE-embedded DNN-based filtering scheme when compared with the extended Kalman filter [8] and particle filter [11].
Suppose an incoming ballistic missile will attack a target at the origin of Cartesian coordinate and is measured by the sensor of radar located at (x, y, z) = (300, 200, 0) . The dynamic motion equation of incoming ballistic missile is given as follows [20], [35]: where x(t), y(t) and z(t) are the target-centered Cartesian coordinates of the incoming ballistic missile, β is the ballistic coefficient, g is the gravity constant, ρ(z(t)) is the density of the atmosphere at the position of the incoming ballistic missile and is defined as follows [20], [35]: Based on the state space dynamic model, the motion equations of incoming ballistic missile in (21) could be rewritten as the following nonlinear dynamic equatioṅ where state vector is defined as Remark 7. In order to avoid the symbol confusion between the state vector x(t) of missile system in (1) and the coordinate x(t), y(t) and z(t) of missile position, in the following, we use X(t) to temporarily represent the state vector of missile system until (26).
Suppose that the incoming missile employs a guidance control law and suffers from stochastic external disturbance v(t) as follows:Ẋ Let Y (t) denote the sensor measurement of radar system and n(t) denote the stochastic measurement noise of radar sensor. Then the measurement equation of radar system is given by where Then the nonlinear stochastic signal system of radar detection of incoming missile can be rewritten as follows: where A(X(t)) = 4 (t)+x 2 5 (t)+x 2 6 (t) 2β 4 (t)+x 2 5 (t)+x 2 6 (t) 2β Based on the flow chart of H ∞ HJIE-embedded DNNbased filter scheme for radar detection of incoming missile stochastic system in Fig. 3, for practical design, the nonlinear stochastic radar detection system is modified to a nonlinear stochastic sample-data signal system. Then from (18) we obtain the following sample-data stochastic signal system of radar detection of incoming missile system, X(t + ∆t) = (X(t) + ∆tA(X(t))) + ∆tB(X(t))v(t) Y (t) = C(X(t)) + n(t) (26) where the sampling time ∆t is 0.01s.
Suppose we want to design the H ∞ HJIE-embedded DNN-based filter in Fig. 3 for the sample-data radar detection system in (26) to achieve the minmax H ∞ state estimation performance in (5) of incoming missile with a desired filtering ability ρ = 0.03. According to Theorem 1, we need to solve ∂V (x(t)) ∂x(t) of the following HJIE for Luenberger gain k * (x(t)) in (8) of robust H ∞ Luenberger-type filter in (2),
From the flow chart of H ∞ HJIE-embedded DNN-based filter scheme in Fig. 3, HJIE-embedded DNN is trained offline via Adam learning algorithm in (14) and (15) by the error HJIE = (θ k (t)) in (16) to achieve HJIE = 0 in (27) for DNN to output ∂V (x(t)) ∂x(t) . In this situation the robust H ∞ filter gain k * (x(t)), v * (t) and n * (t) can be obtained, which are sent to nonlinear stochastic missile detection system in (26), Luenberger-type filter in (19) and estimation error dynamic system in (20) to generate the corresponding signals y(t), x(t) andx(t) as well as x(t) =x(t) +x(t) for the next step t + ∆t of Adam training process as shown in Fig. 3.
Remark 8. At present, there exists no study to combine the theoretical result of nonlinear H ∞ filter and deep learning algorithm to implement the H ∞ deep neural network-based filter of nonlinear signal systems. It is not easy to implement this co-design of nonlinear H ∞ filtering algorithm and DNN learning algorithm for the state estimation of nonlinear stochastic signal system because the conventional deep learning algorithms need lots of empirical data to train DNN as image classifier or recognizer. Further, in the nonlinear H ∞ filter design, we need to solve HJIE, which is a function of the state x(t) of nonlinear stochastic system. However, the real state x(t) is unavailable, so we could not train the DNN model directly. We need filter model to generatex(t) and estimation error model to generatex(t). Therefore, we could obtain x(t) =x(t) +x(t) for solving HJIE. Further, since external disturbance v(t) and measurement noise n(t) are unavailable. Therefore, it is very difficult to generate y(t) by signal system model in (1) and estimation error model in (4) to generatex(t). Therefore, we use the worst-case v * (t) in (6) and n * (t) in (7) to replace v(t) and n(t) for signal system model in (1) to generate y(t) for filter model in (2) to generatex(t), and estimation error dynamic system in (4) to generatex(t) and therefore x(t) =x(t) +x(t) for HJIE for training DNN by Adam learning algorithm at each time t in the off-line training phase. In addition, in order to train DNN byx(t) to output ∂V (x(t)) ∂x(t) for solving HJIE for filter gain k * (x(t)) in (8), v * (t) in (6) and n * (t) in (7), an HJIE block is embedded in the DNN structure to approach to our desired output ∂V (x(t)) ∂x(t) . Therefore, we can use the output of DNN to produce the filter gain k * (x(t)), the worst-case external disturbance v * (t) and measurement noise n * (t) as shown in Fig. 1.
In this example, the architecture of DNN in Fig. 2 contains input layer, four hidden layers, one HJIE layer and output layer. The input layer consists of 6 inputs, four hidden layers consist of 256, 128, 32 and 6 hidden neural units sequentially which can be regarded as the concept of data compression. We want to train DNN to fit the nonlinear function ∂V (x(t)) ∂x(t) of HJIE for H ∞ filter gain k * (x(t)) through the fewer and fewer hidden neurals in each hidden layer sequentially. Further, if the learning rate l in (14) is assumed to be too large at the beginning, the gradient may be trapped in the local minimum, otherwise, the training speed will be too slow. The parameters λ 1 , λ 2 in Adam learning algorithm in Eq. (15) can also represent the impact in current direction from previous direction to avoid trapping in the local minimum and speed up the learning rate. These parameters can not be assumed to be small due to the important information in the previous direction. Finally, these parameters l, λ 1 , λ 2 and τ in Adam learning algorithm in (14) and (15) are selected as 0.001, 0.9, 0.999 and 10 −7 . The number of training steps and batch size are K = 30, M = 800, and the number of random sample data is N = 2000 as the number of training inputs. In the H ∞ filtering design strategy of the radar detection system in (5), the weighting matrix Q and ρ in (5) are specified as Q = 10 −5 I 6 and ρ = 0.03, respectively. The sampling time ∆t used in this example is 0.01s. The external disturbance v(t) and measurement noise n(t) are both assumed to be Gaussian random noise of 20N (0, 1). After H ∞ DNN-based filter being trained by Adam learning algorithm, the trajectory x 1 (t), · · · , x 6 (t) of the incoming missile in (25) is simulated with the initial condition X(0) = [150000, 210000, 120000, −2500, −2500, −2500] T .

Remark 9.
In the real application with DNN method, at the beginning of off-line training phase, we need to randomly select the initial training data near the errorx(t) which will significantly affect the training performance. If the domain is limited by the random selection in the off-line training phase, the state x(t) =x(t) +x(t) may be far from the training data during the on-line operation phase. Hence, it will limit the domain of ∂V (x(t))

∂x(t)
to approach to the real ∂V (x(t)) ∂x(t) of HJIE. In this situation, the error (θ k (t)) will be larger than a small prescribed δ, (i.e., | (θ k (t))| > δ) so that we need to start on Adam learning algorithm again for updating weighting parameters of DNN or change the method of random selections to train the weighting parameters of DNN again.
Remark 10. In this simulation, we have tested three differ- Remark 11. For the weighting matrix Q of robust H ∞ filter design strategy in (5), if the estimation errorx i (t) in the ith state is more significant for our incoming ballistic missile simulation, then we will set a larger parameter in the term (i, i) of Q. In contrast, the less parameter in the term (j, j) of Q if we consider the estimation errorx j (t) in the jth state is insignificant. In this simulation, we consider the estimation error in each element as equal importance with Q = 10 −5 I 6 in the incoming ballistic missile.
Based on the proposed H ∞ HJIE-embedded DNN-based filter, the location and the velocity of the incoming ballistic missile system and their corresponding estimations are shown in Fig. 5 and Fig. 6, respectively. From the simulation results in Fig. 5 and Fig. 6, although the state estimation of incoming missile has a transient response in the beginning, we can see that it still achieves the robust H ∞ filtering performance under the effect of the uncertain external distur-  [8] and particle filter in [11], respectively, in comparison. Obviously, the estimation performance of extended Kalman filter is very poor in the trajectory estimation of incoming ballistic missile. The mean error and standard deviation (µ, σ) in the trajectory estimation of H∞ DNN-based filter and particle filter at the estimation of x(t), y(t) and z(t) are given as follows: H∞ DNN-based filter :(-11.11, 1.32), (-11.11, 1.32), (-38.78, 3.61), and particle filter :(7.25, 10.51), (-59.42, 21.86), (11.79, 11.75). Obviously, particle filter has a good mean trajectory estimation performance but with a very large random fluctuation, which is not suitable for the high precise and robust trajectory estimation of incoming missile.
bance and measurement noise. The real H ∞ state estimation performance of the incoming missile by the proposed robust H ∞ DNN-based filter is calculated as follows: 70 0x It is seen that the proposed HJIE-embedded DNN filter scheme in Fig. 1 could achieve the prescribed H ∞ filtering performance to efficiently attenuate the effect of random external disturbance and measurement noise. Obviously, the filtering performance of the proposed robust H ∞ HJIEembedded DNN filter is better than the prescribed ρ = 0.03. The reason is that the H ∞ filter is designed based on minimizing the effect of the worst-case external disturbance v * (t) in (6) and the worst-case measurement noise n * (t) in (7), which may not exist in the real situation. However, in this simulation, we use v(t) and n(t) of stochastic process of 20N (0, 1) as external disturbance and measurement noise, respectively, which are not the worst-case v * (t) and n * (t). From the simulation results in Fig. 5 and Fig. 6, it is seen that the extended Kalman filter has a poor trajectory estimation performance of incoming missile and the particle filter has a good mean trajectory estimation but with a large random fluctuation at every time step of estimation because its estimation is based on the condition probability of state x(t) under measurement y(t). Comparing with the results of extented Kalman filter with exact statistical knowledge of v(t) and n(t) [8] and particle filter with exact statistical knowledge of v(t) and n(t) [11] in Fig. 5 and Fig. 6 for the trajectory estimation of incoming missile, we found that the proposed robust H ∞ HJIE-embedded DNN-based filter without the knowledge of external disturbance and measurement noise has better estimation performance than the conventional extented Kalman filter and particle filter for nonlinear stochastic signal systems with exact covariance matrices of external disturbance and measurement noise.
Remark 12. In the filter design field at present, there exists no study to use the conventional big data-driven DNN method to deal with this nonlinear filter design problem due to the unavailable empirical data for training DNN. In general, the conventional DNN is based on the input/output empirical data pairs to train the weighting parameters of hidden layers in DNN by optimizer. However, in the nonlinear H ∞ filter design problem, x(t) is unavailable and needs to be estimated from output measurement y(t). Therefore, we can not compare with the conventional big data-driven DNN method in nonlinear H ∞ filter design. Instead, we show the two nonlinear filtering methods of extended Kalman filter and particle filter to compare the filtering performance with the proposed robust H ∞ HJIE-embedded DNN filter.
In the extended Kalman filter [8], at every time step, we need to update the linearization of the nonlinear functions A(x(t)), B(x(t)) and C(x(t)) of nonlinear stochastic signal system in (1), update the estimation error covariance prediction for state prediction and then update the filtering error covariance for state estimation (filtering). There are a large amount of computations to be performed at every time step. If the stochastic signal system is highly nonlinear, the linearization error will propagate to the subsequent step. Further, if the covariance matrices of v(t) and n(t) are unknown or uncertain, it will deteriorate the performance of extended Kalman filter. For the particle filter design [11], we need to predict the conditional probability of state x(t) with the observation according to P (x t |y 1:t−1 ) = dxt−1 P (x t |x t−1 )P (x t−1 |y 1:t−1 )dx t−1 where y 1:t−1 denotes the partial observations y 1 , . . . , y t−1 , and update the filtering of particle filter according to the conditional probability P (x t |y 1:t ) = P (yt|xt)P (xt|y1:t−1) P (yt|y1:t−1) . Further, we still need to update the importance distribution for particle filter. There are a lot of complicated computations in every step of particle filter. However, in the proposed robust H ∞ HJIE-embedded DNN-based filter scheme, after the HJIE-embedded DNN has been trained by Adam learning algorithm in the off-line training phase, the proposed H ∞ DNN-based filter could perform with a quite good trajectory estimation performance in the incoming missile detection system without the knowledge of external disturbance and measurement noise. The E{V (x(0)) − V (x(t f )) + t f 0 (x T (t)Qx(t) + ∂V (x(t)) ∂x(t) T (A(x(t)) − A(x(t))) + ∂V (x(t)) ∂x(t) T B(x(t))v(t) − ∂V (x(t)) ∂x(t) T k(x(t)) × (C(x(t)) − C(x(t))) − ∂V (x(t)) ∂x(t) T k(x(t))n(t)) − ρv T (t)v(t) − ρn T (t)n(t))dt} (A8) By completing square technique of v(t) and n(t), we get max v(t),n(t) E{V (x(0)) − V (x(t f )) ∂x(t) T k(x(t))(C(x(t)) − C(x(t))))dt} (A9) By completing square techniques of k(x(t)), we get max v(t),n(t) E{V (x(0)) − V (x(t f )) ∂x(t) T (A(x(t)) − A(x(t)))

Appendix B: Proof of Theorem 2
Suppose there exists an error function e(x(t)) between ∂V (x(t)) ∂x(t) of HJIE in (16) and ∂V (x(t))