Risk-Aware Maximum Hands-Off Control Using Worst-Case Conditional Value-at-Risk

With the view of risks, this article deals with the problems of maximum hands-off control that aims at minimizing the length of nonzero control input. More specifically, we consider stochastic systems and seek sparse control inputs that bring the system state to a ball centered at the origin, such that the expected value of the states that are further than a given threshold from the origin is small, thus minimizing the risk that the system state is outside of the ball. To deal with this problem, we employ the worst-case conditional value-at-risk under the assumption that the first two moments of the disturbance distribution are known. In particular, we consider two kinds of risk-aware maximum hands-off control problems: one enhances the sparsity within a given risk threshold, and the other minimizes the risk subject to a sparsity constraint. We also derive a risk-constrained sparse model predictive control and provide a numerical example that shows the effectiveness of the proposed approach in networked control systems.


I. INTRODUCTION
The maximum hands-off control introduced in [1] aims at minimizing the length of time during which the control input value is nonzero while achieving given control objectives.Such a control approach provides a sparse control input, which is valuable for electric/hybrid vehicles and electric locomotives because it helps to reduce fuel and/or electric energy consumption.Maximum hands-off control also has significant advantages in networked control systems.This is because real networks are rate-limited and transmitted signals should be compressed to small data to meet the network limitation [2].In [3], it was shown that sparse signals can be more effectively compressed than densely represented signals.Motivated by these various applications, the maximum hands-off control has been proposed for discrete-time systems [4], [5], [6], uncertain systems [7], stochastic systems [8], [9], and infinite-dimensional systems [10].A good survey of the maximum hands-off control was given in [11].
This article considers the maximum hands-off control problems for stochastic systems with the view of risks, or more specifically, tail risks.For safety and reliability, it is important to consider tail risks Manuscript received 5 April 2022; revised 15 August 2022; accepted 26 December 2022.Date of publication 9 January 2023; date of current version 27 September 2023.This work was supported by JST, PRESTO, Japan, under Grant JPMJPR22C3 and JSPS KAKENHI under Grant JP20H02172, Grant JP20K21008, and Grant JP19H02301.Recommended by Associate Editor P. G. Mehta.(Corresponding author: Masako Kishida.)Masako Kishida is with the National Institute of Informatics, Tokyo 101-8430, Japan (e-mail: kishida@nii.ac.jp).
Color versions of one or more figures in this article are available at https://doi.org/10.1109/TAC.2023.3235246.
Digital Object Identifier 10.1109/TAC.2023.3235246 in the decision-making processes to quantify the risk that has a low probability of occurring, but if it does occur, it will result in a large loss.However, in most cases, it is very difficult to obtain the exact knowledge of uncertainties; the probability distribution of uncertainties is not always available.This may be because uncertainties come from many different sources.Even in such cases, moments are relatively easy to compute.Therefore, we consider the case where the system is subject to external disturbances whose first and second moments are already computed.With disturbances, it is not possible to bring the system state to a specific location and keep it there.Thus, our goal is to minimize the risk that the system state moves away from a given ball centered at the origin in the worst case as well as to keep the duration of nonzero control inputs short.
In order to deal with disturbances that are only known to be contained in a set of probability distributions with specified moments, we formulate the problems in the form of distributionally robust optimization [12], [13], [14].Namely, we deal with the supremum of the risk over all possible disturbances in the set.As a risk measure, we employ the notion of the conditional value-at-risk (CVaR), or more precisely, the worst-case CVaR.CVaR is defined as the conditional expectation of losses exceeding a certain threshold, which provides a convex conservative approximation for a joint chance constraint [13].The worst-case CVaR is the supremum of CVaR over the set of possible disturbances.Because the (worst-case) CVaR is a coherent risk measure [15], [16], [17], [18], it enjoys nice mathematical properties.In particular, we observe that the problem of maximum hands-off control using the worst-case CVaR on the squared norm of the states using the first two moments of the disturbance allows a beautiful and efficient mathematical formulation of convex optimization if the control input sparsity is promoted by using 1 -norm.
We present the risk-constrained sparsity enhancement finite-horizon control problem using the 0 -norm of the control inputs and worst-case CVaR, which is followed by the convex optimization problem that is obtained by relaxing the 0 -norm with the 1 -norm.The solutions to those two problems are then compared with the solutions to cases without disturbances.Another important contribution is to show that it is not possible for the system states to be in a given ball centered at the origin regardless of the control input if the ball radius is too small compared with the disturbance.Motivated by this, we also investigate the problems of minimizing the risk subject to sparsity constraints with 0 -and 1 -norms.Furthermore, we develop a model predictive control (MPC) [19] that promotes sparsity subject to risk constraints and demonstrate its effectiveness for the quantized control in the networked control systems.We choose to use the networked control problem for the numerical example because although the transmitted signals should suffer from quantization, packet-dropout, noise, and so on [20], to date, only a few studies have investigated maximum hands-off control to deal with such issues.
The rest of this article is organized as follows.After introducing basic notation, definitions and important results in Section II, Section III provides the system description along with some preliminary observations.Two problems of finite-horizon risk-aware maximum hands-off control, risk-constrained sparsity enhancement and sparsity-constrained risk minimization, are discussed in Sections IV and V, respectively.Based on the results in Section IV, a risk-constrained sparse MPC is developed in Section VI, which is followed by a numerical example in Section VII.Finally, Section VIII concludes this article.

A. Notation
The sets of real numbers, real vectors of length n, and real matrices of size n × m are denoted by R, R n , and R n×m , respectively.For M ∈ R n×n , M 0 and M 0 indicate that M is positive definite and positive semidefinite, respectively.The set of positive semidefinite symmetric matrices in R n×n is denoted by S n + .M denotes the transpose of a real matrix M , M − denotes the transpose of the inverse of M , and Tr(M ) denotes the trace of M .I n denotes the identity matrix of size n, and the subscript may be dropped when the size is clear.The Kronecker product of two matrices X and Y is denoted as X ⊗ Y .For x ∈ R, x + = max{x, 0}.For a vector v ∈ R n , v 0 , v 1 , and v 2 denote the number of nonzero entries of the vector v ( 0 -norm), Manhattan norm ( 1 -norm), and Euclidean norm ( 2 -norm), respectively.For a matrix M , M 2 denotes the maximum singular value norm.diag[A 1 , A 2 , . .., A n ] denotes a block diagonal matrix, whose diagonal blocks are A 1 , A 2 ,..., A n .

B. CVaR
Let μ ∈ R n be the mean and Σ ∈ R n×n be the covariance matrix of the random vector ξ ∈ R n under the true distribution P ξ (i.e., P ξ is the probability law of ξ).For simplicity, we drop the subscript ξ and write P instead of P ξ from now on.Thus, we implicitly assume that the random vector ξ has finite second-order moments.Let P denote the set of all probability distributions on R n that have the same first-and second-order moments as P .Define the second-order moment matrix of ξ by Definition II.1 (CVaR [21]): For a given measurable loss function L : R n → R, the probability distribution P on R n and the level ε ∈ (0, 1), the CVaR at ε with respect to P is defined as where E P [•] denotes the expectation with respect to P .The CVaR is the conditional expectation of loss above the (1 − ε)quantile of the loss function [13] and quantifies the tail risk.
Definition II.2 (Worst-case CVaR [13]): The worst-case CVaR over P is given by sup The worst-case CVaR is the supremum of CVaR over a given set of probability distributions.The exchange of the maximization and minimization is justified by a stochastic saddle point theorem [22].In this article, this worst-case CVaR will be used to design control inputs that minimize the potential outlier states that are outside of a ball.Now, we are ready to introduce some basic results.Lemma II.3 (Worst-case CVaR for quadratic function [13], [23]): Let Lemma II.4 (Bounds for worst-case CVaR [24]): Suppose μ = 0 and Remark II.5: Suppose μ = 0.If we only know that the upper bound Σ of the covariance, i.e., Σ Σ, then Lemma II.3 holds by replacing = n be two independent random vectors whose means are zero, and = sup where P, P 1 , and P 2 denote the set of all probability functions on R n , R n 1 , and R n 2 that have the same first-and second-order moments as the probability law of the random vectors ξ, ξ 1 , and ξ 2 have, respectively.Proof: Let Σ 1 and Σ 2 be the covariances of ξ 1 and ξ 2 , respectively.Then, because ξ 1 and ξ 2 are independent, the covariance of We can extend this result as follows: Corollary II.7:Let ξ i ∈ R n , i = 1, . . ., k, be k independent random vectors whose means are zero under the same true distributions P and where P aug and P denote the set of all probability functions on R nk and R n that have the same first-and second-order moments as the probability law of the random vectors ξ and ξ i have, respectively.

C. Sparsity
To deal with sparsity, let us define the set of all s-sparse vectors by Definition II.8 (Restricted isometry property [25]): A matrix M is said to satisfy the restricted isometry property (RIP) of order s if there Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
It is known that no efficient RIP-testing algorithm exists [26].

III. SYSTEM DESCRIPTION
Consider the discrete-time linear time-invariant stochastic system over a finite time horizon where x t ∈ R n is the system state, u t ∈ R nu is the input, and w t ∈ R nw is the process noise or disturbance, respectively, at discrete time instant t.A ∈ R n×n , B ∈ R n×nu , and E ∈ R n×nw are time-invariant system matrices, and the pairs (A, B) and (A, E) are both reachable (i.e., rank It is assumed that an initial state x 0 = ζ is given and T ≥ n is the given final time of interest.w t are independent and identically distributed random vectors with the mean zero and covariance Σ w 0 for all t.The true underlying probability measure P w is not exactly known, but it is known that P w ∈ P 1 , where Here, δ ij is the Kronecker delta.The second-order moment matrix of w t is given by For t ≥ 1, the state evolution of (1) can be expressed by Thus, the state at the final time T can be written as where With this notation, wt ∈ R nw t is a random vector with the mean zero and covariance I t ⊗ Σ w .Thus, the true underlying probability measure P for wt satisfies P ∈ P t , where and the second-order moment matrix of wt is given by We write P for P T and Ω for Ω T .
Before considering the main problems, the rest of this section extends the results of [4] from the case of a single-input system to the case of a multiple-input system.Namely, we show the existence of an n-sparse control input ū that drives the state from the initial state x 0 = ζ to x T = 0 in the case of no disturbance, i.e., E = 0.For this purpose, let us define an allowable control set by The following lemma shows that the set V 0 ζ ∩ Φ n is nonempty.Lemma III.1 (Existence of allowable sparse control input, cf., [4]): Consider the system (1) with E = 0.For any ζ, there exists ū ∈ V 0 ζ ∩ Φ n .
Proof: Let partition the matrix G as where Then, G 2 is full rank because (A, B) is reachable.Thus, by applying QR decomposition to G 2 , it follows that where R ∈ R n×n is an upper triangular and nonsingular matrix and where ũ ∈ R n is to be determined, it follows that Since R is invertible, by choosing it follows that and From now on, we bring back E.

IV. RISK-CONSTRAINED SPARSE CONTROL
This section considers the problem of promoting the sparsity of the control input while keeping the risk small that the outlier states go far away outside of a ball of radius r.More specifically, we deal with the problems of minimizing the 0 -and 1 -norms of the control inputs that satisfy the worst-case CVaR constraint for the system (1).This problem is to see how much control effort is needed to satisfy the risk constraint.We also consider the relationship between the set of allowable control inputs that satisfy the worst-case CVaR constraint and the set of allowable control inputs without disturbances in (10).
For this problem, we define an allowable control input set by where with r ≥ 0. If a control input ū belongs to U r ζ in (19), then the expected value above (1 − ε)-quantile of x T 2 2 is no more than r 2 .Since the worst-case CVaR provides a conservative approximation for the following distributionally robust chance constraint: inf we can also say that any ū ∈ U r ζ brings x T within a ball of radius r centered at the origin with probability no less than 1 − ε.Here, note that U r ζ can be empty, and the condition for nonemptiness is given later in this section.
Throughout this article, we consider the squared norm of the state • 2  2 and the squared radius r 2 , instead of • 2 and r.This squaring operation allows us tractable problem formulation with the worst-case CVaR.
The problem of maximizing the sparsity of the control input ū subject to the worst-case CVaR constraint can be formulated as follows.
Problem IV.1 (Minimization of 0 -norm of allowable control input): The solution to Problem IV.1 can be found by solving the following equivalent optimization problem: Here, we used the fact that and Lemma II.3 to observe that U r ζ is the set of ū that satisfies sup Problem IV.1 or the optimization problem ( 23) is difficult to solve because of the nonconvex 0 -norm for the control input.By relaxing the 0 -norm by the 1 -norm, we obtain the following problem, from which we can still expect that the obtained control input is sparse even if it may not be the sparsest.

Problem IV.2 (Minimization of 1 -norm of allowable control input):
This time, the solution to Problem IV.2 can be found by solving the following equivalent convex optimization problem: Now, we consider the feasibility of Problems IV.1 and IV.2.

Lemma IV.3 (Existence of allowable control input):
The set U r ζ is nonempty if and only if Moreover, if ( 28) is satisfied, then is a feasible solution to both optimization problems (22) and (26).
Proof: Suppose (28) does not hold.Then, Lemma II.4 implies ū in U r ζ must satisfy which is impossible.Thus U r ζ is empty.On the other hand, if (28) holds, then Lemma II.4 implies that a sufficient condition for U r ζ is nonempty is the existence of ū that satisfies where From Lemma III.1, F ζ + Gū 2 = 0 is satisfied with an n-sparse solution (29).Thus, the optimization problems ( 22) and ( 26) are feasible if and only if (28) is satisfied.
This result can be seen as an impossibility result.Namely, this shows that it is impossible for the expected value above (1 − ε)-quantile of x T 2 2 to be less than r 2 unless the covariance Σ w is small enough compared with r regardless of the control input.This motivates us the consideration of minimizing the risk as much as possible with the control input constraint, which we discuss in Section V.
We can design the parameter r for a reasonable control objective without computing the summation in the right-hand-side of (28) using the following result.
Lemma IV.4:For any final time T , where P 0 is the solution to the Lyapunov function Proof: From Lemma II.4 and Corollary II.7, This bound is independent of the time horizon T .
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
Next, consider the relation between the allowable control set in (10) and the allowable control set in (19).
Theorem IV.5 (Relationship between U r ζ and = sup 28) is satisfied with equality, then from Lemma II.4,u ∈ U r ζ implies that sup Thus, u ∈ V 0 ζ .Let us introduce two more closely related problems: Problem IV.6 (Minimization of 0 -norm of allowable control input): Problem IV.7 (Minimization of 1 -norm of allowable control input): Those problems provide control inputs that bring the state to the origin at the final time T if there is no disturbance.From Theorem IV.5, the optimal values of Problems IV.6 and IV.7 provide upper bounds on the optimal values of Problems IV.1 and IV.2.If the disturbance and radius satisfy (28) with equality, then the same control input which brings the state to the origin assuming no disturbance, achieves the worst-case CVaR objective sup Problem IV.6 is in the form of the basis pursuit problem in signal processing.Solutions to these two problems are discussed in [4] for the single-input case, and the following result remains the same for the multi-input case.
We go back to the original problem.The quality of 1 -norm relaxation can be assessed by the following.
Theorem IV.9 (cf., Lemma IV.8): Let ū0 and ū1 be the solutions to Problems IV.1 and IV.2, respectively.If the matrix G satisfies the RIP of order 2s with δ 2s < √ 2 − 1 and ū0 is unique and s-sparse, then where d is defined in (32) and Furthermore, when d = 0, ū0 = ū1 .Proof: From Lemma II.4, the condition sup On the other hand, by [25, Th. where The four problems in Section IV are closely related as summarized in Fig. 1.

V. SPARSITY-CONSTRAINED RISK MINIMIZATION
This section considers the problem of minimizing the risk subject to sparsity constraints for the system (1).More specifically, we deal with the problem of minimizing the worst-case CVaR subject to 0 -norm constraint and 1 -norm constraint.This problem is closely related to the problem in the previous section.Such a problem appears when one needs to check the control performance subject to a strict requirement on the sparsity of the control inputs.
Problem V.1 (Minimization of risk subject to sparsity constraints): min ū sup where s 0 is a given design requirement and as before, Roughly speaking, this problem is to bring the state x T as close as possible to the origin, using a control input that satisfies the sparsity constraint ū 0 ≤ s 0 .
The solution to Problem V.1 is the solution to the following equivalent optimization problem: Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
Lemma V.2: Consider Problem V.1 with s 0 ≥ n.Then, ū in (29) satisfies the sparsity constraint in (47) and provides the optimal value Proof: Follows from Lemma II.4.
In the general case of s 0 < n, solving Problem V.1 is difficult due to the nonconvex constraint for sparsity.Now consider the convex relaxation of Problem V.1.
Problem V.3 (Minimization of risk subject to 1 -norm constraint): min ū sup where s 1 is a given design requirement.
The solution to Problem V.3 is the solution to the following convex optimization problem: From Lemma II.4,we have the following: 1) sup These motivate us to consider minimizing F ζ + Gū 2 .Problem V. 4: Problem V.5: Problem V.4 is in the form of matching pursuit [27] in signal processing and Problem V.5 is its relaxation.
As stated in Lemma V.2, if s 0 ≥ n, then there exists ū such that F ζ + Gū 2 = 0. Thus, the optimal solution to Problem V.4 is also zero with ū in (29).

VI. RISK-CONSTRAINED SPARSE MPC
In this section, we develop a sparse MPC that brings the state inside the ball of radius r at the specific time instance T and keeps it inside the ball afterward with the view of risk.The approach is based on the finite-horizon 1 -optimal control subject to the risk constraint in Problem IV.2 in Section IV.
Definition VI.1 (Risk-constrained sparse MPC): Consider the system (1).Recall that n is the dimension of the state vector x t .Let the time horizon p ≥ n and the target time instance T ≥ n.Without loss of generality, it is assumed p ≥ T .Choose the radius r of the target ball that satisfies the condition At every time instance t, the state x t is observed.Using this state x t , find a sparse control input ût ∈ R pnu for the next p time instances by solving the finite-horizon 1 -optimal control problem subject to the following (possibly soft) constraints: where k 0 = max{1, T − t}.Here, x k|t is the state at the time k + t that is predicted at time t using (1) with the initial condition ζ = x t = x 0|t .
Once the 1 -optimal control vector ût is found, then apply the control input u t defined by Note that, under the condition (55), for any single k ∈ [n, p], the existence of control input is guaranteed such that the state at time k + t predicted at time t satisfies the constraint sup (58) However, this does not mean that there exists a control input ût that satisfies all the constraints in (56) at the same time.We choose to force this constraint to be satisfied only at the end of the prediction horizon and relax for other time instances.Then, the 1 -optimal control vector is the optimal solution to the following problem.Problem VI.2 (Minimization of 1 -norm for MPC): where k 0 = max{1, T − t} and γ > 0 is a weight.This minimizes the worst-case violation of the constraint (56) during the next p − 1 time instances while satisfying the constraint at the end of the prediction horizon.This can be solved using the following convex optimization problem: where k 0 = max{1, T − t} Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

VII. NUMERICAL EXAMPLE
This section provides a numerical example to illustrate the proposed approach.For this purpose, we design a risk-constrained sparse MPC discussed in Section VI for a quantized control for the use of networked control systems.
We consider a stable linear plant model This model is obtained by the zero-order-hold discretization [28] of a continuous-time system 10/(s + 1) 3 with sampling period h = 0.2.
For this system, we introduce a uniform quantizer for the control input u t : where • denotes the floor function and Δ > 0 is the quantization step size.Let w t be the stochastic quantization error at time t, then the quantized control input Q(u t ) can be expressed as Thus, our plant model becomes which is in the form of (1).As mentioned in Section I, the sparse quantized control input is desired for a networked control system.However, the direct sparsity enhancement for Q(u t ) is nontrivial.This motivates us to enhance the sparsity of u t instead of Q(u t ) because if u t = 0, then Q(u t ) = 0.In other words, the sparsity of u t guarantees the sparsity of Q(u t ).Hence, we apply the proposed risk-constrained sparse MPC to the system (67).
For the simulation, the value of ε in the worst-case CVaR is taken as ε = 0.5 and the quantizer's step size is set to Δ = 5.It is assumed that the quantization error is uniformly distributed over (−Δ/2, Δ/2).Then, the mean and variance of the noise w t ∈ R are μ w = 0 and Σ w = Δ 2 /12 = 2.0833.The time horizon and ball radius are set to p = 5 and r = 5.4308, respectively.These parameters satisfy the inequality (28) by Here, we would like to note that the value of r = 5.4308 was chosen only slightly larger than the step size of quantization Δ = 5.We can also see that the inequality (28) for this case can be expressed as This means that the conditional expectation of x p 2 above the 50%quantile of x p 2 (i.e., ε = 0.5) is about half of the quantizer's step  size, which seems reasonable.The weight and target time instance are set to γ = 1 and T = 4, respectively.In Figs. 2 and 3, the proposed risk-constrained sparse MPC is compared with the maximum hands-off MPC designed for the nominal system using (39) [4].Note that the latter controller design does not take into account the control input quantization, but the control input is quantized before it is applied to the system.Fig. 2 shows the resulting quantized control inputs.We can easily see that the proposed MPC provides a much sparser control input than the nominal MPC does, as it takes into account the effect of quantization errors in the design.In particular, the nominal MPC shows a steady oscillation due to the quantization.
The control performance measured by the Euclidean norm x t 2 , t = 0, 1, 2, . . ., is shown in Fig. 3.The performance of the proposed MPC is obviously better than that of the nominal MPC.
In summary, the proposed risk-constrained MPC successfully provided a sparse quantized control input while the nominal MPC failed to produce a sparse control input.

Fig. 2 .
Fig.2.Quantized control Q(u t ) by MPC for a sample run: proposed risk-constrained sparse control (solid) and nominal control (dashed).