Automatic Control, IEEE Transactions on

The authors consider multiarmed bandit problems with switching cost, define uniformly good allocation rules, and restrict attention to such rules. They present a lower bound on the asymptotic performance of uniformly good allocation rules and construct an allocation scheme that achieves the bound. It is found that despite the inclusion of a switching cost the proposed allocation scheme achieves the same asymptotic performance as the optimal rule for the bandit problem without switching cost. This is made possible by grouping together samples into blocks of increasing sizes, thereby reducing the number of switches to O(log n). Finally, an optimal allocation scheme for a large class of distributions which includes members of the exponential family is illustrated.<> View full abstract»

The existence of a fixed internal dynamics, inherent to the noninteraction requirement, is identified. When unstable, this represents an obstruction to the achievement of noninteracting control with stability, by means of static state-feedback. Remarkably, whenever one leaves the class of linear systems, this obstruction cannot, in general, be removed by dynamic state-feedback.<> View full abstract»

A framework for studying asymptotic orders of reachability in perturbed linear, time-invariant systems is developed. The systems of interest are defined by matrices that have asymptotic expansions in powers of a perturbation parameter epsilon about the point 0. The reachability structure is exposed by means of the Smith form of the reachability matrix. This approach is used to provide insight into the kinds of inputs needed to reach weakly reachable target states, into the structure of high-gain feedback for pole placement, and into the types of inputs that steer trajectories arbitrarily close to almost (A,B)-invariant subspaces and almost (A,B)-controllability subspaces.<> View full abstract»

The problem of minimizing a regulated output due to a specific bounded input is solved in the SISO (single-input single-output) discrete-time case. It is shown that, in most situations, the optimal solution will not exist, but given any epsilon >0, it is possible to find a suboptimal solution with performance within epsilon of the optimal performance. The proposed approach is extended to the problem of shaping the error transient response. It is shown how time-domain templates can be used in this problem in the same way that frequency-weighting functions are used in the H/sup infinity / and H/sup 2/ problems.<> View full abstract»

Graph-theoretic conditions are obtained for a structured system to have the property that the supremal L/sub /p-almost invariant (controllability) subspace is generically the entire state space and that the infimal /sup L//sub /p-almost conditional invariant (complementary observability) subspace is generically the zero subspace. The conditions are used to determine stabilizability of structured interconnected systems by means of decentralized feedback control. Although the obtained graph-theoretic conditions are conservative, they are considered satisfactory to the extent that the benefits of easy testability involving only binary calculations outweigh the conservativeness in the results.<> View full abstract»

The continuous-time stationary linear-quadratic-Gaussian (LQG) optimal regulation problem is considered where the measurements are free of white noise components. A simple direct solution in the s-domain is derived for the optimal controller for general linear, time-variant right-invertible or left-invertible systems. The explicit expressions that are found for the controller transference and for the regulator return-ratio matrix can be used to obtain a practical suboptimal design in the s-domain. These expressions are applied to derive simple conditions for the precise recovery of the return-ratio matrix of the LQG regulator.<> View full abstract»

The authors develop a geometric approach to the theory of 2-D (two-dimensional) systems, defining in a suitable way the notion of (A/sub 1.2/, B/sub 1.2/)-invariant subspaces and of controlled invariant subspaces. Such subspaces are shown to have good computational and feedback properties, which make them useful in application. In particular, sufficient conditions and constructive procedures are obtained for the solutions of disturbance decoupling and model matching problems. Moreover, it is shown that certain structural properties of a 2-D system can be described by means of a set of indexes defined in geometric terms, and that such structural indexes can be used to reformulate the sufficient condition for model matching.<> View full abstract»

A quite general approach to the problem of stabilizing a SISO (single-input single-output) bilinear system by output feedback is presented. Following a Lyapunov-like method, a family of admissible controls is defined and an easy way for deriving its general features is proposed. Previous results can be obtained as special cases of this approach. Numerical examples are included to show the feasibility of this algorithm in determining simple control laws.<> View full abstract»

A linear time-invariant optimal estimator for uncertain output matrix systems is considered. Uncertainty of the output matrix is represented by a set of multiple parameters, and the optimal estimator is introduced by minimizing the expected cost function of the estimation error. When the number of measured variables is small, the order of the optimal estimator does not depend on the number of models, and meeting the computational demand becomes quite feasible View full abstract»

The e₁-modification law has been proposed by K.S. Narendra and A.M. Annaswamy (ibid., vol.32, no.2, pp.134-5, 1987) for robust model reference adaptive control in the presence of bounded disturbances. Using this law, it is shown by the present authors that, in the case of regulation, a prototype adaptive control system with a single unknown pole and an unknown constant disturbance at its input possesses multiple equilibria and undergoes various bifurcations. Simulations and local analysis give a complete characterization of the possible dynamics exhibited by the disturbance adaptive system. As the constant disturbance is varied, saddle-node bifurcation, (subcritical) Hopf bifurcation, and saddle-connection bifurcation occur in that order. Consequently, in this case, the e₁-modification law is qualitatively equivalent to the σ-modification law View full abstract»

Deterministic and stochastic Lyapunov theorems are used to demonstrate the robustness of a stable linear, time-variant, discrete-time nominal system to both unknown deterministic and stochastic perturbations. Time-domain conditions are presented on the appropriate deterministic or random characteristics of perturbations to maintain the proper stability behavior of the overall system. It is concluded that the novel robustness conditions proposed can find application in the feedback design of control systems where the closed-loop system is known to be stable with a certain degree, e.g. as in the case of linear quadratic optimal control with α-degree of prescribed stability View full abstract»

The authors analyze the behavior of a standard identifier when the plant contains additional dynamics, called unmodeled dynamics, which invalidate the known order assumption. The first result of the analysis is an input richness condition which does not depend on the order of the unmodeled dynamics to guarantee persistency of excitation of the regressor. Then it is shown that the persistently exciting (PE) condition leads to a BIBO (bounded-input bounded-output) stability property for the identifier. The method of averaging is used to formally define the notion of tuned parameters as the equilibrium of the identifier averaged system. It is shown that the tuned parameters always exist and that the actual parameters converge to some neighborhood of the tuned parameters. From the definition of the tuned parameters, an explicit expression to calculate and interpret them as the fixed parameter values that minimize the mean-squared output error is derived View full abstract»

The main objective of the authors is to provide a necessary and sufficient condition for a polytope of polynomials to have all its zeros inside the unit circle. The criterion obtained serves as a discrete-time counterpart for results in S. Bialas (1985) and F. Fu and B.R. Barmish (1987) for the continuous case. Also, the results are reduced to operations on (n-1)×(n-1) matrices. It is concluded that, by the edge result of A.C. Bartlett et al. (1987), it suffices to check the exposed edges in order to determine whether a polytope of polynomials has all its zeros in a simply connected region D View full abstract»

The problem of identification of a nonstationary stochastic system is considered, and an estimation method based on the functional series modeling (FSM) of the system parameter trajectory is proposed for its solution. It is shown that the parameter-matching properties of FSM estimators can be described in terms of the appropriately defined (time-varying) impulse and frequency responses. It is suggested and verified by means of computer simulation that the averaged frequency characteristics associated with FSM estimators can yield useful information about their parameter-matching abilities View full abstract»

A fundamental robustness theorem for robust eigenvalue assignment of multivariable feedback systems is derived. The theorem determines whether a perturbed multivariable feedback system has its characteristic polynomial zeros located in the same regions as the nominal system does. It can be applied to continuous systems as well as to discrete systems. The theorem can handle nonsquare transfer matrices as well as dynamic output feedback. Conditions are discussed under which the proposed theorem reduces to a fundamental theorem for stability robustness analysis View full abstract»

It is shown that the strict positive realness (SPR) condition for asymptotic stability of the continuous-time parallel model reference adaptive system (MRAS) can be removed provided that the parameter adaption law includes a proportional term and the information matrix is made sufficiently large. The present result a continuous-time counterpart of an earlier result (M. Tomizuka, ibid., vol.27, no.4, p.505-6, 1982) on the discrete-time parallel MRAS View full abstract»

The author states various time-domain and frequency-domain conditions pertaining to multivariate strict positive real systems and establishes the relationship between these conditions. Their relationship is clarified by means of a three-tier diagram; each tier represents a set of equivalent conditions successively stronger than the lower tier. The main theorem in the present study can be considered as the positive realness lemma for strictly positive real systems View full abstract»

It is demonstrated that for arbitrary linear autonomous systems, a quadratic form yielding a Lyapunov function for the continuous-time system can be used to derive rather easily a Lyapunov function of the same type for the corresponding discrete-time system resulting from it by sampling with sampling period τ. Conversely, if for an arbitrary discrete system allowing an interpretation as sampled-data system, i.e. having a regular system matrix F, a quadratic form yielding a Lyapunov function is found, then a Lyapunov function for the corresponding continuous-time system can be computed rather easily from it View full abstract»

A generalized matrix fraction description (MFD) criterion for fixed modes with arbitrarily constrained feedback structure is presented. The efficiency of the new criterion in structure analysis is illustrated by a numerical example. This criterion, which generalizes that of B.O. Anderson and D.J. Clements (Automatica, vol.17, p.703-12, 1981), is shown to be more convenient in some cases than other generalized criteria of fixed modes View full abstract»

The optimal filtering error for a class of nonlinear cone-bounded filtering problems is analyzed for the case of observation noise intensity tending to zero. A sufficient condition for the resulting optimal error covariance matrix to tend to zero is derived by relating the nonlinear filtering problems to linear ones. The results are specialized for the case of nonlinear autoregressive moving-average (ARMA) processes View full abstract»

Recently, several methods have appeared for the approximation of (power) spectra, notably balanced stochastic truncation (BST). It is shown that BST satisfies a relative error bound approximately twice the bound for the relative error method (REM) proper. This offers a quantitative basis for the observation that BST and REM produce similar reduced-order models. Balanced stochastic truncation can therefore be interpreted as providing a computationally simple algorithm for relative error approximation View full abstract»