04 Automatic Control

then also the feedback system
x(t)
= Ax(t) B[Cx(t), t]
ABSOLUTE STABILITY
is exponentially stable.
Analysis of dynamical systems and circuits is mostly done under the assumption that the system is linear and time-invariant. Powerful mathematical techniques are then available for
analysis of stability and performance of the system, for example, superposition and frequency domain analysis. In fact,
even if the system is nonlinear and time-varying, such assumptions can often be used to get a first estimate of the system properties.
The purpose of absolute stability theory is to carry the
analysis one step further and get a bound on the possible influence of the nonlinear or time-varying components. The approach was suggested by the Russian mathematician Lure in
the 1940s and has, since then, developed into a cornerstone
of nonlinear systems theory.
The basic setup is illustrated in Fig. 1, where the linear
time-invariant part is represented by the transfer function
G(s) and the nonlinear parts of the system are represented by
a feedback loop w (v, t). The analysis of the system is
based on conic bounds on the nonlinearity (Fig. 2):
(v, t)/v
for all v = 0
The name of this result comes from its graphical interpretation. The Nyquist plot, that is, the plot of G(i) in the complex plane as R, must not cross or circumscribe the circle
centered on the real axis and passing through 1/ and 1/
(Fig. 3).
An important aspect of the circle criterion is that it demonstrates how frequency domain properties can be used in a
nonlinear setting. It is instructive to compare with the Nyquist criterion, which states that the closed-loop system with
linear feedback w(t) kv(t) is stable for all k [, ], provided that G(i) does not intersect the real axis outside the
interval [1/ , 1/]. The circle criterion replaces the interval condition with a circular disk. As a consequence, the stability assertion is extended from constant feedback to nonlinear and time-varying feedback.
The proof of the circle criterion is based on a quadratic
Lyapunov function of the form
V (x) = x Px
(1)
where the matrix P is positive definite. It can be verified that

V(x) is decreasing along all possible trajectories of the system,
provided that the frequency condition [Eq. (3)] holds. As a
consequence, the state must approach zero, regardless of the
initial conditions.
This problem was studied in Russia during the 1950s. In

particular, a conjecture by Aiserman was discussed, hoping
that the system would be stable for all continuous functions
in the cone [Eq. (1)], if and only if it was stable for all linear
functions in the cone. This conjecture was finally proved to
be false, and it was not until the early 1960s that a major
breakthrough was achieved by Popov (1).
THE POPOV CRITERION

In the case that has no explicit time dependence, the circle
criterion can be improved. For simplicity, let 0 and hence
0 (v)/v
THE CIRCLE CRITERION
x = Ax Bw
R e[(1 + i)G(i)] >
and the corresponding transfer function is
x(t)
= Ax(t) B[Cx(t)]
Theorem 1 (Circle criterion). Suppose that the system

x Ax is exponentially stable and that : R2 R is Lipschitz
continuous and satisfies Eq. (1). If
R
(6)
then the system
G(s) = C(sI A)1 B
G(i) + 1
0 < Re
G(i) + 1
(5)
Theorem 2 (Popov criterion). Suppose that : R R is

Lipschitz continuous and satisfies Eq. (5). Suppose the system x Ax is exponentially stable and let G(i) C(iI
A)1B. If there exists R such that
(2)
v = Cx
for all v = 0
The Popov criterion can then be stated as follows.
Popov and his colleagues made their problem statements in

terms of differential equations. The linear part then has the
form
(4)
(7)
is exponentially stable.
Note that the circle criterion is recovered with 0. Also the
Popov criterion can be illustrated graphically. Introduce the
Popov plot, where ImG(i) is plotted versus ReG(i). Then
(3)
1
J. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering. Copyright # 1999 John Wiley & Sons, Inc.
ABSOLUTE STABILITY
15
(, t)
10
v
5
Im
G(s)
1/
1/
G(i)
Figure 1. Absolute stability deals with linear time-invariant systems

in interconnection with nonlinear functions.
10
15
stability can be concluded from the Popov criterion if and only

if there exists a straight line separating the plot from the
point 1/. The slope of the line corresponds to the parameter . See the following example.
10
0
Re
10
(a)
Example. To apply the Popov criterion to the system
15
x1 = 5x1 4x2 + (x1 )
10
x2 = x1 2x2 + 21(x1 )
s + 82
G(s) = C(i A)1 B = 2
s + 7s + 6
Im
let
1/
1/
The plot in Fig. 4 then shows that the Popov criterion is satisfied for 1.
The theory of absolute stability had a strong development
in the 1960s, and various improvements to the circle and Popov criteria were generated, for example, by Yakubovich.
Many types of nonlinearities were considered and stronger
criteria were obtained in several special cases (24). Important aspects of the theory were summarized by Willems (5),
using the notions of dissipativity and storage function.
G(i)
10
15
10
0
Re
10
(b)
Figure 3. The circle criterion proves stability as long as the Nyquist
plot does not cross or circumscribe the circle corresponding to the
conic bounds on the nonlinearity. (a) 0 . (b) 0 .
GAIN AND PASSIVITY

In the results described so far, the results were stated in
terms of differential equations. A parallel theory was developed by Zames (6) avoiding the state-space structure and
studying stability purely in terms of inputoutput relations.
2
0
1/
w
w= (v)
Im G(i)
2
4
6
8
10
12
5
v
Figure 2. The nonlinearity is bounded by linear functions.
5
Re G(i)
10
15
Figure 4. The Popov criterion can be applied when there exists a

straight line separating the Popov plot from the point 1/.
ABSOLUTE STABILITY
For this purpose, a dynamical system is viewed as a map

F from the input u to the output Fu. The map F is said to be
bounded if there exists C 0 such that
T
0
The only property of saturation that would be exploited by

the Popov criterion is that 0 sat(v)/v 1 and, consequently,
w(t)[v(t) w(t)] dt
0
|u|2 dt
0
for all T 0. The gain of F is denoted F and defined as the

minimal such constant C. The map is said to be causal if two
inputs that are identical until time T will generate outputs
that are also identical until time T.
This makes it possible to state the following well-known
result:
for all w sat(v), T 0. However, the inequality will remain

valid even if some perturbation of amplitude smaller than one
is added to the factor w in the product w(v w). One way to
do this is to introduce a function h(t) with the property

h(t)dt 1 and replace the previous expression by (w h
w) (v w), where h w is a convolution. The integral inequality then becomes
F G < 1
(8)
then the feedback equations
(w + h w)(v w) dt
(9)
Using this inequality, the Popov criterion [Eq. (6)] can be replaced by the condition
Re[(1 + i + H(i))(G(i) + 1)]] > 0
v = Gw + f
wR
(10)
w = Fv + e
define a bounded causal map from the inputs (e, f) to the outputs (v, w).
It is worthwhile to make a comparison with the circle criterion. Consider the case when . Then the condition [Eq.
(3)] becomes
[0, ]
This is the same as Eq. (8), since is the gain of and

maxG(i) is the gain of the linear part [Eq. (2)].
Another important notion closely related to gain is passivity. The inputoutput map F is said to be passive if
Theorem 3 (Small gain theorem). Suppose that the input

output maps F and G are bounded and casual. If
|G(i)| < 1
|Fu|2 dt C2
where H(i) ei h(t)dt. The factor 1 i H(i) is

called multiplier.
The theory and applications of absolute stability have recently had a revival since new computer algorithms make it
possible to optimize multipliers numerically and to address
applications of much higher complexity than previously. The
inequality [Eq. (9)] is a special case of what is called an integral quadratic constraint, IQC. Such constraints have been
verified for a large number of different model components
such as relays, various forms of hysteresis, time delays, time
variations, and rate limiters. In principle, all such constraints
can be used computationally to improve the accuracy in stability and performance analysis. A unifying theory for this
purpose has been developed by Megretski and Rantzer (8),
while several other authors have contributed with new IQCs
that are ready to be included in a computer library for system analysis.
u(t)y(t) dt
for all T and all y = Fu
For example, if the input is a voltage and the output is a

current, then passivity property means that the system only
can consume electrical power, not produce it.
Stability criteria can also be stated in terms of passivity.
For example, the circle criterion can be interpreted this way,
if 0 and is large.
MULTIPLIERS AND INTEGRAL QUADRATIC CONSTRAINTS
Less conservative stability criteria can often be obtained by
exploiting more information about the nonlinearity. One way
to do this is to introduce so-called multipliers. Consider, for
example, a system with a saturation nonlinearity:
Ax + B
x = Ax Bsat(Cx) = Ax + BCx
Ax B
if Cx 1
if |Cx| < 1
if Cx 1
BIBLIOGRAPHY
1. V. M. Popov, Absolute stability of nonlinear systems of automatic
control, Autom. Remote Control, 22: 857875, 1962. (Original in
Russian, August, 1961.)
2. V. A. Yakubovich, Absolute stability of nonlinear control systems
in critical cases, parts 13, Avtomaika i Telemechanika, 24 (3):
293302; 24 (6): 717731, 1963; 25 (25): 601612, 1964. (English
translation in Autom. Remote Control.)
3. V. A. Yakubovich, On an abstract theory of absolute stability of
nonlinear systems, Vestnik Leningrad Univ. Math., 10: 341361,
1982. (Original in Russian, 1977.)
4. S. Lefschetz, Stability of Nonlinear Control Systems, New York:
Academic Press, 1963.
5. J. C. Willems, Dissipative dynamical systems, part 1, General theory; part 2, Linear systems with quadratic supply rates, Arch. Rational Mech. Anal., 45 (5): 321393, 1972.
6. G. Zames, On the inputoutput stability of nonlinear time-varying
feedback systemspart 1, Conditions derived using concepts of
loop gain; part 2, Conditions involving circles in the frequency
plane and sector nonlinearities, IEEE Trans. Autom. Control, 11:
228238, 1966.
ABSTRACT DATA TYPES
7. G. Zames and P. L. Falb, Stability conditions for systems with

monotone and slope-restricted nonlinearities, SIAM J. Control, 6
(1): 89108, 1968.
8. A. Megretski and A. Rantzer, System analysis via Integral Quadratic Constraints, IEEE Trans. Autom. Control, 47: 819830,
1997.
ANDERS RANTZER
Lund Institute of Technology
ABSORBER. See ELECTROMAGNETIC FERRITE TILE ABSORBER.

ABSORPTION MODULATION. See ELECTROABSORPTION.
J. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering

c 1999 John Wiley & Sons, Inc.
Copyright
ADAPTIVE CONTROL
According to Websters dictionary, to adapt means to change (oneself) so that ones behavior will conform to
new or changed circumstances. The words adaptive systems and adaptive control were used as early as
1950 (1). The design of autopilots for high-performance aircraft was one of the primary motivations for active
research on adaptive control in the early 1950s. Aircraft operate over a wide range of speeds and altitudes,
and their dynamics are nonlinear and conceptually time-varying. For a given operating point, specified by the
aircraft speed (Mach number) and altitude, the complex aircraft dynamics can be approximated by a linear
model of the form (2)
where Ai , Bi , Ci , and Di are functions of the operating point i. As the aircraft goes through different flight
conditions, the operating point changes, leading to different values for Ai , Bi , Ci , and Di . Because the output
response y(t) carries information about the state x as well as the parameters, one may argue that in principle,
a sophisticated feedback controller should be able to learn about parameter changes by processing y(t) and
use the appropriate gains to accommodate them. This argument led to a feedback control structure on which
adaptive control is based. The controller structure consists of a feedback loop and a controller with adjustable
gains as shown in Fig. 1. The way the controller parameters are adjusted on line leads to different adaptive
control schemes.
Gain Scheduling
The approach of gain scheduling is illustrated in Fig. 2. The gain scheduler consists of a lookup table
and the appropriate logic for detecting the operating point and choosing the corresponding value of the control
parameter vector . For example, let us consider the aircraft model in Eq. (1) where for each operating point i,
i = 1, 2, . . ., N, the parameters Ai , Bi , Ci , and Di are known. For each operating point i, a feedback controller
with constant gains, say i , is designed to meet the performance requirements for the corresponding linear
model. This leads to a controller, say C(), with a set of gains {1 , 2 , . . ., i , . . ., N } covering N operating points.
Once the operating point, say i, is detected, the controller gains can be changed to the appropriate value of i
obtained from the precomputed gain set. Transitions between different operating points that lead to significant
parameter changes may be handled by interpolation or by increasing the number of operating points. The two
elements that are essential in implementing this approach are a lookup table to store the values of i and the
plant auxiliary measurements that correlate well with changes in the operating points.
Direct and Indirect Adaptive Control
A wide class of adaptive controllers is formed by combining an on-line parameter estimator, which provides
estimates of unknown parameters at each time instant, with a control law that is motivated from the knownparameter case. The way the parameter estimator, also referred to as the adaptive law, is combined with the
control law gives rise to two different approaches. In the first approach, referred to as indirect adaptive control
1
ADAPTIVE CONTROL
Fig. 1. Controller structure with adjustable controller gains.
Fig. 2. Gain scheduling.
(shown in Fig. 3), the plant parameters are estimated online and used to calculate the controller parameters.
This approach has also been referred to as explicit adaptive control, because the design is based on an explicit
plant model.
In the second approach, referred to as direct adaptive control (shown in Fig. 4), the plant model is
parametrized in terms of the controller parameters, which are estimated directly without intermediate calculations involving plant parameter estimates. This approach has also been referred to as implicit adaptive
control, because the design is based on the estimation of an implicit plant model.
The principle behind the design of direct and indirect adaptive control shown in Figs. 3 and 4 is conceptually simple. The design of C(c ) treats the estimates c (t) (in the case of direct adaptive control) or the estimates
(t) (in the case of indirect adaptive control) as if they were the true parameters. This design approach is
called certainty equivalence and can be used to generate a wide class of adaptive control schemes by combining
different on-line parameter estimators with different control laws.
The idea behind the certainty equivalence approach is that as the parameter estimates c (t) and (t)
converge to the true ones c and , respectively, the performance of the adaptive controller C(c ) tends to that
achieved by C( c ) in the case of known parameters.
Model Reference Adaptive Control
Model reference adaptive control (MRAC) is derived from the model-following problem or model reference
control (MRC) problem. The structure of an MRC scheme for a LTI, single-input single-output (SISO) plant is
shown in Fig. 5. The transfer function W m (s) of the reference model is designed so that for a given reference
input signal r(t) the output ym (t) of the reference model represents the desired response the plant output y(t)
should follow. The feedback controller, denoted by C( c ), is designed so that all signals are bounded and the
closed-loop plant transfer function from r to y is equal to W m (s). This transfer function matching guarantees
that for any given reference input r(t), the tracking error e1 y ym , which represents the deviation of the
ADAPTIVE CONTROL
Fig. 3. Indirect adaptive control.
Fig. 4. Direct adaptive control.
plant output from the desired trajectory ym , converges to zero with time. The transfer function matching is
achieved by canceling the zeros of the plant transfer function G(s) and replacing them with those of W m (s)
through the use of the feedback controller C( c ). The cancellation of the plant zeros puts a restriction on the
plant to be minimum-phase, that is, have stable zeros. If any plant zero is unstable, its cancellation may easily
lead to unbounded signals.
The design of C( c ) requires the knowledge of the coefficients of the plant transfer G(s). When is
unknown we use the certainty equivalence approach to replace the unknown c in the control law with its
estimate c (t) obtained using the direct or the indirect approach. The resulting control schemes are known
as MRAC and can be classified as indirect MRAC of the structure shown in Fig. 3 and direct MRAC of the
structure shown in Fig. 4.
ADAPTIVE CONTROL
Fig. 5. Model reference control.
Adaptive Pole Placement Control

Adaptive pole placement control (APPC) is derived from the pole placement control (PPC) and regulation
problems used in the case of LTI plants with known parameters. In PPC, the performance requirements are
translated into desired locations of the poles of the closed-loop plant. A feedback control law is then developed
that places the poles of the closed-loop plant at the desired locations. The structure of the controller C( c ) and
the parameter vector c are chosen so that the poles of the closed-loop plant transfer function from r to y are
equal to the desired ones. The vector c is usually calculated using an algebraic equation of the form
where is a vector with the coefficients of the plant transfer function G(s).
As in the case of MRC, we can deal with the unknown-parameter case by using the certainty equivalence
approach to replace the unknown vector c with its estimate c (t). The resulting scheme is referred to as
adaptive pole placement control (APPC). If c (t) is updated directly using an on-line parameter estimator, the
scheme is referred to as direct APPC. If c (t) is calculated using the equation
where (t) is the estimate of generated by an on-line estimator, the scheme is referred to as indirect APPC.
The structure of direct and indirect APPC is the same as that shown in Figs. 3 and 4, respectively, for the
general case. The design of APPC schemes is very flexible with respect to the choice of the form of the controller
C(c ) and of the on-line parameter estimator.
Design of On-Line Parameter Estimators
As we mentioned in the previous sections, an adaptive controller may be considered as a combination of
an on-line parameter estimator with a control law that is derived from the known-parameter case. The way this
combination occurs and the type of estimator and control law used give rise to a wide class of different adaptive
controllers with different properties. In the literature of adaptive control the on-line parameter estimator has
often been referred to as the adaptive law, update law, or adjustment mechanism.
Some of the basic methods used to design adaptive laws are
(1) Sensitivity methods
(2) Positivity and Lyapunov design
(3) Gradient method and least-squares methods based on estimation error cost criteria
The sensitivity method is one of the oldest methods used in the design of adaptive laws and is briefly
explained below.
ADAPTIVE CONTROL
Sensitivity Method. This method became very popular in the 1960s (3), and it is still used in many
industrial applications for controlling plants with uncertainties. In adaptive control, the sensitivity method is
used to design the adaptive law so that the estimated parameters are adjusted in a direction that minimizes a
certain performance function. The adaptive law is driven by the partial derivative of the performance function
with respect to the estimated parameters multiplied by an error signal that characterizes the mismatch between
the actual and desired behavior. This derivative is called the sensitivity function, and if it can be generated
online, then the adaptive law is implementable. In most earlier formulations of adaptive control, the sensitivity
function cannot be generated online, and this constitutes one of the main drawbacks of the method. The use
of approximate sensitivity functions that are implementable leads to adaptive control schemes whose stability
properties are either weak or cannot be established.
Positivity and Lyapunov Design. This method of developing adaptive laws is based on the direct
method of Lyapunov and its relationship to positive real functions. In this approach, the problem of designing
an adaptive law is formulated as a stability problem where the differential equation of the adaptive law is
chosen so that certain stability conditions based on Lyapunov theory are satisfied. The adaptive law developed
is very similar to that based on the sensitivity method. The only difference is that the sensitivity functions in
the approach are replaced with ones that can be generated online. In addition, the Lyapunov-based adaptive
control schemes have none of the drawbacks of the MIT rule-based schemes.
The design of adaptive laws using Lyapunovs direct method was suggested by Grayson (4) and Parks
(5) in the early 1960s. The method was subsequently advanced and generalized to a wider class of plants by
Phillipson (6), Monopoli (7), and others (8,9,10,11).
Gradient and Least-Squares Methods Based on Estimation Error Cost Criteria. The main drawback of the sensitivity methods used in the 1960s is that the minimization of the performance cost function led
to sensitivity functions that are not implementable. One way to avoid this drawback is to choose a cost function
criterion that leads to sensitivity functions that are available for measurement. A class of such cost criteria is
based on an error, referred to as the estimation error (12), that provides a measure of the discrepancy between
the estimated and actual parameters. The relationship of the estimation error with the estimated parameters
is chosen so that the cost function is convex, and its gradient with respect to the estimated parameters is implementable. Several different cost criteria may be used, and methods such as the gradient and least-squares
may be adopted to generate the appropriate sensitivity functions.
On-Line Parameter Estimation

The purpose of this section is to present the design and analysis of a wide class of schemes that can be used for
on-line parameter estimation. The essential idea behind on-line estimation is the comparison of the observed
system response y(t) with the output of a parametrized model y (,t) whose structure is the same as that of the
plant model. The parameter vector (t) is adjusted continuously so that y (,t) approaches y(t) as t increases. The
on-line estimation procedure involves three steps: In the first step, an appropriate parametrization of the plant
model is selected. The second step involves the selection of the adjustment law, referred to as the adaptive law,
for generating or updating (t). The third and final step is the design of the plant input so that the properties
of the adaptive law imply that (t) approaches the unknown plant parameter vector as t .
We start by considering the SISO plant
ADAPTIVE CONTROL
where x Rn and only y, u are available for measurement. Note that the plant equation can also be written as
an nth-order differential equation
The constants ai , bi are the plant parameters. We can express yn as
where
We can avoid the use of differentiators by filtering with our nth-order stable filter 1/(s) to obtain
where
In a similar way we can express the plant dynamics as follows:
where W(s) is an appropriate proper stable transfer function and , are defined similarly to , .
In Eqs. (5), (6) the unknown vectors , appear linearly in equations where all the other terms are
known a priori or can be measured online. We use these parametric models to estimate the unknown vectors
or by using the following approaches.
SPRLyapunov Design Approach. We start by rewriting Eq. (6) as follows (for simplicity we drop
the subscript ):
where = L 1 (s)and L(s) is chosen so that L 1 (s) is a proper transfer function and W(s)L(s) is a proper strictly
positive real (SPR) transfer function. Let (t) denote the estimate of at time t. Then the estimated value of z
ADAPTIVE CONTROL
based on (t) is given by
and the estimation error is given by
Let
denote the normalized estimation error, where n2 s is the normalizing signal, which we design to satisfy
Typical choices for ns that satisfy this condition are n2 s = T P for any P = PT > 0, and the like. When L ,
the condition is satisfied with m = 1, that is, ns = 0, in which case = 1 . We express in terms of the parameter
error = :
For simplicity, let us assume that L(s) is chosen so that WL is strictly proper and consider the following state
space representation of Eq. (8):
where Ac , Bc , and Cc are the matrices associated with a state-space representation that has a transfer function
W(s)L(s) = CT c (sI Ac ) 1 Bc .
Let us now consider the following Lyapunov-like function for the differential equation (9):
where = T > 0 is a constant matrix and Pc = PT c > 0 satisfies the algebraic equations
for some vector q, matrix Lc = LT c > 0, and small constant > 0. The existence of Pc = PT c > 0 satisfying the
above equations is guaranteed by the SPR property (12) of W(s)L(s) = CT c (sI Ac ) 1 Bc . The time derivative of
ADAPTIVE CONTROL
V is given by
are
We now need to choose = as a function of signals that can be measured so that the indefinite terms in V
canceled out. Because e is not available for measurement, cannot depend on e explicitly.
We know that Pc Bc = Cc , which implies that eT Pc Bc = eT Cc = . Therefore
0 is now obvious, namely, for

The choice for = to make V
we have
Using the above inequality, we can prove the following theorem.

Theorem 1. The adaptive law in Eq. (4) guarantees that:
(i) , L .
(ii) , ns , L 2 , independent of the boundedness of .
(iii) If ns , , L , and is perstistently exciting (PE)that is, there exist positive constant 1 , 0 , T 0 such
that
then (t) exponentially fast.

The proof of the theorem can be found in Ref. 12.
Gradient Method. In this method, we consider the parametric model in Eq. (5). Similar to the previous
subsection, we define (t) to be the on line estimate of and the normalized estimation error as
where z = T (t) and m2 = 1 + n2 s and n2 s is chosen so that /m L . The adaptive law is designed to minimize
the performance index J(), i.e.,
ADAPTIVE CONTROL
which gives
Different choices for the performance index lead to different adaptive laws.
Let us consider the simple quadratic cost function (instantaneous cost function)
Applying the gradient method, the minimizing trajectory (t) is generated by the differential equation
where = T > 0 is a scaling matrix that we refer to as the adaptive gain. We have
The following theorem holds:

(i) , ns , L .
(ii) , ns , L 2 , independent of the boundedness of .
(iii) If ns , L and is PE, then (t) exponentially fast.
Least Squares. Let (t), , z be defined as above, and let m2 = 1 + n2 s , (t) be the estimate of at time
t, and m satisfy /m L . We consider the following cost function:
where Q0 = QT 0 > 0, 0, 0 = (0), which includes discounting of past data and a penalty on the initial estimate
0 of . Because z/m, /m L , we have that J() is a convex function of over Rn at each time t. Hence, any
local minimum is also global and satisfies
which yields the so-called nonrecursive least-squares algorithm
10
ADAPTIVE CONTROL
where
We can show that P, satisfy the differential equations
We refer to Eqs. (22) and (23) as the continuous-time recursive least-squares algorithm with forgetting factor.
The stability properties of the least-squares algorithm depend on the value of the forgetting factor .
In the identification literature, Eqs. (22) and (23) with = 0 are referred to as the pure least-squares
algorithm and have a very similar form to the Kalman filter. For this reason, the matrix P is usually called
the covariance matrix. The pure least-squares algorithm has the unique property of guaranteeing parameter
convergence to constant values as described by the following theorem:
Theorem 3. The pure least-squares algorithm guarantees that:
(i)
(ii)
(iii)
(iv)
P L .
, ns , , ,
, ns , L 2 .
where is a constant vector.
limt (t) = ,
If ns , L and is PE, then (t) converges to as t .
The proof of the theorem can be found in 12.
Bilinear Parametric Model. As will be shown in the next sections, a certain class of plants can be
parametrized in terms of their desired controller parameters, which are related to the plant parameters via
a Diophantine equation. Such parametrizations and their related estimation problem arise in direct adaptive
control and, in particular, direct MRAC, which is discussed in the next section.
In these cases appears in the form
where is an unknown constant, z, , z0 are signals that can be measured, and W(s) is a known proper transfer
function with stable poles. Because the unknown parameters , appear in a special bilinear form, we refer
to Eq. (24) as the bilinear parametric model.
Known Sign of . The SPRLyapunov design approach and the gradient method with an instantaneous
cost function discussed in the linear parametric case extend to the bilinear one in a rather straightforward
manner.
Let us start with the SPRLyapunov design approach. We rewrite Eq. (24) in the form
ADAPTIVE CONTROL
11
where z1 = L 1 (s)z0 , = L 1 (s), and L(s) is chosen so that L 1 (s) is proper and stable and WL is proper and
SPR. The estimate z of z and the normalized estimation error are generated as
where ns is designed to satisfy
and (t), (t) are the estimates of , at time t, respectively. Letting , , it follows from
Eqs. (25) to (27) that
Now T T = T T + T T = T
T , and therefore,
By choosing
we can see that the following theorem holds.

(i)
(ii)
(iii)
(iv)
, , L .
L 2 .
, ns , ,
If , L , is PE, and L 2 , then (t) converges to as t .
If L 2 , the estimate converges to a constant independent of the properties of .
The proof of the theorem can be found in Ref. 12. The case where the sign of is unknown is also given
in Ref. 12.
Model Reference Adaptive Control

Problem Statement. Consider the SISO LTI plant described by
12
ADAPTIVE CONTROL
where Gp (s) is expressed in the form
where Zp , Rp are monic polynomials and kp is a constant referred to as the high-frequency gain.
The reference model, selected by the designer to describe the desired characteristics of the plant, is
described by
which is expressed in the same form as Eq. (32), that is,
where Zm (s), Rm (s) are monic polynomials and km is a constant.

The MRC objective is to determine the plant input up so that all signals are bounded and the plant output
yp tracks the reference model output ym as close as possible for any given reference input r(t) of the class defined
above. We refer to the problem of finding the desired up to meet the control objective as the MRC problem. In
order to meet the MRC objective with a control law that is implementable (i.e., a control law that is free of
differentiators and uses only measurable signals), we assume that the plant and reference model satisfy the
following assumptions:
Plant Assumptions.
P1. Zp (s) is a monic Hurwitz polynomial of degree mp .
P2. An upper bound n on the degree np of Rp (s) is known.
P3. The relative degree n = np mp of Gp (s) is known.
P4. The sign of the high-frequency gain kp is known.
Reference Model Assumptions.

M1. Zm (s), Rm (s) are monic Hurwitz polynomials of degree qm , pm , respectively, where pm n.
M2. The relative degree n m = pm qm of W m (s) is the same as that of Gp (s), that is, n m = n.
MRC Schemes: Known Plant Parameters. In addition to assumptions P1 to P4 and M1, M2, let us
also assume that the plant parameters, that is, the coefficients of Gp (s), are known exactly. Because the plant
is LTI and known, the design of the MRC scheme is achieved using linear system theory.
We consider the feedback control law
ADAPTIVE CONTROL
13
where
c 0 , 3 R1 , 1 , 2 Rn 1 are constant parameters to be designed, and (s) is an arbitrary monic Hurwitz

polynomial of degree n 1 that contains Zm (s) as a factor, i.e., (s) = 0 (s)Zm (s), which implies that 0 (s) is
monic, Hurwitz, and of degree n0 = n 1 qm . The controller parameter vector
is to be chosen so that the transfer function from r to yp is equal to W m (s).

The inputoutput properties of the closed-loop plant are described by the transfer function equation
where
We can now meet the control objective if we select the controller parameters 1 , 2 , 3 , c 0 so that the
closed-loop poles are stable and the closed-loop transfer function Gc (s) = W m (s) is satisfied for all s C. Choosing
and using (s) = 0 (s)Zm (s), the matching equation Gc (s) = W m (s) becomes
Equating the coefficients of the powers of s on both sides of Eq. (38), we can express it in terms of the algebraic
equation
= [T 1 , T 2 , 3 ]T ; S is an (n + np 1) (2n 1) matrix that depends on the coefficients of Rp , kp Zp ,

where
to satisfy Eq. (39)
and and p is an n + np 1 vector with the coefficients of Rp Zp 0 Rm . The existence of
and, therefore, Eq. (38) will very much depend on the properties of the matrix S. For example, if n > np , more
will satisfy Eq. (39), whereas if n = np and S is nonsingular, Eq. (39) will have only one solution.
than one
Lemma 1. Let the degrees of Rp , Zp , , 0 and Rm be as specified in Eq. (34). Then
of Eq. (38) or (39) always exists.
(i) The solution
is unique.
(ii) In addition, if Rp , Zp are coprime and n = np , then the solution
14
ADAPTIVE CONTROL
The proof of the lemma can be found in Ref. 12.
MRAC for SISO Plants. The design of MRAC schemes for the plant in Eq. (31) with unknown parameters is based on the certainty equivalence approach and is conceptually simple. With this approach, we develop
a wide class of MRAC schemes by combining the MRC law, where is replaced by its estimate (t), with
different adaptive laws for generating (t) online. We design the adaptive laws by first developing appropriate
parametric models for , which we then use to pick up the adaptive law of our choice from the preceding
section.
Let us start with the control law
whose state-space realization is given by
where = [T 1 , T 2 , 3 , c0 ]T and = [T 1 , T 2 , yp , r]T , and search for an adaptive law to generate (t), the estimate
of the desired parameter vector .
It can be seen that under the above control law, the tracking error satisfies
where = 1/c 0 , = [T 1 , T 2 , 3 , c 0 ]T . The above parametric model may be developed by using the matching
Eq. (38) to substitute for the unknown plant polynomial Rp (s) in the plant equation and by canceling the Hurwitz
polynomial Zp (s). The parametric model in Eq. (42) holds for any relative degree of the plant transfer function.
A linear parametric model for may also be developed from Eq. (42). Such a model takes the form
where
The main equations of several MRAC schemes formed by combining Eq. (41) with an adaptive law based
on Eq. (42) or (43). The following theorem gives the stability properties of the MRAC scheme.
Theorem 5. The closed-loop MRAC scheme in Eq. (41), with (t) adjusted with any adaptive law with normalization based on the model in Eq. (42) or (43) as described in the preceding section, has the following
properties:
(i) All signals are uniformly bounded.
(ii) The tracking error e1 = yp ym converges to zero as t .
ADAPTIVE CONTROL
15
(iii) If the reference input signal r is sufficiently rich of order 2n, r L , and Rp , Zp are coprime, then the
tracking error e1 and parameter error = converge to zero for the adaptive law with known sgn(kp ).
Adaptive Pole Placement Control

In the preceding section we considered the design of a wide class of MRAC schemes for LTI plants with stable
zeros. The assumption that the plant is minimum-phase, that is, it has stable zeros, is rather restrictive in
many applications. A class of control schemes that is popular in the known-parameter case are those that
change the poles of the plant and do not involve plant zeropole cancellations. These schemes are referred
to as pole placement schemes and are applicable to both minimum- and nonminimum-phase LTI plants. The
combination of a pole placement control law with a parameter estimator or an adaptive law leads to an adaptive
pole placement control (APPC) scheme that can be used to control a wide class of LTI plants with unknown
parameters.
Problem Statement. Consider the SISO LTI plant
where Gp (s) is proper and Rp (s) is a monic polynomial. The control objective is to choose the plant input up so
that the closed-loop poles are assigned to those of a given monic Hurwitz polynomial A(s). The polynomial
A(s), referred to as the desired closed-loop characteristic polynomial, is chosen according to the closed-loop
performance requirements. To meet the control objective, we make the following assumptions about the plant:
P1. Rp (s) is a monic polynomial whose degreen n is known.
P2. Zp (s), Rp (s) are coprime, and degree(Zp ) < n.
Assumptions P1 and P2 allow Zp , Rp to be non-Hurwitz, in contrast to the MRC case, where Zp is required
to be Hurwitz. If, however, Zp is Hurwitz, the MRC problem is a special case of the general pole placement
problem defined above with A(s) restricted to have Zp as a factor.
In general, by assigning the closed-loop poles to those of A(s), we can guarantee closed-loop stability and
convergence of the plant output yp to zero provided there is no external input. We can also extend the PPC
objective to include tracking, where yp is required to follow a certain class of reference signals ym , by using the
internal model principle as fol8lows: The reference signal ym L is assumed to satisfy
where Qm (s), the internal model of ym , is a known monic polynomial of degree q with nonrepeated roots on the
j axis and satisfies
P3. Qm (s), Zp (s) are coprime.
For example, if yp is required to track the reference signal ym = 2 + sin 2t, then Qm (s) = s(s2 + 4) and
therefore, according to assumption P3, Zp (s) should not have s or s2 + 4 as a factor.
16
ADAPTIVE CONTROL
In addition to assumptions P1 to P3, let us also assume that the coefficients of Zp (s), Rp (s), i.e., the plant
parameters are known exactly.
We consider the control law
where P(s), L(s), M(s) are polynomials [with L(s) monic] of degree q + n 1, n 1, q + n 1, respectively, to
be found, and Qm (s) satisfies Eq. (45) and assumption P3.
Applying Eq. (46) to the plant in Eq. (44), we obtain the closed-loop plant equation
whose characteristic equation
has order 2n + q 1. The objective now is to choose P, L such that
is satisfied for a given monic Hurwitz polynomial A(s) of degree 2n + q 1. It can be seen that assumptions
P2 and P3 guarantee that L, P satisfying Eq. (49) exist and are unique. The solution for the coefficients of L(s),
P(s) of Eq. (49) may be obtained by solving the algebraic equation
where Sl is the Sylvester matrix of Qm Rp , Zp of dimension 2(n + q) 2(n + q),
and l, p, a are the vectors whose entries are the coefficients of L(s), P(s) and A(s), respectively. The coprimeness
of Qm Rp , Zp guarantees that Sl is nonsingular; therefore, the coefficients of L(s), P(s) may be computed from
the equation
The tracking error e1 = yp ym is given by
ADAPTIVE CONTROL
17
For zero tracking error, Eq. (51) suggests the choice of M(s) = P(s) to null the first term. The second term in Eq.
(51) is nulled by using Qm ym = 0. Therefore, the pole placement and tracking objective are achieved by using
the control law
which is implemented using n + q 1 integrators to realize C(s) = P(s)/Qm (s)L(s). Because L(s) is not necessarily
Hurwitz, the realization of Eq. (52) with n + q 1 integrators may have a transfer function, namely C(s), with
poles outside C . An alternative realization of Eq. (52) is obtained by rewriting it as
where is any monic Hurwitz polynomial of degree n + q 1. The control law (53) is implemented using 2(n
+ q 1) integrators to realize the proper stable transfer functions ( LQm )/, P/.
APPC. The APPC scheme that meets the control objective for the unknown plant is formed by combining the control law in Eq. (53) with an adaptive law based on the parametric model in Eq. (5). The adaptive law generates on-line estimates a , b of the coefficient vectors a of Rp (s) = sn + T a n 1 (s) and b
p (s, t) = sn + T a n 1 (s), Z p (s,
of Zp (s) = T b n 1 (s), respectively, to form the estimated plant polynomials R
T
t) = b n 1 (s). The estimated plant polynomials are used to compute the estimated controller polynomials L(s,
t), P(s,
t) by solving the Diophantine equation
P pointwise in time, or the algebraic equation

for L,
l is the Sylvester matrix of R

p Qm , Z p ; l contains the coefficients of L,
P;
and l contains the
for l , where S
coefficients of A(s). The control law in the unknown-parameter case is then formed as
Because different adaptive laws may be picked up from the section On-Line Parameter Estimation above, a
wide class of APPC schemes may be developed.
P or
The implementation of the APPC scheme requires that the solution of the polynomial Eq. (54) for L,
of the algebraic Eq. (55) for l exists at each time. The existence of this solution is guaranteed provided that
l (t) is nonsingular at each time t.
p (s, t)Qm (s), Z p (s, t) are coprime at each time t, that is, the Sylvester matrix S
R
p Qm , Z p are strongly coprime at each time t. Then
Theorem 6. Assume that the estimated plant polynomials R
all the signals in the closed-loop APPC scheme are u.b., and the tracking error converges to zero asymptotically
with time.
The proof of the theorem can be found in Ref. 12. The assumption that the estimated polynomials are
strongly coprime at each time t is restrictive and cannot be guaranteed by the adaptive law. Methods that relax
this assumption are given in Ref. 12.
18
ADAPTIVE CONTROL
Adaptive Control of Nonlinear Systems

In the previous sections we dealt with the problem of designing controllers for linear time-invariant systems.
In this section, we show how the techniques of adaptive control of linear systems can be extended or modified
for nonlinear systems. Although the techniques presented can be applied to a variety of nonlinear systems, we
will concentrate our attention on adaptive state feedback control of SISO feedback-linearizable systems.
Feedback-Linearizable Systems in Canonical Form. We start with an nth-order SISO feedbacklinearizable system in canonical form, whose dynamics are as follows:
where y, u R are the scalar system output and input, respectively, f , g are smooth vector fields, and x
[x1 , x2 , . . ., xn ]T is the state vector of the system. In order for the system in Eq. (57) to be controllable and
feedback-linearizable we assume that
A1. A lower bound for g(x) [i.e., |g(x)| > > 0 x Rn ] and the sign of g(x) are known.
The control objective is to find the control input u that guarantees signal boundedness and forces y to
follow the output ym of the reference model
where A is a Hurwitz n n matrix, r L , and therefore xm L . In order to have a well-posed problem, it is

assumed that the relative degree of the reference model is equal to n. If e ym y is the tracking error, then
its nth time derivative satisfies
= sn + k1 sn 1 + + kn be a Hurwitz polynomial (here s denotes the d/dt operator). Also let [e,
Let h(s)
e , . . ., e(n 1) ]T . Under assumption A1, the system Eq. (57) is a feedback-linearizable system. Therefore, if we
know the vector fields f and g, we can apply the static feedback
where k [kn , kn 1 , . . ., k1 ]T . Then the error system in Eq. (59) becomes
which implies that e, L and therefore all closed-loop signals are bounded, and limt e(t) = 0.
ADAPTIVE CONTROL
19
In many cases, the vector fields f and g are not completely known and thus adaptive versions of the
feedback law (60) have to be applied. For instance, using the usual assumption of linear parametrization, if the
vector fields f and g are of the form
where i , i = 1, 2, are vectors with unknown constant parameters, one may replace the feedback law in Eq. (60)
with the certainty-equivalent one [the certainty-equivalent feedback-linearizing (CEFL) controller]
where i , i = 1, 2, are the estimates of the unknown parameter vectors i , i = 1, 2. These estimates are generated
by an on-line adaptive law. We next propose the following adaptive laws for updating i :
where i , i = 1, 2, are symmetric positive definite matrices and 1 = bc f , 2 = ubc g , bc = [0, . . ., 0, 1] . The
next theorem summarizes the properties of the proposed control law.
Theorem 7. Consider the system in Eq. (57) and the feedback control law in Eqs. (62) and (63). Let assumption
A1 hold. Then, if T 2 (t)g (x(t)) = 0 for all t, all the closed-loop signals are bounded and the tracking error
converges to asymptotically to zero.
Parametric-Pure-Feedback Systems. Let us now try to extend the results of the previous section to
nonlinear systems that take the form
where u, zi R, f ij , gnj are smooth functions, and Rp is the vector of constant but unknown system parameters.
Let us rewrite Eq. (64) as
where f i1 () = f i0 () zi+1 . Systems of the form in Eq. (65) are called parametric-pure-feedback (PPF) systems
(13,14). Note that the above class of systems includes as a special case the system in Eq. (57) of the previous
section.
20
ADAPTIVE CONTROL
The control objective is to force the system output y to asymptotically track a reference signal ym . We
assume that the first n 1 time derivatives of ym are known. Also it will be assumed that ym and its first n
1 time derivatives are bounded and smooth signals.
Let us now assume that the parameter vector is known and construct a control law that meets the
control objectives. Before we design the feedback law, we will transform the system in Eq. (64) into a suitable
form. The procedure we will follow is based on the backstepping integrator principle (13).
Step 0. Let 1 z1 ym . Let also c1 , . . ., cn be positive constants to be chosen later.
Step 1. Using the chain-of-integrators method, we see that, if z2 were the control input in the z1 part of Eq.
(65) and were known, then the control law
would result in a globally asymptotically stable tracking, since such a control law would transform the z1
part of Eq. (65) as follows:
However, the state z2 is not the control. We therefore define 2 to be the difference between the actual z2
and its desired expression in Eq. (66):
Step 2. Using the above definition of 2 , the definition of 1 , and the z1 part of Eq. (65), we find that
Step 2. Using the above definitions of 1 , 2 , we have that
where (2) is a (p + p2 )-dimensional vector that consists of all elements that are either of the form 2,i or
of the form 2,i j , where by 2,i we denote the ith entry of the vector . In the system (69) we will think of
z3 as our control input. Therefore, as in step 1, we define the new state 3 as
ADAPTIVE CONTROL
21
Substituting Eq. (70) into Eq. (69) yields
Step i (2 < i n 1). Using the definitions of 1 , . . ., i and working as in the previous steps, we may express
the derivative of i as
where the vector (i) contains all the terms of the form i1 i2 i with 1 j i. Defining now i+1 as
we obtain that
Step n.Using the definitions of 1 , . . ., n 1 and working as in the previous steps, we may express the derivative
of n as follows:
where the vector contains all the terms of the form i1 i2 ij with 1 j n, Y m [ym , y m , y m , . . .,
y(n 1) m ]T , and [ 0 (z1 , . . ., zn ) + T 1 (z1 , . . ., zn )] is given by
Using the definitions of 1 , . . ., n , and rearranging terms, we may rewrite Eq. (75) as follows:
22
ADAPTIVE CONTROL
Using the above methodology, we have therefore transformed the system in Eq. (65) into the following
one:
The above system is feedback-linearizable if the following assumption holds.

A1. 0 (z) + T 1 (z) = 0 for all z.
Note now that in the case where (and thus ) is known, a controller that meets the control objective is
the controller of the form
Under the above control law, the closed-loop dynamics become
It can be shown that the matrix A0 is a stability matrix, provided that ci > 2.
Theorem 8. The control law in Eq. (79) guarantees that all the closed-loop signals are bounded and that the
tracking error converges to zero exponentially fast, provided that the design constants ci satisfy ci > 2.
In the case where the vector is not known, the certainty-equivalence principle can be employed in order
to design an adaptive controller for the system. However, the problem of designing parameter estimators for
the unknown parameters is not as easy as it was in the linear case. This can be seen from the fact that
the states i , i > 1, are not available for measurement, since they depend on the unknown parameters. To
overcome this problem a recursive design approach similar to the approach above can be constructed. The
difference between this approach [called adaptive backstepping (13)] and the approach presented above is the
following: in the approach presented above the states i , i > 1, depend on the unknown vector of parameters
; in the new approach they are appropriately redefined so they depend on the parameter estimate vector .
ADAPTIVE CONTROL
23
In order to overcome
Then the derivatives of i , i > 1, depend on the derivatives of the parameter estimates .
this problem, the adaptive backstepping approach makes use of the so-called tuning functions (13).
Next we present the adaptive controller that results from applying the adaptive backstepping procedure
to the system in Eq. (65) for the case where
A2. T f i2 () are independent of zi+1 and T gn2 = 0.
Also for simplicity, and without loss of generality, we will assume that f i1 () = 0. The case where assumption
A2 is not valid will be treated in the next subsection. The adaptive controller that results from applying the
adaptive backstepping procedure is recursively defined as follows:
Control law:
Parameter update law:
Tuning functions:
Regressor vectors:
Here ci > 2, i are positive design constants, and = T > 0 is a positive definite design matrix. The next
theorem summarizes the properties of the above control law.
Theorem 9. Suppose that assumptions A1 , A2 hold. Then the above adaptive control law guarantees that all
the closed-loop signals are bounded and that the tracking error converges asymptotically to zero.
24
ADAPTIVE CONTROL
A Control Law That Overcomes the Loss-Of-Controllability Problem. A significant problem that
arises in adaptive control of linear-in-the-parameters feedback-linearizable systems is the computation of
the feedback control law when the identification model becomes uncontrollable although the actual system
is controllable; so far, there is no known solution to this problem. For instance, for the case of the system
in Eq. (57) the parameter estimation techniques used in adaptive control cannot guarantee, in general, that
| 2 (t)T g (x(t))| > 0 for each time t, that is, they cannot guarantee that the identification model is controllable.
Also, for the case of PPF systems presented in the previous subsection, the adaptive backstepping techniques
guarantee global stability only in the case where assumption A2 is valid. Such restrictions are made because
the computation of the adaptive control law depends on the existence of the inverse of the matrix that consists
of the estimated input vector fields (or the Lie derivatives of the output functions along those vector fields).
Even in the case of known parameters where the inverse of the corresponding matrix exists (this is trivially
satisfied for feedback-linearizable systems), the inverse of the estimate of this matrix might not exist at each
time due to insufficiently rich regressor signals, large initial parameter estimation errors, and so on.
We next show how one can overcome the problem where the estimation model becomes uncontrollable, by
appropriately using switching adaptive control. We will apply the switching adaptive control methodology to
the PPF system of the previous subsection, by removing assumption A2.
Consider the Lyapunov function for the PPF system of the previous subsection.
By differentiating V with respect to time, we obtain that
Let us define
Note now that, using the definition of i , we can rewrite the i s as follows:
where i and w
i are appropriately defined known functions. Therefore, we have that
where is defined to be the vector whose entries are the elements i j , and 0 , 1 areappropriately defined
known functions.
We are now ready to present the proposed controller. The control input is chosen as follows:
ADAPTIVE CONTROL
where:
One has
where denotes the estimate of .

k() is a positive design function satisfying k(1 , ) = 0 iff 1 = 0 and
is a continuous-switching signal that is used to switch from control u1 to control u2 and vice versa:
s is hysteresis-switching variable defined as follows:
where s (t) lim t s(), where t.

The parameter estimates are updated using the following smooth projection update law (15)
where is a symmetric positive definite design matrix and PC is defined as follows (15):
25
26
ADAPTIVE CONTROL
where
where 0 < < 1, q 2, and j are positive design constants.

The following theorem summarizes the properties of the control law in Eq. (83, 84, 85, 86, 87, 88, 89).
Theorem 10. Consider the system in Eq. (65) and the control law in Eqs. (83, 84, 85, 86, 87, 88, 89). Let
assumption A1 hold. Moreover assume that the following hold:
C1. K > 1; k() satisfies Eq. (85).
C, where
C2. j are sufficiently small. Moreover, (0)
Then for any compact X 0 Rn and for any positive constant c the following holds: there exist a positive
constant K such that, for any initial state x0 X 0 , the control law in Eqs. (83, 84, 85, 86, 87, 88, 89) with K > K
guarantees that all the closed-loop signals are bounded and, moreover, that the tracking error 1 converges in
finite time to the residual set
The idea of using switching adaptive controllers of the form presented above was first introduced in Ref.
16, where the proposed methodology was applied to systems of the form in Eq. (57). The controller of Ref. 16
was extended in Ref. 17 for PPF systems of the form in Eq. (65).
Acknowledgment
This article was supported by NASA grant NAGW-4103.
BIBLIOGRAPHY
1. J. A. Aseltine, A. R. Manchini, and C. W. Sartune, A survey of adaptive control systems, IRE Trans. Automat. Control,
6 (3): 1958.
2. D. McRuer, I. Ashkenas, D. Graham, Aircraft Dynamics and Automatic Control, Princeton, NJ: Princeton Univ. Press,
1973.
3. J. B. Cruz, Jr., System Sensitivity Analysis, Stroudsburg, PA: Dowden, Hutchinson & Ross, 1973.
4. L. P. Grayson, Design via Lyapunov second method, Proc. 4th JACC, 1963.
5. P. C. Parks, Lyapunov redesign of model reference control systems, IEEE Trans. Autom. Control, 11: 1966.
6. P. H. Phillipson, Design methods for model reference adaptive systems, Proc. Inst. Mech. Eng., 183 (35):, 695706, 1968.
7. R. V. Monopoli, Lyapunovs method for adaptive control design, IEEE Trans. Autom. Control, 3: 1967.
8. I. D. Landau, Adaptive Control: The Model Reference Approach, New York: Marcel Dekker, 1979.
ADAPTIVE CONTROL
27
9. K. S. Narendra, A. M. Annaswamy, Stable Adaptive Systems, Englewood Cliffs, NJ: Prentice-Hall, 1989.
10. S. Sastry, M. Bodson, Adaptive Control: Stability, Convergence and Robustness, Englewood Cliffs, NJ: Prentice-Hall,
1989.
11. K. J. Astrom, B. Wittenmark, Adaptive Control, Reading, MA: Addison-Wesley, 1989.
12. P. A. Ioannou, J. Sun, Robust Adaptive Control, Upper Saddle River, NJ: Prentice-Hall, 1996.
13. M. Krstic, M. I. Kanellakopoulos, and R. Kokotovic, Nonlinear and Adaptive Control Design, New York: Wiley, 1995.
14. D. Seto, A. M. Annaswamy, J. Baillieul, Adaptive control of nonlinear systems with a triangular structure, IEEE Trans.
Autom. Control, 7: 1994.
15. J.-B. Pomet, L. Praly, Adaptive nonlinear stabilization: Estimation from the Lyapunov equation, IEEE Trans. Autom.
Control, 37: 729740, 1992.
16. E. B. Kosmatopoulos and P. A. Ioannou, A switching adaptive controller for feedback linearizable systems, IEEE Trans.
Autom. Control, to be published.
17. E. B. Kosmatopoulos and P. A. Ioannou, Robust switching adaptive control using control Lyapunov functions, Proc.
IEEE Conf. Decision and Control, 1997.
PETROS A. IOANNOU
ELIAS B. KOSMATOPOULOS
University of Southern California
BILINEAR SYSTEMS
An electrical circuit or other engineering system often communicates with its external environment by input signals
that control its behavior and output signals; it is then called
a control system. If the components of a control system all
obey Ohms Law or one of its analogs, such as Hookes Law,
the system is called linear. In linear control systems, the
effect of the controls is additive and the output measurement is linear. They are discussed in the article in this
encyclopedia on MULTIVARIABLE SYSTEMS.
What distinguishes a bilinear system (BLS) is that although it is linear in its state variables, some control signal
u(t) exerts its effect multiplicatively. BLS may be given as
mathematical models of circuits or plants or may be chosen by a designer to obtain better system response than is
possible with a linear system. Their study is a rst step toward nonlinear control theory. Industrial process control,
economics, and biology provide examples of BLS with multiplicative controls such as valve settings, interest rates,
and neural signals, respectively. This topic of research began in the early 1960s with independent work in the USSR
and in the USA; see the surveys of Bruni et al. (1) and
Mohler (2, 3) for historical development and reviews of the
early literature.
with interaction matrix F and a linear output function.

Here is a single-input single-output example, in which the
coefcient vector g describes the control transducer, the
output transducer is described by the row vector h , and
v = v(t) is a control signal:
x = F x + vg, y = h x
or written out in full,

dxi
(Fi j x j ) + vgi , i = 1, . . . , n; y =
h i xi
=
dt
n
j=1
i=1
Control Systems: Facts and Terminology

The following discussion is a brief reminder of state-space
methods; see Sontag (4). The state variables in an electrical
circuit are currents through inductors and voltages across
capacitors; in mechanics, they are generalized positions
and momenta; and in chemistry, they are concentrations
of molecules. The state variables for a given plant constitute a vector function depending on time x(t). Knowledge of
an initial state x(0), of future external inputs, and the rstorder vector differential equation that describes the plant
determine the trajectory {x(t), t 0}. For the moment, we
will suppose that the plant elements, such as capacitors,
inductors, and resistors, are linear (Ohms Law) and constant in value. The circuit equations can usually be combined into a single rst-order vector differential equation,
(1)
(It is customary to suppress in the notation the timedependence of x, y and often the control v.)
As written, equation 1 has constant coefcients, and
such a control system is called time-invariant, which
means that its behavior does not depend on where we
choose the origin of the independent variable t; the system
can be initialized and control v exerted at any time. When
the coefcients are not constant, that is, made explicit in
the notation, e.g.,
x = F (t)x + vg(t), y = h (t)x
which is called a time-variant linear system.
For both linear and bilinear systems, we will need the
solution of x = Fx, which for a given initial condition x(0)
is x(t) = e Ft x(0). The matrix exponential function is dened
by
eFt = I + Ft +
Notation in This Article. The symbol R means the real

numbers and Rn the n-dimensional real linear space; C
means the complex plane, with R(s) the real part of s C.
The bold symbols ah, x, z will represent elements (column vectors) of Rn ; x (x transposed) is a row vector; x .
is the complex conjugate transpose of x. Given a vector function x(t) = col[x1 (t), . . . , xn (t)], its time derivative is
def
= dx(t)/dt. Capital letters A, B, F, X are square matrix(t)
ces and I the identity matrix diag (1, 1, . . . , 1). The trace of
matrix A is the sum of its diagonal elements, written tr(A);
det(A) is its determinant. in are integers; r, s, t are real
scalars, as are lowercase Greek letter quantities. German
type g, sl, . . . will be used for Lie algebras.
(1)
F k tk
F 2t2
+ ... +
+ ...
2
k!
(2)
it is the inverse Laplace transform of the matrix (sI F )1

and can be computed by numerical methods described by
Golub and Van Loan (5). Its most familiar use in electrical
engineering is to solve equation 1
x(t) = e x(0) +
e(tr)F v(r)gdr
Ft
The polynomial
def
PF (s) = det(sI F ) = (s 1 )(s 2 ) (s n )

is called the characteristic polynomial of F, and its roots
are called the eigenvalues of F. The entries in the matrix
e Ft are linear combinations of terms like
t mi ei t , i = 1, . . . , n
If the eigenvalues are distinct, the integers m i vanish, but if i is a multiple eigenvalue, m i may be
positive. For a given F, the matrices {exp(Ft), t R}
are a group under matrix multiplication: exp((r + t)F ) =
exp(rF )exp(tF ), (exp(tF ))1 = exp(tF ).
For different applications, different restrictions may be
placed on the class U of admissible control signals. In this
article, U will usually be the class of piecewise constant
(PWC) signals. The value of a control at an instant of transition between pieces need not be dened; it makes no difference. If there is a single number > 0 that is an upper
bound for all admissible controls, call the class U as a reminder.
J. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering. Copyright 2007 John Wiley & Sons, Inc.
Bilinear Systems
A control system, linear or not, is said to be controllable

x,
a time
on its state space if for any two states x,
T > 0 exists and a control in U for which the trajectory
For time-invariant
starting at x(0) = |x ends at x(T ) = x.
linear systems, there is a simple and useful condition
necessary for controllability. Stated for equation 1, this
Kalman rank condition is
x = Ax +
Precisely what are BLS and why should one use bilinear
systems in control engineering? This section will give some
answers to those questions, starting with a formal denition, and give some examples.
Denition. A bilinear system is a rst-order vector differential equation x = p(x, u) in which x is a state vector;
u is a control signal (scalar or vector); and the components
of p(x, u) are polynomials in (x, u) that are linear in x, u
separately, but jointly quadratic, with constant real coefcients. Restating that, for any real numbers ,
p(x, u) p(x, u) + p(x, 0) + p(0, u)
The (optional) output is linear, y = h x.
To begin with, BLS are simpler and better understood
than most other nonlinear systems. Their study involves a
constant interplay between two protable viewpoints: looking at BLS as time-invariant nonlinear control systems and
as time-variant linear systems.
Another answer is that BLS are useful in designing control systems that use a very small control signal to modulate a large current of electricity of uid, apply brakes, or
change rates of growth.
A third answer is that the usual linearization of a nonlinear control system near an equilibrium point can be improved by using a BLS approximation; thus,
=
a(x) + ug(x), with a(xe ) = 0; let

a
q
A =
|x=xe , b = q(xe ), B =
|x=xe
x
x
Translating the origin so that xe = 0, to rst order in x
and u separately:
x = Ax + u(Bx + b), y = h x
(3)
Although some results will be stated for equation 3,

usually we will suppose that b = 0; such BLS are called
homogeneous bilinear systems, and for later reference,
they are given here in both their single-input and k-input
versions
x = Ax + uBx
k
(5)
B jx
j=1
As a control system, equation 4 is time-invariant, and we

need to use controls that can start and stop when we wish.
The use of PWC controls not only is appropriate for that
reason but also allows us to consider switched linear systems as BLS, and in that case, only a discrete set of control
values such as {1, 1} or {0, 1} is used.
Later we will be concerned with state-dependent feedback controls, u = u(x), which may have to satisfy conditions that guarantee that differential equations like x =
Ax + u(x)Bx have well-behaved solutions.
Discrete time bilinear systems (DBLS) are described by
difference equations, rather than ordinary differential
equations. DBLS applications have come from the discretetime dynamics common in economic and nancial modeling, in which the control is often an interest rate. DBLS also
are used for digital computer simulation of continuous time
systems like equation 3: Using Eulers point-slope method
with a time-step , for times k = 0, 1, 2, . . . , the discrete-time
system is
x(k + 1) = (I + A)x(k) + u(k)(Bx(k) + b)
(6)
(5)
(We will see later how to recover facts about equation 3

from the homogeneous case.) If A = 0 in equations 4 or 5,
the BLS is called symmetric:
x =
BILINEAR SYSTEMS: WHAT, WHY, WHERE?
u jB jx
j=1
rank(g, Fg, . . . , F n1 g) = n
If it is satised, the matrix-vector pair {F, g} is called a
controllable pair. If the controls are not bounded, this condition is sufcient for controllability, but if they are in some
U , the control may be small compared with Fx for large x
and have an insufcient effect.
k
(4)
with output y(k) = h x(k). DBLS will be discussed briey

at appropriate places below. Their solutions are obtained
by recursion from their initial conditions using their difference equations.
Some Application Areas
Reference (2) lists early applications of BLS to nuclear
reactors, immunological systems, population growth, and
compartmental models in physiology. For a recently analyzed BLS from a controlled compartmental model, see the
work on cancer chemotherapy by Ladzewicz and Schattler
(6) and its references.

Using linear feedback u = Kx in a BLS results in a
quadratic autonomous system. Recently some scientically
interesting quadratic systems, exhibiting chaotic behavior,
have been studied by decomposing them into BLS of this
type. Celikovsk
y and Vanec ek (7) have studied the thirdorder Lorenz system as a BLS with output feedback:
x = Ax + uBx, u(x) = x1 , with > 0, > 0, > 0, and

0
0 0 0
A=
, B = 0 0 1
1 0
0
0
0 1 0
For small all eigenvalues of A are negative, but for > 1
one becomes positive and B generates a rotation. This description seems to be characteristic of several such examples of strange attractors and bounded chaos, including the
Rossler attractor.
Bilinear Systems
In electrical engineering, BLS viewpoints can be used

to discuss switched and clocked circuits, in which the control takes on only a discrete set of values like {0, 1} and
the plant is linear in each switched condition. The solutions are then easy to compute numerically. Sometimes the
duty cycle of a switch is under control, as in motors and
DC-to-DC power conversion systems. A simple example is
a conventional ignition sparking system for automobiles,
in which the primary circuit can be modeled by assigning
x 1 to voltage across capacitor of value C, x 2 to current in
the primary coil of inductance L. The control is a distributor or electronic switch, either open (innite resistance)
or closed (small resistance R) with duty cycle specied by
the crankshaft rotation and timing controls. Then with a
battery of emf V,
1
u
x 1 = x2 (x1 V ),
C
C
1
x 2 = x1 ,
L
u=
1/R, closed
0,
open
Other automotive BLS include mechanical brakes and

controlled suspension systems, among the many applications discussed in Reference 3.
One advantage of piecewise constant control in BLS
is that the solutions, being piecewise linear, are readily
computed. For that reason, in aerospace and process engineering, control designs with gain scheduling (see GAIN
SCHEDULING) are an area where BLS methods should be
useful; such control schemes change the F matrix and equilibrium point to permit locally linear control. Recent work
on hybrid systems (nite-state machines interacting with
continuous plants) also falls into the category of switched
linear systems.
STABILIZATION I: CONSTANT CONTROLS
This section will introduce an important engineering design goal, stability, and the beginning of a running discussion of stabilization. Stabilization is an active area of
research, in an effort to nd good design principles.
A matrix F is called a Hurwitz matrix if all n of
the eigenvalues of F lie in the left half of the complex plane; i.e., R(i ) < < 0. Then x = F x is said to be
exponentially stable (ES); as time increases, all solutions
are bounded and ||x(t)|| < ||x(0)||et . If even one eigenvalue
lies in the right half plane, almost all solutions will grow
unboundedly and the system is called unstable. Multiple imaginary eigenvalues = j can give t m cos(t) (resonance) terms that also are unstable. Warning: Even if the
time-varying eigenvalues of a time-variant linear differential equation all lie in the left half plane, that does not
guarantee stability!
If A is a Hurwitz matrix, equation 4 is ES when u = 0.
Suppose that A is not Hurwitz or that is too small; nding
a feedback control u such that equation 4 is ES with a
desirable is called stabilization.
The problem of stabilization of BLS and other nonlinear control systems is still an active area of engineering research. In this section, we consider only the
use of constant controls u = in equation 4; the result
of applying this feedback is a linear dynamical system
x = (A + B)x. To nd a range of values for that will
stabilize, the BLS is somewhat difcult, but for small

n one can nd PA+B () and test possible values of
by the RouthHurwitz stability test for polynomials (see
STABILITY THEORY, ASYMPTOTIC). For n = 2
PA+B () = 2 (tr(A) + tr(B)) + det(A + B), so
tr(A) + tr(B) < 0 and det(A + B) > 0
(7)
guarantee stability. Graphing these two expressions

against is an appropriate method for nding good values of . A complete set of conditions A and B that are
necessary and sufcient for stabilizability of second-order
BLS with constant feedback were given by Chabour et al.
(8).
Other criteria for stabilization by constant control have
been found, such as this one from Luesink and Nijmeijer
(9). Suppose the eigenvalues of A are i , i = 1,. . . , n, and the
eigenvalues of B are i . If there is some nonsingular matrix
P, real or complex, for which P 1 AP and P 1 BP are simultaneously upper triangular, then the eigenvalues of A + B
are i + i , i = 1, . . . , n. If some real satises the n linear
inequalities R(i + i ) < 0, it will be the desired constant
control. For more about such triangularizable BLS, see the
section below on The Lie Algebra of a BLS.
SOLUTIONS OF BILINEAR SYSTEMS
From one viewpoint a BLS with a specic nonconstant
control history {u(t), t 0} should be thought of as a timevariant linear differential equation. We will use the singleinput case of equation 4 as an example, with A + u(t)B as
time-variant matrix. The solution depends on the initial
time t 0 at which the state is x(t0 ), and is of the form x(t) =
(t, t0 )x(t0 ), where

(t, t0 ) is called a transition matrix.
For more about the general theory of these matrices, see
MULTIVARIABLE SYSTEMS, Chap. 9 of Kailath 10, or vol. 1,
Chap. 4 of Reference 3. Once having written that expression for x(t), it can be observed that
must satisfy the
matrix differential equation
= (A + u(t)B)
,
(t0 , t0 ) = I
It has the composition property, also called the semigroup

property
(t, t1 )
(t1 , t0 ) =
(t, t0 ).
However, as a control system, equation 4 is timeinvariant, by denition. Then the most convenient families of admissible controls for BLS are the PWC and other
piecewise-dened controls; such a control can be specied
by its values on an interval of denition of duration , for
instance {u(t), t (t0 , t0 + )}. From the time-invariance of
the system, a basic interval of denition, (0,), can be used
without any loss. Given a particular control signal u on
(0,), its time shift by can be denoted u (t) = u(t ), on
(, + ), as is usual in system theory.
The concatenation of two controls u and v with respective durations 1 and 2 is written u o v and is another
admissible control with duration 1 + 2 :
(u v)(t) =
u(t),
t [0, 1 )
v1 (t), t [1 < t 2 ]
For the general multi-input BLS of equation 5, the control

is a k-component vector u =
[u1 , . . . , uk ], and the transition
= (A + k u j B j )
. Concatenation is
matrix satises
j=1
Bilinear Systems
dened in the same way as for scalar controls: u o v is u

followed by the translate of v.
The time-invariance of the BLS leads to useful properties of the transition matrices. The transition matrix depends on the control u and its starting time, so the matrix
should be labeled accordingly as

(u; t, t0 ), and the state
trajectory corresponding to u is
x(t) =
(u; t, t0 )x(0)
Given two controls and their basic intervals
{u, 0 < t < } and {v, 0 < t < }, the composition property for BLS transition matrices can be written in a nice
form that illustrates concatenation (u followed by the
translate of v)
(v ; , )
(u; , 0) =
(u v; , 0)
(8a)
A transition matrix always has an inverse, but it is not

always a transition matrix for the BLS. However, if the BLS
is symmetric (5) and the admissible controls U are signsymmetric (i.e., if u U, then u U), the transition matrix
(u; , 0) resulting from control history {u(t), 0 t } has

an inverse that is again a transition matrix, obtained by
using the control that reverses what has been done before,
u (t) = u( t), t 2;
(u ; 2, )
(u; , 0) = I
(8b)
An asymmetric BLS, x = (A + u1 B + + uk Bk )x, is like a

symmetric one in which one of the matrices B0 = A has a
constant control u0 1, whose sign cannot be changed.
The controllability problem for asymmetric BLS involves
nding ways around this obstacle by getting to I some other
way.
From a mathematical viewpoint, the set of transition
matrices for equation 4 is a matrix semigroup with identity
element I. See the last section of this article.
MORE ABOUT TRANSITION MATRICES
Transition matrices for BLS have some additional properties worth mentioning. For instance, in the rare situation
that A, B 1 ,. . . , B k all commute, the transition matrix has a
comforting formula
(u; , 0) = e
At+
t k
i=1
ui (s)Bi ds
Warning: If the matrices do not commute, this formula is

invalid!
The solution of a single-input inhomogeneous BLS like
equation 3, x = Ax + u(Bx + b), is much like the solution
of a linear system. If
(u; t, 0) is the solution of the homogeneous matrix system
= A
+ uB
,
(0) = I
then for equation 3 with initial condition x(0),
x(t) =
(u; t, 0)x(0) +
(u; t, s)u(s)bds
0
One advantage of using piecewise constant controls is

that they not only approximate other signals, but suggest
a construction of the transition matrix. For a PWC control
u given by m constant pieces {u(t) = u(k1 ), k1 t < k }

on intervals that partition {0 t < m = T }, the transition
= (A + uB)X is clearly
matrix for X
(u; T, 0) =
m
e(A+u(k1 )B)k
(9)
k=1
This idea can be carried much further with more analysis: More general (measurable) inputs can be approximated by PWC controls, and in analogy to the denition
of an integral as a limit of sums, the solution to equation 4
for measurable inputs can be written as (in an appropriate sense) the limit of products like equation 9, called a
product-integral.
The representation given by equation 9 generalizes to
the multi-input BLS equation 5 in the obvious way. With
equation 9, one can also easily verify the composition and
(for A = 0) inverse properties. To emphasize that exponential formulas for noncommuting matrices have surprising
behavior, here is a standard example in which you should
notice that A 2 = 0 and B 2 = 0.

0 1
0 0
1 0
A=
and B =
; AB BA =
;
0 0
1 0
0 1
eAt eBt =
1
0
e(A+B)t = exp
t
1
0
t
1
t
t
0
0
1
=
1 + t2
t
cosh(t)
sinh(t)
1
, but
1
sinh(t)
cosh(t)
OBSERVABILITY AND OBSERVERS

This section is concerned with BLS that have an output
measurement y with m < n components, so that x(t) is
not directly available. For many purposes in control (stabilization, prediction of future outputs, and optimization),
it is necessary 1) to ensure that different states can be distinguished and 2) to obtain estimates of x(t) from available information. An important question is whether an
inputoutput history
HT = {u(t), y(t)|0 t T }
will uniquely determine the initial or nal state.
Let C be an m n matrix (think of it as m row vectors);
its null-space {x|Cx = 0} is denoted by C . The BLS is the
m-output system given by
x = (A + uB)x, y = Cx.
(10)
The initial state is not known, only the history HT .

Suppose that u(t), t 0, is given. Call two states x, x Rn
u-indistinguishable on the interval (0, T) if the two corresponding outputs are equal on that interval, i.e., if
0 t T . This relation, written
C
(u; t, 0)x = C
(u; t, 0)x,
is transitive, reexive, and symmetric (an equivax u x,
lence relation) so it partitions the state space into disjoint
sets (see Chap. 5 of Reference 4); it is also linear in the
state. Therefore we need only be concerned with the set of
states u-indistinguishable from the origin, namely,
def
Iu = {x|x u 0} = {x|C
(u; t, 0)x = 0, 0 t T }
which is a linear subspace called the u-unobservable subspace of the BLS; the u-observable subspace is the quotient
Bilinear Systems
space Ou = Rn /Iu . That can be rephrased as Rn = Iu Ou .

If Iu = 0, we say that the given system is u-observable. In
Grasselli and Isidori (11) u-observability is given the name
observability under single experiment.
If two states x, z are u-indistinguishable for all admissible u, we say they are indistinguishable and write x z.
The set of unobservable states is the linear subspace
State Observers
Given a u-observable system with A, B known, it is possible to estimate the initial state (or current state) from
the history HT . The theory of time-variant linear systems
(see Reference 10 or vol. 1 of Reference 3) shows that Ou is
the range of the time-variant version of the observability
Gramian,
I = {x|C
(u; t, 0)x = 0 for all u U}
Its quotient subspace (complement) O = R /I is called the

observable subspace, and the system is called observable if
I = 0. The unobservable subspace is invariant for the BLS;
trajectories that begin in I remain there. If the largest invariant linear subspace of C is 0, the BLS is observable;
this is also called observability under multiple experiments, because to test it one would have to have duplicate systems, each initialized at x(0) but using its own
control u.
Theorem 1 of Reference 11 states that for PWC controls
and piecewise continuous controls, the BLS is observable
if and only if a u exists for which it is u-observable. The
proof constructs a universal input u that distinguishes all
states from 0 by concatenating at most n + 1 inputs: u =
u0 o o un . At the kth stage in the construction, the set of
states indistinguishable from 0 is reduced in dimension by
a well-chosen u k . The test for observability that comes out
of this analysis is that the rank of the matrix (C; A, B) is
n, where
n
(C; A, B) = col[C, CA, CB, CA2 , CAB, CBA, CB2 , . . .]

That is, (C; A, B) contains C and all matrices obtained by
repeated multiplications on the right by A and B. This is
the rst theorem on the existence of universal inputs, and
the idea has been extended to other nonlinear systems by
Sontag and Sussmann.
The simplest situation in which to look for observability criteria is for a system with input zero, an autonomous
time-invariant linear system x = Ax, y = Cx. It is no surprise that the Kalman rank criterion for observability is
appropriate for such systems. The (time-invariant) observability Gramian is W = col[C, CA, . . . , C(A)n1 ]; we say {C,
A} is an observable pair if rank(W) = n, and that is both
necessary and sufcient for linear system observability.
How can we extend this to the case where the input
is unknown? To derive the answer, from williamson (12),
choose our admissible controls to be polynomials in t of
degree n on any xed time interval. Assume x(0) = 0. It
is still necessary that rank(W) = n, to preserve observability when u = 0. Repeatedly differentiate y = Cx at
t = 0 (not that one would do this in practice) to obtain
def
Y = col{y0 , y 0 , . . . , y0(n1) }. If y 0 = CAx(0) + u(0)CBx(0) = 0
for some u(0), the information from y 0 would be lost; this
gives a necessary condition that CB = 0; continuing this
way, necessarily CAB = 0, and so on. All the necessary conditions for observability can be summarized as
rank(W) = n and c Ak B = 0, 0 k n 2
(11)
To show the sufciency of these conditions for observability,

just note that no matter what control u is used, it and its
derivatives do not appear in any of the output derivatives,
so Y = Wx(0) and x(0) = W 1 Y from the rank condition.
def
(u; t, 0)C C
(u; t, 0)
WT =
If rank(WT ) = n, the initial state can be recovered; in our

notation

x(0) =
WT1 (T )
(u; t, 0)C y(t)dt

0
The current state can be obtained from

(u; t, 0)x(0) or by
more efcient means. Even though observability may fail
for any constant control, it still may be possible, using some
piecewise constant control u, to achieve u-observability.
Frelek and Elliott (13) pointed out that the Gramian could
be optimized in various ways (e.g., by minimizing its condition number) with PWC controls, using a nite sequence
of u values, to permit accurate recovery of the entire state
or certain preferred state variables. The larger the linear
span of the trajectory, the more information is acquired
about x(0).
One recursive estimator of the current state is an
asymptotic state observer. There are many variations on
this idea; see KALMAN FILTERS AND OBSERVERS. A state observer can be regarded as a simplication of the Kalman
lter in which no assumptions about noise statistics are
made, nor is a Riccati equation used, and so can be extended to nonlinear systems for which no Kalman lter
can be found.
The asymptotic state observer to be described is sufciently general for BLS. For a given control system, it is
a model of the plant to be observed, with state vector denoted by z and an input proportional to the output error.
(Grasselli and Isidori (14) showed that there is nothing
to be gained by more general ways of introducing an error term.) To show how this works, we generalize slightly
to allow an inhomoge-neous BLS. Here are the plant, observer, and error equations; K is an n m gain matrix at
our disposal.
x = Ax + u(Bx + b), y = Cx
(12a)
z = Az + u(Bz + b) + uK(y Cz); let e = z x
(12b)
e = (A + uB uKC)e
(12c)
Observer design for linear systems is concerned with nding K for which the observer is convergent, meaning that
e(t) 0, under some assumptions about u. (Observability is more than what is needed for the error to die out; a
weaker concept, detectability, will do. Roughly put, a system is detectable if what cannot be observed is asymptotically stable.) At least three design problems can be posed
for equation 12.
Bilinear Systems
1. Design an observer that will converge for all choices

of (unbounded) u in U. This requires only the conditions of equation 11, replacing B with B KC, but
then the convergence depends only on the eigenvalues of A; using the input has gained us no information.
2. Assume that u is known and xed; in which case, the
methods of nding K for observers of time-variant
linear systems are employed, such as Riccati-type
equations. There is no advantage to the BLS form
in this problem.
3. Design a control and simultaneously choose K to get
best convergence. Currently, this is a difcult nonlinear programming problem, although Sen (15) shows
that a random choice of the values of PWC u should
sufce; from the invariance of the problem under dilation (zooming in toward the origin), it seems likely
that a periodic control would be a good choice. This
observer design problem is much like an identication problem for a linear system, but identication
algorithms are typically not bilinear.
Using digital computer control, one is likely to have not
a continuous history H but the history of a PWC input u d
and a sampled output
Hd = {(ud (t1 ), y(t1 )), (ud (t2 ), y(t2 )), . . .}
and the BLS can be dealt with as a discrete-time BLS. Using only a, nite sequence of N > n values, the initial state
z [or current state x(t N )] can be estimated by the leastsquares method, which at best projects z onto the closest z
in the observable subspace:
z = arg min
N
y(tk ) C
(ud ; tk , 0)z2
k=1
CONSEQUENCES OF NONCOMMUTIVITY
The noncommutativity of the coefcient matrices A, B of
a BLS is crucial to its controllability, raising questions
that suggest for their answers some interesting mathematical tools: Lie algebras and Lie groups, named for Norwegian mathematician Sophus Lie, pronounced lee. See the
Reading List at the end of this article for books about them.
Lie Brackets
If A and B do not commute, then solving equation 4 is
more interesting and difcult. In addition to A, B, we will
need AB BA, which is written [A, B] and is called the
Lie bracket of A and B. To see how this matrix might come
into the picture, and obtain a geometric interpretation of
the Lie bracket, consider the two-input BLS with piecewise
constant controls
x = uAx + vBx; u, v {1, 0, 1}
(13a)
Starting at any x(0), use a control with four control segments (u, v), each of small duration > 0:
{1, 0}, {0, 1}, {1, 0}, {0, 1} on intervals
(13b)
{0, }, {, 2}, {2, 3}, {3, 4}, respectively
(13c)
The nal state is (using the Taylor series of Eq. (7) for the
exponential and keeping only terms up to degree 2)
x(4) = 7eB eA eB eA x(0)
= (I + 2 [A, B] + 3 (higher order terms) + )x(0)
(13d)
The Lie Algebra of a BLS
In much the same way that controllability of the linear system equation 1 is related to the linear span of
{g, Fg, , F n1 g}, controllability of BLS is related to a linear space of matrices generated from A and B by repeated
bracketing, called the Lie algebra of the BLS. A survey of
Lie algebra facts is given in Belinfante and Kolman (16).
The primary concern of this subsection is homogeneous
BLS of the types given in equations 4 and 5. To make clear
what is being discussed, we need a denition.
Denition. A linear space g over a eld K (usually the
real or complex numbers) with a multiplication g g g:
{X, Y } [X, Y ] g will be called a Lie algebra if it satises
the properties
1. [X, Y ] = [X, Y ] = [X, Y ], K.
2. [X, Y ] + [X, X] = 0.
3. [X, [Y, Z]] + [Y, [Z, X]] + [Z, [X, Y ]] =
0 (Jacobi identity).
In mathematical writing, Lie algebras are usually given abstractly, by relations among their elements, and only then
does one represent the elements by matrices acting on some
vector space. In contrast, the Lie algebras of control theory
have specied generators. (In this article, the Lie algebras
are all matrix Lie algebras; for nonlinear vector eld Lie
algebras, see CONTROLLABILITY AND OBSERVABILITY.)
The Lie algebra generated by a BLS is constructed as
follows. Let Mn denote the linear space of all n n real
matrices; its dimension is r 2 . A real linear subspace g Mn
that is closed under the bracket [X, Y ] = XY YX is called a
(real) matrix Lie algebra. The space Mn itself can be identied with a Lie algebra of dimension n 2 over R called the
general linear algebra gl(n, R).
Two matrix Lie algebras g, g are said to be equivalent if
their elements are related by a common similarity transfor = P 1 XP. This denition is justied by the idenmation X
tity
P 1 [A, B]P = [P 1 AP, P 1 BP]
For homogeneous BLS with system matrices A and B as
in equation 4, we generate a Lie algebra in the following
way, based on the theory of free Lie algebras. {A, B}LA is
the subspace of Mn containing A and B and closed under
the Lie bracket and linear span operations. It is not hard to
compute {A, B}LA , using the properties 13 of Lie brackets.
Start with two generators A and B, we write down a tree
Bilinear Systems
of brackets as a data structure T (A, B):

BA
[A, B]
[A, [A, B]]
[B, [B, A]]
[A, [A, [A, B]]] [B, [A, [A, B]]] [A, [B, [B, A]]] [B, [B, [B, A]]]
.
.
.
.
.
.
.
.
.
.
.
.
The trees indicated structure depends only on the denition of a Lie algebra; T (A, B) is built level by level, bracketing each terminal leaf by A and by B; as shown it has already been pruned of obviously linearly dependent leaves
by the identities [X, Y ] = [Y, X] and [X, X] = 0. By using the Jacobi identity, more leaves can be pruned; e.g.,
note that [B, [A, [A, B]]] = [A, [B, [B, A]]] at level 4 before
building level 5, and so forth. It is a known feature of Lie
algebras that all higher order Lie brackets generated by A
and B can be obtained from those in this tree using the Jacobi identity. The linear span of the members of T (A, B) is
the desired Lie algebra g = {A, B}LA . As our matrices are in
Mn there can be no more than n 2 of them that are linearly
independent; working with the specic matrices A, B more
dependencies can be found (the Cayley-Hamilton Theorem
provides them) so the construction of the tree stops when
the matrices at some level are linearly dependent on their
ancestors.
In this process, we nd enough matrices to obtain the
dimension l and a basis B for the linear space g; we shall
write it as an array B = {C1 , . . . , Cl }. As we assume A and B
are not linearly dependent, it is convenient to take C1 = A,
C2 = B, C3 = [A, B]. If the entries of A and B are rational
numbers, symbolic algebra computer programs can generate T (A, B) and produce B.
It is known that generic pairs A, B will generate a tree
of matrices whose span is Mn . In other words, if you ll
out the entries with randomly chosen real numbers, then
for almost all sample pairs, {A, B}LA = gl(n, R). However,
control systems have structural relationships among their
components that may lead to smaller Lie algebras.
For example, here are some properties of generators that
are preserved under bracketing and linear combination,
and the name of the Lie subalgebra g gl(n, R) with the
given property and largest dimension d.
Commutativity: g is called abelian and has d = 2.

Zero trace: the special linear algebra sl(n, R) has d =
n2 1.
Skew-symmetry: the orthogonal algebra so(n, R) has

d = n(n 1)/2.
Simultaneously triangularizable over R or C: solvable

Lie algebras have d n(n + 1)/2.
In the stabilizability criterion of Reference 9 discussed
above, solvable Lie algebras were mentioned; they are characterized by the property that the sequence of Lie algebras
g1 = [g, g], fg2 = [g, g1 ], . . . , g j+1 = [g, g j ], . . .
terminates at the trivial Lie algebra {0}.
For multi-input BLS like equation 5 or symmetric systems (5), a similar (if harder to diagram) tree construction
can be carried out. If the dimension n is low and there

are many independent matrices A, B1 , . . . , Bk , only a few
brackets may be needed to obtain the basis B of the Lie
algebra.
One of the more useful ideas in computing with Lie
brackets is to notice that [A, X] is a linear operation on
def
the matrix X; dene adA (X) = [A, X] and powers of adA
0
k
recursively: adA (X) = X, adA (X) = [A, adk1
A (X)], k > 0. This
simplies the discussion of an important part of the tree
T(A, B) because its leftmost leaves adkA (B) are used in an important rank condition (the ad-condition, discussed in the
Controllabity Conditions section below). There are many
useful formulas involving adA , such as
etadA (B) = etA BetA
Accessibility and the Lie Rank Condition. What state
space is most appropriate for a bilinear system? For inhomogeneous systems, Rn is appropriate. However, for homogeneous BLS, a trajectory starting at 0 can never leave
it, and 0 can never be reached in nite time; the state space
may as well be punctured at 0. This punctured n-space is
denoted by Rn \0 or Rn0 , and it is of interest in understanding controllability. For n = 1, it is the union of two open
half-lines; in the scalar BLS = u, the state can never
change sign. For n = 2, the punctured plane is not simply
connected. For n 3 the puncture has negligible effect.
In some applications, other unusual state spaces may be
appropriate. There is an n-dimensional generalization of
our scalar example, the diagonalizable BLS, which are (up
to similarity) given by x i = ui xi , i = 1, . . . , n. If the initial
state is on a coordinate half-axis, a quadrant of a coordinate plane, . . . , or on one of the 2n orthants, the state must
stay there forever. Other BLS that live this way on positive
orthants occur rather often, and their controllability properties will be discussed in the Positive Systems section
below. In economics, chemistry, ecology, and probability applications, the state variables are usually positive and the
dynamical model must respect that; bilinear systems and
quadratic systems are the simplest models needed in such
applications.
Sometimes all we need to establish or can establish will
be a property weaker than controllability but still very useful: if the set of states {
(u; t, 0)x, t > 0} has an open interior, the BLS is said to satisfy the accessibility condition at
x. If that condition is satised for all initial states, we
say the BLS has the accessibility property on its state
space. Controllable systems have this property, always, but
it is not enough to ensure controllability, as is shown in
our next example, which will also motivate the concept of
strong accessibility, which means that {
(u; t, 0)x} has an
open interior at each t.
Example I. On R20 consider a BLS
x = (I + uJ)x, I =
1
0
0
, J=
1
0
1
1
0
In polar coordinates, this system becomes r = r, = u

and its trajectories are, for constant control values, expanding spirals, clockwise or counterclockwise. Starting at r0 , 0
Bilinear Systems
at the set of states that can be reached at , any xed positive time, it is a circle with radius r() = e r0 , which is not
an open set, so the BLS does not have the strong accessibility property. No state with r() < r0 can be reached, which
establishes that the system is not controllable. The set
{r(t), (t)|0 < t < } is the open annulus {(r, )|r0 < r < r0 e },
so the control system does have the accessibility property.
Reverse the roles of the two generators. The BLS becomes x = (J + uI)x; in polar coordinates r = ur, = 1. At
later time T, the radius is arbitrary but (T ) = 2T + 0 .
This system has the accessibility property (but not strong
accessibility) and is controllable. If the trajectory misses
a target state, it takes 2 seconds before it has a second
chance. This peculiar state of affairs reects the fact that
our state space is punctured at the origin, and as we remarked before, it is topologically a cylinder.
Sometimes it is easy to see from the structure or known
symmetries of generators that a given system cannot have
the accessibility property; the most obvious of these, for
equation 5, are 1) the dimension of {A, B}LA is less than
n2 1 or 2) A and B are simultaneously triangularizable.
How can we guarantee accessibility on Rn0 ? For
a given homogeneous BLS, e.g., equation 4 whose
Lie algebra is g = {A, B}LA , construct the n l matrix
def
Bx = [C1 x, C2 x, . . . , Cl x], where l is the dimension of g. Dene the Lie rank (x) as the dimension of the linear span
of {Xx |X g}, or more constructively, (x) = rank(Bx).
For homogeneous BLS, a necessary and sufcient condition for the accessibility property is the Lie rank condition
introduced by J. Kucera (17):
(x) = n, for all x = 0
(14)
If (x(0)) = k, then (x(t)) = k, t R. If at some point x the

BLS can move in (x) directions, this must remain true at
all points that the trajectories can reach from x. (Consider
the diagonal systems mentioned above, for example.) Due
to the radial scaling properties of BLS, the Lie rank actually needs to be checked only on the unit sphere and is
the same at antipodal points. If g satises the condition
in equation 14, it is called transitive on Rn0 ; see the Matrix Groups section to see why the word transitive is used.
To check a BLS for transitivity, nd the n n minors of Bx;
these are nth degree polynomials. If 0 is their only common
zero, the Lie rank is n. This algebraic task is performed by
symbolic algebra; see the book by Elliott in the Reading
List.
For symmetric systems (Eq. 5), transitivity of the Lie algebra generated by {B1 , . . . , Bk } is necessary and sufcient
for controllability on Rn0 ; the Matrix Groups section will
explain why. For asymmetric systems (Eq. 4) and (Eq. 5)
transitivity is necessary but far from sufcient; that can
be seen from Example I. Its Lie algebra is the span of I
and J, det(x, Jx) = x12 + x22 so (x) = 2; but all paths have
x(t) = et x(0).
def
At each state x Rn0 , the set hx = {X g|Xx = 0} is a linear subspace of g and contains the Lie bracket of any two
of its elements, so it is called the isotropy Lie algebra at x;
it is the same, up to similarity equivalence, at all points
reachable from x by trajectories of the BLS. Transitivity
of the Lie algebra g on state space Rn0 also means that for
every state x the quotient space of its isotropy subalgebra

hx in g satises g/hx Rn0 .
Returning once more to Example I, g = {I +
J|, R}, so
Bx = {Ix, Jx} =
x1
x2
x2
x1
, det(Bx) = x12 + x22
As, x12 + x22 = 0, the equation Ix + Jx = 0 has the

unique solution = 0, = 0; therefore, the isotropy algebra is {0}; on R20 the Lie rank is (x) = 2.
Those interested in computability will notice that the
Lie rank criterion (Eq. 14) is computationally of exponential complexity as n increases, but as a negative criterion, it
can be easily checked by choosing a random state x Rn0 ; if
(x) < n, the system cannot have the accessibility property
nor controllability. However, if (x) = n, there still may be
states, as in our diagonal-system example, which cannot be
reached from x.
Proving accessibility or establishing useful tests was
made easier through the work of Boothby and Wilson
(18), which lists, for each state-space dimension n, the
(transitive Lie algebras) and provides a rational algorithm
to determine whether a homogeneous BLS has a Lie algebra on the list. This list was completed recently by the
independent work of Kramer (19) with an additional Lie
algebra known as spin (9, 1).
Example II. Another Lie algebra helps relate linear and
bilinear systems; its system theoretic interpretation was
pointed out by Brockett (20).
aff (n, R) =
X M n+1 |X =
x
, X Mn , x Rn
0
X
0
The appropriate state space will be an n-dimensional

hyperplane
P = {z Rn+1 |zn+1 = 1}
To see what is happening, consider rst a linear system on
Rn , x = Ax + u(t)b. Now on Rn let
x1
z = ...

, A = A 0 , B = 0
0
0
b
0
zn+1
with z n+1 = 0 and zn+1 (0) = 1. On Rn+1 , the bilinear system
z = Ax + u(t)Bx has aff (n, R) for its Lie algebra but on P
and is equivalent to the linear system we started with. Note
that
adA (B) =
0
0
Ab
2
, adA
(B) =
0
0 A2 b
, etc.
0 0
Brackets containing the factor B twice will vanish. The hyperplane zn+1 = 1 is invariant, and the BLS is controllable
on that hyperplane under the usual Kalman rank condition
that rank{b, Ab, . . .} = n.
The controllability properties of inhomogeneous BLS
like equation 3 x = Ax + u(Bx + b) can be studied using
the idea and notation of Example II. This BLS system can,
using n + 1 coordinates, be given as a homogeneous BLS
with an invariant hyperplane,
z = Az + u(t)Bz, where A =
A
0
0
, B=
0
B
0
b
0
Bilinear Systems
For equation 4, in Cheng et al. (24), the ad-condition

A Left Inverse System. A problem closely related to observability can be posed at this point: left invertibility of
a single-input single-output BLS x = Ax + uBx, y = cx.
In the terminology of Hirschorn (21), this system is called
left invertible if the output history on some interval [0, T)
uniquely determines the input history on [0,T).
The relative order of the BLS is the least positive integer k such that
cadkA (B) = 0
(15)
or = if all cadk1
A B = 0, k > 0. The BLS is invertible if
< and cadk1
Bx(0)
= 0. Invertibility fails if and only if
A
every control results in the same output.
The conditions of Hirschorn (Eq. 15) and Williamson
(Eq. 11) are interestingly related when one takes into account that in the system-inverse problem x(0) is known.
Assume that the Williamson condition holds, i.e., the rank
of W is n and cAk B = 0, 0 k n 2. They do not force
B = 0, because we can have cAn1 B = 0. Now evaluating
the Hirschorn conditions, in turn
cB = 0; c[A, B] = cAB = 0; c[A, [A, B]] = cA2 B = 0; . . .
n1
B = 0; so the relative order is = n.
but cadn1
A B = cA
Reference 21 points out, in this case the inverse system
becomes an observer, tracking x(t) when y() is supplied as
its input.
CONTROLLABILITY PROBLEMS
Criteria for Controllability. The necessary conditions for
controllability of homogeneous BLS begin with the Lie
rank condition (Eq. 14). Using it is made easier by the list
of transitive Lie algebras in References 1819. A necessary
condition for controllability given by Elliott (22) is that controllable homogeneous BLS on Rn0 , n > 2 have the strong
accessibility property. (The peculiarity of Example I occurs
because the punctured plane is not simply connected.)
For inhomogeneous BLS like equation 3, at x = 0, the
Kalman rank condition on {b, Ab, . . .} is a sufcient condition for local controllability with unbounded controls; as
that rank condition is an open one in the space of coefcients, for sufciently small x, the Bx term does no harm.
There is usually a family of possible equilibria corresponding to solutions of (A + B)x = b and at each of these
one can make such a test. In the rst paper on BLS, Rink
and Mohler (23) assumed such local controllability conditions to show controllability with bounded control u of
x = Ax +
m
(Bk x + bk )uk
k=1
provided that the equilibrium set, for

madmissible u, is connected and all eigenvalues of A + k=1 uk Bk can be made
strictly stable and strictly unstable using admissible constant values of u. This use of bilinear terms to make up for
the deciencies of linear control methods, when controls
are bounded, is emphasized by Mohler.
rank(Ax, Bx, adA (B)x, . . . , adAn 1 (B)x) = n on Rn0
(16)
plus the hypothesis that A is similar to a skew-symmetric

matrix are shown sufcient for controllability with
bounded control. A partial converse is that controllability
of equation 4 for arbitrarily small bounds on the controls
implies that all eigenvalues of A are imaginary.
Many papers on controllability and stabilization cite Jurdjevic and Quinn (25). For BLS their condition specializes
as follows, for piecewise continuous signals. If A has eigenvaiues that are purely imaginary and distinct and the Lie
rank condition is satised, then x = Ax + uBx is controllable. This extends to multiple-input BLS immediately. A
stabilization result using the ad-condition is also obtained.
For symmetric systems, the Lie rank condition is a sufcient condition for controllability, which is connected to the
fact that in that case the transition matrices constitute a
Lie group (discussed below).
For asymmetric BLS, the set of transition matrices is
only a semigroup, meaning that it may not contain the inverses of some transition matrices. A broad principle of
controllability theory is that you must be able to get back
to an open neighborhood of the state where you started,
somehow.
Several conditions sufcient for controllability of BLS
were given by Jurdjevic and Kupka (26), for unbounded
controls. Here is a sample. Assume that the eigenvalues of
B are simple and real, in the order q1 > q2 . . . > qn ; choose
coordinates so that B = diag(q1 , . . . , qn ). Then if the numbers qi q j are all distinct, if the elements of A satisfy
Ai j = 0 for all i, j such that |i j| = 1, and if A1n An1 < 0,
then x = (A + uB)x is controllable on Rn \0. Jurdjevic and
Sallet (27) and others have extended the approach of Reference 24 to nonhomogeneous BLS like equation 3, in the
m-input case.
Positive Systems. There are several ways in which BLS
models arise that involve positive variables. As a simple
example, diagonal systems
x i = ui xi , i = 1, . . . , n, xi > 0 on the orthant
Rn+ = {x Rn |xi > 0, i = 1, . . . , n}
can obviously be transformed by the substitution xi =
exp(zi ) to the system of coordinate-translations z i = ui .
Controllability for systems of a more interesting nature
that have positive orthants as their natural state spaces
was studied by Boothby (28). Here the BLS is
x = Ax + uBx on Rn+
(17)
under the hypothesis that the n n matrix A is

essentially positive: ai j > 0, i = j, written A > 0. It is well
known and easy to show that, for A > 0, if x(0) Rn+ , then
x(t) Rn+ , t > 0.
The conditions on B used in Reference 28 are that B is
nonsingular and diagonal; B = diag[1 , . . . , n ]; and for all
i = j, i j = 0, which is an invertibility condition for adB .
If none of the eigenvalue differences is repeated, {A, B}LA
is sl(n, R) if the trace of A is zero, gl(n, R) otherwise, so the
10
Bilinear Systems
Lie rank condition is satised. Controllability and noncontrollability results are established for several families of A,
B pairs, especially for n = 2.
Bacciotti (29) completed the study for n = 2. He assumes
the same conditions: A > 0; B is diagonal and nonsingular,
with no repeated eigenvalues; and 2 > 0 (if not, reverse the
sign of the control). Then for the BLS x = Ax + uBx:
1. If 1 > 0, the BLS is completely controllable on Rn+ .
2. If 1 < 0 but = (2 a11 1 a22 )2 + 41 2 a12 a21 > 0 and
1 a22 2 a11 > 0, then the BLS is completely controllable on Rn+ . In any other case, controllability fails.
Sachkov (30) gives answers for m-input problems
(Eq. 5) with m = n 1 or m = n 2, using an idea that
has been successful in other
m controllability problems: if the
symmetric system x = i=1 ui Bi x is controllable on hypersurfaces V (x) = that ll up the (simply connected) state
space Rn+ , and if tne zero-control trajectories of equation
5 can cross all the hypersurfaces in both directions, then
equation 5 is globally controllable on Rn+ .
Stabilization II. At this point it is appropriate to look at
stabilization by state feedback controls that are not constant. For BLS the usual way of attacking this has been
by quadratic Lyapunov functions. Given a vector differential equation x = f(x), f (0) = 0, the basic idea behind
Lyapunovs Direct Method is to nd a family of nested
smooth hypersurfaces around 0, such as concentric ellipsoids, which the trajectories enter and never leave. For example, let us start with a test for stability of the differential equation x = Ax. Choose a symmetric matrix Q that is
positive denite, which means that all its eigenvalues are
positive and is easy to check by the criterion that each of
the n leading minor determinants of Q is positive:

Q11 Q12
> 0, . . . , det(Q) > 0
Q11 > 0,
Q12
Q22
Let V (x) = xQx, whose level surfaces are ellipsoids; then

along trajectories of the differential equation, V = x(QA +
AQ)x. If (QA + AQ) is negative denite (all eigenvalues
negative), then V < 0, which is what we wanted. There are
various recipes for choosing Q, such as solving a linear Lyapunov equation QA + AQ = I for Q and testing for positive deniteness afterward.
One of the applications of BLS is to controlled
switched linear systems
x = Au x, Au A
where A is (in the simplest version) a nite family of n n
matrices indexed by an integer valued control u(t) that can
be assigned in any way that enforces some delay between
different values. A basic question is to nd conditions are
needed to ensure that the switched system will be ES for
arbitrary control sequences. Necessarily all the A u are Hurwitz; otherwise a xed u would not provide ES. Agrachev
and Liberzon (31) showed that it is sufcient to also impose
the condition that the Lie algebra generated by the Au A
is solvable, as in Reference 9. In the coordinates (real or
complex) in which the matrices are triangular, their common Lyapunov function is x x.
Returning to equation 4, suppose A has all its eigenvalues on the imaginary axis. Then there is a positive denite Q for which QA + AQ = 0; by a change of variables,
Q = I. That is the assumption in Reference 25, which therefore uses the Lyapunov function V (x) = xx; the the adcondition on A and B (Eq. 16) is assumed to hold. The
proposed feedback control is u(x) = xBx; so along the
trajectories of the closed-loop system x = Ax (xBx)Bx,
we have V = (xBx)2 0 x = 0. From the ad-condition,
trajectories cannot remain in the set {xBx = 0|x = 0},
and V (x(t)) 0 as t . However, looking at the onedimensional case, one sees that for the differential equation = 3 , the approach to the origin is of order t 1/2 ,
and that this is also true for the n-dimensional problem
using quadratic feedback. Using the same hypotheses and
nearly the same short proof as in Reference 25, the reader
can verify that the bounded feedback u = xBx/xx provides exponential stability.
In linear system work, the choice of feedback control
is linear, u = c x. Applied to BLS x = Ax + uBx, this results in a quadratic system, such as the Lorenz system,
and is the topic of References 6 and 7. Reference 7 is concerned with linear feedback for a class of BLS in which B
generates a rotation and, for some constant , A + B has
real eigenvalues of mixed sign. For n = 2, after a similarity
transformation
A + B = diag(1 , 2 ); 1 > 0, 2 < 0, B = J
Using a constant control > , the BLS can be globally
practically stabilized; i.e., the trajectory eventually enters a ball of radius of order 1/. For n = 3, the same type of
system [now with 3 < 0, B so (3)] can be globally asymptotically stabilized given certain polynomial inequalities
in the i and B. Given some simpler inequalities, such as
tr(A) < 0 but allowing 1 > 0, the system is practically stabilized by a family of linear feedbacks with parameter .
Stabilization of homogeneous BLS in the plane has been
fully analyzed; it is not necessary that A have imaginary
eigenvalues. Bacciotti and Boieri (32) using constant, linear, and quadratic feedbacks, and Reference 8 using feedbacks differentiable except perhaps at 0, have given complete classications of the possibilities for stabilizability of
equation 4 on R20 . The methods of analysis in these papers
include Lyapunov functions, center manifolds, and properties of plane curves, depending on the various cases. These
cases are specied by the determinants and traces of A and
B, diagonalizability, and a few other structural features. In
Reference 8, a feedback control, useful even when A is not
stable, is u = xRx/xT x where matrices R, T are found
on a case by case basis. These controls are homogeneous
of degree zero, and if not constant are discontinuous at 0
but differentiate for x = 0. The stabilized trajectories typically approach the origin in a spiraling fashion, often with
large excursions; over each revolution the distance to 0 decreases in a constant ratio. In that sense they generalize
the constant controls; the signum controls are essentially
homogeneous of degree zero, and there may be other possibilities.
Recent work on more general problems of nonlinear system stabilization suggests that a time-periodic feedback
Bilinear Systems
will be needed for three or more dimensions, in order to

bring into play the higher order Lie brackets of A and B.
A Note on Optimal Control. At the time that work began on BLS one of the motivations for such studies (2, 17)
was their application to optimal control; see OPTIMAL CONTROL. Reference 9 treats optimal control of a BLS arising
in cancer chemotherapy. Case studies for vehicle control
and nuclear reactor control were summarized in Reference
3, vol. 2; its bibliography lists some studies of biological
and ecological systems with BLS models, in which there
is some evidence that certain biosystems switch behaviors
optimally; see the paper of Oster (33) on bees.
The most studied cost was the time-to-target; that is,
given an initial state and a target (a state or closed set),
use controls in some admissible class of bounded controls
U to nd the trajectory connecting the initial state and
the target in least elapsed time [See Jurdjevics book (34)].
In linear system versions of this problem, it was known
that the set accessible from a given state was the same for
U as for the set of PWC controls with values in {, },
called bang-bang controls. One formulation of the bangbang principle is the set attainable for bounded controls
can be obtained by using only the extreme values of the
controls. It holds true for BLS in which all matrices commute. The computation of time-optimal controls, as well
as the rst attempts at studying controllability (17), assumed that one could simplify their solution to the problem of nding optimal switching times (open-loop) or nding hypersurfaces on which the control would switch values. However, early on it was discovered by Sussmann (35)
that the bang-bang principle did not apply to bilinear systems in general. A value between the extremes may be required. There are examples of simply formulated optimal
control problems for which the control may have to switch
innitely often in a nite time interval.
MATRIX GROUPS
The set of all n n nonsingular matrices equipped with the
usual matrix multiplication, identity I, and inverse constitutes a group, called GL(n, R) where GL stands for general
linear. A subgroup of GL(n,R) is called a matrix group; the
matrix groups we need for BLS are matrix Lie groups.
The derivation of the results of References 17 and 23
other nonlocal results for BLS depends not only on Lie algebras, but also the corresponding matrix Lie groups. Begin
by considering the k-input symmetric homogeneous BLS
of equation 5, where it is assumed that the matrices B i are
linearly independent. Again write u = (u1 , . . . , uk ). The Lie
algebra of this BLS is {B1 , . . . , Bk }LA . We let {
} be the set
of transition matrices for this BLS, i.e., the solutions of the
matrix system
=
k

ui Bi )
,
(0, 0
=I
(18)
As {
} contains I and is closed under composition [eq. 8a]
and inverse [eq. 8b], we see that it constitutes a group, in
fact a subgroup of GL(n,R). As all matrices in {
} have
11
positive determinants, {
} actually lies in GL+ (n, R).
On the other hand, corresponding to any matrix Lie algebra g, a Lie group we shall call G(g) can be constructed,
consisting of all products of exponentials of matrices in g;
see Reference 16 or Rossmanns book in the Reading List. If
a basis of g is {C1 , . . . , Cl }, in a neighborhood of the identity
G(g) has coordinates {exp(C1 t1 ), . . . , exp(C1 tl )} (t1 , . . . , tl ).
(Thus, g is the tangent space to G at I.) Using elements
of G to translate this coordinate patch anywhere on the
group, it can be observed that G(g) has an atlas of coordinate charts that are related by differentiable transformations where they overlap, like a geographic atlas, and on
which the group multiplication and inverse are differentiable functions. Lie groups occur in many applications of
mathematics to classical mechanics, quantum mechanics,
chemistry, and the control of robots and aerospace vehicles.
At this point we can note that the mathematics of controllability for symmetric BLS is rather simple. If a matrix
Lie group G has the property that given any two states x
and z in Rn0 , X G exists such that Xx = z, then G is called
transitive on Rn0 and g is a transitive Lie algebra.
In Reference 20, it was shown that G({B1 , . . . , Bk }LA ) =
{
}; that is, all matrices in the Lie group can be obtained as
transition matrices. This is a simple version of the Chow
Theorem of nonlinear control. Its meanings for BLS are
that, once the Lie algebra has been identied, the structure
of the group of transition matrices is completely known;
and that any matrix M G({B1 , . . . , Bk }LA ) can be written
as a product
M = et1 B1 es1 B2 . . . etk B1 esk B2
for some nite sequence of reals {t1 , s1 , . . . , tk , sk }. Thus, a
few generators are as good as l of them.
Some examples of Lie groups are useful; in some of those
listed below, the identity det(eA ) = etr(A) is relevant:
Lie algebra g
gl(n, R) = Mn
sl(n, R) = {X|tr(X) = 0}
so(n) = {X|X = X}
G GL(n, R)
GL+ (n, R) = {Q|det(Q) > 0}
SL(n, R) = {Q|det(Q) = 1}
SO(n) = {Q|Q Q = I}
and if J 2 = I and is irrational,

GL(4, R)
{X|X = diag{J, J} M4 } R
is a densely wound curve that lls up a 2-torus,
where R
so it is not a closed (Lie) subgroup of GL(4, R).
Any element Z G is a product of exponentials of elements of the corresponding Lie algebra g but not necessarily an exponential of any such element; the standard counterexample is Z = diag(e, e1 ) SL(2, R). Here I =
exp(J), diag(e, e1 ) = exp(diag(1, 1)), but their product Z
is not the exponential of any real matrix.
For asymmetric systems x = (A + uB)x, in GL(n, R) the
set of transition matrices S = {
(u; t, 0)|u U, t 0} contains I and is closed under multiplication but not under matrix inversion; it is a matrix semigroup. Example
I showed an uncontrollable system for which S is a semigroup but with the accessibility property. Here is another
two-dimensional example with bounded controls, showing that controllability and stabilizability are possible for
semigroups acting on R20 .
12
Bilinear Systems
Figure 1. Phase portrait and path from p to q for Example III.

From p use u = 1 to reach the x 2 axis; then use u = 1; switch to
u = 1 at the arc of the spiral that leads to q.
Example III. Consider this BLS with u = 1 and

0.5 > > 0:
x = Ax + uBx; A =
AB =
1 + 1
1
1
2+ 0
0
2
1 1
;
1
1
2
A+B =
2
( ) is not of the form 2ki for any pair of eigenvalues

, of A and integer k.
However, Sontag (36) has shown that for a controllable
BLS (eq. 4), the sampled system (eq. 20) will be controllabile if this condition is imposed: ( + ) = 2k,
k = 0 for any four eigenvalues of A.
The discretization of an uncontrollable BLS can be artifactually controllable,. depending on the BLS and the numerical method used. For the Euler discretization shown
in equation 6, here is a two-dimensional example. The BLS
is x = u(2J + I)x, which has A = 0, B = 2J + I, and is not
controllable. The system can move back and forth along one
spiral trajectory through x(0). The discrete-time approximation is xs = x + uBx. This DBLS is controllable on R02 ;
the trajectories move on the tangent lines to the spiral.
Control and optimal control problems for DBLS xs =
Ax + Bxu are investigated in Swamy and Tarn (37); in
the optimal control area, the case that B has rank one
is notable, because it can be reduced to a linear system
problem by the factorization B = bc . Perform the optimization, with whatever cost, for the linear system xs = Ax + vb
and obtain the optimal control v then in the DBLS, let
u = v /(c x),
so
, B=
As tr(A B) = 2, for either choice of control this is not a

stable system. Paths with u = 1 are spirals that provide
rotations. With u = 1 the paths are hyperbolas, which
near the coordinate axes permit movement toward the origin x2 -axis) or away from it (x1 -axis). System l9 is controllable, because dilations and rotations of arbitrary extent
are possible. Furthermore, system 19 satises the conditions tr B = 0, tr A > 0, and tr AB = 0 of Th. 2.2.3 of Reference 33, so it can be stabilized with a feedback control
u(x) = x Rx/x x.
Other Aspects of Bilinear System Theory.
Discrete-Time Systems. There is a large literature on
DBLS on most of the topics we have covered, sometimes
employing very different mathematical tools and sometimes highly parallel to continuous-time systems. To emphasize this we use a successor (time-shift) notation;
instead of x(t + 1) = (A + uB)x(t), for DBLS, we use xs =
(A + uB)x.
The observability theory of discrete-time bilinear systems is very much like that of the continuous version, with
the same criteria, e.g., equation 11 to assure observability for all u. Inverting an observaole discrete-time system
requires no time-derivatives of the output y = c x.
When the DBLS is obtained by sample-and-hold operations with interval on a continuous-time system like equation 4, the sampled system is
xs = e(A+uB)
xs = Ax + bc x
(19)
(20)
If the BLS is observable to begin with, observability

is preserved when the sampling interval is sufciently
small; the condition (the same as for linear systems) is that
v
c x
when c x = 0, and perturb the computation slightly to avoid

its vanishing. This division controller reects the fact that
the discrete motion can avoid the hyperplane c x = 0.
Controllability with bounded controls is associated (as it
is in continuous-time systems) with zero-control motion on
an ellipsoid and an accessibility condition. For the DBLS
xs = Ax + uBx, Reference 24 has a DBLS version of the adcondition obtained from the Jacobian of the map from the
input history to output history
rank{Ax, Am Bx, Am1 BAx, . . . , ABAm1 x, BAm x} = n
on Rn0 , where m need be no larger than n2 1. When the
rank condition holds, the condition for controllability with
controls bounded by any positive constant is that A is similar to an orthogonal matrix. As a partial converse, it was
shown that requiring controllability for inputs with arbitrarily small bounds implies the spectrum of A lies on the
unit circle, and that the DBLS ad-condition is sufcient for
controllability when some power of A is orthogonal.
The solution of a DBLS xs = (A + uB)x is given by the
right-to-left ordered product
(u; t, 0)x(0) = (0k=t (A + u(k)B))x(0)

The set of states attainable from x(0) may be pathological.
For example take
A=
cos() sin()
, B=I
sin() cos()
For rational, S is a nite set of rays and for irrational,

a dense set of rays.
For linear systems, the realization of an inputoutput
map as a system on a state space is essentially the same in
continuous and discrete time, using the concept of Hankel
Bilinear Systems
matrix
cb
c Ab
cA b
..
.
..
.
..
.
c Ab c A2 b c A3 b

..
.
For discrete-time bilinear systems, realization theory

involves a different looking but highly analogous Hankel
matrix; Isidori (38), Fliess (39), and Tarn and Nonoyama
(40) introduced variations on this idea. The situation differs from linear systems in that the inputputput map is
u c
(u; k, 0), with the single output a row vector, or it
may be multiplied on the right by the initial state. Discretetime bilinear systems were then, like linear systems, used
as examples of adjoint systems in category theory, with the
Hankel matrix being the fundamental description. A category, loosely speaking, is a collection of objects and maps
between them; for instance, the category Lin has linear
spaces and linear maps. A machine (adjoint system) in a
category has input, state, and output objects; dynamics
(iterated maps on the state space) and input and output
maps; and Hankel matrices. The category of discrete sets
underlies automata theory; Lin leads to discrete-time linear system theory, etc., and discrete-time bilinear systems
occur naturally; see Arbib and Manes (41). Continuoustime dynamics does not t into category theory.
Transforming Nonlinear Systems to Bilinear Form. As an
application of BLS ideas to nonlinear control theory, consider the nonlinear system
z 1
z 2
= z1 + (z2 + z21 )u
= z2 + 3z21 + (z1 2z1 z2 2z31 )u
This was cooked by using the one-to-one coordinate transformation x1 = z1 , x2 = z2 + z21 , from the BLS x = Ax + uBx
where
A=
1
0
0
, B=
1
0
1
1
0
maps is surveyed in Reference 1. Volterra series are used

extensively in the identication of nonlinear systems, especially in biological work, because algorithms exist to evaluate the rst few terms purely from inputoutput data when
no state-space model is known. The series is
y(t)
A Note on Volterra Series. The connection between BLS

and the Volterra series representation of inputoutput
t
W0 (t) + n=1 0 0 n1
Wn (t, 1 , . . . , n )u(1 ) u(n )d1 dn
Treat our usual system x = Ax + u(t)Bx, as if it were inhomogeneous to get the integral equation

x(t) = eAt x(0) +
u(t1 )eA(tt1 ) Bx(t1 )dt1

0
then the Volterra kernels are easily observed, by iteration,

to be
= c eAt x(0)
W0 (t)
W1 (t, 1 )
= c eA(t1 ) BeA1 x(0)
W2 (t, 1 , 2 ) = c eA(t1 ) BeA(1 2 ) BeA2 x(0)
For bounded u, the series converges on any time interval
and represents the solution. The Volterra series has a nite
number of terms precisely when the systems Lie algebra
g is nilpotent, which means that all brackets with a sufciently high number of factors must vanish. Several approximation theorems for analytic nonlinear systems have
been based on this approach.
For an account of this theory in its general form for analytic systems, see Isidori (43); the book is also a source
for many other topics omitted here, such as Fliess functional expansions and continuous-time realization theory.
Also see NONLINEAR CONTROL SYSTEMS: ANALYTICAL METHODS.
Systems on Lie Groups; Quantum Systems. The transition
matrices for Eq. (14) satisfy
The Lie algebra of the BLS is sl(2, R). The matrices A,B are
the Jacobians at 0 of the nonlinear vector functions in the
rst system, so the BLS is also the classic linearization
of the nonlinear system, but what we have here is not an
approximation but a system equivalence. The Lie algebra
of nonlinear vector elds (see CONTROLLABILITY AND OBSERVABILITY, Section Nonlinear Finite-Dimensional Systems)
generated by our example is also sl(2, R).
It has been shown by Sedwick and Elliott (42) that given
a family F of real-analytic nonlinear vector elds on Rn that
vanish at 0, if F and the family F1 of their linear terms isomorphically generate a transitive Lie algebra, then a realanalytic coordinate transformation exists that transforms
F system to its equivalent BLS F1 , and that can be found
by solving linear partial differential equations. Of Lie algebras not on that list, the compact and the semisimple ones
also permit this linearization, but all others either fail to
have differentiable transformations or do not have any, as
far as is known; and nonlinear systems axe prone to have
innite-dimensional Lie algebras.
13
= A
+
k
Bi
(20)
i=1
with the initial condition

(0) = I. It is also worthwhile
to look at equation 20 as a control system on GL(n,R) or
on one of its Lie subgroups such as (if A and the B i are
skew-symmetric) SO(3), the group of rigid-body rotations.
Such control systems are also called bilinear and have
inspired much work such as References 20 and 26 and especially Reference 34. The controllability problem for systems on Lie groups is closely related to the study of matrix
semigroups; see Lawson (44) for a survey of this area.
Typical applications have been to the angular attitude control of spacecraft (satellites) and undersea vehicles. A new application in which electronic engineers have
been active is in quantum control; see QUANTUM SYSTEMS.
DAlessandro and Dahleh (45) and much subsequent work
by DAlessandro on quantum bits has made the study of
BLS on complex Lie groups an attractive subject of study.
BIBLIOGRAPHY
1. Bruni, C.; DiPillo, G.; Koch, G., Bilinear Systems: An Appealing
Class of Nearly Linear Systems in Theory and Application.
IEEE Trans. Automat. Contr. 1974, AC-19, pp 334348.
14
Bilinear Systems
2. Mohler, R. R., Bilinear Control Processes; Academic: New York,

1973.
3. Mohler, R. R., Nonlinear Systems: Volume 1, Dynamics and
Control, Volume II, Applications to Bilinear Control; PrenticeHall, Englewood Cliffs, NJ: 1991.
4. Sontag, E. D., Mathematical Control Theory; Springer-Verlag:
New York; 1990.
5. Golub, G. H.; Van Loan, C. F., Matrix Computations, 2nd ed.;
Baltimore, NJ, Johns Hopkins: 1989.
6. Ledzewicz, U.; Schattler,
H., Optimal Bang-Bang Controls for a

Two Compartment Model in Cancer Chemotherapy. J. Optim.
Theory Appl. 2002, 114, pp 609637.
7. Celikovsk
y,
S.; Vanec ek, A., Bilinear Systems and Chaos. Kybernetika (Prague) 1994, 30, pp 403424.
8. Chabour, R.; Sallet, G.; Vivalda, J. C., Stabilization of Nonlinear Systems: A Bilinear Approach. Math. Contr. Signals Syst.
1993, 6, pp 224246.
9. Luesink, R.; Nijmeijer, H., On the Stabilization of Bilinear
Systems via Constant Feedback. Linear Algebra Appl. 1989,
122124, pp 457474.
10. Kailath, T., Linear Systems; Prentice-Hall: Englewood Cliffs,
NJ, 1980.
11. Grasselli, O. M.; Isidori, A., Deterministic State Reconstruction and Reachability of Bilinear Control Processes;
Proc. Joint Automatic Control Conf. San Francisco, June
2225, 1977; IEEE: New York, pp 14231427.
12. Williamson, D., Observation of Bilinear Systems with Application to Biological Control. Automatica: J. IFAC 1977, 13, pp
243254.
13. Frelek, B. A.; Elliott, D. L., Optimal Observation for VariableStructure Systems; Proc. VI IFAC Congress, Boston, Mass.;
vol I, Paper 29.5, 1975.
14. Grasselli, O. M.; Isidori, A., An Existence Theorem for Observers of Bilinear Systems. IEEE Trans. Autom. Contr. 1981,
AC-26, pp 12991300;Erratum 1982, AC-27, 284.
15. Sen, P., On the Choice of Input for Observability in Bilinear
Systems. IEEE Trans. Autom. Contr. 1981, AC-26, pp 451
454.
16. Belinfante, J. G. F.; Kolman, B., A Survey of Lie Groups and
Lie Algebras with Applications and Computational Methods;
Philadelphia: SIAM, 1972.
17. Kucera, J., Solution in Large of Control Problem x = (A(1
u) + Bu)x. Czechoslovak Math. J. 1966, 16, pp 600623.
18. Boothby, W. M.; Wilson, E. N., Determination on the Transitivity of Bilinera System. SIAM J. Contr. Optim. 1979, 17, pp
212221.
19. Kramer, L., Twotransitive Lie groups. J. Reine Angew Math.
2003, 563, pp 83113.
20. Brockett, R. W., System Theory on Group Manifolds and Coset
Spaces. SIAM J. Contr. 1972, 10, pp 265284.
21. Hirschorn, R. M., Invertibility of Control Systems on Lie
Groups. SIAM J. Contr. Optim. 1977, 15, pp 10341049.
22. Elliott, D. L., A Consequence of Controllability. J. Diff. Equations 1971, 10, pp 364370.
23. Rink, R. E.; Mohler, R. R., Completely Controllable Bilinear
Systems. SIAM J. Contr. 1968, 6, pp 477486.
24. Cheng, G.-S. J.; Tarn, T.-J.; Elliott, D. L., Controllability of Bilinear Systems. In Variable Structure Systems with Applications to Economics and Biology;Roberti, A.;Mohler, R. R., Eds.;
Springer-Verlag: Berlin, 1975; pp 83100.
25. Jurdjevic, V.; Quinn, J., Controllability and Stability. J. Diff.
Equation 1978, 28, pp 381389.
26. Jurdjevic, V.; Kupka, I., Control Systems Subordinated to a

Group Action: Accessibility. J. Diff. Equations 1981, 39, pp
186211.
27. Jurdjevic, V.; Sallet, G., Controllability Properties of Afne
Systems. SIAM J. Contr. Optim. 1984, 22, pp 501508.
28. Boothby, W. M., Some Comments on Positive Orthant Controllability of Bilinear Systems. SIAM J. Contr. Optim. 1982, 20,
pp 634644.
29. Bacciotti, A., On the Positive Orthant Controllability of TwoDimensional Bilinear Systems. Syst. Contr. Lett. 1983, 3, pp
5355.
30. Sachkov, Y. L., On Positive Orthant Controllability of Bilinear
Systems in Small Codimensions. SIAM J. Contr. Optim. 1974,
35, pp 2935.
31. Agrachev, A. A.; Liberzon, D., Lie-Algebraic Stability Criteria for Switched Systems. SIAM J.Contr. Optim 2001, 40, pp
253259.
32. Bacciotti, A.; Boieri, P., A Characterization of Single-Input Planar Bilinear Systems which Admit a Smooth Stabilizer. Syst.
Contr. Lett. 1991, 16, pp 139144.
33. Oster, G., Bilinear Models in Ecology. In Recent Developments in Variable Structure Systems, Economics and Biology;Mohler, R. R.;Ruberti, A., Eds., Springer-Verlag: Berlin,
1978; pp 260271.
34. Jurdjevic, V., Geometric Control Theory; Cambridge University
Press: New York, 1996.
35. Sussmann, H. J., The Bang-Bang Problem for Certain Control Systems in GL (n,R). SIAM J. Contr. Optim. 1972, 10, pp
470476.
36. Sontag, E. D., A Chow Property for Sampled Bilinear Systems. In Analysis and Control of Nonlinear Systems;Byrnes,
C. I.;Martin, C. F.;Sacks, R., Eds.; North Holland: Amsterdam,
1988, pp 205211.
37. Swamy, K. N.; Tarn, T.-J., Deterministic and Stochastic Control of Discrete-Time Bilinear Systems. AutomaticaJ. IFAC
1979, 15, pp 677682.
38. Isidori, A., Direct Construction of Minimal Realizations from
Nonlinear Input-Output Maps. IEEE Trans. Autom. Contr.
1973, AC-18, pp 626631.
39. Fliess, M., Matrices de Hankel. J. Math. Pures Appl. 1974, 53,
pp 197224.
40. Tarn, T.-J.; Nonoyama, S., Realization of Discrete-Time Internally Bilinear Systems; Proc. IEEE Conf. Decision and Control; Clearwater, FL, 1977, pp 125133.
41. Arbib, M. A.; Manes, E. G., Foundations of System Theory: The
Hankel Matrix. J. Comp. Sys. Sci. 1980, 20, pp 330378.
42. Sedwick, J. L.; Elliott, D. L., Linearization of Analytic Vector
Fields in the Transitive Case. J. Diff. Equations 1977, 25, pp
377390.
43. Isidori, A., Nonlinear Control Systems, 3rd ed.; SpringerVerlag: London, 1995.
44. Lawson, J., Germetric Control and Lie Semigroup Theory. In
Differential Geometry and Control, (Proc. Symp. Pure Math.
Vol. 64);Ferreyra, G.;Gardner, R.;Hermes, H.;Sussmann, H.,
Eds.; Amer. Math. Soc.: Providence, RI, 1999.
45. DAlessandro, D.; Dahleh, M., Optimal Control of Two Level
Quantum Systems. IEEE Trans. Autom. Contr. 2001, 46, pp
866876.
Bilinear Systems
Further Reading
The following books and articles are listed more or less in the order
of the topics of this article.
Lipschutz, S., Schaums Outline of Linear Algebra, 2nd ed.;

McGraw-Hill: New York, 1991. (Fine self-study text for students and as a refresher; the accounts of the CayleyHamilton
theorem, the Jordan canonical form, and quadratic forms will
be helpful in studying BLS.)
Bellman, R., Introduction to Matrix Analysis SIAM: Philadelphia,
PA, 1995. (This is a reprint of the 1965 Second Edition, by popular demand, a true classic. Covers exponential matrices, positive matrices, and much more.)
Rossmann, W., Lie Groups: An Introduction Through Linear
Groups, ser., Oxford Grad. Texts Math., 5; Oxford University
Press: Oxford: 2005. (This books emphasis on matrix groups
makes it well suited for the study of bilnear control systems.)
Varadarajan, V. S., Lie Groups, Lie Algebras, and Their Representations (GTM 102); Springer-Verlag: New York: 1988. (Wellknown and respected graduate text.)
Jurdjevic, V.; Sussmann, H. J., Controllability on Lie Groups. J.
Diff. Equations 1972, 12, pp 313329.(A foundational paper on
modern nonlinear control. State space generalized to groups
and group manifolds.)
Jurdjevic, V., Geometric Control Theory; Cambridge University
Press: New York, 1996.
Eilliott, D. L., Bilinear Control Systems; Kluwer Academic Publishers: Dordrecht. In Press.
DAVID L. ELLIOTT
University of Maryland
15
CONTROL SYSTEM DESIGN, CONTINUOUS-TIME
281
as performance indexes in terms of state variables, then modern control approaches must be used.
The systems that may be designed by a conventional or
classical approach are usually limited to single-inputsingleoutput, linear time-invariant systems. The designer seeks to
satisfy all performance specifications by means of educated
trial-and-error repetition. After a system is designed, the designer checks to see if the designed system satisfies all the
performance specifications. If it does not, then he or she repeats the design process by adjusting parameter settings or
by changing the system configuration until the given specifications are met. Although the design is based on a trial-anderror procedure, the ingenuity and know-how of the designer
will play an important role in a successful design. An experienced designer may be able to design an acceptable system
without using many trials.
The primary objective of this article is to present procedures for the design and compensation of single-inputsingleoutput linear time-invariant control systems. Compensation
is the modification of the system dynamics to satisfy the given
specifications. The methods to the control system design and
compensation used in this article are the root-locus method
and frequency-response method. These methods are commonly called the classical or conventional methods of control
systems design. Note that in designing control systems by the
root-locus or frequency-response methods the final result is
not unique, because the best or optimal solution may not be
precisely defined if the time-domain specifications or frequency-domain specifications are given.
SYSTEM COMPENSATION

This article discusses a means of improving performance of
existing control systems and of designing new control systems
with satisfactory performance. The most common approach to
improving the performance of single-inputsingle-output control systems is to insert a suitable compensator in the system.
In this article we are concerned with the design of various
types of compensators.
Actual control systems are generally nonlinear. However,
if they can be approximated by linear mathematical models,
we may use one or more of the well-developed design methods. In a practical sense, the performance specifications given
to the particular system suggest which method to use. If the
performance specifications are given in terms of transient-response characteristics and/or frequency-domain performance
measures, then we have no choice but to use a conventional or
classical approach based on the root-locus and/or frequencyresponse methods. If the performance specifications are given
Setting the gain is the first step in adjusting the system for
satisfactory performance. In many practical cases, however,
the adjustment of the gain alone may not provide sufficient
alteration of the system behavior to meet the given specifications. As is frequently the case, increasing the gain value will
improve the steady-state behavior but will result in poor stability or even instability. It is then necessary to redesign the
system by modifying the structure or by incorporating additional devices or components to alter the overall behavior so
that the system will behave as desired. A device inserted into
the system for the purpose of satisfying the specifications is
called a compensator. The compensator compensates for deficit performance of the original system.
In discussing compensators, we frequently use such terminologies as lead network, lag network, and lag-lead network.
If a sinusoidal input ei is applied to the input of a network
and the steady-state output eo (which is also sinusoidal) has
a phase lead, then the network is called a lead network. (The
amount of phase lead angle is a function of the input frequency.) If the steady-state output eo has a phase lag, then
the network is called a lag network. In a lag-lead network,
both phase lag and phase lead occur in the output but in different frequency regions; phase lag occurs in the low-frequency region and phase lead occurs in the high-frequency
region. A compensator having a characteristic of a lead network, lag network, or lag-lead network is called a lead compensator, lag compensator, or lag-lead compensator.
In this article we specifically consider the design of lead
compensators, lag compensators, and lag-lead compensators.
282
In such design problems, we place a compensator in series

with the unalterable plant transfer function G(s) to obtain desirable behavior. The main problem then involves the judicious choice of the pole(s) and zero(s) of the compensator
Gc(s) to alter the root loci or frequency response so that the
performance specifications will be met.
In the actual design of a control system, whether to use an
electronic, pneumatic, or hydraulic compensator is a matter
that must be decided partially based on the nature of the controlled plant. For example, if the controlled plant involves
flammable fluid, then we have to choose pneumatic components (both a compensator and an actuator) to avoid the possibility of sparks. If, however, no fire hazard exists, then electronic compensators are most commonly used. In fact, we
often transform nonelectrical signals into electrical signals
because of the simplicity of transmission, increased accuracy,
increased reliability, ease of compensation, and the like.
Lead, Lag, and Lag-Lead Compensation
Lead compensation essentially yields an appreciable improvement in transient response and a small change in steadystate accuracy. It may accentuate high-frequency noise
effects. Lag compensation, on the other hand, yields an appreciable improvement in steady-state accuracy at the expense
of increasing the transient response time. Lag compensation
will suppress the effects of high-frequency noise signals. Laglead compensation combines the characteristics of both lead
compensation and lag compensation. The use of a lead or lag
compensator raises the order of the system by 1 (unless cancellation occurs between the zero of the compensator and a
pole of the uncompensated open-loop transfer function). The
use of a lag-lead compensator raises the order of the system
by 2 [unless cancellation occurs between zero(s) of the laglead compensator and pole(s) of the uncompensated open-loop
transfer function], which means that the system becomes
more complex and it is more difficult to control the transient
response behavior. The particular situation determines the
type of compensation to be used.
R(s)
C(s)
G(s)
H(s)
Figure 1. Control system.
rameter is usually the gain, but any other variable of the

open-loop transfer function may be used. Unless otherwise
stated, we shall assume that the gain of the open-loop transfer function is the parameter to be varied through all values,
from zero to infinity.
Angle and Magnitude Conditions
The basic idea behind the root-locus method is that the values
of s that make the transfer function around the loop equal 1
must satisfy the characteristic equation of the system. Consider the system shown in Fig. 1. The closed-loop transfer
function is
G(s)
C(s)
=
R(s)
1 + G(s)H(s)
The characteristic equation for this closed-loop system is obtained by setting the denominator of the right-hand side of
this last equation equal to zero. That is,
1 + G(s)H(s) = 0
or
G(s)H(s) = 1
(1)
Here we assume that G(s)H(s) is a ratio of polynomials in s.

Since G(s)H(s) is a complex quantity, Eq. (1) can be split into
two equations by equating the angles and magnitudes of both
sides, respectively, to obtain the following:
ROOT-LOCUS METHOD
The basic characteristic of the transient response of a closedloop system is closely related to the location of the closed-loop
poles. If the system has a variable loop gain, then the location
of the closed-loop poles depends on the value of the loop gain
chosen. It is important, therefore, that the designer know how
the closed-loop poles move in the s-plane as the loop gain is
varied.
From the design viewpoint, in some systems simple gain
adjustment may move the closed-loop poles to desired locations. Then the design problem may become the selection of
an appropriate gain value. If the gain adjustment alone does
not yield a desired result, addition of a compensator to the
system will become necessary.
A simple method for finding the roots of the characteristic
equation has been developed by W. R. Evans and used extensively in control engineering. This method, called the rootlocus method, is one in which the roots of the characteristic
equation are plotted for all values of a system parameter. The
roots corresponding to a particular value of this parameter
can then be located on the resulting graph. Note that the pa-
Angle condition:
&
G(s)H(s) = 180 (2k + 1)
(k = 0, 1, 2, . . . )
Magnitude condition:
|G(s)H(s)| = 1
The values of s that fulfill both the angle and magnitude conditions are the roots of the characteristic equation, or the
closed-loop poles. A plot of the points of the complex plane
satisfying the angle condition alone is the root locus. The
roots of the characteristic equation (the closed-loop poles) corresponding to a given value of the gain can be determined
from the magnitude condition.
FREQUENCY-RESPONSE METHOD
By the term frequency response, we mean the steady-state response of a system to a sinusoidal input. In frequencyresponse methods, we vary the frequency of the input signal
over a certain range and study the resulting response.
The Nyquist stability criterion enables us to investigate

both the absolute and relative stabilities of linear closed-loop
systems from a knowledge of their open-loop frequencyresponse characteristics. An advantage of the frequencyresponse approach is that frequency-response tests are, in
general, simple and can be made accurately by use of readily
available sinusoidal signal generators and precise measurement equipment. Often the transfer functions of complicated
components can be determined experimentally by frequencyresponse tests. In addition, the frequency-response approach
has the advantage that a system may be designed so that
the effects of undesirable noise are negligible and that such
analysis and design can be extended to certain nonlinear control systems.
Frequency-Response Approach to the
Design of Control Systems
It is important to note that in a control system design, transient-response performance is usually most important. In the
frequency-response approach, we specify the transient-response performance in an indirect manner. That is, the transient-response performance is specified in terms of the phase
margin, gain margin, and resonant peak magnitude (they
give a rough estimate of the system damping); the gain crossover frequency, resonant frequency, and bandwidth (they give
a rough estimate of the speed of transient response); and
static error constants (they give the steady-state accuracy).
Although the correlation between the transient response and
frequency response is indirect, the frequency-domain specifications can be conveniently met in the Bode diagram approach.
After the open loop has been designed by the frequencyresponse method, the closed-loop poles and zeros can be determined. The transient-response characteristics must be
checked to see whether the designed system satisfies the requirements in the time domain. If it does not, then the compensator must be modified and the analysis repeated until a
satisfactory result is obtained.
Design in the frequency domain is simple and straightforward. The frequency-response plot indicates clearly the
manner in which the system should be modified, although
the exact quantitative prediction of the transient-response
characteristics cannot be made. The frequency-response approach can be applied to systems or components whose dynamic characteristics are given in the form of frequency-response data. Note that because of difficulty in deriving the
equations governing certain components, such as pneumatic
and hydraulic components, the dynamic characteristics of
such components are usually determined experimentally
through frequency-response tests. The experimentally obtained frequency-response plots can be combined easily with
other such plots when the Bode diagram approach is used.
Note also that in dealing with high-frequency noise we find
that the frequency-response approach is more convenient
than other approaches.
A common approach to the design by use of the Bode diagram is that we first adjust the open-loop gain so that the
requirement on the steady-state accuracy is met. Then the
magnitude and phase curves of the uncompensated open loop
(with the open-loop gain just adjusted) is plotted. If the specifications on the phase margin and gain margin are not satis-
283
fied, then a suitable compensator that will reshape the openloop transfer function is determined. Finally, if there are any
other requirements to be met, we try to satisfy them, unless
some of them are contradictory to the other.
ROOT-LOCUS APPROACH TO THE
DESIGN OF CONTROL SYSTEMS
The root-locus approach to design is very powerful when the
specifications are given in terms of time-domain quantities,
such as the damping ratio and undamped natural frequency
of the desired dominant closed-loop poles, maximum overshoot, rise time, and settling time.
Consider a design problem in which the original system
either is unstable for all values of gain or is stable but has
undesirable transient-response characteristics. In such a
case, the reshaping of the root locus is necessary in the broad
neighborhood of the j axis and the origin in order that the
dominant closed-loop poles be at desired locations in the complex plane. This problem may be solved by inserting an appropriate lead compensator in cascade with the feedforward
transfer function.
If it is desired to improve steady-state performance (such
as to reduce the error in following the ramp input), insertion
of a lag compensator in the feedforward path will do the job.
If it is desired to improve both the transient-response and
steady-state performance, insertion of a lag-lead compensator
will accomplish the job. In what follows we discuss the lead,
lag, and lag-lead compensation techniques.
Lead Compensation
The procedure for designing a lead compensator for the system shown in Fig. 2 by the root-locus method may be stated
as follows:
1. From the performance specifications, determine the desired location for the dominant closed-loop poles.
2. By drawing the root-locus plot, ascertain whether or not
the gain adjustment alone can yield the desired closedloop poles. If not, calculate the angle deficiency . This
angle must be contributed by the lead compensator if
the new root locus is to pass through the desired locations for the dominant closed-loop poles.
3. Assume the lead compensator Gc(s) to be
1
s+
Ts + 1
T
, (0 < < 1)
= Kc
Gc (s) = Kc
1
Ts + 1
s+
T
(2)
where and T are determined from the angle deficiency. Kc is determined from the requirement of the
open-loop gain.
Gc(s)
G(s)
284
j
j3
j2
Closed-loop poles
j1
5 4
R(s)
4
s(s + 2)
C(s)
1
j1
j2
j3
(a)
Figure 3. (a) Control system; (b) root-locus plot.
4. If static error constants are not specified, determine the

location of the pole and zero of the lead compensator so
that the lead compensator will contribute the necessary
angle . If no other requirements are imposed on the
system, try to make the value of as large as possible.
A larger value of generally results in a larger value
of Kv, which is desirable. (If a particular static error constant is specified, it is generally simpler to use the frequency-response approach.)
5. Determine the open-loop gain of the compensated system from the magnitude condition.
Once a compensator has been designed, check to see
whether all performance specifications have been met. If the
compensated system does not meet the performance specifications, then repeat the design procedure by adjusting the
compensator pole and zero until all such specifications are
met. If a large static error constant is required, cascade a lag
network or alter the lead compensator to a lag-lead compensator.
Example 1. Consider the system shown in Fig. 3(a). The
feedforward transfer function is
G(s) =
4
s(s + 2)
The root-locus plot for this system is shown in Fig. 3(b). The
closed-loop poles are located at
s = 1 j 3
The damping ratio of the closed-loop poles is 0.5. The undamped natural frequency of the closed-loop poles is 2 rad/s.
The static velocity error constant is 2 s1.
It is desired to modify the closed-loop poles so that an undamped natural frequency n 4 rad/s is obtained, without
changing the value of the damping ratio, 0.5. In the present example, the desired locations of the closed-loop poles are
s = 2 j2 3
In some cases, after the root loci of the original system have
been obtained, the dominant closed-loop poles may be moved
(b)
to the desired location by simple gain adjustment. This is,

however, not the case for the present system. Therefore, we
shall insert a lead compensator in the feedforward path.
A general procedure for determining the lead compensator
is as follows: First, find the sum of the angles at the desired
location of one of the dominant closed-loop poles with the
open-loop poles and zeros of the original system, and determine the necessary angle to be added so that the total sum
of the angles is equal to 180 (2k 1). The lead compensator
must contribute this angle . (If the angle is quite large,
then two or more lead networks may be needed rather than a
single one.)
If the original system has the open-loop transfer function
G(s), then the compensated system will have the open-loop
transfer function
Gc (s)G(s) =
1
T
Kc
1
s+
T
s+
G(s)
where
1
s+
Ts + 1
T
= Kc
Gc (s) = Kc
,
1
Ts + 1
s + T
(0 < < 1)
Notice that there are many possible values for T and that
will yield the necessary angle contribution at the desired
closed-loop poles.
The next step is to determine the locations of the zero and
pole of the lead compensator. There are many possibilities for
the choice of such locations. (See the comments at the end of
this example problem.) In what follows, we shall introduce a
procedure to obtain the largest possible value for . (Note that
a larger value of will produce a larger value of Kv. In most
cases, the larger the Kv is, the better the system performance.) First, draw a horizontal line passing through point
P, the desired location for one of the dominant closed-loop
poles. This is shown as line PA in Fig. 4. Draw also a line
connecting point P and the origin. Bisect the angle between
the lines PA and PO, as shown in Fig. 4. Draw two lines PC
and PD that make angles /2 with the bisector PB. The
intersections of PC and PD with the negative real axis give
the necessary location for the pole and zero of the lead net-
j
P
A
1
T
1
T
285
where K 4Kc. The root-locus plot for the compensated system is shown in Fig. 5. The gain K is evaluated from the magnitude condition as follows: Referring to the root-locus plot for
the compensated system shown in Fig. 5, the gain K is evaluated from the magnitude condition as
"
"
" K(s + 2.9) "
"
"
=1
" s(s + 2)(s + 5.4) "
s=2+ j2 3
or
K = 18.7
Figure 4. Determination of the pole and zero of a lead network.
It follows that
Gc (s)G(s) =
work. The compensator thus designed will make point P a

point on the root locus of the compensated system. The openloop gain is determined by use of the magnitude condition.
In the present system, the angle of G(s) at the desired
closed-loop pole is
"
"
"
4
"
= 210
"
s(s + 2) "
"
The constant Kc of the lead compensator is

Kc =
18.7
= 4.68
4
Hence, Kc 2.51. The lead compensator, therefore, has the

transfer function
s=2+ j2 3
Gc (s) = 2.51
Thus, if we need to force the root locus to go through the desired closed-loop pole, the lead compensator must contribute
30 at this point. By following the foregoing design procedure, we determine the zero and pole of the lead compensator,
as shown in Fig. 5, to be
18.7(s + 2.9)
s(s + 2)(s + 5.4)
s + 2.9
0.345s + 1
= 4.68
0.185s + 1
s + 5.4
The static velocity error constant Kv is obtained from the expression
Kv = lim sGc (s)G(s)

s0
Zero at s = 2.9,
Pole at s = 5.4
s18.7(s + 2.9)
s(s + 2)(s + 5.4)
= 5.02 s1
= lim
s0
or
T=
1
= 0.345,
2.9
T =
1
= 0.185
5.4
Thus 0.537. The open-loop transfer function of the compensated system becomes
K(s + 2.9)
s + 2.9
4
=
Gc (s)G(s) = Kc
s + 5.4 s(s + 2)
s(s + 2)(s + 5.4)
j4
j2
15
6
2
4
5.4
2.9
s(s + 2)(s + 5.4) + 18.7(s + 2.9)
= (s + 2 + j2 3)(s + 2 j2 3)(s + 3.4)
The foregoing compensation method enables us to place

the dominant closed-loop poles at the desired points in the
complex plane. The third pole at s 3.4 is close to the
added zero at s 2.9. Therefore, the effect of this pole on
the transient response is relatively small. Since no restriction
has been imposed on the nondominant pole and no specification has been given concerning the value of the static velocity
error coefficient, we conclude that the present design is satisfactory.
15
Note that the third closed-loop pole of the designed system is

found by dividing the characteristic equation by the known
factors as follows:
j2
j4
Figure 5. Root-locus plot of the compensated system.
Comments. We may place the zero of the compensator at

s 2 and pole at s 4 so that the angle contribution of
the lead compensator is 30. (In this case the zero of the lead
compensator will cancel a pole of the plant, resulting in the
second-order system, rather than the third-order system as
we designed.) It can be seen that the Kv value in this case is
4 s1. Other combinations can be selected that will yield 30
286
phase lead. (For different combinations of a zero and pole of

the compensator that contribute 30, the value of will be
different and the value of Kv will also be different.) Although
a certain change in the value of Kv can be made by altering
the pole-zero location of the lead compensator, if a large increase in the value of Kv is desired, then we must alter the
lead compensator to a lag-lead compensator.
error constant without appreciably altering the original

root loci. (Note that the ratio of the value of gain required in the specifications and the gain found in the
uncompensated system is the required ratio between
the distance of the zero from the origin and that of the
pole from the origin.)
6. Draw a new root-locus plot for the compensated system.
Locate the desired dominant closed-loop poles on the
root locus. (If the angle contribution of the lag network
is very smallthat is, a few degreesthen the original
and new root loci are almost identical. Otherwise, there
will be a slight discrepancy between them. Then locate,
on the new root locus, the desired dominant closed-loop
poles based on the transient-response specifications.)
c of the compensator from the magnitude
7. Adjust gain K
condition so that the dominant closed-loop poles lie at
the desired location.
Lag Compensation
Consider the case where the system exhibits satisfactory
transient-response characteristics but unsatisfactory steadystate characteristics. Compensation in this case essentially
consists of increasing the open-loop gain without appreciably
changing the transient-response characteristics. This means
that the root locus in the neighborhood of the dominant
closed-loop poles should not be changed appreciably, but the
open-loop gain should be increased as much as needed. This
can be accomplished if a lag compensator is put in cascade
with the given feedforward transfer function.
The procedure for designing lag compensator for the system shown in Fig. 2 by the root-locus method may be stated
as follows:
1. Draw the root-locus plot for the uncompensated system
whose open-loop transfer function is G(s). Based on the
transient-response specifications, locate the dominant
closed-loop poles on the root locus.
2. Assume the transfer function of the lag compensator to
be
1
s+
Ts + 1
T
= Kc
Gc (s) = Kc
1
Ts + 1
s+
T
Example 2. Consider the system shown in Fig. 6(a). The

root-locus plot for the system is shown in Fig. 6(b). The closedloop transfer function becomes
1.06
C(s)
=
R(s)
s(s + 1)(s + 2) + 1.06
=
1.06
(s + 0.3307 j0.5864)(s + 0.3307 + j0.5864)(s + 2.3386)
The dominant closed-loop poles are

s = 0.3307 j0.5864
( > 1)
(3)
Then the open-loop transfer function of the compensated system becomes Gc(s)G(s).
3. Evaluate the particular static error constant specified
in the problem.
4. Determine the amount of increase in the static error
constant necessary to satisfy the specifications.
5. Determine the pole and zero of the lag compensator that
produce the necessary increase in the particular static
The damping ratio of the dominant closed-loop poles is

0.491. The undamped natural frequency of the dominant
closed-loop poles is 0.673 rad/s. The static velocity error constant is 0.53 s1.
It is desired to increase the static velocity error constant
Kv to about 5 s1 without appreciably changing the location of
the dominant closed-loop poles. To meet this specification, let
us insert a lag compensator as given by Eq. (3) in cascade
with the given feedforward transfer function. To increase the
static velocity error constant by a factor of about 10, let us
choose 10 and place the zero and pole of the lag compenj
j2
Closed-loop poles
j1
3
+
1.06
s(s + 1) (s + 2)
j1
j2
Figure 6. (a) Control system; (b) root-locus plot.
(a)
(b)
sator at s 0.05 and s 0.005, respectively. The transfer

function of the lag compensator becomes
Gc (s) = K c
s + 0.05
s + 0.005
s + 0.05
1.06
s + 0.005 s(s + 1)(s + 2)
K(s + 0.05)
=
s(s + 0.005)(s + 1)(s + 2)
1.5
Uncompensated system
1
Imag axis
K = 1.06K c
1.06
s(s + 1) (s + 2)
(a)
Gc (s)G(s) = K c
where
s +0.05
s + 0.005
Kc = 0.966
The angle contribution of this lag network near a dominant

closed-loop pole is about 4. Because this angle contribution
is not very small, there is a small change in the new root
locus near the desired dominant closed-loop poles.
The open-loop transfer function of the compensated system
then becomes
The block diagram of the compensated system is shown in

Fig. 7(a). The root-locus plot for the compensated system near
the dominant closed-loop poles is shown in Fig. 7(b), together
with the original root-locus plot. Figure 7(c) shows the rootlocus plot of the compensated system near the origin.
If the damping ratio of the new dominant closed-loop poles
is kept the same, then the poles are obtained from the new
root-locus plot as follows:
s1 = 0.31 + j0.55,
Kc
287
0.5
Original closed-loop pole

Compensated system
New
closed-loop
pole
0
0.5
1
1.5
2
3 2.5 2 1.5 1 0.5
Real axis
0.5
(b)
s2 = 0.31 j0.55
The open-loop gain K is

"
"
" s(s + 0.005)(s + 1)(s + 2) "
"
"
K="
"
s + 0.05
s=0.31+ j0.55
0.5
0.4
= 1.0235
0.3
c is determined as
Then the lag compensator gain K
Thus the transfer function of the designed lag compensator is

20s + 1
s + 0.05
= 9.656
Gc (s) = 0.9656
s + 0.005
200s + 1
Then the compensated system has the following open-loop
transfer function:
0.2
Imag axis
1.0235
K
=
= 0.9656
K c =
1.06
1.06
0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
1.0235(s + 0.05)
G1 (s) =
s(s + 0.005)(s + 1)(s + 2)
5.12(20s + 1)
=
s(200s + 1)(s + 1)(0.5s + 1)
The static velocity error constant Kv is
Kv = lim sG1 (s) = 5.12 s1
s0
In the compensated system, the static velocity error constant

has increased to 5.12 s1, or 5.12/0.53 9.66 times the origi-
0.4
0.2
0
0.2
Real axis
0.4
0.6
(c)
Figure 7. (a) Compensated system; (b) root-locus plots of the compensated system and the uncompensated system; (c) root-locus plot of
compensated system near the origin.
s3 = 2.326,
1.4
1.2
Outputs c1 and c2
nal value. (The steady-state error with ramp inputs has decreased to about 10% of that of the original system.) We have
essentially accomplished the design objective of increasing
the static velocity error constant to about 5 s1.
Note that, since the pole and zero of the lag compensator
are placed close together and are located very near the origin,
their effect on the shape of the original root loci has been
small. Except for the presence of a small closed root locus
near the origin, the root loci of the compensated and the uncompensated systems are very similar to each other. However, the value of the static velocity error constant of the compensated system is 9.66 times greater than that of the
uncompensated system.
The two other closed-loop poles for the compensated system are found as follows:
Compensated system
1
0.8
0.6
0.4
0.2
0
10
15
20
s4 = 0.0549
25
30
35
40
t (s)
The addition of the lag compensator increases the order of the

system from 3 to 4, adding one additional closed-loop pole
close to the zero of the lag compensator. (The added closedloop pole at s 0.0549 is close to the zero at s 0.05.)
Such a pair of a zero and pole creates a long tail of small
amplitude in the transient response, as we will see later in
the unit-step response. Since the pole at s 2.326 is very
far from the j axis compared with the dominant closed-loop
poles, the effect of this pole on the transient response is also
small. Therefore, we may consider the closed-loop poles at
s 0.31 j0.55 to be the dominant closed-loop poles.
The undamped natural frequency of the dominant closedloop poles of the compensated system is 0.631 rad/s. This
value is about 6% less than the original value, 0.673 rad/s.
This implies that the transient response of the compensated
system is slower than that of the original system. The response will take a longer time to settle down. The maximum
overshoot in the step response will increase in the compensated system. If such adverse effects can be tolerated, the lag
compensation as discussed here presents a satisfactory solution to the given design problem.
Figures 8(a) and 8(b) show the unit-step response curves
and unit-ramp response curves, respectively, of the compensated and uncompensated systems.
(a)
50
45
40
Outputs c1 and c2
288
35
30
Compensated system
25
20
15
10
5
0
10
15
20
25
30
35
40
45
50
t (s)
(b)
Figure 8. (a) Unit-step response curves for the compensated and uncompensated systems; (b) unit-ramp response curves for both
systems.
Lag-Lead Compensation
Lead compensation basically speeds up the response and increases the stability of the system. Lag compensation improves the steady-state accuracy of the system but reduces
the speed of the response.
If improvements in both transient response and steadystate response are desired, then both a lead compensator and
a lag compensator may be used simultaneously. Rather than
introducing both a lead compensator and a lag compensator
as separate elements, however, it is economical to use a single
lag-lead compensator.
Consider the system shown in Fig. 2. Assume that we use
the following lag-lead compensator:
1
1
s+
s+
(T1 s + 1)(T2 s + 1)
T1
T2

Gc (s) = Kc
= Kc

T1
1
s + 1 (T2 s + 1)
s+
s+
T1
T2
(4)
where 1. The design procedure may be stated as follows:

1. From the given performance specifications, determine
the desired location for the dominant closed-loop poles.
2. If the static velocity error constant Kv is specified, determine the value of constant Kc from the following equation:
Kv = lim sGc (s)G(s)

s0
= lim sKc G(s)

s0
3. To have the dominant closed-loop poles at the desired

location, calculate the angle contribution needed from
the phase lead portion of the lag-lead compensator.
The open-loop transfer function of the compensated system is
4
s(s + 0.5)
289
1
1
s+
4
T1
T2

Gc (s)G(s) = Kc
1
s(s + 0.5)
s+
s+
T1
T2
s+
Since the requirement on the static velocity error constant

Kv is 80 s1, we have
"
1
"
" s+
"
T
2
"
"
"s + 1
" 1 T
s0
1
s1 +
T1
s1 +
T1
Kc = 10
Noting that
"
"
"
"
G(s1 )"" = 1
"
"
1
T1
=
s1 +
T1
s1 +
5. Using the value of just determined, choose T2 so that
"
1
"
" s1 +
"
T
2
"
"
"s + 1
" 1 T
5 <
s0
Thus
"
"
"
"
"
"
"
"
"
"
"
"
"
4
"
s(s + 0.5) ""
"
"
is approximately unity, where s s1 is one of the dominant closed-loop poles. Determine the values of T1 and
from the magnitude and angle conditions:
"
"
"
"
"Kc
"
"
"
4
= 8Kc = 80
0.5
Kv = lim sGc (s)G(s) = lim Kc
4. For the lag-lead compensator, we later choose T2 sufficiently large so that
"
"
"
"
"+1
"
"
"
= 235
s=2.50+ j4.33
the time constant T1 and the value of are determined from

"
"
"
"
1 "
1 "
"
"
"
"s +
""
"
"s +
"
"
"
40
T1 "" ""
T1 "" 8
"
"
"
=1
=
"
"
""
"
"
" s + " s(s + 0.5) s=2.5+ j4.33 " s + " 4.77
"
"
T1 "
T1 "
"
"
1 "
"
s+
T1 ""
= 55
""
s+
T1 ""
s=2.5+ j4.33
Referring to Fig. 10, we can easily locate points A and B such

that
&
1
s1 +
T2
< 0
1
s1 +
T2
APB = 55 ,
PA
PB
4.77
8
j
The value of T2, the largest time constant of the laglead compensator, should not be too large to be physically realized.
j5
j4
j3
55
j2
Example 3. Consider the control system shown in Fig. 9. It

is desired to make the damping ratio of the dominant closedloop poles equal to 0.5 and to increase the undamped natural
frequency to 5 rad/s and the static velocity error constant to
80 s1. Design an appropriate compensator to meet all the design specifications.
Let us use a lag-lead compensator of the form given by Eq.
(4). The desired locations for the dominant closed-loop poles
are at
s = 2.50 j4.33
j1
B
10 9 8
A
7 6 5 4 3 2 1 0
1
j1
j2
j3
j4
Figure 10. Determination of the desired pole-zero location.
290
(Use a graphical approach or a trigonometric approach.) The

result is
AO = 2.38,
1.8
1.6
1.4
BO = 8.34
Outputs
1.2
or
1
= 0.420,
T1 =
2.38
Compensated
system
0.8
0.6
= 8.34T1 = 3.503
0.4
0.2
The phase lead portion of the lag-lead network thus becomes
10
s + 2.38
0.5
1.5
s + 8.34
2.5
t (s)
(a)
3.5
4.5
For the phase lag portion, we may choose

4
T2 = 10
3.5
3
Outputs
Then
1
1
= 0.0285
=
T2
3.503 10
Gc (s) = (10)
s + 8.34
s + 0.1
s + 0.0285
Compensated
system
The compensated system will have the open-loop transfer

function
Gc (s)G(s) =
2
1.5
Thus, the lag-lead compensator becomes
s + 2.38
2.5
0.5
0
0.5
1.5
2
t (s)
(b)
2.5
3.5
Figure 11. (a) Unit-step response curves for the compensated and
uncompensated systems; (b) unit-ramp response curves for both
systems.
40(s + 2.38)(s + 0.1)

(s + 8.34)(s + 0.0285)s(s + 0.5)
No cancellation occurs in this case, and the compensated system is of fourth order. Because the angle contribution of the
phase lag portion of the lag-lead network is quite small, the
dominant closed-loop poles are located very near the desired
location. In fact, the dominant closed-loop poles are located at
s 2.4539 j4.3099. The two other closed-loop poles are
located at
uncompensated systems are shown in Fig. 11(a). The unitramp response curves for both systems are depicted in Fig.
11(b).
FREQUENCY-RESPONSE APPROACH
TO THE DESIGN OF CONTROL SYSTEMS
Lead Compensation
s = 0.1003,
s = 3.8604
Since the closed-loop pole at s 0.1003 is very close to a

zero at s 0.1, they almost cancel each other. Thus, the
effect of this closed-loop pole is very small. The remaining
closed-loop pole (s 3.8604) does not quite cancel the zero
at s 2.4. The effect of this zero is to cause a larger overshoot in the step response than a similar system without such
a zero. The unit-step response curves of the compensated and
We shall first examine the frequency characteristics of the

lead compensator. Then we present a design technique for the
lead compensator by use of the Bode diagram.
Characteristics of Lead Compensators. Consider a lead compensator defined by
Kc
jT + 1
jT + 1
(0 < < 1)
Im
Gc(s)
291
G(s)
m
m
1
(1 )
2
=0
0
Re
1 (1 + )
2
Figure 12. Polar plot of a lead compensator ( jT 1)/( jT 1),
where 0 1.
Figure 12 shows the polar plot of this compensator with Kc

1. For a given value of , the angle between the positive real
axis and the tangent line drawn from the origin to the semicircle gives the maximum phase lead angle, m. We shall call
the frequency at the tangent point m. From Fig. 12 the phase
angle at m is m, where
1
1
sin m = 2 =
1+
1+
2
(5)
Equation (5) relates the maximum phase lead angle and the
value of .
Figure 13 shows the Bode diagram of a lead compensator
when Kc 1 and 0.1. The corner frequencies for the lead
compensator are 1/T and 1/(T) 10/T. By examining Fig. 13, we see that m is the geometric mean of the two
corner frequencies, or
log m =

1
log
1
1
+ log
T
T
1. Assume the following lead compensator:
1
s+
Ts + 1
T
= Kc
Gc (s) = Kc
1
Ts + 1
s+
T
(0 < < 1)
(7)
Define
Kc = K
Then
Gc (s) = K
Ts + 1
Ts + 1
Hence,
1
m =
aT
Lead Compensation Techniques Based on the Frequency-Response Approach. The primary function of the lead compensator is to reshape the frequency-response curve to provide sufficient phase-lead angle to offset the excessive phase lag
associated with the components of the fixed system.
Consider the system shown in Fig. 14. Assume that the
performance specifications are given in terms of phase margin, gain margin, static velocity error constants, and so on.
The procedure for designing a lead compensator by the frequency-response approach may be stated as follows:
(6)
As seen from Fig. 13, the lead compensator is basically a highpass filter. (The high frequencies are passed, but low frequencies are attenuated.)
Ts + 1
Ts + 1
G(s) =
KG(s)
Ts + 1
Ts + 1
Ts + 1
=
G (s)
Ts + 1 1
Gc (s)G(s) = K
where
G1 (s) = KG(s)
10
dB
0
10
20
90
m
0
0.1
T
1
T
10
T
in rad/s
10
T
100
T
Figure 13. Bode diagram of a lead compensator ( jT 1)/

( jT 1), where 0.1.
Determine gain K to satisfy the requirement on the

given static error constant.
2. Using the gain K thus determined, draw a Bode diagram of G1( j), the gain-adjusted but uncompensated
system. Evaluate the phase margin.
3. Determine the necessary phase lead angle to be added
to the system.
4. Determine the attenuation factor by use of Eq. (5).
Determine the frequency where the magnitude of the
uncompensated system G1( j) is equal to 20 log (1/
). Select this frequency as the new gain crossover
frequency. This frequency corresponds to m 1/
(T), and the maximum phase shift m occurs at this
frequency.
292
5. Determine the corner frequencies of the lead compensator as follows:
Zero of lead compensator:

Pole of lead compensator:
1
T
1
=
T
Example 4. Consider the system shown in Fig. 15. The

open-loop transfer function is
4
s(s + 2)
It is desired to design a compensator for the system so that

the static velocity error constant Kv is 20 s1, the phase margin is at least 50, and the gain margin is at least 10 dB.
We shall use a lead compensator of the form defined by Eq.
(7). Define
G1 (s) = KG(s) =
4K
s(s + 2)
where K Kc.
The first step in the design is to adjust the gain K to meet
the steady-state performance specification or to provide the
required static velocity error constant. Since this constant is
given as 20 s1, we obtain
Kv = lim sGc (s)G(s) = lim s

s0
s0
dB
0
20
40
0
90
7. Check the gain margin to be sure it is satisfactory. If

not, repeat the design process by modifying the polezero location of the compensator until a satisfactory result is obtained.
G(s) =
20
6. Using the value of K determined in step 1 and that of

determined in step 4, calculate constant Kc from
Kc =
40
Ts + 1
G (s)
Ts + 1 1
s4K
= 2K = 20
= lim
s0 s(s + 2)
17
180
1
810
in rad/s
Figure 16 shows the magnitude and phase angle curves of

G1( j). From this plot, the phase and gain margins of the
system are found to be 17 and dB, respectively. (A phase
margin of 17 implies that the system is quite oscillatory.
Thus, satisfying the specification on the steady state yields a
poor transient-response performance.) The specification calls
for a phase margin of at least 50. We thus find the additional
phase lead necessary to satisfy the relative stability requirement is 33. To achieve a phase margin of 50 without decreasing the value of K, the lead compensator must contribute
the required phase angle.
Noting that the addition of a lead compensator modifies
the magnitude curve in the Bode diagram, we realize that the
gain crossover frequency will be shifted to the right. We must
offset the increased phase lag of G1( j) due to this increase
in the gain crossover frequency. Considering the shift of the
gain crossover frequency, we may assume that m, the maximum phase lead required, is approximately 38. (This means
that 5 has been added to compensate for the shift in the gain
crossover frequency.)
Since
sin m =
With K 10, the compensated system will satisfy the steadystate requirement.
We shall next plot the Bode diagram of
20
40
=
j( j + 2)
j(0.5 j + 1)
G1 ( j) =
4
s(s + 2)
40 60 100
Figure 16. Bode diagram for G1( j) 10G( j) 40/[j( j 2)].
or
K = 10
20
1
1+
m 38 corresponds to 0.24. Once the attenuation factor

has been determined on the basis of the required phase lead
angle, the next step is to determine the corner frequencies
1/T and 1/(T) of the lead compensator. To do so, we
first note that the maximum phase lead angle m occurs at
the geometric mean of the two corner frequencies, or
1/(T). [See Eq. (6).] The amount of the modification in the
magnitude curve at 1/(T) due to the inclusion of the
term (Ts 1)/(Ts 1) is
"
"
" 1 + jT "
"
"
" 1 + jT "
=1/( aT )
"
"
1 "
"
" 1 + j "
"
1
""
= ""
" =
1
" 1 + j "
"
"
Note that
+
1
1
1
= 6.2 dB
=
=
0.49
0.24
41.7(s + 4.41)
s + 18.4
and G1( j) 6.2 dB corresponds to 9 rad/s. We shall

select this frequency to be the new gain crossover frequency
c. Noting that this frequency corresponds to 1/(T), or
c 1/(T), we obtain
Figure 18. Compensated system.
and
Gc (s)G(s) = 41.7
1
c
= = 18.4
T
The lead compensator thus determined is

s + 4.41
0.227s + 1
= Kc
s + 18.4
0.054s + 1
where the value of Kc is determined as

Kc =
10
K
=
= 41.7
0.24
Thus, the transfer function of the compensator becomes

Gc (s) = 41.7
0.227s + 1
s + 4.41
= 10
s + 18.4
0.054s + 1
Note that
Gc (s)
Gc (s)
G1 (s) =
10G(s) = Gc (s)G(s)
K
10
20
Gc
10
6.2 dB
4
s + 4.41
s + 18.4 s(s + 2)
The solid curves in Fig. 17 show the magnitude curve and

phase-angle curve for the compensated system. The lead compensator causes the gain crossover frequency to increase from
6.3 to 9 rad/s. The increase in this frequency means an increase in bandwidth. This implies an increase in the speed of
response. The phase and gain margins are seen to be approximately 50 and dB, respectively. The compensated system
shown in Fig. 18 therefore meets both the steady-state and
the relative-stability requirements.
Note that for type 1 systems, such as the system just considered, the value of the static velocity error constant Kv is
merely the value of the frequency corresponding to the intersection of the extension of the initial 20 dB/decade slope
line and the 0 dB line, as shown in Fig. 17.
Figures 19 and 20 show, respectively, the unit-step and
unit-ramp responses of both the compensated system and uncompensated system.
Lag Compensation
Characteristics of Lag Compensators. Consider the lag compensator given by Eq. (3). Figure 21 shows a polar plot of
the lag compensator. Figure 22 shows a Bode diagram of the
compensator, where Kc 1 and 10. The corner frequencies of the lag compensator are at 1/T and 1/(T).
As seen from Fig. 22, where the values of Kc and are set
equal to 1 and 10, respectively, the magnitude of the lag com-
40
dB
4
s(s + 2)
The magnitude curve and phase-angle curve for Gc( j)/10 are
shown in Fig. 17. The compensated system has the following
open-loop transfer function:
1
= c = 4.41
T
Gc (s) = Kc
293
Kv
GcG
20
1.4
Compensated system
1.2
G1 = 10G
1
Outputs
40
0
Gc
10
90
G1 = 10G
0.6
0.4
0.2
GcG
50
180
1
0.8
10
20
40 60 100
in rad/s
Figure 17. Bode diagram for the compensated system.
3
t (s)
Figure 19. Unit-step response curves of the compensated and uncompensated systems.
294
30
5
4.5
dB
10
3.5
Compensated system
3
Outputs
20
2.5
90
0.01
T
2
1.5
1
T
10
T
in rad/s
Figure 22. Bode diagram of a lag compensator ( jT 1)/( jT

1), with 10.
0.5
0
0.1
T
0.5
1.5
2.5
3.5
4.5
Define
t (s)
Figure 20. Unit-ramp response curves of the compensated and uncompensated systems.
Kc = K
Then
pensator becomes 10 (or 20 dB) at low frequencies and unity

(or 0 dB) at high frequencies. Thus, the lag compensator is
essentially a low-pass filter.
Lag Compensation Techniques Based on the Frequency-Response Approach. The primary function of a lag compensator
is to provide attenuation in the high-frequency range to give
a system sufficient phase margin. The phase lag characteristic is of no consequence in lag compensation.
The procedure for designing lag compensators for the system shown in Fig. 14 by the frequency-response approach
may be stated as follows:
1. Assume the following lag compensator:
1
s+
Ts + 1
T
= Kc
Gc (s) = Kc
1
Ts + 1
s+
T
( > 1)
Im
Kc
Kc
0
=0
Re
Figure 21. Polar plot of a lag compensator Kc( jT 1)/( jT 1).
Gc (s) = K
Ts + 1
Ts + 1
Ts + 1
Ts + 1
G(s) =
KG(s)
Ts + 1
Ts + 1
Ts + 1
=
G (s)
Ts + 1 1
Gc (s)G(s) = K
where
G1 (s) = KG(s)
Determine gain K to satisfy the requirement on the
given static error constant.
2. If the uncompensated system G1( j) KG( j) does not
satisfy the specifications on the phase and gain margins, then find the frequency point where the phase
angle of the open-loop transfer function is equal to
180 plus the required phase margin. The required
phase margin is the specified phase margin plus 5 to
12. (The addition of 5 to 12 compensates for the phase
lag of the lag compensator.) Choose this frequency as
the new gain crossover frequency.
3. To prevent detrimental effects of phase lag due to the
lag compensator, the pole and zero of the lag compensator must be located substantially lower than the new
gain crossover frequency. Therefore, choose the corner
frequency 1/T (corresponding to the zero of the lag
compensator) 1 octave to 1 decade below the new gain
crossover frequency. (If the time constants of the lag
compensator do not become too large, the corner frequency 1/T may be chosen 1 decade below the new
gain crossover frequency.)
4. Determine the attenuation necessary to bring the magnitude curve down to 0 dB at the new gain crossover
frequency. Noting that this attenuation is 20 log ,
determine the value of . Then the other corner frequency (corresponding to the pole of the lag compensator) is determined from 1/(T).
295
40
G1
20
5. Using the value of K determined in step 1 and that of

determined in step 4, calculate constant Kc from
dB
K
Kc =
GcG
0
20
Example 5. Consider the system shown in Fig. 23. The

open-loop transfer function is given by
40
0
1
G(s) =
s(s + 1)(0.5s + 1)
90
11 dB
Gc
Gc
G1
40
180
GcG
It is desired to compensate the system so that the static velocity error constant Kv is 5 s1, the phase margin is at least 40,
and the gain margin is at least 10 dB.
We shall use a lag compensator of the form
1
s+
Ts + 1
T
= Kc
Gc (s) = Kc
1
Ts + 1
s+
T
Kc = K
Define also
K
s(s 1)(0.5s + 1)
The first step in the design is to adjust the gain K to meet the
required static velocity error constant. Thus,
Kv = lim sGc (s)G(s) = lim s

s0
= lim
s0
s0
0.1
0.2 0.4 0.6 1
in rad/s
Figure 24. Bode diagrams for the uncompensated system, the compensator, and the compensated system. (G1: uncompensated system,
Gc: compensator, GcG: compensated system.)
( > 1)
Define
G1 (s) = KG(s) =
270
0.004 0.01 0.02 0.04
Ts + 1
G (s) = lim sG1 (s)
s0
Ts + 1 1
sK
=K=5
s(s + 1)(0.5s + 1)
With K 5, the compensated system satisfies the steadystate performance requirement.

We shall next plot the Bode diagram of
The magnitude curve and phase-angle curve of G1( j) are

shown in Fig. 24. From this plot, the phase margin is found
to be 20, which means that the system is unstable.
Noting that the addition of a lag compensator modifies the
phase curve of the Bode diagram, we must allow 5 to 12 to
the specified phase margin to compensate for the modification
of the phase curve. Since the frequency corresponding to a
phase margin of 40 is 0.7 rad/s, the new gain crossover frequency (of the compensated system) must be chosen near this
value. To avoid overly large time constants for the lag compensator, we shall choose the corner frequency 1/T
(which corresponds to the zero of the lag compensator) to be
0.1 rad/s. Since this corner frequency is not too far below the
new gain crossover frequency, the modification in the phase
curve may not be small. Hence, we add about 12 to the given
phase margin as an allowance to account for the lag angle
introduced by the lag compensator. The required phase margin is now 52. The phase angle of the uncompensated openloop transfer function is 128 at about 0.5 rad/s. So we
choose the new gain crossover frequency to be 0.5 rad/s. To
bring the magnitude curve down to 0 dB at this new gain
crossover frequency, the lag compensator must give the necessary attenuation, which in this case is 20 dB. Hence,
20 log
5
G1 ( j) =
j( j + 1)(0.5 j + 1)
1
= 20
or
= 10
+
s(s + 1) (0.5s + 1)
The other corner frequency 1(T), which corresponds

to the pole of the lag compensator, is then determined as
1
= 0.01 rad/s
T
296
Thus, the transfer function of the lag compensator is
20
18
1
s+
10s + 1
10
= Kc
Gc (s) = Kc (10)
1
100s + 1
s+
100
16
Since the gain K was determined to be 5 and was determined to be 10, we have
5
K
=
= 0.5
Kc =
10
12
10
8
6

Gc (s)G(s) =
Outputs
14
5(10s + 1)
s(100s + 1)(s + 1)(0.5s + 1)
Compensated system
The magnitude and phase-angle curves of Gc( j)G( j) are

also shown in Fig. 24.
The phase margin of the compensated system is about 40,
which is the required value. The gain margin is about 11 dB,
which is quite acceptable. The static velocity error constant is
5 s1, as required. The compensated system, therefore, satisfies the requirements on both the steady state and the relative stability.
Note that the new gain crossover frequency is decreased
from approximately 2 to 0.5 rad/s. This means that the bandwidth of the system is reduced.
Figures 25 and 26 show, respectively, the unit-step and
unit-ramp responses of the compensated and uncompensated
systems. (The uncompensated system is shown in Fig. 23.)
Lag-Lead Compensation
Lag-Lead Compensation Based on the Frequency-Response Approach. The design of a lag-lead compensator by the frequency-response approach is based on the combination of the
design techniques discussed under lead compensation and
lag compensation.
1.4
10
t (s)
12
14
16
18
20
Let us assume that the lag-lead compensator is of the following form:
1
1
s+
(T1 s + 1)(T2 s + 1)
T1
T2

Gc (s) = Kc
= Kc

T1
1
s + 1 (T2 s + 1)
s+
s+
T1
T2
(8)
s+
where 1. The phase lead portion of the lag-lead compensator (the portion involving T1) alters the frequency-response
curve by adding phase lead angle and increasing the phase
margin at the gain crossover frequency. The phase lag portion
(the portion involving T2) provides attenuation near and
above the gain crossover frequency and thereby allows an increase of gain at the low-frequency range to improve the
steady-state performance.
Figure 27 shows a Bode diagram of a lag-lead compensator
when Kc 1, 10, and T2 10T1. Notice that the magnitude curve has the value 0 dB at both low-frequency and highfrequency regions.
10
0
dB
Outputs
Figure 26. Unit-ramp response curves for the compensated and uncompensated systems.
Compensated system
1.2
0.8
10
20
30
0.6
90
0.4
0
0.2
0
10
15
20
25
30
35
40
t (s)
Figure 25. Unit-step response curves for the compensated and uncompensated systems.
90
0.001
T1
0.01
T1
0.1
T1
1
T1
10
T1
100
T1
in rad/s
Figure 27. Bode diagram of a lag-lead compensator given by Eq. (8)
with Kc 1, 10, and T2 10T1.
60
40
G
20
dB
20
16 dB
GcG
0
Gc
Once we choose the gain crossover frequency to be 1.5 rad/

s, we can determine the corner frequency of the phase lag
portion of the lag-lead compensator. Let us choose the corner
frequency 1/T2 (which corresponds to the zero of the
phase-lag portion of the compensator) to be 1 decade below
the new gain crossover frequency, or at 0.15 rad/s.
Recall that for the lead compensator the maximum phase
lead angle m is given by Eq. (5), where in Eq. (5) is 1/ in
the present case. By substituting 1/ in Eq. (5), we have
40
90
1
1
=
sin m =
1
+1
1+
0
Gc
90
50
180
270
0.01
297
GcG
32
0.04 0.1 0.2 0.4
4 6 10
Notice that 10 corresponds to m 54.9. Since we need

a 50 phase margin, we may choose 10. (Note that we will
be using several degrees less than the maximum angle, 54.9.)
Thus,
in rad/s
Figure 28. Bode diagrams for the uncompensated system, the compensator, and the compensated system. (G: uncompensated system,
Gc: compensator, GcG: compensated system.)
We shall illustrate the details of the procedure for designing a lag-lead compensator by an example.
= 10
Then the corner frequency 1/T2 (which corresponds to
the pole of the phase lag portion of the compensator) becomes
0.015 rad/s. The transfer function of the phase lag portion of the lag-lead compensator then becomes
s + 0.15
6.67s + 1
= 10
s + 0.015
66.7s + 1
Example 6. Consider the unity-feedback system whose

open-loop transfer function is
G(s) =
K
s(s + 1)(s + 2)
It is desired that the static velocity error constant be 10 s1,

the phase margin be 50, and the gain margin be 10 dB or
more.
Assume that we use the lag-lead compensator given by Eq.
(8). The open-loop transfer function of the compensated system is Gc(s)G(s). Since the gain K of the plant is adjustable,
let us assume that Kc 1. Then lims0 Gc(s) 1.
From the requirement on the static velocity error constant,
we obtain
The phase lead portion can be determined as follows: Since

the new gain crossover frequency is 1.5 rad/s, from Fig.
28, G( j1.5) is found to be 13 dB. Hence, if the lag-lead compensator contributes 13 dB at 1.5 rad/s, then the new
gain crossover frequency is as desired. From this requirement, it is possible to draw a straight line of slope 20 dB/
decade, passing through the point (13 dB, 1.5 rad/s). The
intersections of this line and the 0 dB line and 20 dB line
determine the corner frequencies. Thus, the corner frequencies for the lead portion are 0.7 rad/s and 7 rad/s.
Thus, the transfer function of the lead portion of the lag-lead
compensator becomes
s + 0.7
1
=
s+7
10
K
K
=
= 10
Kv = lim sGc (s)G(s) = lim sGc (s)
s0
s0
s(s + 1)(s + 2)
2
Hence,
K = 20
We shall next draw the Bode diagram of the uncompensated
system with K 20, as shown in Fig. 28. The phase margin
of the uncompensated system is found to be 32, which indicates that the incompensated system is unstable.
The next step in the design of a lag-lead compensator is to
choose a new gain crossover frequency. From the phase angle
curve for G( j), we notice that G( j) 180 at 1.5
rad/s. It is convenient to choose the new gain crossover frequency to be 1.5 rad/s so that the phase-lead angle required
at 1.5 rad/s is about 50, which is quite possible by use
of a single lag-lead compensator.
1.43s + 1
0.143s + 1
Combining the transfer functions of the lag and lead portions

of the compensator, we obtain the transfer function of the laglead compensator. Since we chose Kc 1, we have
Gc (s) =
s + 0.7
s+7
s + 0.15
s + 0.015

=
1.43s + 1
0.143s + 1
6.67s + 1
66.7s + 1
The magnitude and phase-angle curves of the lag-lead compensator just designed are shown in Fig. 28. The open-loop
transfer function of the compensated system is
(s + 0.7)(s + 0.15)20
(s + 7)(s + 0.015)s(s + 1)(s + 2)
10(1.43s + 1)(6.67s + 1)
=
s(0.143s + 1)(66.7s + 1)(s + 1)(0.5s + 1)
Gc (s)G(s) =
(9)
298
1.6
1.4
3.
1.2
Output
1
0.8
0.6
4.
0.4
0.2
0
0
10 12
t (s)
14 16 18 20
Figure 29. Unit-step response of the compensated system.
The magnitude and phase-angle curves of the system of Eq.

(9) are also shown in Fig. 28. The phase margin of the compensated system is 50, the gain margin is 16 dB, and the
static velocity error constant is 10 s1. All the requirements
are therefore met, and the design has been completed.
The unit-step response and unit-ramp response of the designed system are shown in Figs. 29 and 30, respectively.
COMPARISON OF LEAD, LAG, LAG-LEAD COMPENSATION
1. Lead compensation achieves the desired result through
the merits of its phase-lead contribution, whereas lag
compensation accomplishes the result through the merits of its attenuation property at high frequencies. (In
some design problems both lag compensation and lead
compensation may satisfy the specifications.)
2. Lead compensation is commonly used for improving stability margins. Lead compensation yields a higher gain
crossover frequency than is possible with lag compensation. The higher gain crossover frequency means larger
bandwidth. A large bandwidth means reduction in the
settling time. The bandwidth of a system with lead compensation is always greater than that with lag compensation. Therefore, if a large bandwidth or fast response
is desired, lead compensation should be employed. If,
however, noise signals are present, then a large bandwidth may not be desirable, since it makes the system
20
5.
6.
more susceptible to noise signals because of increase in

the high-frequency gain.
Lead compensation requires an additional increase in
gain to offset the attenuation inherent in the lead network. This means that lead compensation will require
a larger gain than that required by lag compensation. A
larger gain, in most cases, implies larger space, greater
weight, and higher cost.
Lag compensation reduces the system gain at higher
frequencies without reducing the system gain at lower
frequencies. Since the system bandwidth is reduced, the
system has a slower speed to respond. Because of the
reduced high-frequency gain, the total system gain can
be increased, and thereby low-frequency gain can be increased and the steady-state accuracy can be improved.
Also, any high-frequency noises involved in the system
can be attenuated.
If both fast responses and good static accuracy are desired, a lag-lead compensator may be employed. By use
of the lag-lead compensator, the low-frequency gain can
be increased (which means an improvement in steadystate accuracy), while at the same time the system
bandwidth and stability margins can be increased.
Although a large number of practical compensation
tasks can be accomplished with lead, lag, or lag-lead
compensators, for complicated systems, simple compensation by use of these compensators may not yield satisfactory results. Then different compensators having different pole-zero configurations must be employed.
MULTI-DEGREES-OF-FREEDOM CONTROL
In the classical design approaches presented in this article,
we design control systems such that the response to the reference input is satisfactory. If the control system is subjected to
other inputs, such as disturbance input and noise input, it is
not possible to design the system such that the responses to
the disturbance input and noise input are also satisfactory, in
addition to the primary requirement that the response to the
reference input is satisfactory. This is because the systems we
considered so far simply do not have the freedom to satisfy
requirements on the responses to disturbances and noises.
If we wish to design high-performance control systems in
the presence of disturbances and sensor noises, we must
change the configuration of the control system. This means
that we must provide additional degrees of freedom to the
control system to handle additional requirements.
18
16
D(s)
Output
14
12
R(s)
10
Gc(s)
U(s) +
+
Y(s)
Gp(s)
6
4
B(s)
2
0
0
10 12 14 16 18 20
t (s)
Figure 30. Unit-ramp response of the compensated system.
N(s)
+
Figure 31. One-degree-of-freedom control system.
299
D(s)
D(s)
Gc2(s)
R(s)
+
Gc1(s)
U(s) +
+
Y(s)
Gp(s)
R(s)
Gc1(s)
+ U(s) +
+
Y(s)
Gp(s)
Gc2(s)
B(s)
B(s)
B(s)
N(s)
N(s)
Figure 32. Two-degrees-of-freedom control system.
Figure 33. Two-degrees-of-freedom control system.
In what follows we first discuss the single-degree-of-freedom control systems and then discuss the two-degrees-offreedom control systems. Finally, we present an example of
three-degrees-of-freedom control systems that can satisfy the
requirements on the responses to the reference input, disturbance input, and noise input.
Among the three closed-loop transfer functions Gyr, Gyn, and

Gyd, if one of them is given, the remaining two are fixed. This
means that the system shown in Fig. 31 is a one-degree-offreedom system.
Single-Degree-of-Freedom Control
Two-Degrees-of-Freedom Control
Consider the system shown in Fig. 31, where the system is

subjected to the disturbance input D(s) and noise input N(s).
Gc(s) is the transfer function of the controller and Gp(s) is the
transfer function of the plant. We assume that Gp(s) is fixed
and unalterable.
For this system, three closed-loop transfer functions
Y(s)/R(s) Gyr, Y(s)/D(s) Gyd, and Y(s)/N(s) Gyn may be
derived. They are
Next consider the system shown in Fig. 32, where Gp(s) is the
transfer function of the plant and is assumed to be fixed and
unalterable. For this system, closed-loop transfer functions
Gyr, Gyn, and Gyd are given, respectively, by
Gyr
Gyd
Gyn
Gc Gp
Y (s)
=
=
R(s)
1 + Gc Gp
Gp
Y (s)
=
=
D(s)
1 + Gc Gp
Gc Gp
Y (s)
=
=
N(s)
1 + Gc Gp
Gyn =
Hence, we have
Gyr = Gc1 Gyd
Gp Gyd
Gp
Gyd Gp
Gp
R(s)
Gc1
Gyn =
Gyd Gp
Gp
In this case, if Gyd is given, then Gyn is fixed, but Gyr is not
fixed, because Gc1 is independent of Gyd. Thus, two closed-loop
transfer functions among three closed-loop transfer functions
Gyr, Gyd, and Gyn are independent. Hence, this system is a twodegrees-of-freedom control system.
Similarly, the system shown in Fig. 33 is also a twodegrees-of-freedom control system, because for this system
D(s)
Gc2
Gyd
Gyn
[In deriving Y(s)/R(s), we assumed D(s) 0 and N(s) 0.

Similar comments apply to the derivations of Y(s)/D(s) and
Y(s)/N(s).] The degrees of freedom of the control system refers
to how many of these closed-loop transfer functions are independent. In the present case, we have
Gyr =
Gc1 Gp
Y (s)
=
R(s)
1 + (Gc1 + Gc2 )Gp
Gp
Y (s)
=
=
D(s)
1 + (Gc1 + Gc2 )Gp
Gc1 + Gc2 )Gp
Y (s)
=
=
N(s)
1 + (Gc1 + Gc2 )Gp
Gyr =
Gc3
Y(s)
G1
G2
N(s)
+
Figure 34. Three-degrees-of-freedom system.
300
CONVEX OPTIMIZATION
Gc1 Gp
Y (s)
Gc2 Gp
=
+
R(s)
1 + Gc1 Gp
1 + Gc1 Gp
Gp
Y (s)
=
=
D(s)
1 + Gc1 Gp
Gc1 Gp
Y (s)
=
=
N(s)
1 + Gc1 Gp
Gyr =
Gyd
Gyn
K. Ogata, Modern Control Engineering, 3rd ed., Upper Saddle River,

NJ: Prentice Hall, 1997.
K. Ogata, System Dynamics, 3rd ed., Upper Saddle River, NJ: Prentice Hall, 1998.
K. Ogata, Solving Control Engineering Problems with MATLAB, Upper Saddle River, NJ: Prentice Hall, 1994.
C. L. Phillips and R. D. Harbor, Feedback Control Systems, Upper
Saddle River, NJ: Prentice Hall, 1988.
Hence,
Gyr = Gc2 Gyd +

Gyn =
KATSUHIKO OGATA
Gp Gyd
University of Minnesota
Gp
Gyd Gp
Gp
CONTROL SYSTEM SYNTHESIS. See DELAY SYSTEMS;

SERVOMECHANISMS.
Clearly, if Gyd is given, then Gyn is fixed, but Gyr is not fixed
because Gc2 is independent of Gyd.
Three-Degrees-of-Freedom Control
In the control system shown in Fig. 34, the transfer functions
Gc1, Gc2, and Gc3 are controllers, and transfer functions G1 and
G2 are plant transfer functions that are unalterable. It can be
shown that this control system is a three-degrees-of-freedom
system. If a system has this configuration, then it is possible
to design three controllers by use of the root-locus method
and/or frequency-response method (or other methods) such
that the responses to all three inputs are acceptable.
CONCLUDING COMMENTS
This article has presented easy-to-understand procedures for
designing lead compensators, lag compensators, and lag-lead
compensators by use of the root-locus method or frequencyresponse method. The systems are limited to single-input
single-output, linear time-invariant control systems. For such
systems various design methods are available in addition to
the root-locus method and frequency-response method. Interested readers are referred to specialized books on control systems, as listed in the Reading List.
Toward the end of this article we included discussions on
multi-degrees-of-freedom control systems for the informed
specialist.
Most of the materials presented in this article were taken,
with permission, from Katsuhiko Ogata, Modern Control Engineering 3/e, 1997. Prentice Hall, Upper Saddle River,
New Jersey.
Reading List
H. W. Bode, Network Analysis and Feedback Design, New York: Van
Nostrand Reinhold, 1945.
R. C. Dorf, Modern Control Systems, 6th ed., Reading, MA: AddisonWesley, 1992.
W. R. Evans, Graphical analysis of control systems, AIEE Trans. Part
II, 67: 547551, 1948.
W. R. Evans, Control system synthesis by root locus method, AIEE
Trans. Part II, 69: 6669, 1950.
W. R. Evans, The use of zeros and poles for frequency response or
transient response, ASME Trans., 76: 11351144, 1954.
G. F. Franklin, J. D. Powell, and A. Emami-Naeini, Feedback Control
of Dynamic Systems, Reading, MA: Addison-Wesley, 1986.
B. C. Kuo, Automatic Control Systems, 6th ed., Upper Saddle River,
NJ: Prentice Hall, 1991.
CONVERSION, THERMIONIC. See THERMIONIC CONVERSION.
CONVERTERS. See ANALOG-TO-DIGITAL CONVERSION.

CONVERTERS, AC-AC. See AC-AC POWER CONVERTERS.
CONVERTERS, AC-DC. See AC-DC POWER CONVERTERS.
CONVERTERS, DC-AC. See DC-AC POWER CONVERTERS.
CONVERTER TO BOOST VOLTAGE. See SYNCHRONOUS CONVERTER TO BOOST BATTERY VOLTAGE.
DISCRETE TIME SYSTEMS DESIGN METHODS
643
out the extensive calculations required for their design. These

advances in implementation and design capability can be obtained at low cost because of the widespread availability of
inexpensive and powerful digital computers and their related devices.
The focus of discussion is the modern, state space-based
design of linear discrete-time control systems with an appreciation for classical viewpoints and methods. To begin, we
present an overview of the classical approach to discrete-time
tracking system design. The key concepts, involving specification of transient and steady state response requirements,
are also much a part of the modern approach.
Two important classes of control systems are the tracking
system and the regulator. A tracking system is one in which
the plant outputs are controlled so that they become and remain nearly equal to a set of externally applied reference signals. In a regulator, the objective is to bring the systemtracking outputs near zero in an acceptable manner, often in
the face of disturbances. Thus regulation is a special case of
tracking, in which the externally applied reference signals
are zero.
In the classical approach, whether in the discrete-time domain or the continuous-time domain, the designer begins with
a lower-order controller and raises the controller order as necessary to meet the feedback system performance requirements. The digital controller parameters are chosen to give
feedback system pole locations that result in acceptable zeroinput (transient) response. At the same time, requirements
are placed on the overall systems zero-state response components for representative discrete-time reference inputs, such
as steps or ramps.
In general, tracking control system design has two basic
concerns:
1. Obtaining acceptable zero-input response, that due to
initial conditions
2. Obtaining acceptable zero-state system response to reference inputs
In addition, if the plant to be controlled is continuous-time
and the controller is discrete-time, a third concern is:
3. Obtaining acceptable between-sample response of the
continuous-time plant

This article discusses fundamental concepts in discrete-time
control system design. The rapid advancements in digital system technology have radically altered the boundaries of control system design options. Currently, it is routinely practicable to design very complicated digital controllers and to carry
Using the superposition theorem, the zero-input response,

and the individual zero-state response contributions of each
input can be dealt with separately. The first concern of
tracking system design is met by selecting a controller that
places all of the overall system poles at desired locations inside the unit circle on the complex plane. Having designed a
feedback structure to achieve the desired character of zeroinput response, additional design freedom can then be used
to obtain good tracking of reference inputs.
The first two concerns of discrete-time tracking system design are the subject of this chapter. The third concern, however, is beyond our scope but is covered thoroughly in Ref. 1.
CLASSICAL CONTROL SYSTEM DESIGN METHODS
The tools of classical linear discrete-time control system design, which parallel the tools for continuous-time systems, are
644
R(z)
Compensator
Gc(z)
Plant
Gp(z)
Y(z)
TE (z) =
the z-transform, stability testing, root locus, and frequency

response methods.
The standard classical control system design problem is to
determine the transfer function Gc(z) of a compensator that
results in a feedback tracking system with prescribed performance requirements. The basic system configuration for this
problem is shown in Fig. 1. There are many variations on this
basic theme, including situations where the system structure
is more involved, where there is a feedback transmittance
H(z), and where there are disturbance inputs to be considered. Usually, these disturbances are undesirable inputs that
the plant should not track.
The character of a systems zero-input response is determined by its pole locations, so the first concern of tracking
system design is met by choosing a compensator Gc(z) that
results in acceptable pole locations for the overall transfer
function:
T (z) =
(2)
Similar to continuous-time systems, there are three reference

inputs for which steady-state errors are commonly defined.
They are the step (position), ramp (velocity), and parabolic
(acceleration) inputs. The step input has the form
r(kT ) = A u(kT )
or, in the z domain,
R(z) =
Az
z1
The ramp input is given by
R(z) =
ATz
(z 1)2
and the parabolic input is given by

r(kT ) =
R(z) =
E(z) = R(z) Y (z)H(z)

but
Y (z) = KE(z)G(z)
E(z)
G(z)
z1
(1)
If n is nonnegative, the system is said to be type n.

The error between the input and the output of the system
is
z1
1
A(kT )2 u(kT )
2
or
KG(z)H(z) =
(1 z1 )R(z)
1 + KG(z)H(z)
or
Root locus is an important design tool because, with it, the

effects on closed-loop system pole location of varying design
parameters are quickly and easily visualized.
The second concern of tracking system design is obtaining
acceptable closed-loop zero-state response to reference inputs.
For the discrete-time system shown in Fig. 2, the open-loop
transfer function may be expressed as
limit e(k) = limit (1 z1 )E(z) = limit
r(kT ) = AkTu(kT )
Gc (z)Gp (z)
Y (z)
=
R(z)
1 + Gc (z)Gp (z)
K(z + 1 )(z + 2 ) . . . (z + l )
(z 1)n (z + 1 )(z + 2 ) . . . (z + m )
KN(z)
=
(z 1)n D(z)
1
E(z)
=
R(z)
1 + KG(z)H(z)
Assuming that all the closed-loop poles of the system are inside the unit circle on the z plane, the steady-state error to a
power-of-time input is given by the final value theorem:
Figure 1. Cascade compensation of a unity feedback system.
R(z)
then
Y(z)
T 2 Az(z + 1)
2 (z 1)3
Table 1 summarizes steady-state errors using Eqs. (1) and (2)

for various system types for power-of-time inputs.
There are two basic ways of approaching classical discretetime control. In the sampled data approach, discrete-time signals are represented by continuous-time impulse trains so
that all signals in a plant and controller model are continuous-time signals. This was appealing in the early days of digital control when digital concepts were new and most designers had backgrounds that were solidly in continuous-time
control. Currently, however, there is little to recommend this
complexity. In the conventional approach, which is used here,
discrete-time signals are represented as sequences of
numbers.
Root Locus Design Methods
H(z)
Figure 2. A discrete-time control system.
We now present an overview of classical discrete-time control

system design using an example. Similar to continuous-time
systems, a root locus plot consists of a polezero plot of the
open-loop transfer function of a feedback system, upon which
645
Table 1. Steady-State Errors to Power-of-Time Inputs

Steady-State Error
to Step Input
Az
R(z)
z1
System
Type
Steady-State Error
to Ramp Input
ATz
R(z)
(z 1)2
Steady-State Error to
Parabolic Input
ATz
R(z)
(z 1)2
A
0
1K
N(1)
D(1)
AT
N(1)
K
D(1)
AT 2
N(1)
K
D(1)
is superimposed the locus of the poles of the closed-loop transfer function as some parameter is varied. For the configuration shown in Fig. 2 where the constant gain K is the parameter of interest, the overall transfer function of this system is
T (z) =
function G(s) relates the applied voltage to the drive motor

armature and the tape speed at the recording and playback
heads. The delay term accounts for the propagation of speed
changes along the tape over the distance of physical separation of the tape drive mechanism and the recording and playback heads. The pole term in G(s) represents the dynamics of
the motor and tape drive capstan. Tape position is sensed by
a recorded signal on the tape itself.
It is desired to design a digital controller that results in
zero steady-state error to any step change in desired tape position. Also, the system should have a zero-input (or transient) response that decays to no more than 10% of any initial
value within a 1/30 s interval, which is the video frame rate.
The sampling period of the controller is chosen to be T 1/
120 s in order to synchronize the tape motion control with the
1/60 s field rate (each frame consists of two fields of the recorded video). As shown in Fig. 3(b), the diagram of Fig. 3(a)
has been rearranged to emphasize the discrete-time input
KG(z)
1 + KG(z)H(z)
and the poles of the overall system are the roots of

1 + KG(z)H(z) = 0
which depend on the parameter K. The rules for constructing
the root locus of discrete-time systems are identical to the
rules for plotting the root locus of continuous-time systems.
The root locus plot, however, must be interpreted relative to
the z plane.
Consider the block diagram of the commercial broadcast
videotape-positioning system shown in Fig. 3(a). The transfer
R(z)
Tape motion
dynamics
Digital controller
Desired
tape
position
E(z)
Gc(z)
G(s) =
D/A
with
S/H
(1/120)s
40e
s + 40
Positionvelocity
relation
Tape
position
1
s
Position
sensor
A/D
(a)
Gp(z)
R(z) +
E(z)
Gc(z)
D/A
with
S/H
Gp(z)
(b)
1
s
P(s)
P(z)
A/D
Figure 3. Videotape-positioning system. (a) Block diagram. (b) Relation between discrete-time signals.
646
R(z) and the discrete-time samples P(z) of the tape position.

The open-loop transfer function of the system is

1 e(1/120)s 40e(1/120)s 1
G(z) = Z
s
s + 40
s

1
1/40
(1/120)s (1/120)s 1/40
+ 2+
= Z [1 e
]e
s
s
s + 40

z/40
z/120
z/40
= (1 z1 )z1
+
+
z1
(z 1)2
z 0.72
0.00133(z + 0.75)
=
z(z 1)(z 0.72)

R(z)
E(z)
Gp(z) =
0.00133 (z + 0.75)
z (z 1) (z 0.72)
Gc(z) = K
P(z)
(a)
Im
Unit
circle
K = 1000
The position error signal, in terms of the compensators ztransfer function Gc(z), is given by

Gc (z)Gp (z)
R(z)
1 + Gc (z)Gp (z)

E(z) = R(z) Y (z) = 1
95
K=
1000 K=100
1
=
R(z)
1 + Gc (z)Gp (z)
0.00133
Re
0.75
10
0.72
For a unit step input sequence we have

E(z) =
1
1 + Gc (z)Gp (z)
z
z+1

Circle of
radius
0.56
Assuming that the feedback system is stable, we obtain

limit e(k) = limit (1 z1 )E(z) = limit
k
z1
z1
(b)
1
1 + Gc (z)Gp (z)
Provided that the compensator does not have a zero at z 1,

the system type is 1 and, therefore according to Table 1, the
steady-state error to a step input is zero. For the feedback
system transient response to decay at least by a factor 1/10
within 1/30 s, the desired closed loop poles must be located
such that a decay of at least this amount occurs every 1/120
s steps. This implies that the closed-loop poles must lie within
a radius c of the origin on the z plane, where
c4 = 0.1,
Im
Unit
circle
0.00133
0.75
0.72
Re
c = 0.56
Similar to continuous-time systems, one usually begins with

the simplest compensator consisting of only a gain K. The
feedback system is stable for 0 K 95; but as shown in
Fig. 4, this compensator is inadequate because there are always poles at distances from the origin greater than the required c 0.56 regardless of the value of K. As shown in Fig.
5(a), another compensator with z-transfer function
Figure 4. Constant-gain compensator. (a) Block diagram. (b) Root

locus for positive K. (c) Root locus for negative gain.
the feedback system z-transfer function is
K(z 0.72)
Gc (z) =
z
T (z) =
which cancels the plant pole at z 0.72 is tried. The root

locus plot for this system is shown in Fig. 5(b). For K 90,
the design is close to meeting the requirements, but it is not
quite good enough. However, if the compensator pole is moved
from the origin to the left as shown in Fig. 6, the root locus is
pulled to the left and the performance requirements are met.
For the compensator with z-transfer function
150(z 0.72)
Gc (z) =
z + 0.4
(c)
(3)
0.2(z + 0.75)
Gc (z)Gp (z)
= 3
1 + Gc (z)Gp (z)
z 0.6z2 0.2z + 0.15
As expected, the steady-state error to a step input is zero:

limit
z1
z3 0.6z2 0.4z
=0
z3 0.6z2 0.2z + 0.15
The steady-state error to a unit ramp input is
1
(z2 + 0.4z)
1
120
=
limit 3
z1 z 0.6z2 0.2z + 0.15
30
R(z)
+
Gp(z) =
0.00133 (z + 0.75)
z (z 1) (z 0.72)
Gc(z) =
K(z 0.72)
z
E(z)
647
Im
P(z)
Circle of
radius 0.56
Unit circle
(a)
a = 0.2
Im
Circle of
radius
0.56
K=1000
a = 0.4
K
K
0.75
K
a = 0.6
370
a = 0.6
Unit circle
90
0.00133
Re
K 90
370
Figure 7. Root locus plot as a function of the compensator pole location.
(b)
Figure 5. Compensator with zero at z 0.72 and pole at z 0. (a)

Block diagram. (b) Root locus for positive K.
For a compensator with a z-transfer function of the form we

obtain
Gc (z) =
150(z 0.72)
z+a
The feedback system has the z-transfer function:

T (z) =
1
Re
K = 1000
0.2(z + 0.75)
Gc (z)Gp (z)
= 3
1 + Gc (z)Gp (z)
z z2 + 0.2z + 0.15 + a(z2 z)
A root locus plot in terms of positive a is shown in Fig. 7, from

which it is seen that choices of a between 0.4 and 0.5 give a
controller that meets the performance requirements.
Classical discrete-time control system design is an iterative process just like its continuous-time counterpart. Increasingly complicated controllers are tried until both the steadystate error and transient performance requirements are met.
Root locus is an important tool because it easily indicates
qualitative closed-loop system pole locations as a function of
a parameter. Once feasible controllers are selected, root locus
plots are refined to show quantitative results.
Frequency Domain Methods
R(z)
E(z)
Gp(z) =
0.00133 (z + 0.75)
z (z 1) (z 0.72)
Gc(z) =
K(z 0.72)
z + 0.4
P(z)
(a)
Gc (z)Gp (z) =
Im
Circle of
radius 0.56
Frequency response characterizations of systems have long

been popular because of the ease and practicality of steadystate sinusoidal response methods. Furthermore, frequency
response methods do not require explicit knowledge of system
transfer function models.
For the videotape-positioning system, the open loop ztransfer function, which includes the compensator given by
Eq. (3), is
(150)(0.00133)(z + 0.75)
z(z + 0.4)(z 1)
K = 1000
Unit circle
K = 1000
K = 150
0.75 0.4
K = 150
0.00133
Re
(b)
Figure 6. Compensator with zero at z 0.72 and pole at z 0.4.
(a) Block diagram. (b) Root locus for positive K.
Substituting z ejT, we obtain

Gc (e jT )Gp (e jT ) =
0.1995(e jT + 0.75)
+ 0.4)(e jT 1)
jT (e jT
(4)
which has the frequency response plots shown in Fig. 8. At

the phase crossover frequency (114.2 rad/s) the gain margin is
about 11.5 dB, and at the gain crossover frequency (30 rad/s)
the phase margin is about 66.5s.
For ease of generating frequency response plots and to
gain greater insight into the design process, the frequency domain methods such as Nyquist, Bode, Nichols, and so on, for
discrete-time systems can be developed using the w transform. This is because in the w plane the wealth of tools and
648
6. Finally, transform the controller Gc(w) to Gc(z) according to the mapping
Gain (dB)
50
50 1
10
100
101
102
103
102
103
Frequency (rad/s)
Phase (deg)
0
90
Phase margin
180
270
360
101
w=
Gain margin
100
101
Frequency (rad/s)
Figure 8. Frequency response plots of the videotape-positioning

system.
techniques that were developed for continuous-time systems

are directly applicable to discrete-time systems as well. The
w transform is given by
w=
z1
,
z+1
z=
w+1
1w
which is a bilinear transformation between the w plane and

the z plane.
The general procedure for analyzing and designing discrete-time systems using the w transform is summarized as
follows:
z1
z+1
Control system design for discrete-time systems using Bode,

Nyquist, or Nichols methods can be found in Refs. 2 and 3.
Frequency response methods are most useful in developing
models from experimental data, in verifying the performance
of a system designed by other methods, and in dealing with
those systems and situations in which rational transfer function models are not adequate.
The extension of the classical single-input/single-output
control system design methods to the design of complicated
feedback structures involving many loops, each of which
might include a compensator, is not easy. Put another way,
modern control systems require the design of compensators
having multiple inputs and multiple outputs. Design is iterative, and it can involve considerable trial and error. Therefore
when there are many design variables, it is important to deal
efficiently with those design decisions that need not be iterative. The powerful methods of state space offer insights about
what is possible and what is not. They also provide an excellent framework for general methods of approaching and accomplishing design objectives.
EIGENVALUE PLACEMENT WITH STATE FEEDBACK
Consider a linear, step-invariant nth-order system described
by the state equations
x (k + 1) = Ax
Ax(k) + Bu
Bu(k)
where x(k) is an n vector, and u(k) is an r vector. When the
state x(k) of this system is available and is used for feedback,
the input vector u(k) is given the form
u (k) = Ex
Ex(k) + (k)
1. Replace each z in the open-loop transfer function

G(z)H(z) with
z=
w+1
1w
to obtain G(w)H(w).
2. Substitute w j into G(w)H(w) and generate frequency response plots in terms of the real frequency ,
such as Nyquist, Bode, Nichols, and so on. The w plane
can be thought of as if it were the s plane.
3. Determine the stability margins, crossover frequencies,
bandwidth, closed-loop frequency response, or any other
desired frequency response characteristics.
4. If it is necessary, design a compensator Gc(w) to satisfy
the frequency domain performance requirements.
where (k) is a vector of external inputs as shown in Fig. 9,

and E is a gain matrix. The state equation for the plant with
feedback becomes
A + BE
x (k + 1) = (A
BE)xx (k) + B
B(k)
If the plant is completely controllable, the eigenvalues of the
feedback system, those of A BE, can be placed at any desired locations selected by the designer by appropriately
choosing the feedback gain matrix E. This is to say that the
Plant
(k)
u(k)
+
x(k + 1) = Ax(k) + Bu(k)
5. Convert critical frequencies in the w plane to frequencies in the z domain according to

E
2
= tan1
T
Figure 9. State feedback.
designer can freely choose the character of the overall systems transient performance. When the plant state vector is
not available for feedback, as is usually the case, an observer
can be designed to estimate the state vector. As we shall see
later, the observer state estimate can be used for feedback in
place of the state itself.
649
Letting
Ev i = i
we obtain
v i = (iI A )1 B i
Eigenvalue Placement for Single-Input Systems

If a single-input plant is in controllable canonical form, finding the feedback gains for arbitrary eigenvalue placement is
especially simple because, in that form, each element of the
feedback gain vector determines one coefficient of the feedback systems characteristic equation. In general, however,
the plant is not in controllable form. One way for calculating
the state feedback gain for eigenvalue placement for plants
that are not in controllable canonical form is to transform the
plant to controllable form, calculate the state feedback gain
for the transformed system, and then transform the state
feedback gain of the transformed system back to the original
system (see Ref. 1).
There are a number of other methods for finding the state
feedback gain vector of single-input plants. Two of these
methods are summarized below. Additional ones can be found
in Refs. 1 and 48. The state feedback gain vector is given by
Ackermanns formula:
A)
e = j n M 1
c c (A
(5)
where jn is the transpose of the nth-unit coordinate vector

j n = [0
...
v 1v 2 . . . v n ] = EV = = [ 1 2 n ]
E [v
and the desired feedback gain matrix is
V 1
E = V
For the previous example, choosing 1 1 and 2 1, we

obtain
e = [1.5

1 x1 (k)
1
+
u(k)
0 x2 (k)
2
If the plant for eigenvalue placement has multiple inputs and

if it is completely controllable from one of the inputs, then
that one input alone can be used for feedback. If the plant is
not completely controllable from a single input, a single input
can usually be distributed to the multiple ones in such a way
that the plant is completely controllable from the single input.
For example, for the system
x1 (k + 1)
0.5
0
1
x1 (k)
0.5 2 x2 (k)
x2 (k + 1) = 0
x3 (k + 1)
x3 (k)
1
1
0

1
0
u1 (k)
+ 0 2
u2 (k)
1
1
c (z) = z(z + 0.5) = z2 + 0.5z

and
A ) = A 2 + 0.5A
A
c (A
we let
u1 (k) = 3(k)
Using Ackermanns formula, the state feedback gain vector is
= [1.5
1]
1
2
1
2
1
3
3

1
1
+ 0.5
3
3
1
0

and
u2 (k) = (k)
0]
Yet another method for calculating E is as follows. If i is an

eigenvalue of (A BE), then there exists an eigenvector vi
such that
A + BE
v i = i v i
(A
BE)v
0]
which is the same result obtained using Ackermanns

formula.
The results of the above development can be extended to
situations where A BE is required to have repeated eigenvalues as discussed in Ref. 1.
it is desired to place the feedback system eigenvalues at z

0, 0.5. Then,
e = [0
(6)
Eigenvalue Placement with Multiple Inputs
1]
Mc is the controllability matrix of the system, and c(A) is the

desired characteristic equation with the matrix A substituted
for the variable z.
For example, for the completely controllable system

x1 (k + 1)
1
=
x2 (k + 1)
3
If i is not an eigenvalue of A, the inverse matrix exists. If

the eigenvalues i are distinct, the eigenvectors are linearly
independent. Choosing the i to give n linearly independent
eigenvectors, we obtain
Thus

x1 (k + 1)
0.5

x2 (k + 1) = 0
x3 (k + 1)
1
0
0.5
1
1
3
x1 (k)

2 x2 (k) + 2 (k)
x3 (k)
0
2
650
which is a controllable single input system. If the desired eigenvalues are located at z1 0.1, z2 0.15, and z3 0.1,
Ackermanns formula gives
e = [0.152
0.0223
0.2807]
and hence the feedback gain matrix for the multiple input
system is

E=
0.4559
0.1520
0.0669
0.0223

0.8420
0.2807
Equation (6) can also be applied to multiple input systems.

Continuing with the example, if for each eigenvector, we
choose 1 3 and 2 1, then
6.5871
v 1 = 4.5506 ,
0.3652
6.9304
v 2 = 4.8442 ,
0.5744
5.5076
v 3 = 3.4772
0.3046
and therefore

3
E=
1
3
1

3
0.4559
1
V =
1
0.1520
0.0669
0.0223

0.8420
0.2807
which agrees with the previous results.

Eigenvalue Placement with Output Feedback
It is the measurement vector of a plant, not the state vector,
that is available for feedback. We now consider what eigenvalue placement can be performed with output feedback
alone. With enough linearly independent outputs, the plant
state can be recovered from the outputs and inputs and the
state feedback results applied. With a single plant input and
only a few outputs, the designers options for placing feedback
system eigenvalues could be (and often are) severely limited.
Multiple plant inputs can also be used to advantage for eigenvalue placement with output feedback, but it still may not be
possible to achieve an acceptable design. If the nth-order
plant with state and output equations
will place the feedback system eigenvalues arbitrarily, provided that the matrix I EC1D is nonsingular. If it is singular, a small change in the feedback gain matrix E which corresponds to small changes in the desired feedback system
eigenvalue locations eliminates the singularity. If the nth-order system has more than n outputs, only n of these can be
linearly independent, so excess linearly dependent output
equations can simply be ignored when recovering a systems
state from its output. To improve a feedback systems reliability and its performance in the presence of noise, one may wish
instead to combine linearly dependent outputs with other outputs rather than ignore them.
When the nth-order plant does not have n linearly independent measurement outputs, it still might be possible to
select a feedback matrix E in

u (k) = E y (k) Du
Du(k) + (k) = ECx
ECx(k) + (k)
to place all of the feedback system eigenvalues, those of (A
BEC), acceptably. Generally, however, output feedback alone
does not allow arbitrary feedback system eigenvalue placement.
Pole Placement with Feedback Compensation
We now present another viewpoint for placing the feedback
system poles using a transfer function approach. Although
our discussion is limited to single-input and single-output
plants, the results can be generalized to the case of plants
with multiple inputs and multiple outputs. Similar to output
feedback, pole placement with feedback compensation assumes that the measurement outputs of a plant, not the state
vector, are available for feedback.
For an nth-order, linear, step-invariant, discrete-time system described by the transfer function G(z), arbitrary pole
placement of the feedback system can be accomplished with
an mth-order feedback compensator as shown in Fig. 10.
Let the numerator and denominator polynomials on G(z)
be Np(z) and Dp(z), respectively. Also, let the numerator and
denominator of the compensator transfer function H(z) be
Nc(z) and Dc(z), respectively. Then, the overall transfer function of the system is
T (z) =
x (k + 1) = Ax
Ax(k) + Bu
Bu(k)
Np (z)Dc (z)
P(z)
G(z)
=
=
1 + G(z)H(z)
Dp (z)Dc (z) + Np (z)Nc (z)
Q(z)
y (k) = Cx
Cx(k) + Du
Du(k)
has n linearly independent outputs, that is, if the output coupling matrix C has n linearly independent rows, then the
plant state can be recovered from the plant inputs and the
measurement outputs:
x (k) = C 1 {yy (k) Du
Du(k)}
The output feedback
u (k) = Ex
Ex(k) + (k) = EC 1 {yy (k) Du
Du(k)} + (k)
Plant with feedback T(z)

Plant
P(z)
G(z) =
Np(z)
Y(z)
Dp(z)
Feedback
compensator
Nc(z)
H(z) =
Dc(z)
or
u (k) = (II + EC 1D )1EC 1y (k) + (II + EC 1D )1 (k)
Figure 10. Pole placement with feedback compensation.
which has closed-loop zeros in P(z) that are those of the plant,
in Np(z), together with zeros that are the poles of the feedback
compensator, in Dc(z).
For a desired set of poles of T(z), given with an unknown
multiplicative constant by the polynomial Q(z), we obtain
Dp (z)Dc (z) + Np (z)Nc (z) = Q(z)
(7)
The desired polynomial Q(z) has the form

Q(z) = 0 (zn+m + n+m1zn+m1 + + 1 z + 0 )
where the s are known coefficients but the 0 is unknown.
In general, for a solution to exist there must be at least as
many unknowns as equations:
n + m + 1 2m + 2
m n1
(8)
where n is the order of the plant and m is the order of the

compensator. Equation (8) states that the order of the feedback controller is at least one less than the order of the plant.
If the plant transfer function has coprime numerator and denominator polynomials (that is, plant polezero cancellations
have been made), then a solution is guaranteed to exist.
For example, consider the second-order plant
Np (z)
(z + 1)(z + 0.5)
=
z(z 1)
Dp (z)
(9)
According to Eq. (8), a first-order feedback compensator of the

form
H(z) =
1 z + 2
Nc (z)
=
z + 3
Dc (z)
places the three closed-loop poles of the feedback system at

any desired location in the z plane by appropriate choice of
1, 2, and 3. Let the desired poles of the plant with feedback
be at z 0.1. Then,
Q(z) = 0 (z 0.1)3 = 0 (z3 0.3z2 + 0.03z 0.001)
(10)
In terms of the compensator coefficients, the characteristic

equation of the feedback system is
Dp (z)Dc (z) + Np (z)Nc (z)

= z(z 1)(z + 3 ) + (z + 1)(z + 0.5)(1z + 2 )
= (1 + 1)z3 + (1.51 + 2 + 3 1)z2
+ (0.51 + 1.52 3 )z + 0.52
(11)
Equating coefficients in Eqs. (10) and (11) and solving for the
unknowns gives
0 = 1.325,
1 = 0.325,
2 = 0.00265,
Therefore, the compensator

H(z) =
0.325z 0.00265
z + 0.1185
will place the closed-loop poles where desired.

As far as feedback system pole placement is concerned, a
feedback compensator of order n 1 (where n is the order of
the plant) can always be designed. It is possible, however,
that a lower-order feedback controller may give acceptable
feedback pole locations even though those locations are constrained and not completely arbitrary. This is the thrust of
classical control system design, in which the increasingly
higher-order controllers are tested until satisfactory results
are obtained.
For the plant given by Eq. (9), for example, a zeroth-order
feedback controller of the form
H(z) = K
or
G(z) =
651
3 = 0.1185
gives overall closed-loop poles at z 0.1428 and z 0.5 for

K 1/6, which might be an adequate pole placement design.
QUADRATIC OPTIMAL CONTROL
We have shown in the previous section that provided the
plant is completely controllable, a feedback gain matrix E can
always be determined so that all of the eigenvalues of the
feedback system can be placed arbitrarily. It can be easily
shown that for a single-input plant, the feedback gain vector
is unique. For multiple-input plants, however, there are many
feedback gain matrices that lead to the same set of feedback
eigenvalues. The process of selecting an optimum feedback
gain matrix from among the many possible gain matrices is
the subject of this section.
The approach of selecting the optimal gain matrix is
termed optimal regulation, in which the plant feedback is chosen to minimize a scalar performance measure that weights
the control input and the error from zero of the plant state at
each step.
Principle of Optimality
The discrete-time, linear-quadratic, optimal control problem
is to find the inputs u(0), u(1), . . ., u(N 1) to the plant
with linear state equations
x (k + 1) A (k)xx (k) + B (k)u
u(k)
such that a scalar quadratic performance measure (or cost
function)
P (N)xx (N) +
J = x (N)P
N1
Q (k)xx (k) + u (k)R

R (k)u
u(k)]
[xx (k)Q
k=0
is minimized. The matrix P(N), the matrices Q(0), Q(1), . . .,

Q(N 1), and the matrices R(0), R(1), . . ., R(N 1) are
each taken to be symmetric because each defines a quadratic
form. Each is assumed to be positive semidefinite, which
means that the contribution to J by each of the individual
terms is never negative.
The solution to the linear-quadratic optimal control problem is obtained by applying the principle of optimality, a tech-
652
Table 2. Procedure for Backward-in-Time Calculation of Optimal

Quadratic Regulator Gains
For the plant
x(k 1) A(k)x(k) B(k)u(k)
with state feedback
u(k) E(k)x(k)
and performance measure
J x(N)P(N)x(N)
[x (i)Q(i)x(i) u (i)R(i)u(i)]
N1
i0
begin with i 1 and the known P(N)

1. E(N i) [B(N i)P(N 1 i)B(N i) R(N i)]1
B(N i)P(N 1 i)A(N i)
2. P(N i) [A(N i) B(N i)E(N i)]P(N 1 i)
[A(N i) B(N i)E(N i)]
E(N i)R(N i)E(N i) Q(N i)
3. Increment i and repeat steps 1, 2, and 3 until E(0) and (if desired) P(0)
have been calculated.
The minimum performance measure is
min
J x(0)P(0)x(0)
u(0), . . ., u(N 1)
nique developed by Richard Bellman in the 1950s in connection with his invention of dynamic programming. To apply
the principle of optimality, one begins at the next-to-last step
N 1 and finds the last input u(N 1) that minimizes the
cost of control from step N 1 to step N, J(N 1, N), as a
function of the beginning state for that step, x(N 1). Then
the input u(N 2) is found that minimizes J(N 2, N) when
u(N 1) is as previously determined. One proceeds in this
manner finding one control vector at a time, from the last to
the first, as a function of the systems state. This results in a
recursive calculation of the optimal feedback gains for the linear-quadratic regulator as given in Table 2. Beginning with
known N, P(N), Q, and R, the last feedback gain matrix
E(N 1) is calculated. Using E(N 1), the matrix P(N 1)
is computed. Then all of the indices are stepped backward one
step and, with P(N 1), the feedback gain matrix E(N 2)
is calculated. Using E(N 2), we can calculate P(N 2). The
cycle is continued until E(0) is found. A formidable amount of
algebraic computation is required; the user should therefore
have digital computer aid for all but the lowest-order
problems.
ing at N and proceeding backward is always the same independent of the value of N. The procedure summarized in Table 3 for calculating the optimal regulator gain can be easily
adapted to the steady-state regulator gain by replacing steps
5, 6, and 7 with the following single step 5:
5. Form
1
1
B W 11W 1
E = [B
21 B + R ] B W 11W 21 A
which gives a steady-state gain matrix.

STEP-INVARIANT DISCRETE-TIME OBSERVER DESIGN
In 1964, David Luenberger of Stanford University put forth
the idea of observers, systems that recursively estimate the
state of other systems. It was soon realized that observers
offer a powerful, unified framework for feedback control system design.
Full-Order Observers
Closed-Form Solution for Optimal Gain

When the matrices A, B, Q, and R are constants, it is possible
to generate an analytical expression for the optimal gain from
which the numerical value can be calculated for any point of
time. A closed-form solution for the optimal gain E(N 1) is
summarized in Table 3. Derivation of this procedure along
with detailed numerical examples can be found in Ref. 1.
Steady-State Regulation
For a completely controllable, step-invariant plant and constant cost weighting matrices Q and R, the optimum feedback
gains, from E(N) backward, are not changed if the final step
N is changed. This is to say that the sequence of gains start-
When the plant state vector is not entirely available for feedback, as is usually the case, the state is estimated with an
observer, and the estimated state can be used in place of the
actual state for feedback (see Refs. 9, 10).
For an nth-order step-invariant discrete-time plant,
x (k + 1) = Ax
Ax(k) + Bu
Bu(k)
y (k) = Cx
Cx(k) + Du
Du(k)
(12)
another nth-order system, driven by the inputs and outputs

of the plant, of the form
(k + 1) = F (k) + Gy
Gy(k) + Hu
Hu(k)
(13)
Table 3. Procedure for Calculation of Optimal

Regulator Gains
or
A GC
x (k) (k) = (A
GC)k [xx (0) (0)] = F k [xx (0) (0)]
For the nth-order plant

x(k 1) Ax(k) Bu(k)
then the system in Eq. (13) is a full-order state observer of

the plant in Eq. (12), if the matrix G can be chosen so that
all the eigenvalues of F A GC are inside the unit circle
in the complex plane. The observer error, then, approaches
zero with step regardless of the initial values of x(0) and
(0). That is, the observer state (k) will approach the plant
state x(k). The full-order observer relations are summarized
in Table 4. If all n of the observer eigenvalues (eigenvalues
of F) are selected to be zero, then the characteristic equation
of F is
with state feedback

u(k) E(k)x(k)
and performance measure
J x(N)P(N)x(N)
[x (i)Qx(i) u (i)Ru(i)]
N1
i0
begin with i 1 and the known P(N)

1. Form the matrix
H
A QA1BR1B
A1BR1B
QA1
A1
n = 0
2. Find the eigenvalues and the corresponding eigenvectors of H

3. Generate the matrix W from eigenvectors such that
W1HW D
and since every matrix satisfies its own characteristic equation, then
0
1
Fn = 0
where is the diagonal matrix of eigenvalues outside the unit

circle on the z plane
At the nth step, the error between the plant state and the
observer state is given by
4. Partition W into four n n submatrices as

W
W11
W21
653
x (n) (n) = F n [xx (0) (0)]
W12
W22
so that
5. Form
x (n) = (n)
G(i) i[P(N)W22 W12]1[W11 P(N)W21]i

6. Form
P(N i) [W11 W12G(i)][W21 W22G(i)]1
7. Form
E(N i) [BP(N 1 i)B R]1BP(N 1 i)A
where
P(N 1 i) [W11 W12G(i 1)][W21 W22G(i 1)]1
is termed a full-order state observer of the plant, provided that

the error between the plant state and the observer state,
x (k + 1) (k + 1) = Ax
Ax(k) + Bu
Bu(k) F (k) Gy
Gy(k) Hu
Hu(k)
= Ax
Ax(k) + Bu
Bu(k) F (k) GCx
GCx(k)
and the observer state equals the plant state. Such an observer is termed deadbeat. In subsequent steps, the observer
state continues to equal the plant state.
There are several methods for calculating the observer
gain matrix g for single-output plants. Similar to the situation with state feedback, if a single-output plant is in observable canonical form, finding the elements of the observer gain
vector g for arbitrary eigenvalue placement is simple, because
each element of the observer gain vector determines one coefficient of the observer characteristic equation. Usually, however, the plant is not in observable canonical form. One way
of designing an observer for a completely observable singleoutput plant that is not in observable form is to change the
Table 4. Full-Order State Observer Relations

Plant Model
GDu
GDu(k) Hu
Hu(k)
x(k 1) Ax(k) Bu(k)

y(k) Cx(k) Du(k)
A GC
= (A
GC)xx (k) F (k)
B GD H )u
u (k)
+ (B
Observer
is governed by an autonomous equation. When F and G are

chosen as
F = A GC
(14)
H = B GD
(15)
so that the error satisfies

A GC
x (k + 1) (k + 1) = (A
GC)[xx (k) (k)]
(k 1) F(k) Gy(k) Hu(k)

where
F A GC
H B GD
Observer Error
x(k 1) k 1) F[x(k) (k)]
x(k) (k) F k[x(0) (0)]
654
plant to the observable form, design an observer in that form,

and then convert back to the original system realization.
Another method of observer design for single-output plants
that does not require transforming the system to observable
canonical form is to use Ackermanns formula. Provided that
(A, c) is completely observable, the eigenvalues of F A
gc can be placed arbitrarily by choice of g according to Ackermanns formula:
A )M
M 1
g = 0 (A
0 jn
Reduced-Order State Observers

If a completely observable nth-order plant has m linearly independent outputs, a reduced-order state observer, of order
n m, having an output that observes the plant state can
be constructed.
For the plant described by Eq. (12), when an observers
state
(k + 1) = F (k) + Gy
Gy(k) + Hu
Hu(k)
(16)
provided that (A, c) is completely observable. In Eq. (16),

0(A) is the desired characteristic equation of the observer
eigenvalues with the matrix A substituted for the variable z,
M0 is the observability matrix
c
...

c A
...
2
c A
M0 =
...
.
...
c A n
estimates a linear combination Mx(k) of the plant state rather

than the state itself, the error between the observer state and
the plant state transformation is given by
Mx
Mx(k + 1) (k + 1)
and jn is the nth-unit coordinate vector
0
. . .

0

. . .

0

jn =
. . .
.

.

.

. . .
1
Another popular form of a full-order observer which is viewed
as an error feedback system can be obtained by expressing
the observer Eqs. (13), (14), and (15) in the form
A GC
B GD
u (k)
(k + 1) = (A
GC) (k) + Gy
Gy(k) + (B
GD)u
= MAx
MAx(k) + MBu
MBu(k) F (k) Gy
Gy(k) Hu
Hu(k)
MA GC
MB GD H )u
u (k)
= (MA
GC)xx (k) F (k) + (MB
where M is m n. For the observer error system to be autonomous, we require
FM = MA GC
H = MB GD
(17)
so that the error is governed by

Mx
Mx
Mx(k + 1) (k + 1) = F [Mx
Mx(k) (k)]
For a completely observable plant, the observer gain matrix
g can always be selected so that all the eigenvalues of F are
inside the unit circle on the complex plane. Then the observer
error
Mx
Mx
Mx(k) (k) = F k [Mx
Mx(0) (0)]
will approach zero asymptotically with step and then
(k) Mx
Mx(k)
If the plant outputs, which also involve linear transformation
of the plant state, are used in the formulation of a state observer, the dynamic order of the observer can be reduced. For
the nth-order plant given by Eq. (12) with the m rows of C
linearly independent, an observer of order n m with n outputs

0
I
D

W (k) =
(k) +
y(k) +
u (k)
I
0
0
observes

C
W (k)
x (k) = Nx
Nx(k)
M
= A (k) + Bu
Bu(k) + G [yy (k) w (k)]
where
w (k) = C (k) + Du
Du(k)
Here, the observer consists of a model of the plant driven by
the input u(k) and the error between the plant output y(k)
and the plant output that is estimated by the model w(k).
This form of a full-order observer is similar to the Kalman
Bucy filter (see Ref. 1).
Except in special cases, the rows of M and the rows of C are

linearly independent. If they are not so, slightly different observer eigenvalues can be chosen to give linear independence.
Therefore, the observer output
w (k) = N 1w (k)
observes x(k).
655
D
+
(k) = 0
+
u(k)
x(k+1)
Unit
delay
y(k)
x(k)
C
+
A
D
+
(k+1)
Unit
delay
(k)
C
Figure 11. Eigenvalue placement with

full-order state observer feedback.
Eigenvalue Placement with Observer Feedback

When observer feedback is used in place of plant state feedback, the eigenvalues of the feedback system are those the
plant would have if the state feedback were used and those of
the observer. This result is known as the separation theorem
for observer feedback. For a completely controllable and completely observable plant, an observer of the form
(k + 1) = F (k) + Gy
Gy(k) + Hu
Hu(k)
w (k) = L (k) + N [y(k) Du
Du(k)]
(18)
(19)
with feedback to the plant given by

u (k) = Ew
Ew(k)
(20)
can be designed such that the overall feedback system eigenvalues are specified by the designer. The design procedure
proceeds in two steps. First, the state feedback is designed
to place the n-state feedback system eigenvalues at desired
locations as if the state vector were accessible. Second, the
state feedback is replaced by feedback of an observer estimate
of the same linear transformations of the state. As an example of eigenvalue placement with observer feedback, Figure
11 shows eigenvalue placement with full order state observer.
The eigenvalues of the overall system are those of the state
feedback and those of the full-order observer.
A tracking system in which the plant outputs are controlled so that they become and remain nearly equal to
externally applied reference signals r(k) is shown in Fig.
12(a). The outputs y(k) are said to track or follow the reference inputs.
As shown in Fig. 12(b), a linear, step-invariant controller
of a multiple-input/multiple-output plant is described by two
transfer function matrices: one relating the reference inputs
to the plant inputs, and the other relating the output feedback vector to the plant inputs. The feedback compensator is
used for shaping the plants zero-input response by placing
the feedback system eigenvalues at desired locations as was
discussed in the previous subsections. The input compensator,
on the other hand, is designed to achieve good tracking of the
reference inputs by the system outputs.
The output of any linear system can always be decomposed
into two parts: the zero-input component due to the initial
conditions alone, and the zero-state component due to the input alone. That is,
y (k) = y zero input (k) + y zero state (k)
Basically, there are three methods for tracking system design:
1. Ideal tracking system design
2. Response model design
TRACKING SYSTEM DESIGN

The second concern of tracking system design, that of obtaining acceptable zero-state system response to reference inputs, is now discussed. It is assumed that the first concern of
tracking system designnamely, satisfactory zero-input response by feedback system eigenvalues placementhas been
achieved.
3. Reference model design

Ideal Tracking System Design
In this first method, ideal tracking is obtained if the measurement output equals the tracking input:
y zero state (k) = r (k)
656

Reference
inputs
r(k)
Plant
inputs
Controller
Tracking and
feedback outputs
y(k)
Plant
u(k)
An input compensator or a reference input filter, as shown in

Fig. 12(d), with transfer function matrix G(z), for which
R (z)
(z) = G (z)R
gives
G (z)R
R (z)
Y (z) = T (z)G
(a)
Ideal tracking is achieved if

Controller
r(k)
G (z) = I
T (z)G
Input
compensator
(k)
u(k)
Plant
y(k)
Feedback
compensator
(b)
r(k)
Input
(k)
compensator
Gr(z)
u(k)
Plant
y(k)
where I is the identity matrix with dimensions equal to the

number of reference inputs and tracking outputs. This is to
say that ideal tracking is obtained if the reference input filter
is an inverse filter of the plant with feedback. Reference input
filters do not change the eigenvalues of the plant with feedback which are assumed to have been previously placed with
output or observer feedback.
When a solution exists, ideal tracking system design
achieves exact zero-state tracking of any reference input. Because it involves constructing inverse filters, the ideal
tracking system design may require unstable or noncausal
filters. An ideal tracking solution can also have other undesirable properties, such as unreasonably large gains, high oscillatory plant control inputs, and the necessity of canceling
plant poles and zeros when the plant model is not known accurately.
Response Model Design
When ideal tracking is not possible or desirable, the designer
can elect to design response model tracking, for which
Feedback
compensator
G (z) = (z)
T (z)G
(c)
Reference
inputs
Reference
r(k)
input filter
Gr(z)
Plant
inputs
r(k)
Plant with
feedback
T(z)
Tracking
outputs
y(k)
(d)
Figure 12. Controlling a multiple-input/multiple-output plant. (a)

The output y(k) is to track the reference input r(k). (b) A tracking
system using the reference inputs and plant outputs. (c) Representing
a controller with a feedback compensator and an input compensator.
(d) Feedback compensator combined with plant to produce a plantwith-feedback transfer function matrix T(z).
The tracking outputs y(k) have initial transient errors due to

any nonzero plant initial conditions; after that they are equal
to the reference inputs r(k), no matter what these inputs are.
As shown in Fig. 12(c), if the plant with feedback has the
z-transfer function matrix T(z) relating the tracking output to
the plant inputs, then
(z)
Y (z) = T (z)
where the response model z-transfer function matrix (z)

characterizes an acceptable relation between the tracking
outputs and the reference inputs. Clearly, the price one pays
for the added design freedom of a reference model can be degraded tracking performance. However, performance can be
improved by increasing the order of the reference input filter.
Response model design is a generalization of the classical design technique of imposing requirements for a controllers
steady-state response to power-of-time inputs.
The difficulty with the response model design method is in
selecting suitable model systems. For example, when two or
more reference input signals are to be tracked simultaneously, the response model z-transfer functions to be selected
include not only those relating plant tracking outputs and the
reference inputs they are to track, but also those relating unwanted coupling between each tracking output and the other
reference inputs.
Reference Model Tracking System Design
The awkwardness of the practical response model performance design arises because of the difficulty in relating performance criteria to the z-transfer functions of response models. An alternative design method models the reference input
signals r(k) instead of the system response. This method,
termed the reference model tracking system design, allows
the designer to specify a class of representative reference inputs that are to be tracked exactly, rather than having to
specify acceptable response models for all the possible inputs.
In the reference model tracking system design, additional
external input signals r(k) to the composite system are applied to the original plant inputs and to the observer state
equations so that the feedback system is, instead of being described by Eqs. (18), (19), and (20), described by Eq. (12) and
(k + 1) = F (k) + Gy
Gy(k) + Hu
Hu(k) + Jr
Jr(k)
w (k) = L (k) + N [yy (k) Du
Du(k)]
Observer of the
signal model
Fictitous
autonomous
reference signal
model
r(k)
Plant with
observer
feedback
y(k)
Figure 13. Observing a reference signal model.
(21)
(22)
with
u (k) = Ew
Ew(k) + Pr
Pr(k)
657
(23)
Then, the overall composite system has the state equations

x (k + 1)
A + BENC
BEL
x (k)
=
(k + 1)
GC + H ENC F + H EL
(k)

BP
+
r (k)
H P + J
= A x (k) + B rr(k)
In the reference model tracking system design, the concept of

an observer is used in a new way; it is the plant with feedback
that is an observer of the fictitious reference input model system as shown in Fig. 13. When driven by r(k), the state of the
composite system observes
x (k) M (k)
where M satisfies, according to Eq. (17),
M A M = B
(25)
The plant tracking output y(k) observes

y (k) = C x (k) + D rr(k) C M (k) + D rr(k)
and the output equation becomes

C + ENC
y (k) = [C
and for
DEL
DEL]xx (k) + DPr
DPr(k)
y (k) r (k)
= C x (k) + D rr(k)
it is necessary that
where
H = H + GD
C M (k) + D rr(k) = r (k)
Examining the composite state coupling matrix A above

shows that the coupling of external inputs r(k) to the feedback
system does not affect its eigenvalues. The input coupling matrix B has matrices P and J which are entirely arbitrary and
thus can be selected by the designer. Our objective is to select
P and J such that the system output y(k) tracks the reference
input r(k).
Consider the class of reference signals which are generated
by the autonomous state variable model of the form
Equations (25) and (26) constitute a set of linear algebraic

equations where the elements of M, P, and J are unknowns.
If, for an initial problem formulation, there is no solution to
the equations, one can reduce the order of the reference signal
model and/or raise the order of the observer used for plant
feedback until an acceptable solution is obtained.
The autonomous reference input model has no physical existence; the actual reference input r(k) likely deviates somewhat from the prediction of the model. The designer deals
with representative reference inputs, such as constants and
ramps, and, by designing for exact tracking of these, obtains
acceptable tracking performance for other reference inputs.
(k + 1) = (k)
r (k) =
(k)
(24)
The output of this reference input model system may consist

of step, ramp, parabolic, exponential, sinusoidal, and other
common sequences. For example, the model

1 (k + 1)
2 1
1 (k)
=
2 (k + 1)
2 (k)
1 0

(k)
r(k) = [1 0] 1
2 (k)
produces the sum of an arbitrary constant plus an arbitrary
ramp:
r(k) = 1 (0) + 1 (0)k
(26)
SIMULATION
One of the most important control system design tools is simulationthat is, computer modeling of the plant and controller to verify the properties of a preliminary design and to test
its performance under conditions (e.g., noise, disturbances,
parameter variations, and nonlinearities) that might be difficult or cumbersome to study analytically. It is usually
through simulation that difficulties with between-sample
plant response are covered and solved.
When a continuous-time plant is simulated on a digital
computer, its response is computed at closely spaced discrete
times. It is plotted by joining the closely spaced calculated
response values with straight line segments in approximation
658
DISPATCHING
of continuous curve. A digital computer simulation of discretetime control of a continuous-time system involves at least two
sets of discrete-time calculations. One runs at a high rate for
simulation of the continuous-time plant. The other runs at a
lower rate (say once every 10 to 50 of the former calculations)
to generate new control signals at each discrete control step.
BIBLIOGRAPHY
1. M. S. Santina, A. R. Stubberud, and G. H. Hostetter, Digital Control System Design, 2nd ed., Philadelphia: Saunders, 1994.
2. J. J. DiStefano III, A. R. Stubberud, and I. J. Williams, Feedback
and Control Systems (Schaums Outline), 2nd ed., New York:
McGraw-Hill, 1990.
3. B. C. Kuo, Digital Control Systems, 2nd ed., Philadelphia: Saunders, 1992.
4. G. F. Franklin, J. D. Powell, and M. L. Workman, Digital Control
of Dynamic Systems, 2nd ed., Reading, MA: Addison-Wesley,
1990.
5. T. Kailath, Linear Systems, Englewood Cliffs, NJ: Prentice-Hall,
1980.
6. C. T. Chen, Linear System Theory and Design, Philadelphia:
Saunders, 1984.
strom and B. Wittenmark, Computer Controlled Systems,
7. K. J. A
Englewood Cliffs, NJ: Prentice-Hall, 1987.
8. K. Ogata, Discrete-Time Control Systems, Englewood Cliffs, NJ:
Prentice-Hall, 1987.
9. B. Friedland, Control System Design, New York: McGraw-Hill,
1986.
10. W .M. Wonham, Linear Multivariable Control: A Geometric Approach, 3rd ed., New York, Springer-Verlag, 1985.
MOHAMMED S. SANTINA
Boeing Defense and Space Group
ALLEN R. STUBBERUD
University of California, Irvine
DISCRETE TIME TRANSFORMS. See Z TRANSFORMS.
264
CONTROLLABILITY AND OBSERVABILITY
one may deduce the evolution of x. We will illustrate these

concepts for three different cases: (1) linear finite-dimensional
systems, (2) nonlinear systems, and (3) linear infinite dimensional systems.
We remark that one could also study a more general type
of dynamic system, namely one given by implicit equations on
a set of abstract variables w:
f ( n w, , w, w) = 0
(3)
Two examples are

u
w=
y

In this article, we will consider dynamic systems described
by a set of first order differential or difference equations, together with a set of algebraic equations:
d
x(t), t R
x = f (x, u) where ( x)(t) := dt
(1)
x(t + 1), t Z
y = h(x, u)
(2)
Here, u is the input or excitation function, and its values

u(t) belong to a Euclidean space m. The state at time t is
x(t) and, depending on the context, may belong to a finite or
an infinite dimensional space. The output or measurement
function y takes values y(t) p. When u(t) 0, one interprets the equation x f(x, 0) as describing the evolution of
the system in the absence of inputs. Observe that, in contrast
to the classical study of dynamic systems, where inputs (or
forcing functions) are fixed, the object of control theory is to
study the effect of different input functions on the system ,
that is, on the solutions of the difference or differential Eq.
(1).
In the sequel questions of state controllability and state
observability will be investigated. Roughly speaking, the former questions have to do with the extent to which the choice
of u can influence the evolution of x, while the latter questions
are concerned with the extent to which by observing y and u
or

u

w = x
y
where u, x, and y are, as stated, the input, state, and output,

respectively. Let B denote the behavior of this system, that
is, the set of all time trajectories w which satisfy Eq. (3). This
leads to the so-called behavioral theory of dynamic systems
(1). Let w and w denote, respectively, the past and future
of a given trajectory with respect to some fixed time t0 (with
w restricted to t t0 and t t0). The system described by Eq.
(3) is said to be controllable, if for any w1, w2 belonging to B ,
any trajectory composed of the concatenation of (w1) and
(w2) belongs to B . This approach has actually been worked
out for the case where f is a linear and time-invariant function of its arguments. This provides a generalization of state
controllability for the case where states are not available, and
without distinguishing between inputs and outputs. It turns
out that this concept is indeed the generalization of the concept of state controllability when Eq. (3) is Eq. (1). This will
be discussed briefly in a later section of this article.
For reasons of brevity and space, the proofs of the various
results are omitted in the sections that follow. The interested
reader is referred to the original sources.
LINEAR FINITE-DIMENSIONAL SYSTEMS
The system described by the state equations Eq. (1) is linear,
if f(x, u) Fx Gu:
x = Fx + Gu
(4)
where as in Eq. (1), denotes the derivative operator for continuous-time systems and the (backwards) shift operator for
discrete-time systems. The input u(t) and state x(t) of the system at time t belong, respectively, to the input space U m
and state space X n. Moreover,
G : U X,
F:X X
are linear maps; the first is called the input map, while the
second describes the dynamics or internal evolution of the
system.
The output Eq. (2), for both discrete- and continuous-time
linear systems, is composed of a set of linear algebraic equations
y = Hx + Ju
(5)
where y(t) is the output (response) at time t, and belongs to

the output space Y p; furthermore:
H : X Y,
are linear maps; H is called the output map. It describes how

the dynamics affect the output, while J describes how the input affects the output directly (i.e., without passing through
the state). For simplicity of exposition, we will assume that
the systems considered are time-invariant, that is to say,
there is no explicit dependence of the system matrices on
time. Thus the term linear will be used in this section to denote a linear, time-invariant, continuous- or discrete-time system which is finite-dimensional. Linearity means: U, X, Y are
linear spaces, and F, G, H, J are linear maps; finite-dimensional means: U, X, Y are all finite dimensional; time-invariant means: F, G, H, J do not depend on time; their matrix
representations are constant n n, n m, p n, p m
matrices. We are now ready to give
Definition 1. (a) A linear system in state space description
is a quadruple of linear maps (matrices)

:=
F
H
G
J
, F Rnn , G Rnm , H R pn , J R pm
(6)
The dimension of the system is defined as the dimension of

the associated state space:

dim = n
(7)
(b) is called (asymptotically) stable if the eigenvalues of F
have negative real parts or lie strictly inside the unit disk,
depending on whether is a continuous-time or a discretetime system.
Let (u; x0; t) denote the solution of the state Eq. (4), that
is, the state of the system at time t attained starting from the
initial state x0 at time t0, under the influence of the input u.
For the continuous-time, time-invariant state equations,
(u; x0 ; t) = eF (tt 0 ) x0 +
t0
eF (t ) Gu( ) d , t t0
(8)
t1

F t1 j Gu( j), t t0
(9)
where J remains unchanged. The corresponding triples are

called equivalent.
Remark. The material that follows was first introduced by
R. E. Kalman (2), (3); see also Refs. 4 and 5. For a more recent
treatment, we refer to the book by Sontag (6); see also Refs.
7,8, and 9.
The State Controllability Problem
There are two fundamental concepts associated with the state
controllability problem: reachability and controllability.
These concepts allow us to answer questions concerning the
extent to which the state of the system x can be manipulated
through the input u.
We will first discuss the concept of state reachability for
linear systems. The related concept of controllability will also
be discussed. Both concepts involve only the state equations.
Consequently, for this subsection, H and J will be ignored.
Definition 2. Given

=
F nxn, G nxm, a state x X is reachable from the zero

state iff there exist an input function u(t) and a time T ,
such that
x = (u; 0; T )
The reachable subspace Xreach X of is the set which contains all reachable states of . We will call the system
(completely) reachable iff Xreach X. Furthermore
Rn (F, G) := [G FG F 2 G
F n1 G]
(11)
will be called the reachability matrix of .
Definition 3. The finite reachability Grammian at time

t is defined as follows. For continuous-time systems:

For both discrete- and continuous-time systems the output is

given by:
= H(0; x0 ; t) + H(u; 0; t) + Ju(t)
j=t 0
y(t) = H(u; x(0); t) + Ju(t)
1
y = HT
x + Ju
A useful concept is that of the reachability grammian.

Complex conjugation and transposition will be denoted by *.
while for the discrete-time state equations,
(u; x0 ; t) = F tt 0 x0 +
then Eqs. (4) and (5) when expressed in terms of the new
state x, will become
1
x = TFT
TG u,
x +
J :U Y
265
P (t) :=
eF GG eF d , t > 0
(12)
while for discrete-time systems

(10)
P (t) := Rt (F, G)Rt (F, G) =
t1

F k GG (F )k , t > 0
(13)
k=0
If we transform the state under a linear change of coordinates, the corresponding matrices describing the system will
change. In particular, if the new state is
x := Tx, det T = 0
We will make use of the following input

w ,T (t) := G eF
(T t )
, Rn , t, T R, 0 t T
(14)
266
for the continuous-time case, and

w ,T (t) := G (F )T t , Rn , t, T Z, 0 t T
(15)
for the discrete-time case.

A concept which is closely related to reachability is that of
controllability. Here, instead of driving the zero state to a desired state, a given nonzero state is steered to the zero state.
More precisely we have:
Proposition 1. The reachability Grammians have the following properties: (a) P (t) P *(t) 0, and (b) their columns
span the reachability subspace, that is
im P (t) = im Rn (F, G)
This relationship holds for continuous-time systems for all
t 0, and for discrete-time systems (at least for t n).
Corollary 2
Definition 4. Given

=
a (nonzero) state x X is controllable to the zero state if and

only if there exist an input function u(t) and a time T ,
such that
(u; x; T ) = 0
The controllable subspace Xcontr of is the set of all controllable states. The system is (completely) controllable if and
only if Xcontr X.
The fundamental result concerning reachability is the following
Theorem 1. Given

=
(16)
where imR n denotes the image (span of the columns) of R n.

X . (b) is (completely) reachCorollary 1. (a) FX
able if, and only if, rank R n(F, G) n. (c) Reachability is
basis independent.
reach
reach
In general, reachability is an analytic concept. The previous theorem, however, shows that for linear, finite-dimensional, time-invariant systems, reachability reduces to an algebraic concept depending only on properties of F, G and in
particular on the rank of the reachability matrix R n(F, G),
but independent of time and the input function. It is also
worthwhile to note that Eq. (16) is valid for both continuousand discrete-time systems. This, together with a similar result on observability [see Eq. (23)], has as a consequence the
fact that many tools for studying linear systems are algebraic.
It should be noticed however that the physical significance of
F and G is different for the discrete- and continuous-time
cases; if for instance we discretize the continuous-time system
d
x(t) = Fcontx(t) + Gcontu(t), to
dt
x(t + 1) = Fdiscrx(t) + Gdiscru(t)
then Fdiscr eFcont.
is reachable if and only if P (t) is positive definite for some

t 0.
The energy f of the vector function f, defined on an interval I or , is defined as

I f ( ) f ( ), t Z
2
f := f, f :=
f ( ) f ( ) d , t R
I
The input function w,T defined by Eqs. (14) and (15) has the
following property:
Proposition 2. Given the reachable state x Xreach, let
u be any input function which reaches x at time T, that is,
(u; 0; T) x. There exists n satisfying:
x = P (T )
for both the continuous- and discrete-time case, Xreach is a linear subspace of X, given by the formula
X reach = im Rn (F, G)
(17)
It follows that w,T defined by Eqs. (14) and (15) reaches x at

time T; moreover this is the minimum energy input which
achieves this:
u
w ,T
(18)
The minimum energy required to reach the state x at time T

is equal to the energy of the input function w,T. If the system
is reachable this energy is equal to:

w ,T = x P (T )1 x
(19)
From the previous considerations the length of time

needed to reach a given reachable state can be derived.
Proposition 3. Given is

=
(a) For discrete-time systems, every reachable state can be

reached in at most n time-steps. (b) For continuous-time systems, every reachable state can be reached in any arbitrary
positive length of time.
The second part of the proposition shows that ideally, in
continuous-time linear systems, every reachable state can be
reached arbitrarily fast. In a practical situation, the extent to
which this is not possible gives a measure of how significant
the nonlinearities of the system are. We conclude this subsection by stating a result on various equivalent conditions for
reachability.
Theorem 2 Reachability Conditions. The following are
equivalent:
1.

=
F nxn, G nm is reachable.
2. The rank of the reachability matrix is full: rank R n(F,
G) n.
3. The reachability Grammian is positive definite, that is,
P (t) 0, for some t 0.
4. No left eigenvector v* of F is in the left kernel of G:
v F = v v G = 0
5. rank(In F G) n, for all
6. The polynomial matrices sI F and G are left coprime.
The fourth and fifth conditions in this theorem are known
as the PHB or PopovHautusBelevich tests for reachability.
The equivalence of the fifth and sixth conditions is a straightforward consequence of the theory of polynomial matrices; it
will not be discussed in this article.
Remark. Reachability is a generic property. This means intuitively that almost every n n, n m pair of matrices F,
G satisfies
rank Rn (F, G) = n
Put in a different way, in the space of all n n, n m pairs
of matrices, the unreachable pairs form a hypersurface of
measure zero.
The next theorem shows that for continuous-time systems,
the concepts of reachability and controllability are equivalent
while for discrete-time systems the latter is weaker. This is
easily seen by considering the system with state equation:
x(t 1) 0. Clearly, for this system all states are controllable, while none is reachable. Often for this reason, only the
notion of reachability is used.
Theorem 3. Given is

=
(a) For continuous-time systems Xcontr X reach. (b) For discretetime systems X reach. X contr; in particular X contr X reach
ker F n.
ability of x1 implies the existence of u1, T1 such that x1

(u1; 0; T1). The function u12 is then the concatenation of u2
with u1, while T12 T1 T2. In general, if x1, x2 are not reachable, there is a trajectory passing through the two points if,
and only if,
x2 f (F, T )x1 X reach,
for some T,
where f(F, T) eFT for continuous-time systems and f(F,

T) FT for discrete-time systems. This shows that if we start
from a reachable state x1 0 the states that can be attained
are also within the reachable subspace.
The State Observation Problem
In order to be able to modify the dynamics of a system, very
often the state x needs to be available. Typically however the
state variables are inaccessible and only certain linear combinations y, given by the output Eqs. (5), are known. Thus, the
problem of reconstructing the state x(T) from observations
y(), where is in some appropriate interval, arises. If
[T, T t], we have the state observation problem, while if
[T t, T] we have the state reconstruction problem.
We will first discuss the observation problem. Without loss
of generality we will assume that T 0. Recall Eqs. (8), (9),
and (10). Since the input u is known, the latter two terms in
Eq. (10) are also known. Therefore, in determining x(0) we
may assume without loss of generality that u( ) 0. Thus,
the observation problem reduces to the following: given
H(0; x(0); t) for t 0 or t 0, find x(0). Since G and J are
irrelevant, for this subsection

F
, F Rnm , H R pxn
=
H
Definition 5. A state x X is unobservable iff y(t) H(0;
x; t) 0, for all t 0, that is, iff x is indistinguishable from
the zero state for all t 0. The unobservable subspace Xunobs
of X is the set of all unobservable states of . is (completely) observable iff Xunobs 0. The observability matrix of
is
H
HF
HF 2
On (H, F ) :=
(20)
.
.
.
HF n1
Definition 6. Let

=
F
H
The finite observability Grammians at time t are:

eF H HeF d , t > 0
(21)
Q(t) := Ot (H, F )Ot (H, F ), t > 0
(22)
Q(t) :=
Remark. It follows from the previous results that for any
two states x1, x2 X reach there exist u12, T12 such that x1
(u12; x2; T12). Since x2 is reachable it is also controllable; thus
there exist u2, T2 such that (u2; x2; T2) 0. Finally, the reach-
267
for continuous- and discrete-time systems, respectively.
268
Definition 7. A state x X is unreconstructible iff y(t)

H(0; x; t) 0, for all t 0, that is, iff x is indistinguishable
from the zero state for all t 0. The unreconstructible subspace X unrecon of X is the set of all unreconstructible states of
. is (completely) reconstructible iff X unrecon 0.
(a) For t , Xunrecon Xunobs. (b) For t , Xunrecon Xunobs, in

particular, Xunrecon Xunobs im Fn.
The Duality Principle in Linear Systems
The dual of a linear system

We are now ready to state the main theorem
F
H
G
J
Theorem 4. Given

=
F
H
is defined as follows. Let U*, X*, Y* be the dual spaces of the

input U, state X, output Y spaces of . Let
F : X X , G : X U , H : Y X , J : Y U
for both continuous- and discrete-time systems, X

ear subspace of X given by
unobs
is a lin-
X unobs = ker On (H, F ) = {x X : HF i1 x = 0, i > 0}
(23)
As an immediate consequence of the last formula we have

Corollary 3. (a) The unobservable subspace Xunobs is F-invariant. (b) is observable iff, rank O n(H, F) n. (c) Observability is basis independent.
Remark. Given y(t), t 0, let Y0 denote the following np
1 vector:
Y0 := (y (0) y (1) y (n 1)) , t Z
be the dual maps to F, G, H, J. The dual system * of is

F
H
:=
R(n+m)(n+ p)
(25)
G J
that is, the input map is given by H*, the output map by G*,
and the dynamics are given by F*. Correspondingly the input,
state, and output spaces of * are Y*, X*, U*. The matrix
representations of F*, H*, G*, J* are the complex conjugate
transposes of F, H, G, J, respectively, computed in appropriate dual bases. One may think of the dual system * as
the system where the role of the inputs and the outputs
has been interchanged, or the flow of causality has been reversed. The main result is the duality principle.
Y0 := (y (0) Dy (0) Dn1 y (0)) , t R
Theorem 5. The orthogonal complement of the reachable

subspace of is equal to the unobservable subspace of its
dual *:
where D : d/dt. The observability problem reduces to the

solution of the linear set of equations
reach
unobs
(X
) = X
On (H, F )x(0) = Y0
Corollary 4. The system is reachable iff its dual * is observable.
This set of equations is solvable for all initial conditions x(0),

that is, it has a unique solution iff, is observable. Otherwise x(0) can only be determined modulo Xunobs, that is, up to
an arbitrary linear combination of unobservable states.
It readily follows that ker Q (t) ker O n(H, F). As in the
case of reachability, this relationship holds for continuoustime systems for t 0 and for discrete-time systems, at least
for t n. The energy of the output function y at time T, generated from the initial state x will be denoted by y. In terms
of the observability Grammian this energy can be expressed
as

y = x Q(T )x
(24)
We now briefly turn our attention to the reconstructibility

problem. The main result which follows shows that while for
continuous-time systems the concepts of observability and reconstructibility are equivalent, for discrete-time systems the
latter is weaker. For this reason, the concept of observability
is used most of the time.
It can also be shown that controllability and reconstructibility are dual concepts. We conclude this subsection by stating the dual to theorem 2.
Theorem 6. Observability conditions. The following are
equivalent:
1.
F
H
F
H
H pn, F nn is observable.
2. The rank of the observability matrix is full: rank
O n(H, F) n.
3. The observability Grammian is positive definite Q (t)
0, for some t 0.
4. No right eigenvector v of F is in the right kernel of H:
Fv = v Hv = 0
5.
In F
rank
H
Proposition 4. Given is

=

for all

=n
6. The polynomial matrices sI F and H are right coprime.

Canonical Forms
A nonreachable system can be decomposed in a canonical way
into two subsystems; one whose states are all reachable and
a second whose states are all unreachable. The precise result
is stated next.
Lemma 1. Reachable canonical decomposition. Given is

Lemma 2 Observable canonical decomposition. Given is

F
=
H
There exists a basis in X such that F, H have the following
matrix representations

Fo Foo
F
= 0 Fo
H
0 Ho
where

There exists a basis in X such that F, G have the following
matrix representations:
Fr
= 0
Gr
0
Frr
Fr
(26)
where the subsystem

:=
Fr
Gr
is reachable.
Thus every system

can be decomposed in a subsystem

=
Fr
Gr
Fo
Ho
is observable.
The reachable and observable canonical decompositions
given in lemmas 1 and 2 can be combined to obtain the following decomposition of the triple (H, F, G):
Lemma 3. Reachable-observable canonical decomposition.
Given

F G
=
H
there exists a basis in X such that F, G, and H have the following matrix representations
269

=
F
H
Fro
0
0
0
0
F12
Fro
0
0
Hro
F13
0
Fro
0
0
Fro
Hro
Gro
F14
F24
F34
Fro
Hro
Gro
Gro
0
0
(27)
where the triple

which is reachable, and in a subsystem

r
Fr

ro
:=
is both reachable and observable.

which is completely unreachable, that is, it cannot be influenced by outside forces. The interaction between r and r is
given by Frr. Since Frr 0, it follows that the unreachable
subsystem r influences the reachable subsystem r but not
vice versa. It should be noticed that although the direct complement X of Xreach is not unique, the form of the reachable
decomposition of Eq. (26) is unique.
Since by duality

is reachable if, and only if,

F
G
is observable, we obtain the following results.
A concept related to, but weaker than reachability, is that

of stabilizability. Its dual is detectability.
Definition 8. The pair

is stabilizable iff in the reachable canonical decomposition,

Fr is stable, that is, all its eigenvalues have either negative
real parts or are inside the unit disk, depending on whether
we are dealing with continuous- or discrete-time systems.

F
H
is detectable iff in the observable canonical decomposition, Fo
is stable.
270
The Infinite Grammians
and is equal to:
Consider a continuous-time linear system

c
F
H
Eo2 = x Qx
which is stable, that is, all eigenvalues of F have negative

real parts. In this case both Eqs. (12) as well as (21) are defined for t ;

P :=
eF GG eF d ,
Q :=
eF H HeF d
(28)
P , Q are the infinite reachability and infinite observability

grammians associated with c. These grammians satisfy the
following linear matrix equations, called Lyapunov equations.
Proposition 5. Given the stable, continuous-time system c
as stated, the associated infinite grammians P , Q satisfy the
continuous-time Lyapunov equations
FP + P F + GG = 0,
F Q + QF + H H = 0
(29)
If the discrete-time system

d
F
H
F t1 GG (F )t1 ,
t>0
Q :=
(F )t1 H HF t1
(30)
t>0
Notice that P can be written as P GG* FP F*; moreover

Q H*H F*Q F. These are the so-called discrete Lyapunov
or Stein equations:
Proposition 6. Given the stable, discrete-time system d as
stated, the associated infinite grammians P , Q satisfy the
discrete-time Lyapunov equations
FP F + GG = P ,
We summarize these results in the following proposition

which is important in the theory of balanced representations
and Hankel-norm model reduction.
Lemma 4. Let P and Q denote the infinite grammians of a
linear stable system.
(a) The minimal energy required to steer the state of the
system from 0 to x is given by Eq. (33).
(b) The maximal energy produced by observing the output
of the system whose initial state is x is given by Eq.
(34).
(c) The states which are difficult to reach, that is, require
large amounts of energy, are in the span of those eigenvectors of P which correspond to large eigenvalues.
Furthermore, the states which are difficult to observe,
that is, produce small observation energy, are in the
span of those eigenvectors of Q which correspond to
small eigenvalues.
(d) The eigenvalues of the product of the reachability and
of the observability grammians are input-output invariants called Hankel singular values of .
Controllability in the Behavioral Framework
is stable, that is, all eigenvalues of F are inside the unit disk,
the grammian Eqs. (13) as well as (22) are defined for t
P :=
(34)
F QF + H H = Q
(31)
Recall Eqs. (18), (19), and (24), valid for both discrete- and
continuous-time systems. From the definition of the grammians follows that:
P (t2 ) P (t1 ), Q(t2 ) Q(t1 ), t2 t1
(32)
irrespective of whether we are dealing with discrete- or continuous-time systems. Hence from Eq. (19) it follows that the
minimal energy Er, required for the transfer of state 0 to x, is
obtained as the alloted time T tends to infinity. Assuming
reachability, this minimal energy is:
Er2 = x P 1 x
(33)
Similarly, the largest observation energy Eo, produced by the

state x is also obtained for an infinite observation interval,
A dynamical system in the classical framework is a mapping

which transforms inputs u into outputs y. In many cases however, the distinction between inputs and outputs is not a priori clear. Consider for example the RLC (ResistorInductor
Capacitor) network presented in the next section, and
suppose that we are interested in the relationship between
the current through the resistor RC and the voltage across the
capacitor C. Is the voltage causing the current or vice versa?
Other than the often encountered inherent difficulty in distinguishing between inputs and outputs, it is desirable to have
a framework in which the different representations of a given
system (for example: input-output and input-state-output) are
treated in a unified way.
The need for a framework at a more abstract level than
is provided by the inputoutput framework gave rise to the
behavioral framework. For a tutorial account see Refs. (1) and
(10). The variables considered are the external or manifest
variables w and (possibly) a second set of variables, the socalled latent variables a. The manifest variables consist of u
and y, without distinguishing between them. In the behavioral theory, a dynamical system is defined as a collection B
of trajectories w. This set B , called the behavior of the system, is the primary object of study for system and control theoretic issues.
In this section we will provide an overview of controllability in the behavioral framework. For further details and
proofs, see Polderman and Willems (1).
The trajectories w composing the behavior B are most often represented as solutions of appropriate equations, called
behavioral equations; these are equations providing relationships between w and w, where is defined by Eq. (1). The
most important type of such equations are the annihilating
behavioral equations. An important special case of such be-
havioral equations are: state variable (SV) equations; SV

equations, in addition to w, make use of the latent variables
a x, which can be assigned the property of state, and are
called state variables. Another is the inputoutput equation
representation, which thus appears as a special case of a more
general system representation.
For linear, time-invariant systems, annihilating behavioral equations representing have the following form. Let
[s] denote the ring of polynomials in the indeterminate s
with coefficients in , and n1n2[s] denote the n1 n2 polynomial matrices. The resulting equation has the form:
R( )w = M( )a, R R pq [s], M R pr [s]
(35)
w (t) =

w 1 (t) for t < 0
w 2 (t) for t > t
In terms of annihilating behavioral equation representations

such as Eq. (35), the corresponding system is controllable if,
and only if, the rank of the (constant) matrix
rank [R() M()] = constant, C
(36)
From this follows the condition for controllability of the SV

and input/output models. There holds, respectively:
rank [I F G] = constant = n,
It relates
rank [N() D()] = constant = p, C

w
B
a
to its shifts or derivatives. This equation can be written explicitly in terms of the coefficient matrices of R and M. Let
R(s) :=
271
1

Ri si , Ri R pq , M(s) :=
i=0
2

Mi si , Mi R pr
i=0
Equation (35) becomes:

1

i=0
Ri ( i w)(t) =
2

Mi ( i a)(t)
i=0
Since the differential or difference operator [R() M()]

annihilates all trajectories

w
B
a
Eq. (35) is referred to as an annihilating behavioral equation.
The special case of SV Eq. (4) is described by w u, a x,
R() G and M(s) I F; while that of input/output equations is described by

u
w=
y
a is nonexistent, and R() [N() D()], where D is a
square, nonsingular polynomial matrix of size p.
A further important aspect at which the behavioral formalism departs from, and generalizes, the classical formalism is
that related to controllability; controllability becomes namely
an attribute of the system (i.e., of a collection of trajectories)
as opposed to an attribute of a system representation (i.e., of
equations generating these trajectories).
Roughly speaking, a system is controllable if its behavior
has the property: whatever the past history (trajectory), it can
always be steered to any desired future trajectory. More precisely, a dynamical system with behavior B is said to be controllable, if for any w1, w2 B , there exists a t 0 and a
w B such that
Notice that the first condition stated here is the same as condition 5 of theorem 2. Thus the behavioral definition of controllability provides a generalization of the classic concept.
Furthermore the second condition provides a way of defining
controllability without the definition of state.
Examples
1. The previous issues are illustrated by means of two examples. First, consider an RLC circuit composed of the
parallel connection of two branches: the first branch is
composed of an inductor L in series with a resistor RL;
the second is composed of a capacitor C in series with a
resistor RC. The driving force u is a voltage source applied to the two branches. Let the state variables x1, x2
be the current through the inductor, the voltage across
the capacitor respectively. The state equations are
RL
1
d
R
L
x =
x + u
0
dt 1
L 1 L
L
F=
1
d
1
1
x =
x +
u
RC C
dt 2
RCC 2 RCC
L
G=
1
RCC
Since this is a continuous-time system, reachability and
controllability are equivalent notions. The reachability
matrix is
L
R2 = [G FG] = 1
RCC
RL
2
L
1
2 2
RCC
Thus, reachability of this system is equivalent with the

nonsingularity of R 2, that is,
det R2 =
1
RCCL
RL
1
L
RCC

= 0
272
It readily follows that the reachability Grammian is
g21
(1 e2 L T )
2L
P (T ) =
g1 g2
(1 e( L + C )T )
L + C
Controllability: Continuous-Time
In continuous time, we consider systems of differential equations of the following general form:
g1 g2
(1 e( L + C )T )
L + C
g22
(1 e2 C T )
2C
where
L =
RL
1
1
, C =
, g = , g 2 = C
L
RCC 1
L
Hence the infinite reachability grammian is
g21
g1 g2
L + C
2L
P =
2
g2
g1 g2
L + C
2C
and it can be verified that FP P F* GG* 0.
Assume now that the variable observed y is the sum
of the voltages across the capacitor and across the resistor RL:
y = RL x1 + x2 H = [RL
d
x(t) = f (x(t), u(t))
dt
where f : n m n is a vector function which specifies,

for the current state variables x(t) n and the current values of the control variables u(t) m, the direction of instantaneous movement. For each fixed vector u m, f( , u) is
thought of as a vector field. Linear (time-invariant) systems
are a particular case, namely those systems for which f is a
linear map,
f (x, u) = Fx + Gu
for some matrices F of size n n and some matrix G of size
n m.
An Example. A simplified model of a front-wheel drive automobile uses a four-dimensional state space. The coordinates
of the state

x1
x1
x x

x = 2 = 2
x 3
x4
1]
The observability matrix is
1
R
L
O2 =
1 det O2 = RL RC LC det R2
R2L
L
RC C
Thus reachability and observability are lost simultaneously. If this happens, then one can reach any given
state x1 and x2 L/RL x1; while only the linear combination RLx1 x2 can be deduced, but not x1 or x2 individually.
2. Consider the system given in input/output form by the
equation
d
d
y=
u, u(t), y(t) R
dt
dt
We will show that this system is not controllable.
This equation can be rewritten as d/dt v 0, where
v y u. All trajectories composing the behavior of
this system are constants. But a trajectory defined by
v(t) c1, t T, and v(t) c2, t T, c1 c2, does not
belong to the behavior, as its derivative is not zero.
Hence, since the trajectories of this system are not concatenable, the conclusion follows.
(37)
denote, respectively, the position of the center of the front

axle (coordinates (x1, x2)), the orientation of the car (angle ,
measured counterclockwise from the positive x-axis), and the
angle of the front wheels relative to the orientation of the car
(, also counterclockwise); see Figure 1.
As controls, we take two-dimensional vectors u col (u1,
u2), whose coordinates are proportional to the steering wheel
velocity (u1) and the engine speed (u2) at each instant. Thus,
a control u2(t) 0 corresponds to a pure steering move, while
one with u1(t) 0 models a pure driving move in which the
steering wheel is fixed in one position. In general, a control is
a function u(t) which indicates, at each time t, the current
steering velocity and engine speed.
Using elementary trigonometry, the following equations
are obtained (choosing units so that the distance between the
x2
(x1,x2)
NONLINEAR FINITE-DIMENSIONAL SYSTEMS

x1
We turn attention now to nonlinear systems. Both continuous- and discrete-time systems will be discussed.
Figure 1. Four-dimensional car model.
front and rear axles is unity):

0
cos( + )
0
sin( + )
d

x = u1 + u2
0
sin
dt
1
0
(38)
This is of the form of Eq. (37), where f : 4 2 4.

Note that, in practice, the angle would be restricted to
some maximal interval (0, 0). For simplicity of exposition,
we do not impose this constraint. Similarly, the orientation
angle only makes sense as a number modulo 2, that is,
angles differing by 2 correspond to the same physical orientation. Nonlinear control theory is usually developed in far
more generality than we do here. The more general formalism
allows states to evolve in general differentiable manifolds, instead of insisting, as we do here, in Euclidean state spaces.
Thus, for instance, a more natural state space than 4 would
be, for this example, 2 1 (0, 0), that is, the angle
is thought of as an element of the unit circle. Analogously, we
assume here that the controls may attain arbitrary values in
2; of course, a more realistic model would also incorporate
constraints on their magnitude.
Technical Assumptions. We will assume, for simplicity of exposition, that the function f is real-analytic. This means that
f(x, u) can be expressed, around each point (x, u) n m,
as a locally convergent power series. Analyticity is a condition
which is satisfied in models derived from physical principles.
The assumption of analyticity allows stating results in an elegant necessary and sufficient, rather than merely sufficient,
manner. A control (or input) function is by definition of
Lebesgue-measurable essentially bounded function u( ) defined on some interval of the form [0, T] and taking values in
m (the reader may substitute piecewise continuous function without much loss of generality). We let Inp be the set
of such controls.
Accessibility. Consider any state n. For each control
u : [0, T] m, we consider the solution of the initial problem
d/dt x(t) f(x(t), u(t)), x(0) . If this solution is well-defined
on the interval t [0, T], we denote the final state x(T) as
(, u). [If the solution does not exist on the entire interval,
we do not define (, u)]. The reachable set from , denoted
as R (), is by definition the set of states reachable from the
origin, that is, the set of states of the form
273
ample, consider the system with equation d/dt x u2 (having

dimension n 1 and input space also of dimension m 1).
Take any initial state (for instance, 0). Clearly, R ()
[, ) [because d/dt x(t) 0, no matter which control function is applied]. Thus the system is accessible from every
state. Observe that, as illustrated by this example, accessibility does not mean that must be in the interior of the set
R () (local controllability), even for an equilibrium state.
The reason that the accessibility question is studied is that
it is far easier to characterize than controllability.
Accessibility Rank Condition. Given any two vector fields f
and g, one can associate the new vector field
[ f, g]
defined by the formula
[ f, g](x) := g [x] f (x) f [x] g(x)
where, in general, the notation h[x] means the Jacobian of a
vector field h, evaluated at the point x. This is called the Lie
bracket of f and g.
The Lie bracket of f and g can be interpreted in terms of
certain trajectories that arise from integrating f and g, as follows. We let eth denote the solution at time t (possibly negative) of the differential equation d/dt x h(x) with initial
value x(0) . (When the differential equation is linear, i.e.,
d/dt x Fx and F is a matrix, eth is precisely the same as
etF, where etF is the exponential of the matrix F. In general,
for nonlinear differential equations, eth is merely a convenient notation for the flow associated to the vector field h.)
Then, for any two vector fields f and g,
etg et f etg et f = et
2 [ f,g]
+ o(t 2 )( )
(40)
as t 0, as a simple computation shows. Therefore, one may

understand the Lie bracket of f and g as the infinitesimal
direction that results from following solutions of f and g in
positive time, followed by f and g in negative time. Another
way to state this fact is by introducing the curve
(t) := e
tg t f
tg
tf
For linear systems d/dt x Fx Gu, and the zero initial

state 0,
Observe that (0) and that the values of (t) are all in the
set of points S attainable by positive and negative time solutions of the differential equations corresponding to f and g;
the above expansion implies that d/dt (0) [f, g](), that is,
there is a curve in S whose tangent is the Lie bracket of the
two vector fields. Thus, Lie brackets provide new directions of
infinitesimal movement in addition to those corresponding to
f and g themselves (and their linear combinations).
Given now a system by Eq. (37), we consider, for each possible control value u m, the following vector field:
R(0) = im (G, FG, . . ., F n1 G)
f u : Rn Rn : x f (x, u)
{( , u) Rn |u Inp}
Definition. The system Eq. (37) is accessible from the state
if the reachable set R () contains an open subset of n (that
is, it has a nonempty interior).
(39)
(where im indicates the span of the columns of the matrix)

is a linear subspace. so accessibility from 0 is equivalent to
complete controllability, that is, R (0) n. However, for nonlinear systems, the accessibility property is weaker. For ex-
The accessibility Lie algebra L associated to the system given

by Eq. (37) is the linear span of the set of all vector fields that
can be obtained, starting with the f us and taking all possible
iterated Lie brackets of them. For instance, if u1, u2, u3, u4 are
274
any four control values, the vector field

[[ f u 1 , [ f u 2 , f u 3 ]], [ f u 3 , f u 4 ]]
is in L . The system Eq. (37) satisfies the accessibility rank
condition at the state if the vector space
L ( ) := {X ( ), X L } Rn
ing (11)) steer and drive respectively:

0
cos( + )
0
sin( + )

g1 = , g2 =
0
sin
1
0
Computing some brackets, we get the two new vector fields
[again, we borrow our terminology from (11)]:
sin( + )
cos( + )
wriggle := [steer, drive] =
cos
0
has the maximal possible dimension, that is, n.

There is a special case, of interest because it appears very
often in applications, especially to mechanical systems. This
is the class consisting of systems for which f(x, u) is affine in
u. That is, the equations can be written as
m

d
x = g0 (x) +
ui gi (x)
dt
i=1
and
(41)
for some vector fields gis. It is easy to verify that, for such
systems, L is the Lie algebra generated by taking all possible
iterated Lie brackets starting from the gis.
For example, consider a linear system
d
x = Fx + Gu
dt
Here g0(x) Fx is a linear vector field and the gi(x)s are the
constant vector fields defined by the m columns of the matrix
G. It then follows that, for each state , L () is the span of
the vector F together with the columns of G, FG, . . .,
Fn1G. In particular, for 0, one has that L (0) is the same
as the right-hand side of Eq. (39). Seen in that context, the
following result, which is valid in general, is not surprising:
Theorem. The system Eq. (37) is accessible from if and
only if the accessibility rank condition holds at .
There is a subclass of systems for which far stronger conclusions can be drawn. This subclass includes all purely kinematic mechanical models. It is the class of affine systems
without drift, that is, systems affine in u (as in Eq. 41, but
for which, in addition, g0 0). We say that a system is completely controllable if R () n for every n, that is to
say, every state can be steered to every other state by means
of an appropriate control action.
Theorem. A system affine without drift is completely controllable if and only if the accessibility rank condition holds
at every state.
sin
cos
slide := [wriggle, drive] =
0
0
(The bracket [wriggle, steer] equals drive, so it is redundant,
in so far as checking the accessibility rank condition is concerned.) It turns out that these four brackets are enough to
satisfy the accessibility test. Indeed, one computes
det(steer, drive, wriggle, slide) 1
so there is accessibility from every state. Moreover, since this
system is affine without drift, it is completely controllable. (Of
course, it is quite obvious from physical reasoning, for this
example, that complete controllability holds.)
Consider in particular the problem of accessibility starting
from the special state 0 (corresponding to the problem of
exiting from a parallel parked spot). For 0, wriggle
is the vector (0, 1, 1, 0), a mix of sliding in the x2 direction
and a rotation, and slide is the vector (0, 1, 0, 0) corresponding to sliding in the x2 direction. This means that one can in
principle implement infinitesimally both of these motions.
The wriggling motion is, based on the characterization of Lie
brackets mentioned earlier, the one that arises, in a limiting
sense, from fast repetitions of the following sequence of four
basic actions:
steer drive reverse steer reverse drive ( )
This characterization of controllability for systems without

drift belongs more properly to classical Lie theory and differential geometry. As far as control theory is concerned, most
interesting questions concern more general classes of systems
as well as the design of explicit algorithms for controllability,
sometimes imposing optimality constraints.
This is, essentially, what one does in order to get out of a

tight parking space. Observe that wriggle(0) equals the sum
of slide and rotate [a pure rotation, col(0, 0, 1, 0)]. Interestingly enough, one could also approximate the pure sliding motion in the x2 direction: wriggle, drive, reverse wriggle, reverse
drive, repeat corresponds to the last vector field.
Note that the term t2 in Eq. (40) explains why many iterations of basic motions (*) are required in order to obtain a
displacement in the wriggling direction: the order of magnitude t2 of a displacement in time t is much smaller than t.
The Car Example. In the notations for systems affine in controls, we have the vector fields g1 and g2, which we call (follow-
Remark. If the right-hand side f in Eq. (37) is assumed

merely to be infinitely differentiable, instead of analytic, the
accessibility rank condition is still sufficient for accessibility,

but it is not a necessary condition. Consider for instance the
system on 2, with 2 also as control space, and having equations as follows:

1
0
d
+ u2
x = u1
dt
0
(x1 )
where ei denotes the ith coordinate vector, and more generally

for all u, i and each integer k 0 let
"
""
k
1
k
(Ad0 Xu,i )(x) :=
f k f u f u+
e i f 0 (x)
" =0 0
The accessibility Lie algebra is now defined in terms of iterated Lie brackets of these vector fields, and the accessibility
rank condition is defined in terms of this, analogously to the
continuous time case. The main fact is, then, as follows.
where is the function defined by

(x) = e1/x
275
Theorem. The system Eq. (36) is accessible from zero if and

only if the accessibility rank condition holds at zero.
for x 0, and (x) 0 for x 0. This system is easily shown

to be accessiblein fact, it is completely controllable (any
state can be steered to any other state)but the accessibility
rank condition does not hold.
As in the continuous-time case, for linear (discrete time)

systems, the condition reduces to the usual reachability test.
The vectors Ad0kXu,i are in fact all of the type FkGu, for vectors
u m.
Controllability: Discrete-Time
Accessibility and Controllability of Linearized Systems
We next consider discrete time systems. These are described

by difference equations analogous to those for Eq. (37):
It is easy to prove that, for both continuous and discrete time

systems, if the linearization about an equilibrium point is
controllable as a linear system, then the accessibility condition holds, and, in fact, the system is locally controllable, that
is, is in the interior of R (); see for example, Ref. 6. For
instance, each state near zero can be reached from zero, for
the system
x(t + 1) = f (x(t), u(t))
(42)
where f : n m n is a function. This function now plays

the role of specifying the state at time t 1, provided that
the state at time t was x(t) and the control vector u(t) was
applied at that instant.
We again suppose that the function f is real-analytic. Now
the set of controls, denoted again Inp, is the set of all possible
sequences u(0), . . ., u(T) consisting of vectors in m. An additional assumption which we make for the discrete time system of Eq. (42) is that it is invertible, meaning that the map
f (, u)
is a diffeomorphism for each fixed u; in other words, this map
is bijective and has a nonsingular differential at each point.
Imposing invertibility simplifies matters considerably, and is
a natural condition for equations that arise from the sampling
of continuous time systems, which is one of the main ways in
which discrete time systems appear in practice.
Accessibility is defined as in the continuous-time case, using the analogous definition of R (). We discuss only the special case 0 (the general case is a bit more complicated),
assuming that this state is in equilibrium for the system, that
is,
f (0, 0) = 0
There is an analogue of the accessibility rank condition for
discrete time systems, and this is discussed next.
The notation f u is as stated earlier, and in particular f 0 is
the map f( , 0). Recall that in the discrete case one assumes
invertibility, so that the inverse maps f 1
u are well-defined and
again analytic. For each i 1, . . ., m and each u m let
"
""
Xu,i (x) :=
"
=0
1
f u f u+
e (x)
i
d
x = x21 x2 + sin x1
dt 1
d
x = x1 ex 2 + u cos x1
dt 1
because, up to first order around 0 one has x12x2 sin
x1 x1 h1, x1ex2 u cos x1 x1 u h1, where h1 and
h2 are higher-order terms in states and controls, which means
that the linearization at the equilibrium 0 is the linear
system with matrices

0 1
0
F=
, G=
1 0
1
which is controllable. This is only a sufficient condition. the
system d/dt x u3, in dimension one, is clearly (even) completely controllable, but its linearization at 0 gives the
noncontrollable linear system d/dt x 0. A necessary and
sufficient condition does exist linking accessibility and linear
controllability, but it is more subtle. It is illustrated next, for
simplicity, only for discrete-time systems. In continuous time,
an analogous result holds, but it is slightly more complicated
to state and prove (12).
Observe that accessibility from corresponds to the requirement that the union of the images of the composed maps
f k ( , ) : (Rm )k Rn k 0
cover an open subset, where we are denoting
f k (x, (u1 , . . ., uk )) := f ( f (. . . f ( f (x, u1 ), u2 ), . . ., uk1 ), uk )
for every state x and sequence of controls u1, . . . uk. A simple
argument, based on a standard result in analysis (Sards the-
276
orem) gives that accessibility is equivalent to the following

property: there exists some positive integer k and some sequence of controls u1, . . . uk so that the Jacobian of f k(, )
evaluated at that input sequence,
f k ( , ) [u1 , . . . uk ],
has rank n. Consequently, accessibility is equivalent to accessibility in time exactly k. This Jacobian condition can be restated as follows: Consider the linearization of the system Eq.
(42) along the trajectory
x1 = , x2 = f (x1 , u1 ), x3 = f (x2 , u2 ), . . .
that is, the linear time-varying system
x(t + 1) = Ft x(t) + Gt u(t)

with
Ft =
f [xt , ut ],
x
Gt =
f [xt , ut ]
u
Then, accessibility is equivalent to the existence of some sequence of controls u1, . . . uk for which this linearization is
controllable as a time-varying linear system.
Observability: Continuous-Time
We present a brief outline of a nonlinear observability test,
for the special case of continuous-time systems affine in Eq.
(41), with an output map h : n p added to the system description. Two states and are distinguishable by input/
output experiments if there is at least some input which,
when applied to the system in initial state , gives a different
output than when applied to the system in state . An observable system is one with the property that every pair of distinct states is distinguishable. Thus an observable system is
one for which, at least in principle, it is possible to distinguish
between internal states by means of input/output measurements alone.
Consider the vector space spanned by the set of all functions of the type
Lg i . . . Lg i h j (x)
1
(43)
over all possible sequences i1, . . ., ik, k 0, out of 0, . . .,

m and all j 1, . . ., p, where Lg .g for any function
and any vector field g (f denotes the gradient of f). This is
called the observation space, which we denote as O , associated
to the system. We say that two states x1 and x2 are separated
by O if there exists some O such that (x1) (x2).
One can prove that if two states are separated by O then
they are distinguishable. A sketch of the argument is as
follows. Assume that 1 is indistinguishable from 2 and consider a piecewise constant control which is equal to u1 on
[0, t1), equal to u2 on [t1, t1 t2), . . ., and equal to uk on
[t1 tk1, t1 tk). By indistinguishability,
we know that the resulting output at time t t1
tk is equal for both. In general, we denote the jth coordinate
of this output value by
h j (t1 , t2 , . . ., tk , u1 , u2 , . . ., uk , )
(44)
if the initial state is . It follows that the derivatives with

respect to the tis of this output are also equal, for 1 and 2,
and for every such piecewise constant control. One may prove
by induction that
"
"
k
"
h (t , t , . . ., tk , u1 , u2 , . . ., uk , )
t1 . . . tk "t =t =...=0 j 1 2
1
= LX LX . . . LX h j ( )
1
where Xl(x) g0(x) uilgi(x). This expression is a multilinear function of the uils, and a further derivation with respect
to these control value coordinates shows that the generators
in Eq. (43) must coincide at x1 and x2. It turns out that (for
analytic vector fields, as considered in this exposition), separability by O is necessary as well as sufficient, because Eq.
(44) can be expressed as a power series in terms of the generators of Eq. (43). Thus, observability is equivalent to separation by the functions in O .
The observability rank condition at a state is the condition that the dimension of the span of
m
i1
{Lg i . . . Lg i h j ( )|i1 , . . ., ik {0, . . ., m}, j = 1, . . ., p} Rn

1
be n. An application of the implicit function theorem shows

that this is sufficient for the distinguishability of states
near .
Remarks
The early 1970s saw the beginnings of the systematic study
of controllability and observability questions for continuous
time nonlinear systems. Building upon previous work (13,14)
on partial differential equations, the papers (15), (16), and
(17), among others, provided many of the basic accessibility
and controllability results. In discrete time, one of the early
papers was (18). For more details on accessibility at an expository level, see for instance Refs. 19, 20, 21, or 6 in continuous
time, and 22 in discrete time. These references should also be
consulted for justifications of all statements given here without proof. For affine systems without drift, for which the accessibility rank condition completely characterizes controllability, Lie techniques can be used to provide efficient
algorithms for constructing controls; see for instance Ref. 23.
Similarly, there are useful controllability algorithms available for special classes of systems such as so-called flat systems, and for systems exhibiting special symmetries which
arise from mechanical constraints; see, e.g., Ref. 24, and references therein.
A complete characterization of controllability, as opposed
to the weaker accessibility property, has eluded solution, even
though substantial progress has been made (see for instance
Ref. 25 and the references there). One way to understand the
difficulty inherent in checking controllability is by formulating the problem in terms of computational complexity. In Ref.
26, it is shown that, for wide classes of systems, testing for
controllability is an NP-hard problem (hence most likely impossible to ever be amenable to an efficiently computable
characterization); this contrasts with accessibility, which, for
the same classes of systems, can be checked in polynomial
time.
A different type of linearization is also of interest in control theory. Instead of merely taking first-order approxima-
tions, one searches for changes of state and input variables

which render a nonlinear system linear. For example, take
the nonlinear system x x2 u. This system becomes linear
if we consider the new input variable v x2 u, since the
equations become x v. Observe that this linearized system
may be stabilized globally by means of the feedback v x.
(In terms of the original variables, this means that we picked
the feedback law u x2 x.) In general, finding linearizing
transformations is not as obvious as in this example, of
course; much theory has been devoted to the search for Liealgebraic conditions which guarantee that a system can be so
transformed, or as one says, feedback linearized. This line of
work started with Brockett in the late 1970s, and major results were obtained by Jakubczyk, Respondek, Hunt, Su, and
Meyer, in the early 1980s. See Ref. 6 for an elementary introduction to the subject. For some related recent developments,
see Ref. 27.
LINEAR INFINITE-DIMENSIONAL SYSTEMS
Distributed parameter systems are those where the state
variable is spatially dependent. Typical examples are systems
described by the heat equation, wave equation, beam equations, or delay-differential equations. Due to such a dependence on spatial variables, the state space generally becomes
infinite-dimensional. The term distributed parameter systems is thus often used synonymously for systems with infinite-dimensional state space. In what follows, we shall also
employ this convention, and discuss controllability/observability of infinite-dimensional systems.
A formal generalization of the finite-dimensional (continuous-time) definitions would lead to the following state equations:
d
x(t) = Fx(t) + Gu(t)
dt
(45)
y(t) = Hx(t) + Ju(t)
(46)
where the input values u(t), state x(t), and output values y(t)
are elements of, for instance, Hilbert spaces. For example, if
we consider the heat equation
2x
x
(t, ) = 2 (t, ) + G( )u(t)
t
with state space L2[0, 1] with boundary condition x(t, 0)

x(t, 1) 0, then F is set to be the differential operator 2 /2
with domain
D(F ) := {x H 2 (0, 1) : x(0) = x(1) = 0}
Here H2(0, 1), which is the natural state space for these equations, is the space of functions in L2[0, 1] whose second-order
derivatives again belong to L2[0, 1]. The important point to
notice is that the F operator is in general, as in this example,
not defined on the whole space, and is discontinuous (unbounded). Thus, some care has to be taken when extending
the finite-dimensional framework. We need to assume that F
is the infinitesimal generator of a strongly continuous semigroup S(t), which plays a role analogous to that played by eFt
in the finite-dimensional case. Typically, as in the previous
u(t)
v(t h)
v(t)
G0
+
+ +
ehs
1/s
277
y(t)
H0
F0
F1
Figure 2. Retarded delay-differential system.
example, F is defined only on a domain D (F) which is a dense

subspace of the state space X.
Another example often encountered in practice is provided
by delay systems. Consider a retarded delay-differential system (see Fig. 2):
d
v(t) = F0 v(t) + F1 v(t h) + G0 u(t)
dt
y(t) = H0 v(t h), v(t) R n
(47)
This system is described by the functional differential

equation
#
$ # $
F1 Eh
G0
v(t)
+
u(t)
0
zt ( )
=: Fx(t) + Gu(t)
#
$
x(t)
y(t) = [0 H0 Eh ]
, zt () (L2 [h, 0])n
zt ()
#
$
F0
d v(t)
=
dt zt ()
0
(48)
(49)
where x(t) : [v(t) zt( )], and zt( ) is the state in the delay
element ehs which is related to v(t) via zt() v(t ). Eh
denotes the point evaluation operator Ehz( ) : z(h). The
domain of F in Eq. (48) is [v z( )] R n (H2(h, 0))n :
z(0) v. Here the input operator G is bounded but the output
operator C [0 C0Eh] is not. (Point evaluation in L2 cannot
be continuous.)
We thus consider the abstract system Eqs. (45), (46) with
x(t) in a Hilbert space X, with input/output variables u(t) and
y(t) being m- and p-valued. The operators G : m X and
H : X p are often assumed to be bounded. However, there
are many systems that do not satisfy this property: G may
not take values in X or H may be only densely defined. The
system described by Eqs. (48), (49) is an example of the latter
case. For brevity of exposition, however, we will not be very
rigorous about this point in what follows. Also, since J plays
no role in controllability/observability, we will assume J 0.
Denote this system by (X, F, G, H).
The solution of Eq. (45) can be written as

x(t) = S(t)x0 +
S(t )Gu( ) d
0
We assume that u is square integrable. Such a solution is

often called the mild solution.
Controllability and Observability
The technical problems that arise due to the infinite-dimensionality include the following:
278
In general, canonical (controllable and observable) realizations are not unique

Controllability need not guarantee stabilizability; similarly, observability need not imply existence of an observer
The problems involved are mostly topological, and this is one
of the reasons why there are several nonequivalent
controllability/observability notions. We start by introducing
controllability concepts.
In analogy with the finite-dimensional case, one wants to
say that a system is controllable (or reachable) if every state
x X can be reached from the origin by suitable application
of an input. For distributed parameter systems, however, this
does not usually occur due to the infinite-dimensionality of
the state space.
For each T 0, define a map (T) : L2[0, T] X by

S(T )Gu( )d
(50)
We say that a state x is controllable from zero in time T if x

belongs to the range of (T). The system is simply said to
be controllable from zero if
%
range (T )
T >0
The system is said to be exactly controllable (reachable)

if
X=
range (T )
(51)
T >0
Unlike the finite-dimensional case, exact controllability does

not occur very often. In fact, when the G operator is bounded
(and takes values in X), it has finite rank, and hence it can
be shown that (T) is a compact operator. Thus the righthand side of Eq. (51) cannot be equal to X as the union of
images of compact operators. (Of course, this argument does
not apply when G does not take values in X; extended study
has been made on systems with boundary controls; see Refs.
2830.)
We are thus interested in a less restrictive condition of approximate controllability (reachability). This requires that, in
place of Eq. (51),
X=
: X L2loc [0, ) : x HS(t)x,
t0
(53)
2
where Lloc
[0, ) is the space of locally square integrable functions. Then is observable if and only if is one-to-one.
For finite-dimensional systems these observability notions
imply the following consequences:
(T )u :=
subspace of X and is indeed extendable to X as a continuous

mapping. While H is often unbounded, (T) still happens to
be continuous in many examples [for example, for delay systems such as Eqs. (48), (49)].
The system is said to be observable in bounded time
T 0 if (T) is a one-to-one mapping, that is, if (T)x 0
(almost everywhere) implies x 0. It is observable if (T)x
0 for all T 0 occur only if x 0. (This concept is often called
approximate observability, indicating its duality to approximate controllability.) To state the observability condition differently, define the observability map
range (T )
(52)
T >0
Here M denotes the closure of M in X. This means that any

state x X can be approximated arbitrarily closely by controllable elements. When X is a finite-dimensional space, approximate controllability coincides with standard controllability.
We now give definitions for several notions of observability. Fix T 0. Define (T) : X L2[0, T] by
(T ) : x HS(t)x,
0tT
Because of the strong continuity of S(t), this mapping is well

defined if H is a bounded operator. If H is unbounded, we
generally require that this mapping be defined on a dense
Observability always implies observability in bounded

time
Initial states can be determined continuously from output data
The latter is an important key to constructing observers and
state estimators. For infinite-dimensional systems, this property does not hold, and we define a topological notion of observability as follows.
The system is said to be topologically observable if is
continuously invertible when its codomain is restricted to
im . It is topologically observable in bounded time T 0 if
the same holds of (T). (For the same reason as in the case
of approximate observability, some authors adopt the terminology exact observability, again indicating its duality to exact
controllability.)
Topological observability requires that the initial state determination be well posed. The system given by Eqs. (48) and
(49) is topologically observable. This property is also crucial
in proving uniqueness of canonical realizations. That is, if we
understand that a canonical realization is one that is approximately controllable and topologically observable, then there
is essentially only one canonical realization for a given input/
output behavior (31).
Duality
Controllability and observability are dual concepts to each
other. Reversing the time axis, we see that the controllable
states are those in the image of the mapping
:
%
T >0
L2 [0, T] X : u
S(t)Gu(t) dt
(54)
The adjoint of is easily computable (at least formally) as

: X L2loc [0, ) : x G S (t)x,
t0
(55)
Define * : (X, F*, H*, G*) as the dual system of . Then

the mapping of Eq. (55) is the observability map of *. Since
a bounded linear f : X Y has dense image if and only if its
adjoint satisfies ker f* 0, we can say that is approximately controllable if and only if * is observable. Similarly,
is observable if and only if * is approximately controllable.

Also, is topologically observable if and only if the adjoint
mapping of in Eq. (53) is a surjective mapping.
Related Concepts and Controllability Criteria
When is exponentially stable in the sense that there exist
M, 0 such that
complete with suitable feedback. Since feedback does not alter controllability, this latter condition preserves controllability.
Similar spectral tests are possible for systems with spectral F operators. For example, consider the case where F is
the self-adjoint operator defined by a spectral representation
(32):
S(t) Met
Fx =
n=1
then it is possible to relate controllability/observability to Lyapunov equations. In this case it is possible to extend the domain of and restrict the codomain of to L2[0, ). Define
P := =
Q := =
S(t)GG S (t) dt
Gu :=
x1 , x2 D(F )
A similar statement holds for controllability.

Null controllability refers to the property that every state
can be steered back to the origin by application of a suitable
input. Its dual concept is the final state observability (or reconstructibility). Similar theory is possible for these properties.
Closely related concepts are those of stabilizability and detectability. However, in the infinite-dimensional context, controllability need not imply stabilizability. The same can be
said of observability and detectability. With a finite-rank input term one may not have enough control freedom to stabilize possibly infinitely many unstable poles. Stabilizability often requires that there be only finitely many unstable poles.
One may also wish to say that a system is spectrally controllable if all its finite-dimensional modal subsystems are
controllable (in the usual sense for finite-dimensional systems). Some care must be exercised to be clear about the
meaning of modal subsystems. Even if there exists a decomposition to modal subsystems, some modes may not be finitedimensional.
For systems where eigenfunction expansion is possible, one
can say that the system is approximately controllable if its
eigenfunctions are complete and it is spectrally controllable.
Systems with self-adjoint F operators are easy examples.
Then several rank conditions are possible in testing controllability of each subsystem. Another class of systems that satisfy
this condition is that of delay differential systems. For example, the retarded system of Eqs. (48), (49) is approximately
controllable if and only if
1. rank[I F0 F1eh G0] n for every and
2. rank[F1 G0] n
The first condition guarantees spectral controllability and the
second guarantees that the system can be made eigenfunction
(x, n j )n j
j=1
m

gi u i ,
gi X
i=1
Qx1 , Fx2 + Fx1 , Qx2 = Hx1 , Hx2 ,
rn

where n : n 1, 2, . . . are distinct real numbers listed in

decreasing order, nj, j 1, 2, . . ., rn, n 1, 2, . . . is an
orthonormal basis in X, and ( , ) denotes the inner product.
Let G be given by
S (t)H HS(t) dt
Then is approximately controllable if and only if P is positive definite, and observable if and only if Q is positive definite. P and Q are called controllability and observability
Grammians. Actually, when is observable, Q is a unique
self-adjoint solution to the Lyapunov equation
279
Then (F, G, H) (H is irrelevant here and hence not defined) is

approximately controllable if and only if
(gm , n 1 )
(g1 , n 1 )
..
..
= rn
rank
.
.
(g1 , n r n ) (gm , n r n )
Remarks
Controllability/observability questions for distributed parameter systems were first addressed by Fattorini (33). Since then
numerous papers have appeared, and the literature is too
vast to be listed or surveyed here. For developments up to
1978, consult Russels survey (34). The recent textbook by
Curtain and Zwart (32) gives an extensive set of further references. For abstract operator settings of linear systems, see
Refs. 35, 36, and 37. There is also a vast amount of literature
concerning controllability of systems described by partial differential equations. For the developments along this line, consult Refs. 28, 29, and 30. We discuss here some limited references that deal with further related subjects.
Controllability/observability for delay systems has received considerable attention: see the work by Datko, Delfour,
Langenhop, Kamen, Manitius, Mitter, OConnor, Pandolfi,
Salamon, Triggiani, Yamamoto, and others [references cited
in (32)]. The spectral condition for approximate controllability
of retarded delay systems cited previously is due to the work
of Manitius and Triggiani (38,39). It is also extended to various situations including neutral systems; see, for example,
Refs. 40, 41, and 42.
Controllability and observability are also important in realization theory. It was shown by Baras, Brockett, and Fuhrmann (43) that an approximately controllable and observable
realization need not be unique. One needs a stronger condition to guarantee uniqueness; the results cited in Ref. 31 are
one example.
As with finite-dimensional systems, controllability and observability are closely related to the question of coprimeness
of factorizations of transfer functions. For example, a factorization is (left) coprime (in an appropriately defined sense) if
a certain system associated with this factorization is approximately controllable (see, for example, Ref. 44). However, in
280
contrast to the finite-dimensional context, there is a large variety of freedom in choosing the algebra over which the factorization is considered. Typical examples are the Callier-Desoer
algebra, the space of H functions, and the algebra of distributions with compact support. Each different choice leads to
a different theory of realization/stabilization/controller parameterization as is the case with the finite-dimensional theory, but with much wider freedom. Each has its advantage in
different contexts, reflecting the variety of distributed parameter systems. The theories can therefore hardly be expected
to be complete.
Another topic that is in close connection with
controllability/observability is the existence and uniqueness
of solutions to Riccati equations. This topic is also related
with H control theory and under active research. The current
trend is in spectral factorization in an abstract functional
equation setting. To this end, various questions concerning
the well-posedness of system equations and transfer functions
arise and are currently under study.
BIBLIOGRAPHY
1. J. W. Polderman and J. C. Willems, Introduction to Mathematical
System Theory: A Behavioral Approach, New York: SpringerVerlag, 1998.
2. R. E. Kalman, On the general theory of control systems, Proc.
1st Int. Cong. IFAC, Moscow 1960, London: Butterworths, 1960,
pp. 481492.
3. R. E. Kalman, Y. C. Ho, and K. Narendra, Controllability of linear dynamical systems, Contributions to Differ. Equations, 1:
189213, 1963.
4. R. E. Kalman, Lectures on controllability and observability,
Centro Internazionale Matematico Estrio, Bologna, 1968.
5. R. E. Kalman, P. L. Falb, and M. A. Arbib, Topics in Mathematical System Theory, New York: McGraw-Hill, 1969.
6. E. D. Sontag, Mathematical Control Theory: Deterministic Finite
Dimensional Systems, 2nd ed., New York: Springer-Verlag, 1998.
7. W. J. Rugh, Linear System Theory, 2nd ed., Upper Saddle River,
NJ: Prentice-Hall, 1996.
8. K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control,
Upper Saddle River, NJ: Prentice-Hall, 1996.
9. W. L. Brogan, Modern Control Theory, Upper Saddle River, NJ:
10. J. C. Willems, Paradigms and puzzles in the theory of dynamical
systems, IEEE Trans. Autom. Control, AC-36: 259294, 1991.
11. E. Nelson, Tensor Analysis, Princeton, NJ: Princeton University
Press, 1967.
12. E. D. Sontag, Universal nonsingular controls, Syst. Control Lett.,
19: 221224, 1992.
ber Systeme von linearen partiellen differen13. W. L. Chow, U
tialgleichungen erster ord-nung, Math. Ann., 117: 98105, 1939.
14. R. Hermann, On the accessibility problem in control theory, in
Int. Symp. Differ. Equations and Mech., New York: Academic
Press, 1973.
15. C. Lobry, Controllabilite des syste`mes non lineaires, SIAM J.
Contr., 8: 573605, 1970.
16. H. J. Sussmann and V. Jurdjevic, Controllability of nonlinear
systems, J. Differ. Equations 12: 95116, 1972.
17. A. Krener, A generalization of Chows theorem and the bang-bang
theorem to nonlinear control systems, SIAM J. Control, 12: 43
52, 1974.
18. M. Fliess and D. Normand-Cyrot, A group-theoretic approach to

discrete-time nonlinear controllability, Proc. IEEE Conf. Dec. Control, 1981.
19. R. Hermann and A. J. Krener, Nonlinear controllability and observability, IEEE Trans. Autom. Control, 22: 728740, 1977.
20. H. J. Sussmann, Lie brackets, real analyticity, and geometric
control, in R. W. Brockett, R. S. Millman, and H. J. Sussmann,
(eds.), Differential Geometric Control Theory, Boston: Birkhauser, 1983.
21. A. Isidori, Nonlinear Control Systems, 3rd ed., London: SpringerVerlag, 1995.
22. B. Jakubczyk and E. D. Sontag, Controllability of nonlinear discrete-time systems: a Lie-algebraic approach, Invited Expository
Article, SIAM J. Control Opt., 28: 133, 1990.
23. I. Kolmanovsky and N. H. McClamroch, Developments in nonholonomic control problems, Control Syst. Mag., 15 (6): 2036, December, 1995.
24. M. Kawski and H. J. Sussmann, Noncommutative power series
and formal Lie-algebraic techniques in nonlinear control theory,
in U. Helmke, D. Praetzel-Wolters, and E. Zerz (eds.), Operators,
Systems and Linear Algebra: Three Decades of Algebraic Systems
Theory, Stuttgart: B. G. Teubner, 1997, pp. 111129.
25. H. J. Sussmann, A general theorem on local controllability, SIAM
J. Control Opt., 25: 158194, 1987.
26. E. D. Sontag, Controllability is harder to decide than accessibility, SIAM J. Control Opt., 26: 11061118, 1988.
27. D. Tilbury, R. M. Murray, and S. S. Sastry, Trajectory generation
for the n-trailer problem using Goursat normal form, IEEE
Trans. Autom. Control, 40: 802819, 1995.
28. C. Bardos, G. Lebeau, and J. Rauch, Sharp sufficient conditions
for the observation, control and stabilization of waves from the
boundary, SIAM J. Control Optimiz., 30: 10241065, 1992.
29. V. Komornik, Exact Controllability and StabilizationThe Multiplier Method, New York: Wiley, 1994.
30. J. E. Lagnese and J.-L. Lions, Modelling Analysis and Control of
Thin Plates, New York: Springer-Verlag, 1989.
31. Y. Yamamoto, Realization theory of infinite dimensional linear
systems, part 1, Math. Syst. Theory, 15: 5577, 1981.
32. R. F. Curtain and H. J. Zwart, An Introduction to Infinite-Dimensional Linear Systems Theory, New York: Springer-Verlag, 1995.
33. H. O. Fattorini, Some remarks on complete controllability, SIAM
J. Control, 4: 686694, 1966.
34. D. L. Russel, Controllability and stabilizability theory for linear
partial differential equations: recent progress and open problems,
SIAM Rev., 20: 639739, 1978.
35. R. F. Curtain and A. J. Prichard, Infinite dimensional linear systems theory, Springer Lecture Notes in Control and Information
Sciences, 8: 1978.
36. P. A. Fuhrmann, Linear Systems and Operators in Hilbert Space,
New York: McGraw-Hill, 1981.
37. G. Weiss, Transfer functions of regular linear systems, part I:
characterization of regularity, Trans. Amer. Math. Soc., 342: 827
854, 1994.
38. A. Manitius and R. Triggiani, Function space controllability of
linear retarded systems: a derivation from abstract operator conditions, SIAM J. Control Optimiz., 16: 595645, 1978.
39. A. Manitius, Necessary and sufficient conditions of approximate
controllability for general linear retarded systems, SIAM J. Control Optimiz., 19: 516532, 1981.
40. D. A. OConnor and T. J. Tarn, On the function space controllability of linear neutral systems, SIAM J. Control Optimiz., 21: 306
329, 1983.
41. D. Salamon, Control and Observation of Neutral Systems, Boston:
Pitman, 1984.

42. Y. Yamamoto, Reachability of a class of infinite-dimensional linear systems: an external approach with applications to general
neutral systems, SIAM J. Control Optimiz., 27: 217234, 1989.
43. J. S. Baras, R. W. Brockett, and P. A. Fuhrmann, State-space
models for infinite-dimensional systems, IEEE Trans. Autom.
Control, AC-19: 693700, 1974.
44. Y. Yamamoto, Pseudo-rational input/output maps and their realizations: a fractional representation approach to infinite-dimensional systems, SIAM J. Control Optimiz., 26: 14151430, 1988.
A. C. ANTOULAS
Rice University
E. D. SONTAG
Rutgers University
Y. YAMAMOTO
Kyoto University
CONTROL, LIGHTING. See LIGHTING CONTROL.

CONTROL, NYQUIST CRITERION. See NYQUIST CRITERION, DIAGRAMS, AND STABILITY.
CONTROL OF EXCITATION. See EXCITATION CONTROL

IN POWER SYSTEMS.
CONTROL OF INDUSTRIAL SYSTEMS. See INDUSTRIAL CONTROL.
CONTROL OF JET ENGINES. See JET ENGINE CONTROL,

IMPLEMENTATIONS.
CONTROL OF PERIODIC SYSTEMS. See PERIODIC

CONTROL.
CONTROL, OSCILLATORY. See OPEN-LOOP OSCILLATORY CONTROL.
CONTROL, POWER SYSTEM. See POWER SYSTEM CONTROL;
POWER SYSTEM TRANSIENTS.
CONTROL, PROCESS. See PROCESS CONTROL.

CONTROL, RELAY. See RELAY CONTROL.
CONTROL, ROBUST. See ROBUST CONTROL; ROBUST CONTROL ANALYSIS.
CONTROL SYSTEM ANALYSIS. See INTERVAL ANALYSIS

FOR CIRCUITS;
SERVOMECHANISMS.
281
194
CHAOS, BIFURCATIONS, AND THEIR CONTROL
which have explicit formulation for a specified physical

system.
For the discrete-time setting, a nonlinear dynamical system is described by either a difference equation
x k+1 = f (xx k , k; p ),
NONLINEAR DYNAMICS
Unlike linear systems, many nonlinear dynamical systems do
not show orderly, regular, and long-term predictable responses to simple inputs. Instead, they display complex, random-like, seemingly irregular, yet well-defined output behaviors. This dynamical phenomenon is known as chaos.
The term chaos, originating from the Greek word ,
was designated as the primeval emptiness of the universe
before things came into being of the abyss of Tartarus, the
underworld. . . . In the later cosmologies Chaos generally
designated the original state of things, however conceived.
The modern meaning of the word is derived from Ovid, who
saw Chaos as the original disordered and formless mass, from
which the maker of the Cosmos produced the ordered universe (1). There also is an interpretation of chaos in ancient
Chinese literature, which refers to the spirit existing in the
center of the universe (2). In modern scientific terminology,
chaos has a fairly precise but rather complicated definition by
means of the dynamics of a generally nonlinear system. For
example, in theoretical physics, chaos is a type of moderated
randomness that, unlike true randomness, contains complex
patterns that are mostly unknown (3).
Bifurcation, as a twin of chaos, is another prominent phenomenon of nonlinear dynamical systems: Quantitative
change of system parameters leads to qualitative change of
system properties such as the number and the stability of system response equilibria. Typical bifurcations include transcritical, saddle-node, pitchfork, hysteresis, and Hopf bifurcations. In particular, period-doubling bifurcation is a route to
chaos. To introduce the concepts of chaos and bifurcations as
well as their control (4,5), some preliminaries on nonlinear
dynamical systems are in order.
Nonlinear Dynamical Systems
A nonlinear system refers to a set of nonlinear equations,
which can be algebraic, difference, differential, integral, functional, and abstract operator equations, or a certain combination of these. A nonlinear system is used to describe a physical device or process that otherwise cannot be well defined by
a set of linear equations of any kind. The term dynamical
system is used as a synonym of mathematical or physical system, in which the output behavior evolves with time and/or
other varying system parameters (6).
In general, a continuous-time dynamical system can be described by either a differential equation
x = f (xx, t; p ),
t [t0 , )
(1)
F : x g (xx, t; p ),
t [t0 , )
(2)
where x x(t) is the state of the system, p is a vector of

variable system parameters, and f and g are continuous (or
differentiable) nonlinear functions of comparable dimensions,
(3)
or a map
F : x k g (xx k , k; p ),
k = 0, 1, . . .
(4)
where notation is similarly defined. Repeatedly iterating the

discrete map F backward yields
x k = F (xx k1 ) = F (F (xx k2 )) = = F k (xx 0 )
where the map can also be replaced by a function, f, if the
system is given via a difference equation, leading to
x k = f f (xx 0 ) = f k (xx 0 )

k times
where denotes composition operation of functions or mappings.

The dynamical system of Eq. (1) is said to be nonautonomous when the time variable, t, appears separately in the system function f (e.g., a system with an external time-varying
force input); otherwise, it is said to be autonomous and is expressed as
x = f (xx; p ),
t [t0 , )
(5)
Classification of Equilibria
For illustration, consider a general two-dimensional autonomous system

x = f (x, y)
(6)
y = g(x, y)
with given initial conditions (x0, y0), where f and g are two
smooth nonlinear functions that together describe the vector
field of the system.
The path traveled by a solution of Eq. (6), starting from
the initial state (x0, y0), is a solution trajectory, or orbit, of the
system and is sometimes denoted by t(x0, y0). For autonomous systems, two different orbits will never cross each other
(i.e., never intersect) in the x-y plane. This x-y coordinate
plane is called the (generalized) phase plane (phase space in
the higher-dimensional case). The orbit family of a general
autonomous system, corresponding to all possible initial conditions, is called solution flow in the phase space.
Equilibria, or fixed points, of Eq. (6), if they exist, are the
solutions of two homogeneous equations:
f (x, y) = 0
or a map
k = 0, 1, . . .
and g(x, y) = 0
An equilibrium is denoted by (x, y). It is stable if all the

nearby orbits of the system, starting from any initial conditions, approach it; it is unstable if the nearby orbits are moving away from it. Equilibria can be classified, according to
their stabilities, as stable or unstable node or focus, saddle
Stable node
Unstable node
Stable focus
Unstable focus
Saddle
Center
point and center, as summarized in Fig. 1. The type of equilibria is determined by the eigenvalues, 1,2, of the system Jacobian

J :=
fx
gx
fy
gy
195
with f x : f /x, f y : f /y, and so on, all evaluated at (x, y).

If the two Jacobian eigenvalues have real parts R 1,2 0,
the equilibrium (x, y) at which the linerization was taken, is
said to be hyperbolic.
Figure 1. Classification of two-dimensional equilibria: Stabilities are determined by Jacobian eigenvalues.
some constant tp 0. The minimum value of such tp is called

the ( fundamental) period of the periodic solution, while the
solution is said to be tp-periodic.
A limit cycle of a dynamical system is a periodic solution
of the system that corresponds to a closed orbit in the phase
space and possesses certain attracting (or repelling) properties. Figure 2 shows some typical limit cycles for the two-dimensional case: (a) an inner limit cycle, (b) an outer limit
cycle, (c) a stable limit cycle, (d) an unstable limit cycle, and
(e) and (f) saddle limit cycles.
Limit Sets and Attractors
This theorem guarantees that for the hyperbolic case, one

can study the linearized system instead of the original nonlinear system, with regard to the local dynamical behavior of the
system within a (small) neighborhood of the equilibrium (x,
y). In other words, there exist some homeomorphic maps that
transform the orbits of the nonlinear system into orbits of its
linearized system in a (small) neighborhood of the equilibrium. Here, a homeomorphic map (or a homeomorphism) is a
continuous map whose inverse exists and is also continuous.
However, in the nonhyperbolic case the situation is much
more complicated, where such local dynamical equivalence
does not hold in general.
The most basic problem in studying the general nonlinear dynamical system of Eq. (1) is to understand and/or to solve for
the system solutions. The asymptotic behavior of a system
solution, as t , is called the steady state of the solution,
while the solution trajectory between its initial state and the
steady state is the transient state.
For a given dynamical system, a point x in the state space
is an -limit point of the system state orbit x(t) if, for every
open neighborhood U of x, the trajectory of x(t) will enter U
at a (large enough) value of t. Consequently, x(t) will repeatedly enter U infinitely many times, as t . The set of all
such -limit points of x(t) is called the -limit set of x(t) and
is denoted x. An -limit set of x(t) is attracting if there exists
an open neighborhood V of x such that whenever a system
orbit enters V then it will approach x as t . The basin of
attraction of an attracting set is the union of all such open
neighborhoods. An -limit set is repelling if the nearby system orbits always move away from it. An attractor is an limit set having the property that all orbits nearby have it as
their -limit sets.
For a given map, F, and a given initial state, x0, an -limit
set is obtained from the orbit Fk(x0) as k . This -limit
set, x, is an invariant set of the map, in the sense that
F(x) x. Thus, -limit sets include equilibria and periodic
orbits.
Periodic Orbits and Limit Cycles
Poincare Maps
A solution orbit, x(t), of the nonlinear dynamical system of

Eq. (1) is a periodic solution if it satisfies x(t tp) x(t) for
Assume that the general n-dimensional nonlinear autonomous system of Eq. (5) has a tp-periodic limit cycle, , and let
Theorem 1 (Grobman-Hartman) If (x, y) is a hyperbolic

equilibrium of the nonlinear dynamical system of Eq. (6), then
the dynamical behavior of the nonlinear system is qualitatively the same as (i.e., topologically equivalent to) that of its
linearized system,

x
x
=J
y
y
in a neighborhood of the equilibrium (x, y).
196
Figure 2. Periodic orbits and limit cycles.
(a)
(b)
(c)
(d)
(e)
(f)
x* be a point on the limit cycle and be an (n 1)-dimensional hyperplane transversal to at x*, as shown in Fig. 3.
Here, the transversality of to at x* means that and the
tangent line of at x* together span the entire n-dimensional
space (hence, this tangent line of cannot be tangent to at
x*). Since is tp-periodic, the orbit starting from x* will return to x* in time tp. Any orbit starting from a point, x, in a
small neighborhood U of x* on will return and hit at a
point, denoted P(x), in the vicinity V of x*. Therefore, a map
P : U V can be uniquely defined by , along with the solution flow of the autonomous system. This map is called the
Poincare map associated with the system and the cross section . For different choices of the cross section , Poincare
maps are similarly defined. Note that a Poincare map is only
locally defined and is a diffeomorphismnamely, a differentiable map that has an inverse and the inverse is also differentiable. If a cross section is suitably chosen, the orbit will
repeatedly return and pass through the section. The Poincare
map together with the first return orbit is particularly important, which is called the first return Poincare map. Poincare
maps can also be defined for nonautonomous systems in a
similar way, where, however, each return map depends on
the initial time in a nonuniform fashion.
Homoclinic and Heteroclinic Orbits

Let x* be a hyperbolic equilibrium of a diffeomorphism P :
Rn Rn, which can be of either unstable, center, or saddle
type; t(x) be a solution orbit passing through x*; and x* be
the -limit set of t(x). The stable manifold of x*, denoted
Ms, is the set of such points x* that satisfy t(x*) x* as
t ; the unstable manifold of x*, Mu, is the set of points
x* satisfying t(x*) x* as t .
Suppose that s(x*) and u(x*) are cross sections of the
stable and unstable manifolds of t(x), respectively, which intersect at x*. This intersection always includes one constant
orbit, t(x) x*. A nonconstant orbit lying in the intersection
is called a homoclinic orbit and is illustrated in Fig. 4(a). For
two equilibria, x1 x2, of either unstable, center, or saddle
type, an orbit lying in s(x1) u(x2), or in u(x1) s(x2), is
called a heteroclinic orbit. A heteroclinic orbit is dipicted in
Fig. 4(b), which approaches one equilibrium as t but converges to another equilibrium as t .
It is known that if a stable and an unstable manifold intersect at a point, x0 x*, then they will do so at infinitely many
points, denoted xkk

counted in both forward and backward
directions, which contains x0. This sequence, xk, is a homo-
P (x)
x*
x
x*
Figure 3. Schematic illustration of the Poincare map and cross

section.
(a)
x1
x2
(b)
Figure 4. Schematic illustration of homoclinic and heteroclinic

orbits.
197
and, by changing variables if necessary, assume the origin,

x 0, is an equilibrium of the system satisfying f(0, t) 0.
Lyapunov stability theory concerns various stabilities of the
zero equilibrium of Eq. (7).
Stability in the Sense of Lyapunov. The equilibrium x 0 of
Eq. (7) is said to be stable in the sense of Lyapunov if for any
0 and any initial time t0 0, there exists a constant,
(, t0) 0, such that
Figure 5. Illustration of a Silinkov-type homoclinic orbit.
xx (t0 ) <

clinic orbit in which each xk is called a homoclinic point. This
special structure is called a homoclinic structure, in which the
two manifolds usually do not intersect transversally. Here,
two manifolds are said to intersect transversally if their tangent planes together span the entire space (hence, these two
tangent planes cannot coincide at the intersection). This
structure is unstable in the sense that the connection can be
destroyed by very small perturbations. If they intersect transversally, however, a transversal homoclinic point will imply
infinitely many other homoclinic points. This eventually leads
to a picture of stretching and folding of the two manifolds.
Such complex stretching and folding of manifolds are key to
chaos, which generally implies the existence of a complicated
Smale horseshoe map and is supported by the following mathematical theory (7).
Theorem 2 (Smale-Birkhoff) Let P : Rn Rn be a diffeomorphism with a hyperbolic equilibrium x*. If the cross sections of the stable and unstable manifolds, s(x*) and u(x*),
intersect transversally at a point other than x*, then P has a
horseshoe map embedded within it.
For three-dimensional autonomous systems, the case of an
equilibrium with one real eigenvalue, , and two complex conjugate eigenvalues, j, is especially interesting. For example, the case with 0 and 0 gives a Silnikov type of
homoclinic orbit, which is illustrated in Fig. 5.
Theorem 3 (Silnikov) Let t be the solution flow of a
three-dimensional autonomous system that has a Silnikovtype homoclinic orbit . If , then t can be extremely
slightly perturbed to t, such that t has a homoclinic orbit ,
near , of the same type, and the Poincare map defined by a
cross section, transversal to , has a countable set of Smale
horseshoes.
Stabilities of Systems and Orbits
Stability theory plays a central role in both dynamical systems and automatic control. Conceptually, there are different
types of stabilities, among which Lyapunov stabilities and the
orbital stability are essential for chaos and bifurcations
control.
xx (t) < ,
t t0
(8)
Here and throughout, denotes the standard Euclidean

norm of a vector.
It should be emphasized that the constant in the preceeding equation generally depends on both and t0. It is particularly important to point out that, unlike autonomous systems,
one cannot simply assume the initial time t0 0 for a nonautonomous system in a general situation. The stability is said
to be uniform, with respect to the initial time, if this constant,
(), is indeed independent of t0 over the entire time interval of interest.
Asymptotic Stability. In both theoretical studies and applications, the concept of asymptotic stability is of most importance.
The equilibrium x 0 of Eq. (7) is said to be asymptotically
stable if there exists a constant, (t0) 0, such that
xx (t0 ) <
xx (t) 0
as t
(9)
This asymptotical stability is said to be uniform if the existing

constant is independent of t0, and is said to be global if the
convergence (x 0) is independent of the starting point
x(t0) over the entire domain on which the system is defined
(e.g., when ).
Orbital Stability. The orbital stability differs from the Lyapunov stabilities in that it concerns the structural stability of
a system orbit under perturbation.
Let t(x0) be a tp-periodic solution of the autonomous system
x (t) = f (x),
x (t0 ) = x 0
(10)
and let be the closed orbit of t(x0) in the phase space

namely,
= {yy|yy = t (xx 0 ),
0 t < tp }
The solution trajectory t(x0) is said to be orbitally stable if for

any 0 there exits a () 0, such that for any x 0
satisfying
d(x 0 , ) := inf x 0 y <
y
the solution t(x 0) of the autonomous system satisfies

Lyapunov Stabilities. In the following discussion of Lyapunov stabilities for the general nonautonomous system of
Eq. (1), the parameters are dropped since they do not affect
the concept and consequence. Thus, consider the general nonautonomous nonlinear system
x = f (xx, t),
x (t0 ) = x 0
(7)
d(t ( x 0 ), ) < ,
t t0
Lyapunov Stability Theorems. Two cornerstones in the Lyapunov stability theory for dynamical systems are the Lyapunov first (or indirect) method and the Lyapunov second (or
direct) method.
198
The Lyapunov first method, known also as the Jacobian or

local linearization method, is applicable only to autonomous
systems. This method is based on the fact that the stability
of an autonomous system, in a neighborhood of an equilibrium, is essentially the same as its linearized model operating
at the same point, and under certain conditions local system
stability behavior is qualitatively the same as that of its linearized model (in some sense, similar to the Grobman-Hartman theorem). The Lyapunov first method provides a theoretical justification for applying linear analysis and linear
feedback controllers to nonlinear autonomous systems in the
study of asymptotic stability and stabilization. The Lyapunov
second method, on the other hand, which originated from the
concept of energy decay (i.e., dissipation) associated with a
stable mechanical or electrical system, is applicable to both
autonomous and nonautonomous systems. Hence, the second
method is more powerful, also more useful, for rigorous stability analysis of various complex dynamical systems.
For the general autonomous system of Eq. (10), under the
assumption that f : D Rn is continuously differentiable in
a neighborhood, D , of the origin in Rn, the following theorem
of stability for the Lyapunov first method is convenient to use.
Theorem 4 (Lyapunov First Method) (for continuoustime autonomous systems)
In Eq. (10), let

f
J0 =
xx x =xx=0
be the Jacobian of the system at the equilibrium x 0. Then
(1) x* 0 is asymptotically stable if all the eigenvalues of J0
have negative real parts; and (2) x 0 is unstable if one of
the eigenvalues of J0 has a positive real part.
Note that the region of asymptotic stability given in this
theorem is local. It is important to emphasize that this theorem cannot be applied to nonautonomous systems in general,
not even locally. For the general nonautonomous system of
Eq. (7), the following criterion can be used. Let
K ={ g(t) : g(t0 ) = 0,
g(t) is continuous and nondecreasing on [t0 , )}
Theorem 5 (Lyapunov Second Method) (for continuoustime nonautonomous systems)

Let x 0 be an equilibrium of the nonautonomous system
of Eq. (7). The zero equilibrium of the system is globally (over
the domain D Rn containing the origin), uniformly (with
respect to the initial time), and asymptotically stable if there
exist a scalar-valued function V(x, t) defined on D [t0, )
and three functions ( ), ( ), ( ) K , such that (1) V(0,
t0) 0; (2) V(x, t) 0 for all x 0 in D and all t t0; (3)
(x, t) (x)
(x) V(x, t) (x) for all t t0; and (4) V
for all t t0.
In this theorem, the uniform stability is usually necessary
since the solution of a nonautonomous system may depend
sensitively on the initial time. As a special case for autonomous systems, the preceding theorem reduces to the following.
Theorem 6 (Lyapunov Second Method) (for continuoustime autonomous systems)

Let x 0 be an equilibrium for the autonomous system of
Eq. (10). This zero equilibrium is globally (over the domain D
Rn containing the origin) and asymptotically stable if there
exists a scalar-valued function V(x) defined on D such that
(x) 0
(1) V(0) 0; (2) V(x) 0 for all x 0 in D ; and (3) V
for all x 0 in D .
In the preceding two theorems, the function V is called a
Lyapunov function, which is generally not unique for a given
system. Similar stability theorems can be established for discrete-time systems (by properly replacing derivatives with differences).
To this end, it is important to remark that the Lyapunov
theorems only offer sufficient conditions for determining the
asymptotic stability. Yet the power of the Lyapunov second
method lies in its generality: It works for all kinds of dynamical systems (linear and nonlinear, continuous-time and discrete-time, autonomous and nonautonomous, time-delayed,
functional, etc.), and it does not require any knowledge of the
solution formula of the underlying system. In a particular application, the key is to construct a working Lyapunov function
for the system, which can be technically difficult if the system
is higher-dimensional and structurally complicated.
CHAOS
Nonlinear systems have various complex behaviors that
would never have been anticipated in (finite-dimensional) linear systems. Chaos is a typical behavior of this kind. In the
development of chaos theory, the first evidence of physical
chaos was Edward Lorenzs discovery in 1963. The first underlying mechanism within chaos was observed by Mitchell
Feigenbaum, who in 1976 found that when an ordered system begins to break down into chaos, a consistent pattern of
rate doubling occurs (3).
What Is Chaos?
There is no unified, universally accepted, rigorous definition
of chaos in the current scientific literature. The term chaos
was first formally introduced into mathematics by Li and
Yorke (8). Since then, there have been several different but
closely related proposals for definitions of chaos, among which
Devaneys definition is perhaps the most popular one (9). It
states that a map F : S S, where S is a set, is said to be
chaotic if
1. F is transitive on S: For any pair of nonempty open sets
U and V in S, there is an integer k 0 such that
Fk(U) V is nonempty.
2. F has sensitive dependence on initial conditions: There
is a real number 0, depending only on F and S, such
that in every nonempty open subset of S there is a pair
of points whose eventual iterates under F are separated
by a distance of at least .
3. The periodic points of F are dense in S.
Another definition requires the set S be compact but drops
condition 3. There is a belief that only the transitive property
is essential in this definition. Although a precise and rigorous
mathematical definition of chaos does not seem to be available anytime soon, some fundamental features of chaos are
well received and can be used to signify or identify chaos in
most cases.
A hallmark of chaos is its fundamental property of extreme

sensitivity to initial conditions. Other features of chaos include the embedding of a dense set of unstable periodic orbits
in its strange attractor, positive leading (maximal) Lyapunov
exponent, finite Kolmogorov-Sinai entropy or positive topological entropy, continuous power spectrum, positive algorithmic
complexity, ergodicity and mixing (Arnolds cat map), Smale
horseshoe map, a statistical-oriented definition of Shilnikov,
as well as some unusual limiting properties (4).
Extreme Sensitivity to Initial Conditions. The first hallmark
of chaos is its extreme sensitivity to initial conditions, associated with its bounded (or compact) region of orbital patterns.
It implies that two sets of slightly different initial conditions
can lead to two dramatically different asymptotic states of the
system orbit after some time. This is the so-called butterfly
effect and says that a single flap of a butterflys wings in
China today may alter the initial conditions of the global
weather dynamical system, thereby leading to a significantly
different weather pattern in Argentina at a future time. In
other words, for a dynamical system to be chaotic it must
have a (large) set of such unstable initial conditions that
cause orbital divergence within a bounded region.
Positive Leading Lyapunov Exponents. Most sensitive dependence on initial conditions of a chaotic system possesses an
exponential growth rate. This exponential growth is related
to the existence of at least one positive Lyapunov exponent,
usually the leading (largest) one. Among all main characteristics of chaos, the positive leading Lyapunov exponent is perhaps the most convenient one to verify in engineering applications.
To introduce this concept, consider an n-dimensional, discrete-time dynamical system described by xk1 f(xk) via a
smooth map f. The ith Lyapunov exponent of the orbit
xkk0
, generated by the iterations of the map starting from
any given initial state x0, is defined to be
i (xx0 ) = lim
pending on the direction (but not the position) of the initial

state vector, the n Lyapunov exponents, 1 n, describe different types of attractors. For example, for some
nonchaotic attractors (limit sets),
i < 0, i = 1, . . ., n
Features of Chaos
1
ln |i (Jk (xx k ) . . . J0 (xx 0 ))|,
k
i = 1, . . ., n
(11)
where Ji( ) f( ) is the Jacobian and i( ) denote the ith

eigenvalue of a matrix (numbered in decreasing order of magnitude). In the continuous-time case, x f(x), the leading
Lyapunov exponent is defined by
(xx 0 ) = lim
1
ln xx (t; x 0 )/xx 0
t
which is usually evaluated by numerical computations. All

Lyapunov exponents depend on the system initial state x0,
and reflect the sensitivity with respect to x0.
Lyapunov exponents are generalizations of eigenvalues of
linear systems and provide a measure for the mean convergence or divergence rate of neighboring orbits of a dynamical
system. For an n-dimensional continuous-time system, de-
199
stable equilibrium
1 = 0, i < 0, i = 2, . . ., n
stable limit cycle
1 = 2 = 0, i < 0, i = 3, . . ., n
stable two-torus
1 = = m = 0,
i < 0, i = m + 1, . . ., n
stable m-torus
Here, a two-torus is a bagel-shaped surface in three-dimensional space, and an m-torus is its geometrical generalization
in (m 1)-dimensional space.
It is now well known that one and two-dimensional continuous-time autonomous dynamical systems cannot produce
chaos. For a three-dimensional continuous-time autonomous
system, the only possibility for chaos to exist is that the three
Lyapunov exponents are
(+, 0, ) := (1 > 0, 2 = 0, 3 < 0) and 3 < 1
Intuitively, this means that the system orbit in the phase
space expands in one direction but shrinks in another direction, thereby yielding many complex (stretching and folding)
dynamical phenomena within a bounded region. The discretetime case is different, however. A prominent example is the
one-dimensional logistic map, discussed in more detail later,
which is chaotic but has (the only) one positive Lyapunov exponent. For four-dimensional continuous-time autonomous
systems, there are only three possibilities for chaos to emerge:
1. (, 0, , ): 1 0, 2 0, 4 3 0; leading to chaos
2. (, , 0, ): 1 2 0, 3 0, 4 0; leading to
hyperchaos
3. (, 0, 0, ): 1 0, 2 3 0, 4 0; leading to a
chaotic two-torus (this special orbit has not been
experimentally observed).
Simple Zero of the Melnikov Function. The Melnikov theory
of chaotic dynamics deals with the saddle points of Poincare
maps of continuous solution flows in the phase space. The
Melnikov function provides a measure of the distance between the stable and unstable manifolds near a saddle point.
To introduce the Melnikov function, consider a nonlinear
oscillator described by the Hamiltonian system
+ f1
p =
q
H
q =
+ f2
p
where f : [f 1(p, q, t), f 2(p, q, t)] has state variables (p(t),
q(t)), 0 is small, and H H(p, q) EK EP is the Hamilton function for the undamped, unforced (when 0) oscillator, in which EK and EP are the kinetic and potential energy
of the system, respectively. Suppose that the unperturbed
(unforced and undamped) oscillator has a saddle-node equilibrium (e.g., the undamped pendulum) and that f is tp-periodic
with phase frequency . If the forced motion is described in
200
the three-dimensional phase space (p, q, t), then the Melnikov function is defined by
F (d ) =
[H( p, q)] f dt
(12)
where (p, q) are the solutions of the unperturbed homoclinic

orbit starting from the saddle point of the original Hamiltonian system, f* f(p, q, t d*), and H [H/p, H/q].
The variable d* gives a measure of the distance between the
stable and unstable manifolds near the saddle-node equilibrium.
The Melnikov theory states that chaos is possible if the
two manifolds interset, which corresponds to the fact that the
Melnikov function has a simple zero: F(d*) 0 at a single
point, d*.
Strange Attractors. Attractors are typical in nonlinear systems. The most interesting attractors, very closely related to
chaos, are the strange attractors. A strange attractor is a
bounded attractor that exhibits sensitive dependence on initial conditions but cannot be decomposed into two invariant
subsets contained in disjoint open sets. Most chaotic systems
have strange attractors; however, not all strange attractors
are associated with chaos.
Generally speaking, a strange attractor is not any of the
stable equilibria or limit cycles, but rather consists of some
limit sets associated with Cantor sets and/or fractals. In other
words, it has a special and complicated structure that may
possess a noninteger dimension (fractals) and has some of the
properties of a Cantor set. For instance, a chaotic orbit usually appears to be strange in that the orbit moves toward a
certain point (or limit set) for some time but then moves away
from it for some other time. Although the orbit repeats this
process infinitely many times it never settles anywhere. Figure 6 shows a typical Chua circuit attractor (2,4,6) that has
such strange behavior.
Fractals. An important concept that is related to Lyapunov
exponent is the Hausdorff dimension. Let S be a set in Rn and
C be a covering of S by countably many balls of radii d1, d2,

, satisfying 0 dk for all k. For a constant 0,
consider k1
dk for different coverings, and let infC dk be the
smallest value of the sum over all possible such coverings. In
the limit 0, this value will diverge if h but tends to
zero if h for some constant h 0 (need not be an integer).
This value, h, is called the Hausdorff dimension of the set S.
If such a limit exists for h, then the Hausdorff measure
of the set S is defined to be
h (S) := lim inf
0 C
dk
k=1
There is an interesting conjecture that the Lyapunov exponents k (indicating the dynamics) and the Hausdorff dimension h (indicating the geometry) of a strange attractor have
the relation
h=k+
k
k+1
i=1
i ,
1 2 n
k
i 0. This
where k is the largest integer that satisfies i1
formula has been mathematically proved for large families of
three-dimensional continuous-time autonomous systems and
of two-dimensional discrete-time systems.
A notion that is closely related to the Hausdorff dimension
is fractal. Fractal was first coined and defined by Mandelbrot
in the 1970s to be a set with Hausdorff dimension strictly
greater than its topological dimension (which is always an
integer). Roughly, a fractal is a set that has a fractional
Hausdorff dimension and possesses certain self-similarities.
An illustration of the concept of self-similarity and fractal is
given in Fig. 13. There is a strong connection between fractal
and chaos. Chaotic orbits often possess fractal structures in
the phase space. For conservative systems, the KolmogorovArnold-Moser (KAM) theorem implies that the boundary between the region of regular motion and that of chaos is fractal. However, some chaotic systems have nonfractal limit sets,
and many fractal structures are not chaotic.
Finite Kolmogorov-Sinai Entropy. Another important feature

of chaos and strange attractors is quantified by the Kolmogorov-Sinai (KS) entropy, a concept based on Shannons information theory.
The familiar statistical entropy is defined by
E = c
Pk ln(Pk )
Figure 6. A typical example of stranger attractor: The double scroll

of Chuas circuit response.
where c is a constant and Pk is the probability of the system

state being at the stage k of the process. According to Shannons information theory, this entropy is a measure of the
amount of information needed to determine the state of the
system. This idea can be used to define a measure for the
intensity of a set of system states, which gives the mean loss
of information on the state of the system when it evolves with
time. To do so, let x(t) be a system orbit and partition its mdimensional phase space into cells of a small volume, m. Let
Pk0. . .ki be the joint probability that x(t 0) is in cell k0,
x(t ts) is in cell k1, . . ., x(t its) is in cell ki, where ts 0
is the sampling time. Then Shannon defined the information

index to be

In :=
Pk ...k n ln Pk ...k n
k 0 ,...,k n
which is proportional to the amount of the information needed

to determine the orbit, if the probabilities are known. Consequently, I n1 I n gives additional information for predicting
the next state if all preceding states are known. This difference is also the information lost during the process. The KS
entropy is then defined by
n1
1
(Ii+1 Ii )
t s 0 0 n nts
i=0
(13)

1
Pk ...k
ln Pk ...k
= lim lim lim
0
n1
0
n1
t s 0 0 n nts
k ,...,k
EKS := lim lim lim
n1
This entropy, EKS, quantifies the degree of disorder: (1) EKS

0 indicates regular attractors, such as stable equilibria, limit
cycles, and tori; (2) EKS implies totally random dynamics
(which has no correlations in the phase space); and (3) 0
EKS signifies strange attractors and chaos.
It is interesting to note that there is a connection between
the Lyapunov exponents and the KS entropy:

EKS
+
i
i
where i are positive Lyapunov exponents of the same

system.
Chaos in Control Systems
Chaos is ubiquitous. Chaotic behaviors have been found in
many typical mathematical maps such as the logistic map,
Arnolds circle map, Henon map, Lozi map, Ikeda map, Bernoulli shift; in various physical systems, including the Duffing
oscillator, van der Pol oscillator, forced pendula, hopping robot, brushless dc motor, rotor with varying mass, Lorenz
model, and Rossler system. They are also found in electrical
and electronic systems (such as Chuas circuit and electric
power systems), digital filters, celestial mechanics (the threebody problem), fluid dynamics, lasers, plasmas, solid states,
quantum mechanics, nonlinear optics, chemical reactions,
neural networks and fuzzy systems, economic and financial
systems, biological systems (heart, brain, and population
models), and various Hamiltonian systems (4).
Chaos also exists in many engineering processes and, perhaps unexpectedly, in both continuous-time and discrete-time
feedback control systems. For instance, in the continuoustime case, chaos has been found in very simple dynamical systems such as a first-order autonomous feedback system with
a time-delay feedback channel, surge tank dynamics under a
simple liquid level control system with time-delayed feedback,
and several other types of time-delayed feedback control systems. Chaos also exists in automatic gain control loops, which
are very popular in industrial applications, such as in many
receivers of communication systems. Most fascinating of all,
very simple pendula can display complex dynamical phenomena; in particular, pendula subject to linear feedback controls
can exhibit even richer bifurcations and chaotic behaviors.
201
For example, a pendulum controlled by a proportional-derivative controller can behave chaotically when the tracking signal is periodic, with energy dissipation, even for the case of
small controller gains. In addition, chaos has been found in
many engineering applications, such as design of control circuits for switched-mode power conversion equipment, highperformance digital robot controllers, second-order systems
containing a relay with hysteresis, and various biochemical
control systems.
Chaos also occurs frequently in discrete-time feedback control systems due to sampling, quantization, and roundoff effects. Discrete-time linear control systems with dead-zone
nonlinearity have global bifurcations, unstable periodic orbits, scenarios leading to chaotic attractors, and crises of chaotic attractors changing to periodic orbits. Chaos also exists in
digitally controlled systems, feedback types of digital filtering
systems (either with or without control), and even the linear
Kalman filter when numerical truncations are involved.
Many adaptive systems are inherently nonlinear, and thus
bifurcation and chaos in such systems are often inevitable.
The instances of chaos in adaptive control systems usually
come from several possible sources: the nonlinearities of the
plant and the estimation scheme, external excitation or disturbances, and the adaptation mechanism. Chaos can occur
in typical model-referenced adaptive control (MRAC) and selftuning adaptive control (STAC) systems, as well as some
other classes of adaptive feedback control systems of arbitrary
order that contain unmodeled dynamics and disturbances. In
such adaptive control systems, typical failure modes include
convergence to undesirable local minima and nonlinear selfoscillation, such as bursting, limit cycling, and chaos. In indirect adaptive control of linear discrete-time plants, strange
system behaviors can arise due to unmodeled dynamics (or
disturbances), bad combination of parameter estimation and
control law, and lack of persistency of excitation. For example, chaos can be found in set-point tracking control of a linear discrete-time system of unknown order, where the adaptive control scheme is either to estimate the order of the plant
or to track the reference directly.
Chaos also emerges from various types of neural networks.
Similar to biological neural networks, most artificial neural
networks can display complex dynamics, including bifurcations, strange attractors, and chaos. Even a very simple recurrent two-neuron model with only one self-interaction can produce chaos. A simple three-neuron recurrent neural network
can also create period-doubling bifurcations leading to chaos.
A four-neuron network and multineuron networks, of course,
have higher chances of producing complex dynamical patterns
such as bifurcations and chaos. A typical example is cellular
neural networks, which have very rich complex dynamical behaviors.
Chaos has also been experienced in some fuzzy control systems. The fact that fuzzy logic can produce complex dynamics
is more or less intuitive, inspired by the nonlinear nature of
the fuzzy systems. This has been justified not only experimentally but also both mathematically and logically. Chaos has
been observed, for example, from a coupled fuzzy control system. The change in the shapes of the fuzzy membership functions can significantly alter the dynamical behavior of a fuzzy
control system, potentially leading to the occurrence of chaos.
Many specific examples of chaos in control systems can be
given. Therefore, controlling chaos is not only interesting as
202
a subject for scientific research but also relevant to the objectives of traditional control engineering. Simply, it is not an
issue that can be treated with ignorance or neglect.
x2 = p
BIFURCATIONS
Associated with chaos is bifurcation, another typical phenomenon of nonlinear dynamical systems that quantifies the
change of system properties (such as the number and the stabilities of the system equilibria) due to the variation of system
parameters. Chaos and bifurcation have a very strong connection; often they coexist in a complex dynamical system.
Basic Types of Bifurcations
To illustrate various bifurcation phenomena, it is convenient
to consider the two-dimensional, parametrized, nonlinear dynamical system

x = f (x, y; p)
(14)
y = g(x, y; p)
Figure 8. The saddle-node bifurcation.
has an equilibrium x1 0 at p0 0 and an equilibrium curve

x2 p at p 0, where x21 p is stable and x22 p is
unstable for p p0 0. This bifurcation, as shown in Fig. 8,
is called the saddle-node bifurcation.
Pitchfork Bifurcation. The one-dimensional system
x = f (x; p) = px x3
where p is a real and variable system parameter.

Let (x, y) (x(t; p0), y(t; p0)) be an equilibrium of the system when p p0, at which f(x, y; p0) 0 and g(x, y; p0) 0.
If the equilibrium is stable (respectively, unstable) for p p0
but unstable (respectively, stable) for p p0, then p0 is a bifurcation value of p, and (0, 0, p0) is a bifurcation point in the
parameter space, (x, y, p). A few examples are given next to
distinguish several typical bifurcations.
has an equilibrium x1 0 at p0 0 and an equilibrium curve

x2 p at p 0. Since x1 0 is unstable for p p0 0 and
stable for p p0 0, and since the entire equilibrium curve
x2 p is stable for all p 0 at which it is defined, this situation, as depicted in Fig. 9, is called the pitchfork bifurcation.
Note, however, that not all nonlinear parametrized dynamical systems have bifurcations. A simple example is
x = f (x; p) = p x3
Transcritical Bifurcation. The one-dimensional system

x = f (x; p) = px x2
has two equilibria: x1 0 and x2 p. If p is varied, then there
are two equilibrium curves, as shown in Fig. 7. Since the Jacobian at zero for this one-dimensional system is simply J
p, it is clear that for p p0 0, the equilibrium x1 0 is
stable, but for p p0 0 it changes to be unstable. Thus,
(x1, p0) (0, 0) is a bifurcation point. In the figure, the solid
curves indicate stable equilibria and the dashed curves, the
unstable ones. (x2, p0) is another bifurcation point. This type
of bifurcation is called the transcritical bifurcation.
Saddle-Node Bifurcation. The one-dimensional system
x = f (x; p) = p x2
which has an entire stable equilibrium curve x p1/3 and

hence does not have any bifurcation.
Hysteresis Bifurcation. The dynamical system
x1 = x1
x2 = p + x2 x32
has equilibria
x1 = 0 and
According to different values of p, there are either one or

three equilibrium solutions, where the second equation gives
p x2 + x32 = 0
x2 = p
t
t
Figure 7. The transcritical bifurcation.
Figure 9. The pitchfork bifurcation.
where R denotes the real part of the complex eigenvalues. Then
x2
p0
1. p p0 is a bifurcation point of the system.

2. For close enough values p p0, the equilibrium x 0
is asymptotically stable.
is unstable.
is surrounded by an emerging limit cycle of magnitude
O(p p0).
p0
Figure 10. The hysteresis bifurcation.
a bifurcation point at p0 23/9, but three equilibria for

p0 23/9.
The stabilities of the equilibrium solutions are shown in
Fig. 10. This type of bifurcation is called the hysteresis bifurcation.
Hopf Bifurcation and Hopf Theorems. In addition to the bifurcations described previously, called static bifurcations, the
parametrized dynamical system of Eq. (14) can have another
type of bifurcation, the Hopf bifurcation (or dynamical bifurcation).
Hopf bifurcation corresponds to the situation where, as the
parameter p is varied to pass the critical value p0, the system
Jacobian has one pair of complex conjugate eigenvalues moving from the left-half plane to the right, crossing the imaginary axis, while all the other eigenvalues remain to be stable
(with negative real parts). At the moment of the crossing, the
real parts of the eigenvalue pair are zero, and the stability of
the existing equilibrium changes to opposite, as shown in Fig.
11. In the meantime, a limit cycle will emerge. As indicated in
the figure, Hopf bifurcation can be classified as supercritical
(respectively, subcritical), if the equilibrium is changed from
stable to unstable (respectively, from unstable to stable). The
same terminology of supercritical and subcritical bifurcations
applies to other non-Hopf types of bifurcations.
Consider the general nonlinear, parametrized autonomous
system
x = f (xx; p),
203
x (t0 ) = x 0
Theorem 7 (Poincare-Andronov-Hopf) Suppose that the

two-dimensional system of Eq. (15) has a zero equilibrium,
x 0, and assume that its associate Jacobian A f /xxx0
has a conjugate pair of purely imaginary eigenvalues, (p0)
and *(p0) for some p0. Also assume that
x
u
x = A(p)x + B(p)u
y = C(p)xx
u = g (yy; p)
(16)
where the matrix A(p) is chosen to be invertible for all values

of p, and g C4 depends on the chosen matrices A, B, and
p < p0
p = p0
p > p0
(15)
where x Rn, p is a real variable parameter, and f is differentiable. The most fundamental result on the Hopf bifurcation of this system is the following theorem, which is stated
here for the special two-dimensional case.

d{(p)}
>0

dp
p= p
Graphical Hopf Bifurcation Theorem. The Hopf bifurcation

can also be analyzed in the frequency-domain setting (10). In
this approach, the nonlinear parametrized autonomous system of Eq. (15) is first rewritten in the following Lure form:
p
p
p = p0
y
y
x
Figure 11. Two types of Hopf bifurcations illustrated in the phase

plane. (a) supercritical; (b) subcritical.
204
{ ( ;p)}
C. Assume that this system has an equilibrium solution, y,

satisfying
( ;p)
y (t; p) = H(0; p)g( y (t; p); p)
( ;p)
where
Let J(p) g/yyy and let ( j; p) be the eigenvalue of

the matrix [H( j; p) J(p)] that satisfies
j=
Then fix p p and let vary. In so doing, a trajectory of the

function (; p), the eigenlocus, can be obtained. This locus
traces out from the frequency 0 0. In much the same way,
a real zero eigenvalue (a condition for the static bifurcation)
is replaced by a characteristic gain locus that crosses the
point (1 j 0) at frequency 0 0.
For illustration, consider a single-input single-output
(SISO) system. In this case, the matrix [H( j; p) J(p)] is
merely a scalar, and
y(t) y +

n
yk e jkt
k=0
where y is the equilibrium solution and the complex coefficients, yk, are determined as follows. For the approximation
with n 2, first define an auxiliary vector
1 ()
=
p)]h
h1
ll T [H( j ;
l Tr
where p is the fixed value of the parameter p, l and r are

the left and right eigenvectors of [H( j ; p) J(p)], respectively,
associated with the eigenvalue ( j ; p), and

1
1
h 1 = D2 z 02 r + r z 22 + D3 r r r
2
8
in which * denotes the complex conjugate, is the frequency
of the intersection between the locus and the negative real
axis that is closest to the point (1 j 0), is the tensor
product operator, and

2 g (y; p)

D2 =
y2 y=y

3g (y; p)
D3 =
y3 y=y
1
p)]
1 G(0; p)D
z 02 = [1 + H(0; p)J(
2r r
4
1
p)]
1 H(2 j ;
z 22 = [1 + H(2 j ;
p)D
2r r
4
y0 = z 02 p p0
1/2
y2 = z 22 p p0
{ ( ;p)}
2 1()
Figure 12. The frequency-domain version of the Hopf bifurcation

theorem.
The graphical Hopf bifurcation theorem (for SISO systems)

formulated in the frequency domain, based on the generalized
Nyquist criterion, is stated as follows (10).
Theorem 8 (Graphical Hopf Bifurcation Theorem)
Suppose that when varies, the vector 1( ) 0. Assume also
that the half-line, starting from 1 j 0 and pointing to the
direction parallel to that of 1( ), first intersects the locus of
the eigenvalue ( j ; p) at the point
;
P = (
p)
= 1 + 1 ()
2
at which and the constant ( ) 0, as shown in
Fig. 12. Suppose, furthermore, that the preceding intersection
is transversalnamely,
I {1 ( j )}
{1 ( j )}

d
d
det
= 0
(;
p)
=
(;
p)
=

I
d
d
(17)
y1 = r | p p0
P = 1 + 2 1( )
H(0; p) = C(p)A1 (p)B(p)
( j0 ; p0 ) = 1 + j0,
where I is the imaginary part of the complex eigenvalue.

Then
1. The nonlinear system of Eq. (16) has a periodic solution
(output) y(t) y(t; y). Consequently, there exists a
unique limit cycle for the nonlinear equation x f(x),
in a ball of radius O(1) centered at the equilibrium x.
2. If the total number of counterclockwise encirclements of
the point p1 P 1( ), for a small enough 0, is
equal to the number of poles of [H(s; p) J(p)] that have
positive real parts, then the limit cycle is stable.
Period-Doubling Bifurcations to Chaos
There are several routes to chaos from a regular state of a
nonlinear system, provided that the system is chaotic in
nature.
It is known that after three Hopf bifurcations a regular
motion can become highly unstable, leading to a strange attractor and, thereafter, chaos. It has also been observed that
even pitchfork and saddle-node bifurcations can be routes to
chaos under certain circumstances. For motion on a normalized two-torus, if the ratio of the two fundamental frequencies 1 /2 p/q is rational, then the orbit returns to the same
point after a q-cycle; but if the ratio is irrational, the orbit
(said to be quasiperiodic) never returns to the starting point.
Quasiperiodic motion on a two-torus provides another common route to chaos.
Period-doubling bifurcation is perhaps the most typical

route that leads system dynamics to chaos. Consider, for example, the logistic map
205
1.00
0.00
(18)
xk+1 = pxk (1 xk )
where p 0 is a variable parameter. With 0 p 1, the

origin x 0 is stable, so the orbit approaches it as k .
However, for 1 p 3, all points converge to another equilibrium, denoted x. The dynamical evolution of the system behavior, as p is gradually increased from 3.0 to 4.0 by small
steps, is mostly interesting, which is depicted in Fig. 13. The
figure shows that at p 3, a (stable) period-two orbit is bifurcated out of x, which becomes unstable at that moment, and,
in addition to 0, there emerge two (stable) equilibria:
x1,2 = (1 + p
p2 2p 3)/(2p)
When p continues to increase to the value of 1 6

3.544090 . . ., each of these two points bifurcates to the other
two, as can be seen from the figure. As p moves consequently
through the values 3.568759 . . ., 3.569891 . . ., , an infinite sequence of bifurcations is generated by such period
doubling, which eventually leads to chaos:
period 1 period 2 period 4
period 2k chaos
It is also interesting to note that certain regions (e.g., the
three windows magnified in the figure) of the logistic map
show self-similarity of the bifurcation diagram of the map,
which is a typical fractal structure.
xn
1.0
0.5
0.0
2.8 3.0
1+ 6
1+ 8
3.56994
4.0
Figure 13. Period doubling of the logistic system with self-similarity.

Reprinted from J. Argyris, G. Faust, and M. Haase, An Exploration
of Chaos, 1994, Fig. 9.6.6, p. 66f, with kind permission from Elsevier
ScienceNL, Amsterdam, The Netherlands.
1.00
2.00
3.00
2.50
2.88
3.25
p
3.63
4.00
Figure 14. Lyapunov exponent versus parameter p for the logistic

map. Reprinted from J. Argyris, G. Faust, and M. Haase, An Exploration of Chaos, 1994, Fig. 5.4.8.(b), p. 172, with kind permission from
Elsevier ScienceNL, Amsterdam, The Netherlands.
Figure 14 shows the Lyapunov exponent versus the parameter p, in the interval [2.5, 4]. This figure corresponds to
the period-doubling diagram shown in Fig. 13.
The most significant discovery about the phenomenon of
period-doubling bifurcation route to chaos is Feigenbaums
observation in 1978: The convergence of the period-doubling
bifurcating parameters has a geometric rate, p pk k,
where
pk+1 pk
= 4.6692 . . .
pk+2 pk+1
(k )
This is known as a universal number for a large class of chaotic dynamical systems.
Bifurcations in Control Systems
Not only chaos but also bifurcations can exist in feedback and
adaptive control systems. Generally speaking, local instability
and complex dynamical behavior can result from feedback
and adaptive mechanisms when adequate process information is not available for feedback transmission or for parameter estimation. In this situation, one or more poles of the linearized closed-loop transfer function may move to cross over
the stability boundary, thereby causing signal divergence as
the control process continues. However, this sometimes may
not lead to global unboundedness, but rather, to self-excited
oscillations or self-stabilization, creating very complex dynamical phenomena.
Several examples of bifurcations in feedback control systems include the automatic gain control loop system, which
has bifurcations transmitting to Smale horseshoe chaos and
the common route of period-doubling bifurcations to chaos.
Surprisingly enough, in some situations even a single pendulum controlled by a linear proportional-derivative controller
can display rich bifurcations in addition to chaos.
Adaptive control systems are more likely to produce bifurcations than a simple feedback control system due to changes
of stabilities in adaptation. The complex dynamics emerging
from an adaptive control system are often caused by estimation instabilities. It is known that certain prototypes of
MRAC systems can experience various bifurcations.
206
Bifurcation theory has been employed for analzying complex dynamical systems. For instance, in an MRAC system, a
few pathways leading to estimator instability have been identified via bifurcation analysis:
1. A sign change in the adaptation law, leading to a reversal of the gradient direction as well as an infinite linear
drift.
2. The instability caused by high control gains, leading to
global divergence through period-doubling bifurcations.
3. A Hopf bifurcation type of instability, complicated by a
number of nonlocal phenomena, leading to parameter
drift and bursting in a bounded regime through a sequence of global bifurcations.
Both instabilities of types 1 and 2 can be avoided by gain
tuning or simple algorithmic modifications. The third instability, however, is generally due to the unmodeled dynamics and
a poor signal-to-noise ratio, and so cannot be avoided by simple tuning methods. This instability is closely related to the
presence of a degenerate set and a period-two attractor.
Similarly, in the discrete-time case, a simple adaptive control system can have rich bifurcation phenomena, such as period-doubling bifurcation (due to high adaptive control gains)
and Hopf and global bifurcations (due to insufficient excitation).
Like the omnipresent chaos, bifurcations exist in many
physical systems (4). For instance, power systems generally
have various bifurcation phenomena. When consumers demands for power reach peaks, the stability of an electric
power network may move to its margin, leading to serious
oscillations and stability bifurcations, which may quickly result in voltage collapse. As another example, a typical double
pendulum can display bifurcations as well as chaotic motions.
Some rotational mechanical systems also have similar behavior. Even a common road vehicle driven by a pilot with driver
steering control can have Hopf bifurcation when its stability
is lost, which may also develop chaos and even hyperchaos. A
hopping robot, or a simple two-degree-of-freedom flexible robot arm, can response strange vibrations undergoing period
doubling, which eventually lead to chaos. An aircraft stalling
for flight below a critical speed or over a critical angle of attack can cause various bifurcations. Dynamics of a ship can
exhibit stability bifurcation according to wave frequencies
that are close to the natural frequency of the ship, which creates oscillations and chaotic motions leading the ship to capsize. Simple nonlinear circuits are rich sources of different
types of bifurcations as well as chaos. Other systems that
have bifurcation properties include cellular neural networks,
lasers, aeroengine compressors, weather systems, and biological population dynamics, to name just a few.
CONTROLLING CHAOS
Understanding chaos has long been the main focus of research in the field of nonlinear dynamics. The idea that chaos
can be controlled is perhaps counterintuitive. Indeed, the extreme sensitivity of a chaotic system to initial conditions once
led to the impression and argument that chaotic motion is in
general neither predictable nor controllable.
However, recent research effort has shown that not only

(short-term) prediction but also control of chaos are possible.
It is now well known that most conventional control methods
and many special techniques can be used for controlling chaos
(4,11,12). In this pursuit, whether the purpose is to reduce
bad chaos or to introduce good ones, numerous control
strategies have been proposed, developed, tested, and applied
to many case studies. Numerical and experimental simulations have demonstrated that chaotic physical systems respond quite well to these controls. In about the same time,
applications are proposed in such diverse fields as biology,
medicine, physiology, chemical engineering, laser physics,
electric power systems, fluid mechanics, aerodynamics, circuits and electronic devices, and signal processing and communication. The fact that researchers from vast scientific and
engineering backgrounds are joining together and aiming at
one central themebringing order to chaosindicates that
the study of nonlinear dynamics and their control has progressed into a new era. Much has been accomplished in the
past decade, yet much more remains a challenge for the
future.
Similar to conventional systems control, the concept of
controlling chaos first means to suppress chaos in the sense
of stabilizing chaotic system responses, often unstable periodic outputs. However, controlling chaos has also encompassed many nontraditional tasks, particularly those of creating or enhancing chaos when it is useful. The process of chaos
control is now understood as a transition between chaos and
order and, sometimes, the transition from chaos to chaos, depending on the application at hand. In fact, the notion of
chaos control is neither exclusive of, nor conflicting with, the
purposes of conventional control systems theory. Rather, it
targets at better managing the dynamics of a nonlinear system on a wider scale, with the hope that more benefits may
be derived from the special features of chaos.
Why Chaos Control?
There are many practical reasons for controlling or ordering
chaos. First, chaotic (messy, irregular, or disordered) system
response with little useful information content is unlikely to
be desirable. Second, chaos can lead systems to harmful or
even catastrophic situations. In these troublesome cases,
chaos should be reduced as much as possible or totally suppressed. Traditional engineering design always tries to reduce
irregular behaviors of a system and, therefore, completely
eliminates chaos. Such overdesign is needed in the aforementioned situations. However, this is usually accomplished
at the price of loosing great benefits in achieving high performance near the stability boundaries, or at the expense of radically modifying the original system dynamics.
Ironically, recent research has shown that chaos can actually be useful under certain circumstances, and there is growing interest in utilizing the very nature of chaos (4). For example, it has been observed (13) that a chaotic attractor
typically has embedded within it a dense set of unstable limit
cycles. Thus, if any of these limit cycles can be stabilized, it
may be desirable to select one that characterizes maximal
system performance. In other words, when the design of a
dynamical system is intended for multiple uses, purposely
building chaotic dynamics into the system may allow for the
desired flexibilities. A control design of this kind is certainly

nonconventional.
Fluid mixing is a good example in which chaos is not only
useful but actually necessary (14). Chaos is desirable in many
applications of liquid mixing, where two fluids are to be thoroughly mixed while the required energy is minimized. For
this purpose, it turns out to be much easier if the dynamics
of the particle motion of the two fluids are strongly chaotic,
since it is difficult to obtain rigorous mixing properties otherwise due to the possibility of invariant two-tori in the flow.
This has been one of the main subjects in fluid mixing, known
as chaotic advection. Chaotic mixing is also important in
applications involving heating, such as plasma heating for a
nuclear fusion reactor. In such plasma heating, heat waves
are injected into the reactor, for which the best result is obtained when the heat convection inside the reactor is chaotic.
Within the context of biological systems, the controlled biological chaos seems to be important with the way a human
brain executes its tasks. For years, scientists have been trying to unravel how our brains endow us with inference,
thoughts, perception, reasoning, and, most fascinating of all,
emotions such as happiness and sadness. It has been suggested that the human brain can process massive information
in almost no time, for which chaotic dynamics could be a fundamental reason: the controlled chaos of the brain is more
than an accidental by-product of the brain complexity, including its myriad connections, but rather, it may be the chief
property that makes the brain different from an artificial-intelligence machine (15). The idea of anticontrol of chaos has
been proposed for solving the problem of driving the system
responses of a human brain model away from the stable direction and, hence, away from the stable (saddle-type) equilibrium. As a result, the periodic behavior of neuronal population bursting can be prevented (16). Control tasks of this type
are also nontraditional.
Other potential applications of chaos control in biological
systems have reached out from the brain to elsewhere, particularly to the human heart. In physiology, healthy dynamics
has been regarded as regular and predictable, whereas disease, such as fatal arrhythmias, aging, and drug toxicity, is
commonly assumed to produce disorder and even chaos. However, recent laboratory studies have seemingly demonstrated
that the complex variability of healthy dynamics in a variety
of physiological systems has features reminiscent of deterministic chaos, and a wide class of disease processes (including drug toxicities and aging) may actually decrease (yet not
completely eliminate) the amount of chaos or complexity in
physiological systems (decomplexification). Thus, in contrast
to the common belief that healthy heartbeats are completely
regular, a normal heart rate may fluctuate in a somewhat
erratic fashion, even at rest, and may actually be chaotic (17).
It has also been observed that, in the heart, the amount of
intracellular Ca is closely regulated by coupled processes
that cyclically increase or decrease this amount, in a way similar to a system of coupled oscillators. This cyclical fluctuation
in the amount of intracellular Ca is a cause of afterdepolarizations and triggered activities in the heartthe so-called
arrhythmogenic mechanism. Medical evidence reveals that
controlling (but not completely eliminating) the chaotic arrhythmia can be a new, safe, and promising approach to regulating heartbeats (18,19).
207
Chaos Control: An Example

To appreciate the challenge of chaos control, consider the onedimensional logistic map of Eq. (18) with the period-doubling
bifurcation route to chaos as shown in Fig. 13. Chaos control
problems in this situation include, but are not limited to, the
following:
Is it possible (and, if so, how) to design a simple (e.g., linear) controller, uk, for the given system in the form
xk+1 = pxk (1 xk ) + uk
such that
1. The limiting chaotic behavior of the period-doubling bifurcation process is suppressed?
2. The first bifurcation is delayed to take place, or some
bifurcations are changed either in form or in stability?
3. When the parameter p is currently not in the bifurcating range, the asymptotic behavior of the system becomes chaotic?
Many of such nonconventional control problems emerging
from chaotic dynamical systems have posed a real challenge
to both nonlinear dynamics analysts and control engineers
they have become, in effect, motivation and stimuli for the
current endeavor devoted to the new research direction in
control systems: controlling bifurcations and chaos.
Some Distinctive Features of Chaos Control
At this point, it is illuminating to highlight some distinctive
features of chaos control theory and methodology, in contrast
to other conventional approaches regarding such issues as objectives, perspectives, problem formulations, and performance measures.
1. The targets in chaos control are usually unstable periodic orbits (including equilibria and limit cycles), perhaps of high periods. The controller is designed to stabilize some of these unstable orbits or to drive the
trajectories of the controlled system to switch from one
orbit to another. This interorbit switching can be either
chaos order, chaos chaos, order chaos, or order
order, depending on the application of interest. Conventional control, on the other hand, does not normally
investigate such interorbit switching problems of a dynamical system, especially not those problems that involve guiding the system trajectory to an unstable or
chaotic state by any means.
2. A chaotic system typically has embedded within it a
dense set of unstable orbits and is extremely sensitive
to tiny perturbations in its initial conditions and system
parameters. Such a special property, useful for chaos
control, is not available in nonchaotic systems and is
not utilized in any forms in conventional controls.
3. Most conventional control schemes work within the
state space framework. In chaos control, however, one
more often deals with the parameter space and phase
space. Poincare maps, delay-coordinates embedding,
parametric variation, entropy reduction, and bifurca-
208
4.
5.
6.
7.
tion monitoring are some typical but nonconventional

tools for design and analysis.
In conventional control, the terminal time for the control is usually finite (e.g., the elementary concept of
controllability is typically defined using a finite and
fixed terminal time, at least for linear systems and affine-nonlinear systems). However, the terminal time for
chaos control is usually infinite to be meaningful and
practical, because many nonlinear dynamical behaviors,
such as equilibrium states, limit cycles, attractors, and
chaos, are asymptotic properties. In addition, in chaos
control, a target for tracking is not limited to constant
vectors in the state space but often is an unstable periodic orbit of the given system.
Depending on different situations or purposes, the performance measure in chaos control can be different from
those for conventional controls. Chaos control generally
uses criteria like Lyapunov exponents, Kolmogorov-Sinai entropy, power spectra, ergodicity, and bifurcation
changes, whereas conventional controls normally emphasize robustness of the system stability or control
performance, optimality of control energy or time, ability of disturbances rejection, etc.
Chaos control includes a unique taskanticontrol, required by some unusual applications such as those in
biomedical engineering mentioned previously. This
anticontrol tries to create, maintain, or enhance chaos
for improving system performance. Bifurcation control
is another example of this kind, where a bifurcation
point is expected to be delayed in case it cannot be
avoided or stabilized. This delay can significantly extend the operating time (or system parameter range) for
a time-critical process such as chemical reaction, voltage collapse of electric power systems, and compression
of stall of gas turbine jet engines. These are in direct
contrast to traditional control tasks, such as the textbook problem of stabilizing an equilibrium position of a
nonlinear system.
Due to the inherent association of chaos and bifurcations with various related issues, the scope of chaos control and the variety of problems that chaos control deals
with are quite diverse, including creation and manipulation of self-similarity and symmetry, pattern formation, amplitudes of limit cycles and sizes of attractor
basins, and birth and change of bifurcations and limit
cycles, in addition to some typical conventional tasks,
such as target tracking and system regulation.
It is also worth mentioning an additional distinctive feature of a controlled chaotic system that differs from an uncontrolled chaotic system. The controlled chaotic system is generally nonautonomous and cannot be reformulated as an
autonomous system by defining the control input as a new
state variable, since the controller is physically not a system
state variable and, moreover, it has to be determined via design for performance specifications. Hence, a controlled chaotic system is intrinsically much more difficult to design than
it appears (e.g., many invariant properties of autonomous systems are no longer valid). This observation raises the question of extending some existing theories and techniques from
autonomous system dynamics to nonautonomous, controlled,
dynamical systems, including such complex phenomena as degenerate bifurcations and hyperchaos in the system dynamics
when a controller is involved. Unless suppressing complex dynamics in a process is the only purpose for control, understanding and utilizing the rich dynamics of a controlled system are very important for design and applications.
Representative Approaches to Chaos Control
There are various conventional and nonconventional control
methods available for bifurcations and chaos control (4, 11,
12). To introduce a few representative ones, only three categories of methodologies are briefly described in this section.
Parametric Variation Control. This approach for controlling
a chaotic dynamical system, proposed by Ott, Grebogi, and
Yorke (13,20) and known as the OGY method, is to stabilize
one of its unstable periodic orbits embedded in an existing
chaotic attractor, via small time-dependent perturbations of
the key system parameter. This methodology utilizes the
special feature of chaos that a chaotic attractor typically
has embedded within it a dense set of unstable periodic
orbits.
To introduce this control strategy, consider a general continuous-time parametrized nonlinear autonomous system
x (t) = f (xx (t), p)
(19)
where, for illustration, x [x y z] denotes the state vector,

and p is a system parameter accessible for adjustment. Assume that when p p* the system is chaotic, and it is desired
to control the system orbit, x(t), to reach a saddle-type unstable equilibrium (or periodic orbit), . Suppose that within a
small neighborhood of p*, that is,
p pmax < p < p + pmax
(20)
where pmax 0 is the maximum allowable perturbation, both

the chaotic attractor and the target orbit do not disappear
(i.e., within this small neighborhood of p*, there are no bifurcation points of the periodic orbit ). Then let P be the underlying Poincare map and be a surface of cross section of .
For simplicity, assume that this two-dimensional hyperplane
is orthogonal to the third axis and thus is given by
= {[ ]T R3 : = z0 (a constant)}
Moreover, let be the coordinates of the surface of cross section; that is, a vector satisfying
k+1 = P(k , pk )
where
pk = p + pk ,
pk pmax
At each iteration, p pk is chosen to be a constant.

Many distinct unstable periodic orbits within the chaotic
attractor can be determined by the Poincare map. Suppose
that an unstable period-one orbit *f has been selected, which
maximizes certain desired system performance with respect
The iteration of the map near the desired orbit are then observed, and the local properties of this chosen periodic orbit
are obtained. To do so, the map is first locally linearized,
yielding a linear approximation of P near *f and p*, as
k+1 f + Lk (k f ) + v k (pk p )
(21)
tabl
em
ani
fold
le
an
ifo
ld
St
Uns
f = P( f , p )
ab
to the dynamical behavior of the system. This target orbit satisfies
209
Target
k + 1 iterate
without
perturbation
kth iterate
or
k+1 Lk k + v k pk
k + 1 iterate
with parameter
perturbation
(22)
Figure 15. Schematic diagram for the parametric variation control

method.
where
k = k f ,
Lk =
P( f ,
pk = pk p ,
p )/k , v k =
P( f ,
p )/ pk
The stable and unstable eigenvalues, s,k and u,k satisfying

s,k 1 u,k, can be calculated from the Jacobian Lk. Let
Ms and Mu be the stable and unstable manifolds whose directions are specified by the eigenvectors es,k and eu,k that are
associated with s,k and u,k, respectively. If gs,k and gu,k are
the basis vectors defined by
gsT,k es ,k = guT ,k eu ,k = 1,
For this case of a saddle-node equilibrium target, this control

procedure is illustrated by Fig. 15.
Now suppose that k has approached sufficiently close to
*f , so that Eq. (21) holds. For the next iterate, k1, to fall onto
the local stable manifold of *f , the parameter pk p* pk
has to be so chosen that
guT ,k k+1 = guT ,k (k+1 f ) = 0
This simply means that the direction of the next iteration is
perpendicular to the direction of the current local unstable
manifold. For this purpose, taking the inner product of Eq.
(22) with gu,k and using Eq. (23) lead to
gsT,k eu ,k = guT ,k es ,k = 0
pk = u ,k
then the Jacobian Lk can be expressed as
Lk = u ,k eu ,k guT ,k + s ,k es ,k gsT,k
(23)
To start the parametric variation control scheme, one may

open a window covering the target equilibrium and wait until
the system orbit travels into the window (i.e., until k falls
close enough to *f ). Then the nominal value of the parameter
pk is adjusted by a small amount pk using a control formula
given below in Eq. (24). In so doing, both the location of the
orbit and its stable manifold are changed, such that the next
iteration, represented by k1 in the surface of cross section, is
forced toward the local stable manifold of the original equilibrium. Since the system has been linearized, this control action
usually is unable to bring the moving orbit to the target at
one iteration. As a result, the controlled orbit will leave the
small neighborhood of the equilibrium again and continue to
wander chaotically as if there was no control on it at all. However, due to the semi-attractive property of the saddle-nose
equilibrium, sooner or later the orbit returns to the window
again, but generally is closer to the target due to the control
effect. Then the next cycle of iteration starts, with an even
smaller control action, to nudge the orbit toward the target.
guT ,k k
guT ,kv k
(24)
where it is assumed that gu,kvk 0. This is the control formula for determining the variation of the adjustable system
parameter p at each step, k 1, 2, . The controlled orbit
thus is expected to approach *f at a geometric rate.
Note that this calculated pk is used to adjust the parameter p only if pk pmax. When pk pmax, however, one
should set pk 0. Also, when k1 falls on a local stable manifold of *f , one should set pk 0 because the stable manifold
might lead the orbit directly to the target.
Note also that the preceding derivation is based on the assumption that the Poincare map, P, always possesses a stable
and an unstable direction (saddle-type orbits). This may not
be the case in many systems, particularly those with high periodic orbits. Moreover, it is necessary that the number of accessible parameters for control is at least equal to the number
of unstable eigenvalues of the periodic orbit to be stabilized.
In particular, when some of such key system parameters are
unaccessible, the algorithm is not applicable or has to be modified. Also, if a system has multiattractors the system orbit
may never return to the opened window but move to another
nontarget limit set. In addition, the technique is successful
only if the control is applied after the system orbit moves into
the small window covering the target, over which the local
210
linear approximation is still valid. In this case, the waiting

time can be quite long for some chaotic systems. While this
algorithm is effective, it generally requires good knowledge of
the equations governing the system, so that computing pk by
Eq. (24) is possible. In the case where only time-series data
of the system are available, the delay-coordinate technique
may be used to construct a faithful dynamical model for control (20,21).
Entrainment and Migration Controls. Another representative
approach for chaos control is the entrainment and migration
control. Originally an open-loop strategy, this approach has
lately been equipped with the closed-loop control technique
and has been applied to many complex dynamical systems,
particularly those with multiattractors. The entrainment and
migration control strategy results in a radical but systematic
modification of the behavior of the given dynamical system,
thereby allowing to introduce a variety of new dynamical motions into the system (22,23). This approach can handle a
multiattractor situation effectively, as opposed to the parametric variation control method.
Entrainment means that an otherwise chaotic orbit of a
system can be purposely entrained so that its dynamics, in
both amplitude and phase, asymptotically tends to a prespecified set or region (e.g., a periodic orbit or an attractor). The
basic formulation of entrainment control is based on the existence of some convergent regions in the phase space of a dynamical system. For a general smooth discrete-time system,
x k+1 = f k (xx k ),
k = 0, 1, . . .
For simplicity, assume that Gk Ck with the goal orbit gk
Gk, k 1, 2, . Let C k1
Ck, and denote the basin of
entrainment for the goal by
B = {xx0 Rn : lim xx k g k = 0}
k
Once a near entrainment is obtained in the sense that

xx k g k
for some small 0, another form of control can be applied
(i.e., to use migration-goal dynamics between different convergent regions, which allows the system trajectory to travel
from one attractor to another). This is the entrainment-migration control strategy.
To describe the entrainment-goal control more precisely,
consider a discrete-time system of the form
x k+1 = f k (xx k ),
Let the goal dynamics be gk and Sk be a switching function

defined by Sk 1 at some desired steps k but Sk 0 otherwise. The controlled dynamical system is suggested as
x k+1 = f k (xx k ) + k Sk [ g k+1 f k ( g k )]
where 0 k 1 are constant control gains determined by
the user. The control is initiated, with Sk 1, if the system
state has entered the basin B; that is, when the system state
enters the basin B at k kb, the control is turned on for k
kb. With k 1, it gives
or continuous-time system,
x (t) = f (xx (t)),
x (0) = x 0
the convergent regions are defined to be
g k ) = x k+1 f k (xx k ) = 0,
g k+1 f k (g
or
C( f ) = {xx Rn : f i (xx )/x j ij i (xx ) = 0,

{i (xx )} < 0 for all i = 1, . . ., n}
where i( ) and i( ) are the eigenvalues of the Jacobians of
the nonlinear maps f k and f, respectively, and ij 1 if i j
but ij 0 if i j.
If these convergent regions exist, the system orbitssay,
xk in the discrete casecan be forced by a suitably designed
external input to approach (a limit set of) the desired goal
dynamics, gk, in the sense that
lim xx k g k = 0
In other words, the system is entrained to the goal dynamics.

One advantage of the entrainment control is that the goal
dynamics can have any topological characteristics, such as
equilibrium, periodic, knotted, and chaotic, provided that the
target orbit gk is located in some goal region Gk satisfying
G k Ck 0
, where Ck (k 1, 2, ) are convergent regions.
k kb
(25)
The desired goal dynamics is then achieved: gk1 f(gk) for

all k kb. Clearly, in this approach,
u k = k Sk [ g k+1 f k ( g k )]
C( f k ) = {xx Rn : f i (xx )/x j ij i (xx ) = 0,

i (xx ) < 1 for all i = 1, . . ., n}
x k Rn
is an open-loop controller, which is directly added to the righthand side of the original system.
A meaningful application of the entrainment control is for
multiattractor systems, to which the parametric variation
control method is not applicable. Another important application is for a system with an asymptotic goal gk g, an equilibrium of the given system. In this case, the basin of entrainment is a convex region in the phase space:
Be = {x0 Rn : xx 0 g < r( g )}
where
r( g ) = max{r : xx 0 g < r lim xx k g = 0}
r
The entrainment-migration control method is straightforward, easily implementable, and flexible in design. However,
it requires the dynamics of the system be accurately described
by either a map or a differential equation. Also, in order for
the system state to be entrained to the given equilibrium, the
equilibrium must lie in a particular subset of the convergent
region. This can be a technical issue, particularly for higherdimensional systems. In addition, due to the open-loop na-
ture, process stability is not guaranteed in most cases. The

main disadvantage of this approach is that it generally employs sophisticated controllers, which may even be more complicated than the given system.
Engineering Feedback Controls. From a control theoretic
point of view, if only suppression of chaos is concerned, chaos
control may be considered as a special deterministic nonlinear
control problem and so may not be much harder than conventional nonlinear systems control. However, this remains to be
a technical challenge to conventional controls when a single
controller is needed for stabilizing the chaotic trajectory to
multiple targets of different periods.
A distinctive characteristic of control engineering from
other disciplines is that it employs some kind of feedback
mechanism. In fact, feedback is pervasive in modern control
theories and technologies. For instance, the parametric variation control method discussed previously is a special type of
feedback control method. In engineering control systems, conventional feedback controllers are used for nonchaotic systems. In particular, linear feedback controllers are often designed for linear systems. It has been widely experienced that
with careful design of various conventional controllers, controlling chaotic systems by feedback strategies is not only possible, but indeed quite successful. One basic reason for this
success is that chaotic systems, although nonlinear and sensitive to initial conditions with complex dynamical behaviors,
belong to deterministic systems by their very nature.
Some Features of Feedback Control. Feedback is one of the
most fundamental principles prevalent in the world. The idea
of using feedback, originated from Isaac Newton and Gottfried Leibniz some 300 years ago, has been applied in various
forms in natural science and modern technology.
One basic feature of conventional feedback control is that,
while achieving target tracking, it can guarantee the stability
of the overall controlled system, even if the original uncontrolled system is unstable. This implies its intrinsic robustness against external disturbances or internal variations
to a certain extent, which is desirable and often necessary for
good performance of a required control task. The idea of feedback control always consuming strong control energy perhaps
led to a false impression that feedback mechanisms may not
be suitable for chaos control due to the extreme sensitive
nature of chaos. However, feedback control under certain
optimality criteria, such as a minimum control energy constraint, can provide the best performance, including the
lowest consumption of control energy. This is not only supported by theory but is also confirmed by simulation with
comparison.
Another advantage of using feedback control is that it normally does not change the structure and parameters of the
given system, and so whenever the feedback is disconnected
the given system retains the original form and properties
without modification. In many engineering applications, the
system parameters are not feasible or not allowed for direct
tuning or replacement. In such cases, state or output feedback
control is a practical and desirable strategy.
An additional advantage of feedback control is its automatic fashion in processing control tasks without further human interaction after being designed and implemented. As
long as a feedback controller is correctly designed to satisfy
the stability criteria and performance specifications, it works
211
on its own. This is important for automation, reducing the

dependence on individual operators skills and avoiding human errors in monitoring the control.
A shortcoming of feedback control methods that employ
tracking errors is the explicit or implicit use of reference signals. This has never been a problem in conventional feedback
control of nonchaotic systems, where reference signals are always some designated, well-behaved ones. However, in chaos
control, quite often a reference signal is an unstable equilibrium or unstable limit cycle, which is difficult (if not impossible) to be physically implemented as a reference input. This
critical issue has stimulated some new research efforts (for
instance, to use another auxiliary reference as the input in a
self-tuning feedback manner).
Engineering feedback control approaches have seen an alluring future in more advanced theories and applications in
controlling complex dynamics. Utilization of feedback is
among the most inspiring concepts that engineering has ever
contributed to modern sciences and advanced technologies.
A Typical Feedback Control Problem. A general feedback approach to controlling a dynamical system, not necessarily chaotic nor even nonlinear, can be illustrated by starting from
the following general form of an n-dimensional control system:
x (t) = f (xx, u , t),
x (0) = x 0
(26)
where, as usual, x is the system state, u is the controller, x0

is a given initial state, and f is a piecewise continuous or
smooth nonlinear function satisfying some defining conditions.
Given a reference signal, r(t), which can be either a constant (set-point) or a function (time-varying trajectory), the
automatic feedback control problem is to design a controller
in, say, the state-feedback form
u (t) = g (xx, t)
(27)
where g is generally a piecewise continuous nonlinear function, such that the feedback-controlled system
x (t) = f (xx, g (xx, t), t)
(28)
can achieve the goal of tracking:

lim xx (t) r (t) = 0
(29)
For discrete-time systems, the problem and notation are

similar: For a system
x k+1 = f k (xx k , u k )
(30)
with given target trajectory rk and initial state x0, find a

(nonlinear) controller
u k = g k (xx k )
(31)
to achieve the tracking-control goal:

lim xx k r k = 0
(32)
212

r
ex
which yields en(t) : xn(t) yn 0 as t . Overall, it results

in a completely controllable linear system, so that the constant control gain kc can be chosen such that x(t) y as t
. Another example is that for the control system
f (.)
g(.)
x (t) = f (xx (t), t) + u (t)
Figure 16. Configuration of a general feedback control system.
using
u (t) = f (xx (t), t) + y (t) + K(xx (t) y (t))
A closed-loop continuous-time feedback control system has

a configuration as shown in Fig. 16, where ex : r g(x), f is
the given system, and g is the feedback controller to be designed, in which f and g can be either linear or nonlinear. In
particular, it can be a linear system in the state-space form
connected with a linear additive state-feedback controller
namely,
It is then mathematically straightforward to use the controller
with a stable constant gain matrix K can drive its trajectory

to the target y(t) as t .
This kind of design, however, is undesirable, and its
practical value is questionable in most cases, because the controller is even more complicated than the given system (it
cancels the nonlinearity by using the given nonlinearity
which means it removes the given plant and then replaces it
by another system). In the discrete-time setting, for a given
nonlinear system, xk1 f k(xk) uk, one may also find a similar nonlinear feedback controller or, even simpler, use u
f k(xk) gk(xk) to achieve any desired dynamics satisfying
xk1 gk(xk) in just one step. This is certainly not an engineering design, nor a valuable methodology, for any realworld application other than artificial computer simulations.
Therefore, in designing a feedback controller, it is very important to come out with a simplest possible working controller: If a linear controller can be designed to do the job, use a
linear controller; otherwise, try the simplest possible nonlinear controllers (starting, for example, from piecewise linear or
quadratic controllers). Whether or not one can find a simple,
physically meaningful, easily implementable, low-cost, and effective controller for a designated control task can be quite
technical: It relies on the designers theoretical background
and practical experience.
A General Approach to Feedback Control of Chaos. To outline
the basic idea of a general feedback approach to chaos suppression and tracking control, consider Eq. (26), which is now
assumed to be chaotic and possess an unstable periodic orbit
(or equilibrium), x, of period tp 0namely, x(t tp) x(t),
t0 t . The task is to design a feedback controller in the
form of Eq. (27), such that the tracking control goal of Eq.
(29), with r x therein, is achieved.
Since the target periodic orbit x is itself a solution of the
original system, it satisfies
u(t) = f (x1 (t), . . ., xn (t)) + kc (xn (t) yn )
x = f (xx, 0, t)
u = Axx + BKc (rr x )

x = Axx + Bu
where Kc is a constant control gain matrix to be determined.
The corresponding closed-loop block diagram is shown in
Fig. 17.
A Control Engineers Perspective. In controllers design, particularly in finding a nonlinear controller for a system, it is
important to emphasize that the designed controller should
be (much) simpler than the given system to make sense of
the world.
For instance, suppose that one wants to find a nonlinear
controller, u(t), in the continuous-time setting, to guide the
state vector x(t) [x1(t), , xn(t)] of a given nonlinear control system,
x1 (t) = x2 (t)
x (t) = x (t)
2
..
x (t) = f (x (t), . . ., x (t)) + u(t)

n
n
1
to a target state, y [y1, . . ., yn] namely,
x (t) y as t
with an arbitrary constant kc 0. This controller leads to

xn (t) = kc (xn (t) yn )
Subtracting Eq. (33) from Eq. (26) then yields the error dynamics:
e x = f e (eex , x , t)
r +
ex
Kc
x
+
Figure 17. Configuration of a state-space feedback control system.
(33)
(34)
where
ex (t) = x (t) xx (t), f e (eex , x , t) = f (xx, g (xx, x , t), t) f (xx, 0, t)
Here, it is important to note that in order to perform correct
stability analysis later on, in the error dynamical system of
Eq. (34) the function f e must not explicitly contain x; if so, x
should be replaced by ex x (see Eq. (38) below). This is because Eq. (34) should only contain the dynamics of ex but not
x, while the system may contain x, which merely is a specified

time function but not a system variable.
Thus, the design problem becomes to determine the controller, u(t), such that
lim eex (t) = 0
(35)
which implies that the goal of tracking control described by

Eq. (29) is achieved.
It is clear from Eqs. (34) and (35) that if zero is an equilibrium of the error dynamical system of Eq. (34), then the original control problem has been converted to the asymptotic stability problem for this equilibrium. As a result, Lyapunov
stability methods and theorems can be directly applied or
modified to obtain rigorous mathematical techniques for controller design (24). This is discussed in more detail next.
Chaos Control via Lyapunov Methods. The key in applying
the Lyapunov second method to a nonlinear dynamical system is to construct a Lyapunov function that describes some
kind of energy and governs the system motion. If this function
is constructed appropriately, so that it decays monotonically
to zero as time evolves, then the system motion, which falls
on the surface of this decaying function, will be asymptotically stabilized to zero. A controller, then, may be designed
to force this Lyapunov function of the system, stable or not
originally, to decay to zero. As a result, the stability of
tracking error equilibrium, and hence the goal of tracking, is
achieved. For a chaos control problem with a target trajectory
x, typically an unstable periodic solution of the given system,
a design can be carried out by determining the controller
u(t) via the Lyapunov second method such that the zero equilibrium of the error dynamics, ex 0, is asymptotically stable.
In this approach, since a linear feedback controller alone
is usually not sufficient for the control of a nonlinear system,
particularly a chaotic one, it is desirable to find some criteria
for the design of simple nonlinear feedback controllers. In so
doing, consider the feedback controller candidate of the form
u (t) = Kc (xx x ) + g (xx x , k c , t)
(36)
where Kc is a constant matrix, which can be zero, and g is a

simple nonlinear function with constant parameters kc, satisfying g(0, kc, t) 0 for all t t0. Both Kc and kc are determined in the design. Adding this controller to the given system gives
x = f (xx , t) + u = f (xx, t) + Kc (xx x ) + g (xx x , k c , t)
(37)
The controller is required to drive the trajectory of the controlled system of Eq. (37) to approach the target orbit x.
The error dynamics of Eq. (34) now takes the form
ex = f e (eex , t) + Kc ex + g (eex , k c , t)
(38)
where
ex = x x ,
f e (eex , t) = f (eex + x , t) f (xx, t)
It is clear that f e(0, t) 0 for all t [t0, )namely, ex 0

is an equilibrium of the tracking-error dynamical system of
Eq. (38).
Next, Taylor expand the right-hand side of the controlled
system of Eq. (38) at ex 0 (i.e., at x x) and remember that
213
the nonlinear controller will be designed to satisfy g(0, kc, t)

0. Then the error dynamics is reduced to
ex = A(xx, t)eex + h (eex , Kc , k c , t)
(39)
where

f e (eex , t)
A(xx, t) =
eex

e x =0
and h(ex, Kc, kc, t) contains the rest of the Taylor expansion.
The design is then to determine both the constant control
gains Kc and kc as well as the nonlinear function g( , , t)
based on the linearized model of Eq. (39), such that ex 0 as
t . When this controller is applied to the original system,
the goal of both chaos suppression and target tracking will be
achieved. For illustration, two controllability conditions established based on the boundedness of the chaotic attractors
as well as the Lyapunov first and second methods, respectively, are summarized next (24).
Suppose that in Eq. (39), h(0, Kc, kc, t) 0 and A(x, t) A
is a constant matrix whose eigenvalues all have negative real
parts, and let P be the positive definite and symmetric solution of the Lyapunov equation
PA + AT P = I
where I is the identity matrix. If Kc is designed to satisfy
h (eex , Kc , k c , t) ceex
h
for a constant c max(P) for t0 t , where max(P) is the
maximum eigenvalue of P, then the controller u(t), defined in
Eq. (36), will drive the trajectory x of the controlled system of
Eq. (37) to the target, x, as t .
For Eq. (39), since x is tp-periodic, associated with the matrix A(x, t) there always exist a tp-periodic nonsingular matrix M(x, t) and a constant matrix Q such that the fundamental matrix (consisting of n independent solution vectors) has
the expression
(xx, t) = M(xx, t)etQ
The eigenvalues of the constant matrix etpQ are called the Floquet multipliers of the system matrix A(x, t).
In Eq. (39), assume h(0, Kc, kc, t) 0 and h(ex, Kc, kc, t)
and h(ex, Kc, kc, t)/ex are both continuous in a bounded
neighborhood of the origin in Rn. Assume also that
lim
e x 0
h (eex , Kc , k c , t)
h
=0
eex
uniformly with respect to t [t0, ). If the nonlinear controller of Eq. (36) is so designed that all Floquet multipliers i
of the system matrix A(x, t) satisfy
i (t) < 1,
i = 1, . . ., n,
t [t0 , )
then the controller will drive the chaotic orbit x of the controlled system of Eq. (37) to the target orbit, x, as t .
Various Feedback Methods for Chaos Control. In addition to
the general nonlinear feedback control approach described pre-
214
viously, adaptive and intelligent controls are two large classes

of engineering feedback control methods that have been shown
to be successful for chaos control. Other effective feedback control methods include optimal control, sliding mode and robust
controls, digital controls, and occasionally proportional and
time-delayed feedback controls. Linear feedback controls are
also useful, but generally for simple chaotic systems. Various
variants of classical control methods that have demonstrated
great potential for controlling chaos include distortion control,
dissipative energy method, absorber as a controller, external
weak periodic forcing, Kolmogorov-Sinai entropy reduction,
stochastic controls, and chaos filtering (4).
Finally, it should be noted that there are indeed many valuable ideas and methodologies that by their nature cannot be
well classified into one of the aforementioned three categories,
not to mention that many novel approaches are still emerging, improving, and developing as of today (4).
CONTROLLING BIFURCATIONS
Ordering chaos via bifurcation control has never been a subject in conventional control. This seems to be a unique approach valid only for those nonlinear dynamical systems that
possess the special characteristic of a route to chaos from bifurcation.
f (xx; p) = J(p)xx + Q(xx, x ; p) + C(xx, x , x ; p) +

where J(p) is the parametric Jacobian, and Q and C are quadratic and cubic terms generated by symmetric bilinear and
trilinear forms, respectively.
This system has the following property (25): A period-doubling orbit can bifurcate from the origin of system of Eq. (40)
at p 0; the period-doubling bifurcation is supercritical and
stable if 0 but is subcritical and unstable if 0, where
= 2ll T [C0 (rr, r , r ; p) 2Q0 (rr, J0 Q0 (rr, r ; p))]
in which l is the left eigenvector and r the right eigenvector
of J(0), respectively, both associated with the eigenvalue 1,
and
Q0 = J(0)Q(xx, x ; p) + Q(J(0)xx, J(0)xx; p)
Why Bifurcation Control?

Bifurcation and chaos are often twins and, in particular, period-doubling bifurcation is a route to chaos. Hence, by monitoring and manipulating bifurcations, one can expect to
achieve certain types of control for chaotic dynamics.
Even bifurcation control itself is very important. In some
physical systems such as a stressed system, delay of bifurcations offers an opportunity to obtain stable operating conditions for the machine beyond the margin of operability in a
normal situation. Also, relocating and ensuring stability of bifurcated limit cycles can be applied to some conventional control problems, such as thermal convection, to obtain better
results. Other examples include stabilization of some critical
situations for tethered satellites, magnetic bearing systems,
voltage dynamics of electric power systems, and compressor
stall in gas turbine jet engines (4).
Bifurcation control essentially means designing a controller for a system to result in some desired behaviors, such as
stabilizing bifurcated dynamics, modifying properties of some
bifurcations, or taming chaos via bifurcation control. Typical
examples include delaying the onset of an inherent bifurcation, relocating an existing bifurcation, changing the shape or
type of a bifurcation, introducing a bifurcation at a desired
parameter value, stabilizing (at least locally) a bifurcated periodic orbit, optimizing the performance near a bifurcation
point for a system, or a combination of some of these. Such
tasks have practical values and great potential in many nontraditional real-world control applications.
Bifurcation Control via Feedback
Bifurcations can be controlled by different methods, among
which the feedback strategy is especially effective. Consider a
general discrete-time parametrized nonlinear system
x k+1 = f (xx k ; p),
where f is assumed to be sufficiently smooth with respect to

both the state xk Rn and the parameter p R, and has an
equilibrium at (x, p) (0, 0). In addition, assume that the
system Jacobian of Eq. (40), evaluated at the equilibrium that
is the continuous extension of the origin, has an eigenvalue
(p) satisfying (0) 1 and (0) 0, while all remaining
eigenvalues have magnitude strictly less than one. Under
these conditions, the nonlinear function has a Taylor expansion
k = 0, 1, . . .
(40)
C0 = J(0)C(xx, x , x ; p) + 2Q(J(0)xx, Q(xx, x ; p))

+ C(J(0)xx, J(0)xx, Q(xx , x; p); p)
J0 = [J T (0)J(0) + ll T ]1 J T (0)
Now consider Eq. (40) with a control input:
x k+1 = f (xx k ; p, u k ),
k = 0, 1, . . .
which is assumed to satisfy the same assumptions when uk

0. If the critical eigenvalue 1 is controllable for the linearized system, then there is a feedback controller, uk(xk), containing only third-order terms in the components of xk, such
that the controlled system has a locally stable bifurcated period-two orbit for p near zero. Also, this feedback stabilizes
the origin for p 0. If, however, 1 is uncontrollable for the
linearized system, then generically there is a feedback controller, uk(xk), containing only second-order terms in the components of xk, such that the controlled system has a locally
stable bifurcated period-two orbit for p near 0. This feedback
controller also stabilizes the origin for p 0 (25).
Bifurcation Control via Harmonic Balance
For continuous-time systems, limit cycles in general cannot
be expressed in analytic forms, and so limit cycles corresponding to the period-two orbits in the period-doubling bifurcation
diagram have to be approximated in applications. In this
case, the harmonic balance approximation technique (10) can
be applied, which is also useful in controlling bifurcations
such as delay and stabilization of the onset of period-doubling
bifurcations (26).
Consider a feedback control system in the Lure form described by
f ( g y + Kc y ) + y = 0
f (.;p)
Kc
g(.;p)
Figure 18. A feedback system in the Lure form.
where and represent the convolution and composition operations, respectively, as shown in Fig. 18. First, suppose that
a system S S(f, g) is given as shown in the figure without
the feedback controller, Kc. Assume also that two system parameter values, ph and pc, are specified, which define a Hopf
bifurcation and a supercritical predicted period-doubling bifurcation, respectively. Moreover, assume that the system has
a family of predicted first-order limit cycles, stable in the
range ph p pc.
Under this system setup, the problem for investigation is
to design a feedback controller, Kc, added to the system as
shown in Fig. 18, such that the controlled system, S*
S*(f, g, Kc), has the following properties:
1. S* has a Hopf bifurcation at p*h ph.
2. S* has a supercritical predicted period-doubling bifurcation for p*c pc.
3. S* has a one-parameter family of stable predicted limit
cycles for p*h p p*c .
4. S* has the same set of equilibria as S.
Only the one-dimensional case is discussed here. First, one
can design a washout filter with the transfer function s/(s
a), where a 0, such that it preserves the equilibria of the
given nonlinear system. Then note that any predicted firstorder limit cycle can be well approximated by
y
(1)
(t) = y0 + y1 sin(t)
In so doing, the controller transfer function becomes

Kc (s) = kc
s(s2 + 2 (ph ))
(s + a)3
where kc is the constant control gain, and (ph) is the frequency of the limit cycle emerged from the Hopf bifurcation
at the point p ph. This controller also preserves the Hopf
bifurcation at the same point. More importantly, since a 0,
the controller is stable, so by continuity in a small neighborhood of kc the Hopf bifurcation of S* not only remains supercritical but also has a supercritical predicted period-doubling
bifurcation (say at pc(kc), close to ph) and a one-parameter
family of stable predicted limit cycles for ph p pc(kc).
The design is then to determine kc such that the predicted
period-doubling bifurcation can be delayed, to a desired parameter value p*c . For this purpose, the harmonic balance approximation method (10) is useful, which leads to a solution
of y(1) by obtaining values of y0, y1, and (they are functions
of p, depending on kc and a, within the range ph p p*c ).
The harmonic balance also yields conditions, in terms of kc, a
215
and a new parameter, for the period-doubling to occur at the

point p*c . Thus, the controller design is completed by choosing
a suitable value for kc to satisfy such conditions (26).
Controlling Multiple Limit Cycles
As indicated by the Hopf bifurcation theorem, limit cycles are
frequently associated with bifurcations. In fact, one type of
degenerate (or singular) Hopf bifurcations (when some of the
conditions stated in the Hopf theorems are not satisfied) determines the birth of multiple limit cycles under system parameters variation. Hence, the appearance of multiple limit
cycles can be controlled by manipulating the corresponding
degenerate Hopf bifurcations. This task can be conveniently
accomplished in the frequency-domain setting.
Again, consider the feedback system of Eq. (16), which can
be illustrated by a variant of Fig. 18. For harmonic expansion
of the system output, y(t), the first-order formula is (10)
y 1 = rr + 3 z 13 + 5 z 15 +
where is shown in Fig. 12, r is defined in Eq. (17), and z13,
, z1,2m1 are some vectors orthogonal to r, m 1, 2, ,
given by explicit formulas (10).
Observe that for a given value of , defined in the graphical Hopf theorem, the SISO system transfer function satisfies
H( j )
= H(s) + ( + j)H (s) +
1
( + j)2 H (s) +
2
(41)
where , with being the imaginary part of the

bifurcating eigenvalues, and H(s) and H(s) are the first and
second derivatives of H(s), defined in Eq. (16), respectively.
On the other hand, with the higher-order approximations, the
following equation of harmonic balance can be derived:
[H( j)J + I]
m
z 1,2i+1 2i+1 = H( j)
i=0
m
r 1,2i+1 2i+1
i=1
where z11 r and r1,2m1 hm, m 1, 2, , in which h1 has

the formula shown in Eq. (17), and the others also have explicit formulas (10).
In a general situation, the following equation has to be
solved:
[H( j )
J + I](rr + z 13 3 + z 15 5 + )
= H( j )[h
h1 3 + h 2 5 + ]
(42)
In so doing, by substituting Eq. (41) into Eq. (42), one obtains

the expansion
( j) = 1 2 + 2 4 + 3 6 + 4 8 + O( 9 )
(43)
in which all the coefficients i, i 1, 2, 3, 4, can be calculated

explicitly (10). Then taking the real part of Eq. (43) gives
= 1 2 2 4 3 6 4 8
where i R i are the curvature coefficients of the
expansion.
To this end, notice that multiple limit cycles will emerge
when the curvature coefficients are varied near the value
216
zero, after alternating the signs of the curvature coefficients

in increasing (or decreasing) order. For example, to have four
limit cycles in the vicinity of a type of degenerate Hopf bifurcation that has 1 2 3 0 but 4 0 at the criticality,
the system parameters have to be varied in such a way that,
for example, 0, 1 0, 2 0, 3 0, and 4 0. This
condition provides a methodology for controlling the birth of
multiple limit cycles associated with degenerate Hopf bifurcations.
One advantage of this methodology is that there is no need
to modify the feedback control path by adding any nonlinear
components, to drive the system orbit to a desired region. One
can simply modify the system parameters, a kind of parametric variation control, according to the expressions of the curvature coefficients, to achieve the goal of controlling bifurcations and limit cycles.
ANTICONTROL OF CHAOS
Anticontrol of chaos, in contrast to the main stream of ordering or suppressing chaos, is to make a nonchaotic dynamical
system chaotic or to retain/enhance the existing chaos of a
chaotic system. Anticontrol of chaos as one of the unique
features of chaos control has emerged as a theoretically attractive and potentially useful new subject in systems control
theory and some time-critical or energy-critical high-performance applications.
Why Anticontrol of Chaos?
Chaos has long been considered as a disaster phenomenon
and so is very fearsome in beneficial applications. However,
chaos is dynamics freed from the shackles of order and predictability. Under good conditions or suitable control, it permits systems to randomly explore their every dynamical possibility. It is exciting variety, richness of choice, a cornucopia
of opportunities (27).
Today, chaos theory has been anticipated to be potentially
useful in many novel and time- or energy-critical applications.
In addition to those potential utilizations of chaos mentioned
earlier in the discussion of chaos control, it is worth mentioning navigation in the multibody planetary system, secure
information processing via chaos synchronization, dynamic
crisis management, and critical decision making in political,
economical, and military events. In particular, it has been observed that a transition of a biological systems state from
being chaotic to being pathophysiologically periodic can cause
the so-called dynamical disease and so is undesirable. Examples of dynamical diseases include cell counts in hematological disorder; stimulant drug-induced abnormalities in the behavior of brain enzymes and receptors; cardiac interbeat
interval patterns in a variety of cardiac disorders; the resting
record in a variety of signal sensitive biological systems following desensitization; experimental epilepsy; hormone release patterns correlated with the spontaneous mutation of a
neuroendocrine cell to a neoplastic tumor; the prediction of
immunologic rejection of heart transplants; the electroencephalographic behavior of the human brain in the presence of
neurodegenerative disorder; neuroendocrine, cardiac, and
electroencephalographic changes with aging; and imminent
ventricular fibrillation in human subjects (28). Hence, pre-
serving chaos in these cases is important and healthy, which

presents a real challenge for creative research on anticontrol
of chaos (4).
Some Approaches to Anticontrolling Chaos
Anticontrol of chaos is a new research direction. Different
methods for anticontrolling chaos are possible (4), but only
two preliminary approaches are presented here for illustration.
Preserving Chaos by Small Control Perturbations. Consider an
n-dimensional discrete-time nonlinear system
x k+1 = f (xx k , p, uk )
xk is the system state, uk is a scalar-valued control input, p is
a variable parameter, and f is a locally invertible nonlinear
map. Assume that with uk 0 the system orbit behaves chaotically at some value of p, and that when p increases and
passes a critical value, pc, inverse bifurcation emerges leading
the chaotic state to periodic.
Within the biological context, such a bifurcation is often
undesirable: There are many cases where loss of complexity
and the emergence of periodicity are associated with pathology (dynamical disease). The question, then, is whether it is
possible (if so, how) to keep the system state chaotic even if
p pc, by using small control inputs, uk.
It is known that there are at least three common bifurcations that can lead chaotic motions directly to low-periodic
attracting orbits: (1) crises, (2) saddle-node type of intermittency, and (3) inverse period-doubling type of intermittency.
Here, crisis refers to sudden changes caused by the collision
of an attractor with an unstable periodic orbit; intermittency
is a special route to chaos where regular orbital behavior is
intermittently interrupted by a finite duration burst in
which the orbit behaves in a decidedly different fashion; and
inverse period-doubling bifurcation has a diagram in reverse
form to that shown in Fig. 13 (i.e., from chaos back to less
and less bifurcating points, leading back to a periodic motion)
while the parameter remains increasing.
In all these cases, one can identify a loss region, G, which
has the property that after the orbit falls into G, it is rapidly
drawn to the periodic orbit. Thus, a strategy to retain the
chaos for p pc is to avoid this from happening by successively iterating G in such a way that
G1 = f 1 (G, p, 0),
G2 = f 1 (G1 , p, 0) = f 2 (G, p, 0),
..
.
Gm = f m (G, p, 0)
As m increases, the width of Gm in the unstable direction(s)
has a general tendency to shrink exponentially. This suggests
the following control scheme (28):
Pick a suitable value of m, denoted m0. Assume that the orbit initially starts outside the region Gm01 Gm0 G1 G. If
the orbit lands in Gm01 at iterate , the control ul is applied to
kick the orbit out of Gm0 at the next iterate. Since Gm0 is thin, this

control can be very small. After the orbit is kicked out of Gm0, it is
expected to behave chaotically, until it falls again into Gm01; at
that moment another small control is applied, and so on. This procedure can keep the motion chaotic.
217
It turns out that this is possible under a natural condition

that all the Jacobians fk(xk) are uniformly bounded:
sup f k (xx k ) f <
(46)
0k
Anticontrol of Chaos via State Feedback. An approach to

anticontrol of discrete-time systems can be made mathematically rigorous by applying the engineering feedback control
strategy. This anticontrol technique is first to make the Lyapunov exponents of the controlled system either strictly positive or arbitrarily assigned (positive, zero, and negative in
any desired order), and then apply the simple mod operations
(4,29). This task can be accomplished for any given higherdimensional discrete-time dynamical system that could be
originally nonchaotic or even asymptotically stable. The argument used is purely algebraic and the design procedure is
completely schematic without approximations.
Specifically, consider a nonlinear dynamical system, not
necessarily chaotic nor unstable to start with, in the general
form
x k+1 = f k (xx k )
(44)
where xk Rn, x0 is given, and f k is assumed to be continuously differentiable, at least locally in the region of interest.
The anticontrol problem for this dynamical system is to
design a linear state-feedback control sequence, uk Bkxk,
with uniformly bounded constant control gain matrices,
Bks u , where s is the spectral norm for a matrix,
such that the output states of the controlled system
behaves chaotically within a bounded region. Here, chaotic

behavior is in the mathematical sense of Devaney described
previouslynamely, the controlled map (a) is transitive, (b)
has sensitive dependence on initial conditions, and (c) has a
dense set of periodic solutions (9).
In the controlled system
x x+1 = f k (xx k ) + Bkx k
let
Jk (xx k ) = f k (xx k ) + Bk
be the system Jacobian, and let
Tk (xx 0 ) = Jk (xx k ) J1 (xx 1 )J0 (xx 0 ), k = 0, 1, 2, . . .
Moreover, let ik i(Tk Tk) be the ith eigenvalue of the kth
product matrix [Tk Tk], where i 1, , n and k 0, 1,
2, .
The first attempt is to determine the constant control gain
matrices, Bk, such that the Lyapunov exponents of the controlled system are all finite and strictly positive:
i = 1, . . ., n
i = lim
1
ln i(k)
k
(for i(k) > 0),
i = 1, . . ., n
(k1)i
Clearly, if (k)
is used in the design, then all (k)
i e
i will
not be zero for any finite values of i, for all i 1, , n and
k 0, 1, . Thus, Tk(x0) is always nonsingular. Consequently, a control-gain sequence Bk can be designed such
that the singular values of the matrix Tk(x0) are exactly equal
n
to ekii1
: At the kth step, k 0, 1, 2, , one may simply
choose the control gain matrix to be
Bk = ( f + ec )In ,
for all k = 0, 1, 2, . . .
where the constants c and f are given in Eqs. (45) and (46),
respectively (29). This ensures Eq. (45) to hold.
Finally, in conjunction with the previously designed controllerthat is,
u k = Bkx k = ( f + ec )xx k
anticontrol can be accomplished by imposing the mod operation in the controlled system:
x k+1 = f k (xx k ) + u k
0 < c i (xx 0 ) < ,
To come up with a design methodology, first observe that

n
if (k)
i i1 are the singular values of the matrix Tk(x0); then
(k)
i 0 for all i 1, , n and k 0, 1, . Let i 0 for
(k)
i 0 and
(45)
x k+1 = f k (xx k ) + u k
(mod 1)
This results in the expected chaotic system whose trajectories remain within a bounded region in the phase space
and, moreover, satisfies the aforementioned three basic
properties that together define discrete chaos. This approach yields rigorous anticontrol of chaos for any given
discrete-time systems, including all higher-dimensional, linear time-invariant systems; that is, with f k(xk) Axk in Eq.
(44), where the constant matrix A can be arbitrary (even
asymptotically stable).
Although uk Bkxk is a linear state-feedback controller,
it uses full-order state variables, and the mod operation is
inherently nonlinear. Hence, other types of (simple) feedback controllers are expected to be developed in the near
future for rigorous anticontrol of chaos, particularly for continuous-time dynamical systems [which is apparently much
more difficult (30), especially if small control input is desired].
BIBLIOGRAPHY
1. The New Encyclopaedia Britannica (Ready Reference and Index),
Micropaedia, Vol. 2, Chicago: Encyclopaedia Britannica, Inc.,
1984, p. 743.
2. T. Matsumoto, Chaos in electronic circuits, Proceedings of the
IEEE, 75: 10331057, 1987.
218
CHAOS TIME SERIES ANALYSIS
3. Academic American Encyclopaedia, Vol. 4, Danbury, CT: Grolier,

1986, p. 283.
4. G. Chen and X. Dong, From Chaos to Order: Methodologies, Perspectives, and Applications, Singapore: World Scientific, 1998.
5. G. Chen and J. L. Moiola, An overview of bifurcation, chaos and
nonlinear dynamics in control systems, J. Franklin Inst., 331B:
819858, 1994.
6. G. Chen, Representation, approximation, and identification, in
W. K. Chen (ed.), The Circuits and Filters Handbook, Boca Raton,
FL: CRC Press, 1995, pp. 9731006.
7. S. Wiggins, Introduction to Applied Nonlinear Dynamical Systems
and Chaos, New York: Springer-Verlag, 1990.
8. T. Y. Li and J. A. Yorke, Period three implies chaos, Amer. Math.
Monthly, 82: 481485, 1975.
9. R. L. Devaney, An Introduction to Chaotic Dynamical Systems,
Reading, MA: Addison-Wesley, 1987.
10. J. L. Moiola and G. Chen, Hopf Bifurcation Analysis: A Frequency
Domain Approach, Singapore: World Scientific Pub., 1996.
11. G. Chen and X. Dong, From chaos to orderperspectives and
methodologies in controlling chaotic nonlinear dynamical systems, Int. J. Bifur. Chaos, 3: 13631409, 1993.
12. M. J. Ogorzalek, Taming chaospart II: Control. IEEE Trans.
Circuits Syst. I, Fundam. Theory Appl., 40: 700706, 1993.
13. E. Ott, C. Grebogi, and J. A. Yorke, Controlling chaos, Phys. Rev.
Lett., 64: 11961199, 1990.
14. J. M. Ottino, The Kinematics of Mixing: Stretching, Chaos, and
Transport, New York: Cambridge Univ. Press, 1989.
15. W. J. Freeman, The physiology of perception, Sci. Amer., 7885,
Feb. 1991.
16. S. J. Schiff et al., Controlling chaos in the brain, Nature, 370:
615620, 1994.
17. A. L. Goldberger, Applications of chaos to physiology and medicine, in J. Kim and J. Stringer (eds.), Applied Chaos, New York:
Academic Press, 1992, pp. 321331.
18. A. Garfinkel et al., Controlling cardiac chaos, Science, 257: 1230
1235, 1992.
19. J. N. Weiss et al., Chaos and chaos control in biology, J. Clinical
Invest., 93: 13551360, 1994.
20. T. Shinbrot et al., Using small perturbations to control chaos,
Nature, 363: 411417, 1993.
21. D. Auerbach et al., Controlling chaos in high dimensional systems, Phys. Rev. Lett., 69: 34793482, 1992.
22. E. A. Jackson, On the control of complex dynamic systems, Physica D, 50: 341366, 1991.
23. E. A. Jackson, OPCL migration controls between five attractors
of the Chua system, Int. J. Bifur. Chaos, 5: 12551260, 1995.
24. G. Chen, On some controllability conditions for chaotic dynamics
control, Chaos, Solitions & Fractals, 8: 14611470, 1997.
25. E. H. Abed, H. O. Wang, and R. C. Chen, Stabilization of period
doubling bifurcations and implications for control of chaos. Physica D, 70: 154164, 1994.
26. A. Tesi et al., Harmonic balance analysis of period-doubling bifurcations with implications for control of nonlinear dynamics, Automatica, 32: 12551271, 1996.
27. J. Ford, What is chaos, that we should be mindful of it?, in P.
Davies (ed.), The New Physics, New York: Cambridge Univ. Press,
1989, pp. 348372.
28. W. Yang et al., Preserving chaos: Control strategies to preserve
complex dynamics with potential relevance to biological disorders. Phys. Rev. E., 51: 102110, 1995.
29. G. Chen and D. Lai, Feedback control of Lyapunov exponents for
discrete-time dynamical systems, Int. J. Bifur., Chaos, 6: 1341
1349, 1996.
30. T. S. Parker and L. O. Chua, Practical Numerical Algorithms for

Chaotic Systems, New York: Springer-Verlag, 1989.
GUANRONG CHEN
University of Houston
300
CONVEX OPTIMIZATION
CONVEX OPTIMIZATION
An optimization problem can be stated in the so-called standard form as follows:
minimize
f (xx ) : R R
subject to g (xx ) 0,
g : Rm Rn
(NLP)
representing the minimization of a function f of n variables

under constraints specified by inequalities determined by
functions g [g1, g2, . . ., gm]T. The functions f and gi are, in
general, nonlinear functions. Note that inequalities can be
handled under this paradigm by multiplying each side by 1,
and equalities by representing them as pairs of inequalities.
The maximization of an objective function f(x) can be
achieved by minimizing f(x). The set F xg(x) 0 that
satisfies the constraints on the nonlinear optimization problem is known as the feasible set, or the feasible region. If F
covers all of (a part of) Rn, then the optimization is said to be
unconstrained (constrained).
Note that the above standard-form formulation may not be
directly applicable to real life design problems, where often,
multiple conflicting objectives must be optimized. In such a
case, multicriterion optimization techniques and Pareto optimality can be used to identify noninferior solutions (1). In
practice, however, techniques to map the problem to the form
in Eq. (NLP) are often used.
When the objective function is a convex function and the
constraint set is a convex set (both terms will be formally defined later), the optimization problem is known as a convex
programming problem. This problem has the remarkable
property of unimodality, i.e., any local minimum of the problem is also a global minimum. Therefore, it does not require
special methods to extract the solution out of local minima in
a quest to find the global minimum. While the convex programming problem and its unimodality property have been
known for a long time, it is only recently that efficient algoJ. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering. Copyright # 1999 John Wiley & Sons, Inc.
CONVEX OPTIMIZATION
301
x1
x2
A
x1
x2
Convex set
Nonconvex set
Figure 1. Convex and nonconvex sets.
rithms for the solution of these problems have been proposed.

The genesis of these algorithms can be traced back to the
work of Karmarkar (2) that proposed a polynomial-time interior-point technique for linear programming, a special case of
convex programming. Unlike the simplex method for linear
programming, this technique was found to be naturally extensible from the problem of linear programming to general convex programming formulations. It was shown that this
method belongs to the class of interior penalty methods proposed by Fiacco and McCormick (3) using barrier functions.
The work of Renegar (4) showed that a special version of the
method of centers for linear programming is polynomial. Gonzaga (5) showed similar results for the barrier function associated with a linear programming problem, with a proof of
polynomial-time complexity. Nesterov and Nemiorvsky (6) introduced the concept of self-concordance, studying barrier
methods in their context. Further improvements in the computational complexity using approximate solutions and rankone updates were shown in the work of Vaidya (7). The work
of Ye (8) used a potential function to obtain the same complexity as Renegars work without following the central path
too closely.
Figure 3. A separating hyperplane (line) in two dimensions between

convex sets A and B.
Elementary Convex Sets

Ellipsoids. An ellipsoid E(x, B , r) Rn centered at point
x Rn is given by the equation
{yy|(yy x )T B(yy x ) r2 }
If B is a scalar multiple of the unit matrix, then the
ellipsoid is called a hypersphere. The axes of the ellipsoid are given by the eigenvectors, and their lengths are
related to the corresponding eigenvalues of B .
Hyperplanes. A hyperplane in n dimensions is given by
the region
c Tx = b,
c Rn ,
bR
Half spaces. A half space in n dimensions is defined by the

region that satisfies the inequality
c Tx b,
c Rn ,
bR
Polyhedra. A (convex) polyhedron is defined as an intersection of half spaces, and is given by the equation
P = {xx|Axx b ),
A Rmn ,
b Rm
corresponding to a set of m inequalities aTi x bi, ai

Rn. If the polyhedron has a finite volume, it is referred
to as a polytope. An example of a polytope is shown in
Fig. 2.
DEFINITIONS OF CONVEXITY
Convex Sets
Definition. A set C Rn is said to be a convex set if, for
every x1, x2 C and every real number , 0 1, the
point x1 (1 )x2 C.
This definition can be interpreted geometrically as stating
that a set is convex if, given two points in the set, every point
on the line segment joining the two points is also a member
of the set. Examples of convex and nonconvex sets are shown
in Fig. 1.
Some Elementary Properties of Convex Sets

Property. The intersection of convex sets is a convex set. The
union of convex sets is not necessarily a convex set.
Property (Separating hyperplane theorem). Given two
nonintersecting convex sets A and B, there exists a separating
hyperplane cTx b such that A lies entirely within the half
space cTx b and B lies entirely within the half space cTx
b. This is pictorially illustrated in Fig. 3.
Property (Supporting hyperplane theorem). Given a convex set C and any point x on its boundary, there exists a
supporting hyperplane cTx b such that C lies entirely
within the half space cTx b. This is illustrated in Fig. 4.
x
Figure 2. An example convex polytope in two dimensions as an intersection of five half spaces.
Figure 4. A supporting hyperplane (line) in two dimensions at the

boundary point of a convex set S.
302
CONVEX OPTIMIZATION
x1
x3
x2
x4
x5
c =1 c = 2
Figure 5. An example showing the convex hull of five points.
c=3
x
Definition. The convex hull of m points, x1, x2, . . ., xm

Rn, denoted co(x1, x2, . . ., xm), is defined as the set of points
y Rn such that
X
m
y=
X
m
i x i ,
i 0 i,
i=1
Figure 7. Level sets of f(x,y) x2 y2.
i = 1
i=1
The convex hull is thus the smallest convex set that contains the m points. An example of the convex hull of five
points in the plane is shown by the shaded region in Fig. 5. If
the set of points xi is of finite cardinality (i.e., m is finite),
then the convex hull is a polytope. Hence, a polytope is also
often described as the convex hull of its vertices.
Convex Functions
Definition. A function f defined on a convex set is said to
be a convex function if, for every x1, x2 and every , 0
1,
f [xx 1 + (1 )xx 2 ] f (xx 1 ) + (1 ) f (xx 2 )
f is said to be strictly convex if the above inequality is strict
for 0 1.
Geometrically, a function is convex if the line joining two
points on its graph is always above the graph. Examples of
convex and nonconvex functions on Rn are shown in Fig. 6.
2 f (xx 0 ) 0
Property. A function f(x) is convex over the set S if and only if

xx, x 0 S
where f corresponds to the gradient of f with respect to the

vector x. Strict convexity corresponds to the case where the
inequality is strict.
i.e., its Hessian matrix is positive semidefinite over S. For

strict convexity, 2f(x0) must be positive definite.
Property. If f(x) and g(x) are two convex functions on the
convex set S, then the functions f g and max( f, g) are convex over S.
Definition. The level set of a function f(x) is the set defined
by f(x) c where c is a constant. An example of the level sets
of f(x, y) x2 y2 is shown in Fig. 7. Observe that the level
set for f(x, y) c1 is contained in the level set of f(x, y) c2
for c1 c2.
Property. If f is a convex function in the space S , then the
level set of f is a convex set in S .
Definition. A function g defined on a convex set is said to

be a concave function if the function f g is convex. The
function g is strictly concave if g is strictly convex.
For a fuller treatment of convexity properties, the reader
is referred to Ref. 9.
f (x)
g(x)
x1
Figure 6. Convex and nonconvex functions.
xx 0 S
This is a very useful property, and many convex optimization algorithms depend on the fact that the constraints are
defined by an intersection of level sets of convex functions.
Some Elementary Properties of Convex Functions
f (xx ) f (xx 0 ) + [ f (xx 0 )]T (xx x 0 )
Property. A function f(x) is convex over the convex set S if

and only if
x2
Convex function
x1
x2
Nonconvex function
CONVEX OPTIMIZATION
CONVEX OPTIMIZATION
Convex Programming
Definition. A convex programming problem is an optimization problem that can be stated as follows:
f (xx )
minimize
where f is a convex function and S is a convex set.

Such a problem has the property that any local minimum
of f over S is a global minimum.
Comment. The problem of maximizing a convex function
over a convex set does not have the above property. However,
it can be shown (10) that the maximum value for such a problem lies on the boundary of the convex set.
For a convex programming problem of the type (CP), we
may state without loss of generality that the objective function is linear. To see this, note that the problem (CP) may
equivalently be written as min t:f(x) t, g(x) 0
Linear programming is a special case of nonlinear optimization, and more specifically, a special type of convex programming problem where the objective and constraints are
all linear functions. The problem is stated as
minimize c Tx
subject to
where
Axx b ,
AR
mn
x0
bR ,
m
Barrier Methods. The barrier technique of Fiacco and

McCormick (3) is a general technique to solve any constrained
nonlinear optimization problem by solving a sequence of unconstrained nonlinear optimization problems. This method
may be used for the specific case of minimizing a convex function over a convex set S described by an intersection of convex
inequalities
(CP)
such that x S
(LP)
cR ,
n
xR
The feasible region for a linear program corresponds to a polyhedron in Rn. It can be shown that an optimal solution to a
linear program must necessarily occur at one of the vertices
of the constraining polyhedron. The most commonly used
technique for solution of linear programs, the simplex method
(11), is based on this principle and operates by a systematic
search of the vertices of the polyhedron. The computational
complexity of this method can show exponential behavior for
pathological cases, but for most practical problems it has been
observed to grow linearly with the number of variables and
sublinearly with the number of constraints. Algorithms with
polynomial-time worst-case complexity do exist; these include
Karmarkars method (2) and the ShorKhachiyan ellipsoidal
method (12). The computational times of the latter, however,
are often seen to be impractical.
In the remainder of this section, we will describe various
methods used for convex programming and for mapping problems to convex programs.
303
S = {xx| gi (xx ) 0, i = 1, 2, . . ., m}
where each gi(x) is a convex function. The computation required of the method is dependent on the choice of the barrier
function. In this connection, the logarithmic barrier function
(abbreviated as the log barrier function) for the set of inequalities is defined as
(xx ) =
( Pn
x )]
i=1 log[gi (x
for x S
otherwise
Intuitively, the idea of the barrier is that any iterative gradient-based method that tries to minimize the barrier function
will be forced to remain in the feasible region, due to the singularity at the boundary of the feasible region. It can be
shown that (x) is a convex function over S and its value
approaches infinity as x approaches the boundary of S. Intuitively, it can be seen that (x) becomes smallest when x is,
in some sense, farthest away from all of the boundaries of S.
The value of x at which the function (x) is minimized is
called the analytic center of S.
Example. For a linear programming problem of the type described in Eq. (LP), with constraint inequalities described by
aTi x bi, the barrier function in the feasible region is given
by
X
n
(xx ) =
log(bi a Ti x )
i=1
The value of bi aTi x represents the slack in the ith inequality, i.e., the distance between the point x and the corresponding constraint hyperplane. The log barrier function, therefore,
is a measure of the product of the distances from a point x to
each hyperplane, as shown in Fig. 8(a). The value of (x) is
n
minimized when i1
(bi aTi x) is maximized. Coarsely speaking, this occurs when the distance to each constraint hyperplane is sufficiently large.
As a cautionary note, we add that while the analytic center
is a good estimate of the center in the case where all con-
Path-Following Methods
This class of methods proceeds iteratively by solving a sequence of unconstrained problems that lead to the solution of
the original problem. In each iteration, a technique based on
barrier methods is used to find the optimal solution. If we
denote the optimal solution in the kth iteration as x*k , then
the path x*1 , x*2 , . . . in Rn leads to the optimal solution, and
hence techniques of this class are known as path-following
methods.
Analytic
center
p
(a)
Analytic
center
p
(b)
Figure 8. (a) Physical meaning of the barrier function for the feasible region of a linear program; (b) the effect of redundant constraints
on the analytic center.
304
CONVEX OPTIMIZATION
straints present an equal contribution to the boundary, the

presence of redundant constraints can shift the analytic center. The effect on the analytic center of repeating the constraint p five times is shown in Fig. 8(b).
We will now consider the convex programming problem
specified as
minimize
f (xx )
such that g(xx ) 0,
i = 1, 2, . . ., m
where each gi(x) is a convex function. The traditional barrier

method (3) used to solve this problem formulates the corresponding unconstrained optimization problem
minimize B() = f (xx ) + (xx )
and solves this problem for a sequence of increasing (constant) values of the parameter . When is zero, the problem
reduces to finding the center of the convex set constraining
the problem. As increases, the twin objectives of minimizing f(x) and remaining in the interior of the convex set are
balanced. As , the problem amounts to the minimization of the original objective function f.
In solving this sequence of problems, the outer iteration
consists in increasing the values of the parameter . The inner iteration is used to minimize the value of B() at that
value of , using the result of the previous outer iteration as
an initial guess. The inner iteration can be solved using Newtons method (10). For positive values of , it is easy to see
that B() is a convex function. The notion of a central path
for a linearly constrained optimization problem is shown in
Fig. 9.
Method of Centers. Given a scalar value t f(x*), the
method of centers finds the analytic center of the set of inequalities f(x) t and gi(x) 0 by minimizing the function
log[t f (xx )] + (xx )
where (x) is the log barrier function defined earlier. The optimal value x* associated with solving the optimization problem associated with finding the analytic center for this barrier
function is found, and the value of t is updated to be a convex
combination of t and f(x*) as
t t + (1 ) f (xx ),
>0
Optimum
= 10
=0
(Analytic
center)
= 100
The Self-Concordance Property and Step Length. The value of

above is an adjustable parameter that will affect the number of Newton iterations required to find the optimum value
of the analytic center. Depending on the value of chosen,
the technique is classified as a short-step, medium-step, or
long-step (possibly even with 1) method. For a short-step
method, one Newton iteration is enough, while for longer
steps, further Newton iterations may be necessary.
Nesterov and Nemirovskii (6) introduced the following idea
of the self-concordance condition:
Definition. A convex function : S R is self-concordant
with parameter a 0 (a-self-concordant) on S if is three
times continuously differentiable on S and for all x S and
h Rm, the following inequality holds:
h, h , h ]| 2a1/2{D2 (xx )[h
h, h]}3/2
|D3 (xx )[h
where Dk(x)[h1, h2, . . ., hk] denotes the value of the kth
differential of taken at x along the collection of directions
[h1, h2, . . ., hk].
By formulating logarithmic barrier functions that are selfconcordant, proofs of the polynomial-time complexity of various interior point methods have been shown. An analysis of
the computational complexity in terms of the number of outer
and inner (Newton) iterations is presented in Refs. 13 and 6.
Other Interior-Point Methods
Affine Scaling Methods. For a linear programming problem,
the nonnegativity constraints x 0 are replaced by constraints of the type X1 (x xc) 1, representing an
ellipsoid centered at xc. The linear program is then relaxed to
the following form, whose feasible region is contained in that
of the original linear program:
min{cc Tx : Axx = b , X 1 (xx x c ) < 1}
Note that the linear inequalities in Eq. (LP) have been converted to equalities by the addition of slack variables. This
form has the following closed-form solution:
x ( ) = x
X PAX Xcc
PAX Xcc
where PAX I XAT(AX2AT)1AX. The updated value of x is

used in the next iteration, and so on. The search direction
XPAXXc is called the primal affine scaling direction and corresponds to the scaled projected gradient with respect to the
objective function, with scaling matrix X. Depending on the
value of , the method may be a short-step or a long-step
(with 1) method, and convergence proofs under various
conditions are derived. For details of the references, the
reader is referred to Ref. 13.
We consider a general convex programming problem of the
type (note that the linear objective function form is used here)
{min f (yy ) = b Ty : gi (yy ) 0}
Figure 9. Central path for a linearly constrained convex optimization problem.
The constraint set here is similarly replaced by the ellipsoidal

constraint (y yc)T H (y yc) 2, where yc is the center of
CONVEX OPTIMIZATION
the current ellipsoid, y is the variable over which the minimization is being performed, and H is the Hessian of the logn
barrier function i1 log[gi(y)]. The problem now reduces
to
{min b Ty : (yy y c )T H(yy y c ) 2 }
which has a closed-form solution of the form
H 1b
y ( ) = y
b T H 1b
This is used as the center of the ellipsoid in the next iteration.
The procedure continues iteratively until convergence.
Potential-Based Methods. These methods formulate a potential function that provides a coarse estimate of how far the
solution at the current iterate is from the optimal value. At
each step of a potential reduction method, a direction and
step size are prescribed; however, the potential may be minimized further by the use of a line search (large steps). This is
in contrast with a path-following method that must maintain
proximity to a prescribed central path. An example of a potential-based technique is one that utilizes a weighted sum of the
gap between the value of the primal optimization problem
and its dual, and of the log barrier function value as the potential. For a more detailed description of this and other
potential-based methods, the reader is referred to Refs. 6
and 13.
Localization Approaches
Polytope Reduction. This method begins with a polytope
P Rn that contains the optimal solution, xopt. The polytope P may, for example, be initially selected to be an ndimensional box described by the set
{xx|xmin x(i) xmax }
where xmin and xmax are the minimum and maximum values of
each variable, respectively. In each iteration, the volume of
the polytope is shrunk while keeping xopt within the polytope,
until the polytope becomes sufficiently small. The algorithm
proceeds iteratively as follows.
Step 1. A center xc deep in the interior of the polytope P
is found.
Step 2. The feasibility of the center xc is determined by
verifying whether all of the constraints of the optimization problem are satisfied at xc. If the point xc is infeasible, it is possible to find a separating hyperplane passing through xc that divides P into two parts, such that
the feasible region lies entirely in the part satisfying the
constraint
c Tx
where c [gp(x)] is the negative of the gradient of
a violated constraint gp, and cTxc. The separating
hyperplane above corresponds to the tangent plane to
the violated constraint. If the point xc lies within the
feasible region, then there exists a hyperplane cT x
T
305
that divides the polytope into two parts such that xopt is
contained in one of them, with c [f(x)]T being the
negative of the gradient of the objective function, and
being defined as cT xc once again. This hyperplane
is the supporting hyperplane for the set f(x) f(xc) and
thus eliminates from the polytope a set of points whose
value is larger than the value at the current center. In
either case, a new constraint cT x is added to the
current polytope to give a new polytope that has roughly
half the original volume.
Step 3. Go to step 1 and repeat the process until the polytope is sufficiently small.
Note that the center in step 1 is ideally the center of gravity of the current polytope, since a hyperplane passing
through the center of gravity is guaranteed to reduce the volume of the polytope by a factor of 1 1/e in each iteration.
However, since finding the center of gravity is prohibitively
expensive in terms of computation, the analytic center is an
acceptable approximation.
Example. The algorithm is illustrated by using it to solve
the following problem in two dimensions:
minimize
f (x1 , x2 )
such that
(x1 , x2 ) S
where S is a convex set and f is a convex function. The shaded

region in Fig. 10(a) is the set S, and the dashed lines show
the level curves of f. The point xopt is the solution to this problem. The expected solution region is first bounded by a rectangle with center xc, as shown in Fig. 10(a). The feasibility of
xc is then determined; in this case, it can be seen that xc is
infeasible. Hence, the gradient of the constraint function is
used to construct a hyperplane through xc such that the polytope is divided into two parts of roughly equal volume, one of
which contains the solution xc. This is illustrated in Fig. 10(b),
where the region enclosed in darkened lines corresponds to
the updated polytope. The process is repeated on the new,
f decreasing
xc
xopt
xc
xopt
(b)
(a)
xc
xc
xopt
xopt
(c)
(d)
Figure 10. Polytope reduction approach.
306
CONVEX OPTIMIZATION
Some elementary examples of such functions are:
Ek
1. Clearly, any convex (concave) function is also quasiconvex (quasiconcave).

2. Any monotone function f : R R is quasilinear.
3. The linear fractional function f(x) (aTx b)/(cTx
d) where a, c, x Rn, is quasilinear over the half space
xcTx d 0.
f(xk )
xk
Ek + 1
Figure 11. The ellipsoidal method.
Other Characterizations of Quasiconvexity. A function f defined on a convex set is quasiconvex if, for every x1, x2 :
smaller polytope. Its center lies inside the feasible region, and
hence the gradient of the objective function is used to generate a hyperplane that further shrinks the size of the polytope,
as shown in Fig. 10(c). The result of another iteration is illustrated in Fig. 10(d). The process continues until the polytope
has been shrunk sufficiently.
Ellipsoidal Method. The ellipsoidal method begins with a
sufficiently large ellipsoid that contains the solution to the
convex optimization problem. In each iteration, the size of the
ellipsoid is shrunk, while maintaining the invariant that the
solution lies within the ellipsoid, until the ellipsoid becomes
sufficiently small. The process is illustrated in Fig. 11.
The kth iteration consists of the following steps, starting
from the fact that the center xk of the ellipsoid E k is known:
Step 1. In case the center is not in the feasible region, the
gradient of the violated constraint is evaluated; if it is
feasible, the gradient of the objective function is found.
In either case, we will refer to the computed gradient
as h(xk).
Step 2. A new ellipsoid containing the half ellipsoid given
by
Ek {xx|h(xx k ) x h(xx k ) x k )
T
Ak+1 =
1
A g
n+1 k k
n2
2
T
A g g A
Ak
n2 1
n+1 k k k k
Property. If f,g are quasiconvex over , then the functions

f for 0 and max( f,g) are also quasiconvex over . The
composed function g( f(x)) is quasiconvex provided g is monotone increasing. In general, the function f g is not quasiconvex over .
As in the case of convex optimization, the gradient of a
quasiconvex objective can be used to eliminate a half space
from consideration. The work in Ref. 14 presents an adaptation of the ellipsoidal method to solve quasiconvex optimization problems.
Semidefinite Programming
The problem of semidefinite programming (15) is stated as
follows:
minimize c Tx
subject to F (xx ) 0
where
is computed. This new ellipsoid, E k1, and its center

xk1, are given by the following relations:
x k+1 = x k
1. For every , 0 1, f( x1 (1 )x2)

max[f(x1), f(x2)].
2. If f is differentiable, f(x1) f(x2) (x2 x1)T f(x1) 0.
where
h(xx k )
g k =
h(xx k )T Ak h(xx k )
Step 3. Repeat the iterations in steps 1 and 2 until the
ellipsoid is sufficiently small.
RELATED TECHNIQUES
Quasiconvex Optimization
Definition. A function f : S R, where S is a convex set, is
quasiconvex if every level set La xf(x) a is a convex set.
A function g is quasiconcave if g is quasiconvex over S. A
function is quasilinear if it is both quasiconvex and quasiconcave.
F (xx ) R
mm
(SDP)
,
c, x R
Here, F(x) F0 F1 x1 . . . Fn xn is an affine matrix

function of x, and the constraint F(x) 0 represents the fact
that this matrix function must be positive semidefinite, i.e.,
zT F(x) z 0 for all z Rn. The constraint is referred to as a
linear matrix inequality. The objective function is linear and
hence convex, and the feasible region is convex, since if
F(x) 0 and F(y) 0, then for all 0 1 it can be readily
seen that F(x) (1 ) F(y) 0.
A linear program is a simple case of a semidefinite program. To see this, we can rewrite the constraint set A x b
(note that the here is a componentwise inequality, and
it is not related to positive semidefiniteness) as F(x)
diag(A x b), i.e., F0 diag(b), Fj diag(aj), j 1, . . ., n,
where A [a1 a2 . . . an] Rmn.
Semidefinite programs may also be used to represent nonlinear optimization problems. As an example, consider the
problem
(cc Tx )2
d Tx
subject to Axx + b 0
minimize
where we assume that dT x 0 in the feasible region. Note

that the constraints here represent, as in the case of a linear
program, componentwise inequalities. The problem is first re-
CONVEX OPTIMIZATION
written as minimizing an auxillary variable t subject to the

original set of constraints and a new constraint
(ccTx )2
t
d Tx
The problem may be cast in the form of a semidefinite program as
minimize t
subject to
2
diag(Axx + b )
64
0
0
0
t
c Tx
3
0
7
c Tx 5 0
d Tx
The first constraint row appears in a manner similar to the

linear programming case. The second and third rows use
Schur complements (16) to represent the nonlinear convex
constraint above as the 2 2 matrix inequality
cT x
0
d Tx
c Tx
The two tricks shown here, namely, the reformulation of linear inequations and the use of Schur complements, are often
used to formulate optimization problems as semidefinite programs.
A simple illustration of this technique is in minimizing the

outer surface area of an open box of fixed volume (say, 4
units) and sides of length x1, x2, x3. The problem can be stated
as
minimize x1 x2 + 2x1 x3 + 2x1 x3
subject to x1 x2 x3 = 4
By setting u1 x1x2, u2 2x1x3, u3 2x1x3, and applying the
condition listed above, the minimum value of the objective
function is 3(u1u2u3)1/3 3(4x12x22x32) 12. It is easily verified
that this corresponds to the values x1 2, x2 2, x3 1.
We add a cautionary note that some, but not all, posynomial programming problems may be solved using this simple
solution technique. For further details, the reader is referred
to Ref. 18.
Optimization Involving Logarithmic Concave Functions
A function f is a logarithmic concave (log-concave) function if
log f is a concave function. Log-convex functions are similarly
defined. The maximization of a log-concave function over a
convex set is therefore a unimodal problem, i.e., any local
minimum is a global minimum. Log-concave functional forms
are seen among some common probability distributions on
Rn, for example:
1. The Gaussian or normal distribution
Geometric Programming
Definition. A posynomial is a function h of a positive variable x Rm that has the form
X Y
n
h(xx ) =
307
j)
x (i,
i
i=1
where the exponents (i,j) R and the coefficients j 0, j

R.
f (xx ) =
1
(2 )n det
e0.5(xxc )
T 1 (xx )
c
where 0.
2. The exponential distribution
f (xx ) =
Y
n
!
(i) e((1)x(1)+(2)x(2)++(n)x(n))
i=1
For example, the function f(x,y,z) 7.4x 2.6y3.18z4.2

3 x2y1.4z5 is a posynomial in the variables x, y, and z.
Roughly speaking, a posynomial is a function that is similar
to a polynomial, except that the coefficients j must be positive, and an exponent (i,j) can be any real number, not necessarily a positive integer.
A posynomial has the useful property (17) that it can be
mapped onto a convex function through an elementary variable transformation, x(i) ez(i). Such functional forms are useful because in the case of an optimization problem where the
objective function and the constraints are posynomial, the
problem can easily be mapped onto a convex programming
problem.
For some geometric programming problems, simple techniques based on the use of the arithmeticgeometric inequality may be used to obtain simple closed-form solutions to the
optimization problems (18). The arithmeticgeometric inequality states that if u1, u2, . . ., un 0, then their arithmetic mean is no smaller than their geometric mean, i.e.,
u1 + u2 + + un
(u1 u2 . . . un )1/n
n
with equality occurring if and only if u1 u2 . . . un.
The following properties are true of log-concave functions:

1. If f and g are log-concave, then their product fg is logconcave.
2. If f(x, y) is log-concave, then the integral S f(x, y) dx is
log-concave provided S is a convex set.
3. If f(x) and g(x) are log-concave, then the convolution
S f(x y) g(y) dy is log-concave if S is a convex set
(this follows from properties 1 and 2).
ENGINEERING PROBLEMS AS
CONVEX OPTIMIZATION PROBLEMS
There has been an enormous amount of recent interest in
applying convex optimization to engineering problems, particularly as the optimizers have grown more efficient. The
reader is referred to Boyd and Vandenberghes lecture notes
(19) for a treatment of this subject. In this section, we present
a sampling of engineering problems that can be posed as convex programs to illustrate the power of the technique in
practice.
308
CONVEX OPTIMIZATION
Design Centering
While manufacturing any system, it is inevitable that process
variations will cause design parameters, such as component
values, to waver from their nominal values. As a result, after
manufacture, the system may no longer meet some behavioral
specifications, such as requirements on the delay, gain, and
bandwidth, that it has been designed to satisfy. The procedure of design centering attempts to select the nominal values
of design parameters to ensure that the behavior of the system remains within specifications with the greatest probability and thus maximize the manufacturing yield.
The random variations in the values of the design parameters are modeled by a probability density function (x,xc) :
Rn [0,1], with a mean corresponding to the nominal value
of the design parameters. The yield of the manufacturing process, Y, as a function of the mean xc is given by
Y (xx c ) =
(xx, x c ) dxx
xF
where F corresponds to the feasible region where all design

constraints are satisfied.
A common assumption made by geometrical design centering algorithms for integrated circuit applications is that F
is a convex bounded body. Techniques for approximating
this body by a polytope P have been presented in Ref. 20.
When the probability density functions that represent variations in the design parameters are Gaussian in nature, the
design centering problem can be posed as a convex programming problem. The design centering problem is formulated
as (21)
maximize Y (xxc ) =
h|h
h = h c + F p , p
p 1}
H = {h
where p (pTp)1/2, h Rk1, F R(k1)q.
The robust version of the optimization problem above must
ensure that the error is minimized over all possible values of
h within the ellipsoid. To consider the worst-case tracking error, the optimization problem may be written as
minimize worst-case error = max
max | y(t) yd (t)|
hH t=1,2,..., M
subject to Ulow u(t) Uhigh,
t = 1, 2, . . ., N
|u(t + 1) u(t)| S,
t = 1, 2, . . ., N 1
The value of p at which the worst-case error is maximized

may be derived analytically (23), and the corresponding
worst-case tracking error at that point is
h T Dt u yd (t)|]
max [F T Dt u + |h
t=1,2,..., M
The problem is therefore described as a specific form of convex

programming problem, called a second-order cone programming (SOCP) problem as follows:
minimize
t = 1, 2, . . ., N
S u(t + 1) u(t) S,
t = 1, 2, . . ., N 1
h Dt u ydes (t)]
F Dt u + [h
T
(xx, x c ) dxx
xP
where P is the polytope approximation to the feasible region

F . Since the integral of a log-concave function over a convex
region is also a log-concave function (22), the yield function
Y(x) is log-concave, and the above problem reduces to a problem of maximizing a log-concave function over a convex set.
Hence, this can be transformed into a convex programming
problem.
Robust Optimal Control
Consider a single-input single-output discrete-time linear dynamic system with a finite impulse response described by
y(t) = h0 u(t) + h1 u(t 1) + + hk u(t k)
where u is the input sequence, y is the output sequence, and
ht is the tth impulse response coefficient. Given a desired response yd(t), the problem is to find an finite bounded input
sequence u(t) for which the output y(t) most closely tracks the
desired response, subject to constraints on the slew rate of
the input signal. The problem may be stated as
minimize error =
If the hks are entirely known, then the problem is a linear

program. However, if they are not known exactly, given the
uncertainty in the hks, then it may be possible to say that
their values lie within the ellipsoid
max |y(t) yd (t)|
t=1,2,..., M

|u(t + 1) u(t)| S,
t = 1, 2, . . ., N
t = 1, 2, . . ., N 1
h T Dt u ydes (t)]
F T Dt u [h
Optimizing Structural Dynamics
Consider a linear elastic structure consisting of a stack of k
linear elastic bars connecting a set of p nodes. The topology
and lengths of the bars and their material are fixed, and the
appropriate cross-sectional widths of the bars are to be determined. The elastic stored energy of this system is given by
f Td, where f is the vector of load forces and d is the vector of
(small) node displacements. The relation between f and d is
given by f A(x) d, where A(x), called the stiffness matrix,
k
is an affine sum of the variables xi, given by A(x) i1xi Ai
with the matrices A being all symmetric positive semidefinite.
The optimization problem of minimizing the elastic stored energy (24) can then be stated as follows:
minimize
f Td
k
subject to
ljxj v
j=1
d
f = A(xx )d
x j,min x j x j,max
for j = 1, 2, . . ., k
Here v is the maximum volume, and zj the length of the jth

bar. The last constraint places simple bounds on the values
of the x variables (clearly, all of these variables must be positive, since they correspond to physical lengths). We can then
CONVEX OPTIMIZATION
rewrite the problem by eliminating the d variables by substitution, as follows:
minimize
f T A(xx )1 f
X
k
subject to
ljxj v
j=1
x j,min x j x j,max
for j = 1, 2, . . ., k
Using Schur complements, this leads to the semidefinite programming formulation in x and t given by
minimize t
subject to
t
f
k
X
fT
0
A(x)
309
the dominant eigenvalue of a matrix G1C, where G and C

are, respectively, matrices representing the conductances
(corresponding to the resistances) and the capacitances referred to above. The entries in both G and C are affine functions of the xis. The dominant time constant can be calculated
as the negative inverse of the largest zero of the polynomial
det(sC G). It is also possible to calculate it using the following linear matrix inequality:
T dom = min{T|TG C 0}
The 0 here refers to the fact that the matrix must be positive definite. To ensure that Tdom Tmax for a specified value
of Tmax, the linear matrix inequality TmaxG(x) C(x) 0 must
be satisfied. This sets up the problem in the form of a semidefinite program as follows (28):
X
n
ljxj v
minimize
j=1
l i xi
i=1
x j,min x j x j,max
subject to Tmax G(xx ) C(xx ) 0
for j = 1, 2, . . ., k
x min x x max
VLSI Transistor and Wire Sizing
Convex Optimization Formulation. Circuit delays in integrated circuits often have to be reduced to obtain faster response times. Given the circuit topology, the delay of a combinational circuit can be controlled by varying the sizes of
transistors, giving rise to an optimization problem of finding
the appropriate areadelay tradeoff. The formal statement of
the problem is as follows:
minimize area
subject to
delay Tspec
(TS)
The circuit area is estimated as the sum of transistor sizes,

i.e.,
X
n
area =
xi
i=1
where xi is the size of the ith transistor and n is the number

of transistors in the circuit. This is easily seen to be a posynomial function of the xis. The circuit delay is estimated using the Elmore delay estimate (25), which calculates the delay
as a maximum of path delays. Each path delay is a sum of
resistancecapacitance products. Each resistance term is of
the form a/xi, and each capacitance term is of the type bi
xi, with the constants a and bi being positive. As a result, the
delays are posynomial functions of the xis, and the feasible
region for the optimization problem is an intersection of constraints of the form
(posynomial function in xi s) Tspec
Since the objective and constraints are both posynomial functions in the xis, the problem is equivalent to a convex programming problem. Various solutions to the problem have
been proposed, for instance, in Refs. 26 and 27.
Largest Inscribed Ellipsoid in a Polytope

Consider a polytope in Rn given by P xaiT x bi, i 1,
2, . . ., L into which the largest ellipsoid E , described as
follows, is to be inscribed:
E = {Byy + d |yy 1},
The center of this ellipsoid is d, and its volume is proportional

to det B. The objective here is to find the entries in the matrix
B and the vector d. To ensure that the ellipsoid is contained
within the polytope, it must be ensured that for all y such
that y 1,
a Ti (Byy + d ) bi
Therefore, it must be true that supy1 (aTi By aTi d) bi, or
in other words, B ai bi aTi d. The optimization problem
may now be set up as
maximize
log det B
subject to B = BT > 0
a i bi a Ti d ,
Ba
i = 1, 2, . . ., L
This is a convex optimization problem (6) in the variables B

and d, with a total dimension of n(n 1)/2 variables corresponding to the entries in B and n variables corresponding to
those in d.
Beamforming
Antenna arrays are often used to detect and process signals
arriving from different directions. Each sensor is associated
with a parameter called its weight, typically a complex number, and the values of these weights determine the beam pattern. For a planar array with N elements, the beam pattern
is given by the expression
X
N
Semidefinite Programming Formulation. In the problem (TS)

above, the circuit delay may alternatively be determined from
B = BT > 0
G( )
i=1
wi gi ( ) exp j
2
(x cos + yi sin )
i
310
CONVEX OPTIMIZATION
The problem of optimal beamforming is to choose the

weights in such a way that the pattern level in given areas
i are minimized, subject to upper bound constraints j for
other angles and a maximum level on the weights. The optimization problem is formally written as (29)
minimize |G(i )|
subject to |G(i )| Ui
|wk | W
for j = 1, 2, . . ., M
for k = 1, 2, . . ., N
G(0 ) = 1
BIBLIOGRAPHY
1. W. Stadler, Multicriteria Optimization in Engineering and in the
Sciences, New York: Plenum, 1988.
2. N. Karmarkar, A new polynomial-time algorithm for linear programming, Combinatorica, 4: 373395, 1984.
3. A. V. Fiacco and G. P. McCormick, Nonlinear Programming, New
York: Wiley, 1968.
4. J. Renegar, A polynomial time algorithm, based on Newtons
method, for linear programming, Math. Programming, 40: 59
93, 1988.
5. C. C. Gonzaga, An algorithm for solving linear programming
problems in O(nL) operations, in Progress in Mathematical Programming: Interior Point and Related Methods, N. Meggido (ed.),
New York: Springer-Verlag, 1988, pp. 128.
This may be rewritten as
minimize t
subject to |G(i )| t,
i = 1, 2, . . ., L
|G(i )| U j
|wk | W
for j = 1, 2, . . ., M
for k = 1, 2, . . ., N
G(0 ) = 1
The last constraint here is a normalization constraint and can
be handled by decreasing the number of variables by one. Recalling that each wj is complex, we may choose the vector x
as
6. Y. Nesterov and A. Nemirovskii, Interior-Point Polynomial Algorithms in Convex Programming, SIAM Studies in Applied Mathematics, Philadelphia: Society for Industrial and Applied Mathematics, 1994.
7. P. M. Vaidya, An algorithm for linear programming which requires O(((m n)n2 (m n)1.5n)L) arithmetic operations, Math.
Programming, 47: 175201, 1990.
8. Y. Ye, An O(n3L) potential reduction algorithm for linear programming, Math. Programming, 50: 239258, 1991.
9. R. T. Rockafellar, Convex Analysis, Princeton, NJ: Princeton University Press, 1970.
x = [Re w1 , Im w1 , Re w2 , Im w2 , . . ., Re wN1 ), Im wN1 , f ]
10. D. G. Luenberger, Linear and Nonlinear Programming, Reading,

MA: Addison-Wesley, 1984.
and rewrite the problem in the corresponding convex form:
11. P. E. Gill, W. Murray, and M. H. Wright, Numerical Linear Algebra and Optimization, Reading, MA: Addison-Wesley, 1991.
subject to Ai x + b i < c Ti x + d i ,
i = 1, 2, . . ., L + M + N
Here, the amplitude of G(i) is represented as the norm of a

vector Aix bi where x R2N1, Ai R2(2N1), bi R2. The
two components of Ai are the real and imaginary parts of
G(i). The vectors ci and di are defined as follows:
c i = {[0, 0, . . ., 0, 1]
[0, 0, 0, . . ., 0]
d i = {[0, 0, . . ., 0, 0]
UiL
W
for i = 1, 2, . . ., L
for i = L + 1, . . ., L + M + N
for i = 1, 2, . . . L
for i = L + 1, . . ., M
for i = L + M + 1, . . ., L + M + N
12. A. J. Schrijver, Theory of Linear and Integer Programming, New

York: Wiley, 1986.
13. D. den Hertog, Interior Point Approach to Linear, Quadratic and
Convex Programming, Boston: Kluwer Academic, 1994.
14. J. A. dos Santos Gromicho, Quasiconvex Optimization and Location Theory, Amsterdam: Thesis Publishers, 1995.
15. L. Vandenberghe and S. Boyd, Semidefinite programming, SIAM
Rev., 38 (1): 4995, 1996.
16. G. H. Golub and C. F. Van Loan, Matrix Computations, 2nd ed.,
Baltimore, MD: The Johns Hopkins University Press, 1989.
17. J. G. Ecker, Geometric programming: methods, computations and
applications, SIAM Rev., 22 (3): 338362, 1980.
18. R. J. Duffin and E. L. Peterson, Geometric Programming: Theory
and Application, New York: Wiley, 1967.
Note that the objective here is linear, and the constraints

quadratic.
19. S. Boyd and L. Vandenberghe, Introduction to Convex Optimization with Engineering Applications, Lecture Notes, Electrical Engineering Department, Stanford University, 1995. Available from
http://www-isl.stanford.edu/boyd.
CONCLUSION
20. S. W. Director and G. D. Hachtel, The simplicial approximation

approach to design centering, IEEE Trans. Circuits Syst., CAS24: 363372, 1977.
This overview has presented an outline of convex programming. The use of specialized techniques that exploit the convexity properties of the problem have led to rapid recent advances in efficient solution techniques for convex programs,
which have been outlined here. The applications of convex optimization to real problems of engineering design have been
illustrated. Convex optimization techniques are used widely
in control, for example, in Youla-based design and in design
by linear matrix inequalities (LMIs). For Youla-based design,
the reader is referred to Refs. 30 and 31. A good sourcebook
for design by LMIs is Ref. 32 and a useful practical design
tool is the LMI Control Toolbox (33).
21. S. S. Sapatnekar, P. M. Vaidya, and S. M. Kang, Convexity-based

algorithms for design centering, IEEE Trans. Comput.-Aided Des.
Integr. Circuits Syst., 13: 15361549, 1994.
22. A. Prekopa, Logarithmic concave measures and other topics, in
Stochastic Programming, M. Dempster (ed.), London: Academic
Press, 1980, pp. 6382.
23. S. Boyd, C. Crusius, and A. Hansson, Control applications of nonlinear convex programming, Electrical Engineering Department,
Stanford University, 1997; J. Process Control, in press.
24. A. Ben-Tal and M. P. Bendsoe, A new method for optimal truss
topology design, SIAM J. Optimiz., 3: 322358, 1993.
CONVOLUTION
25. S. S. Sapatnekar and S. M. Kang, Design Automation for TimingDriven Layout Synthesis, Boston: Kluwer Academic, 1993.
26. J. Fishburn and A. E. Dunlop, TILOS: A posynomial programming approach to transistor sizing, in Proc. IEEE Int. Conf.
Comput.-Aided Des., 1985, pp. 326328.
27. S. S. Sapatnekar et al., An exact solution to the transistor sizing
problem using convex optimization, IEEE Trans. Comput.-Aided
Des. Integr. Circuits Syst., 12: 16211632, 1993.
28. L. Vandenberghe, S. Boyd, and A. El Gamal, Optimal wire and
transistor sizing for circuits with non-tree topology, in Proc. IEEE
Int. Conf. Comput.-Aided Des., 1997, pp. 252259.
29. H. Lebret, Optimal beamforming via interior point methods, J.
VLSI Signal Process., 14 (1): 2941, 1996.
30. S. P. Boyd and C. H. Baratt, Linear Controller Design: Limits of
Performance, Englewood Cliffs, NJ: Prentice-Hall, 1991.
31. M. A. Dahleh and I. J. Diaz-Bobillo, Control of Uncertain Systems:
A Linear Programming Approach, Englewood Cliffs, NJ: PrenticeHall, 1995.
32. S. P. Boyd et al., Linear Matrix Inequalities in System and Control
Theory, SIAM Studies in Applied Mathematics, Philadelphia, PA:
SIAM, 1994.
33. P. Gahinet et al., LMI Control Toolbox, Natick, MA: The MathWorks, 1995.
SACHIN S. SAPATNEKAR
University of Minnesota
311
DELAY SYSTEMS
139
DELAY SYSTEMS
In most applications of mathematics to engineering it is
tacitly assumed that the systems under consideration are
causal. That is, the future state of the system depends
only on its present state. In reality most electrical systems,
particularly control systems, are subject to transportation
and/or processing delays. Usually these delays are ignored,
either because they are considered small or because they
complicate the mathematical model. Thus a dilemma arises.
140
DELAY SYSTEMS
When does the realistic modeling of a physical system require the introduction of time delays into the mathematical
model? One purpose of this article is to introduce the
reader to the fundamental properties of time delay differential equations and to compare these to the corresponding
ones for ordinary differential equations. The other is to sum
up the structure and fundamental properties of delay systems of the type most frequently encountered in electrical
engineering.
Ordinary differential equations (ODE) in which a part of
the past history affects the present state are called delay differential equations or functional differential equations (FDE).
Some examples are

x(t)
= ax(t) + bx(t 1) +
0
1
x(t + ) d
d
(x(t) dx(t 1)) = ax(t) + bx(t 1) + c
dt
(1)
0
1
x(t + ) d
(2)
Although special examples of delay differential equations

were investigated as early as the eighteenth century by Euler
and Lagrange, their systematic development did not occur until this century. The initial impetus was the study of certain
mathematical models in mechanics and the physical sciences
which incorporated time delays in their dynamics. One of the
most interesting control models was constructed by N. Minorsky in 1942 (1) in which he incorporated the fact that the
automatic steering mechanism of a ship was subject to a time
delay between a course deviation and the turning angle of the
rudder. Perhaps the two most important contributors to the
initial mathematical development of delay systems were A. D.
Mishkis (2) and N. N. Krasovskii (3). Myshkis gave the first
systematic treatment of the existence and uniqueness problems for delay systems, and Krasovskii not only extended the
second method of Lyapunov for stability to delay systems but
also showed that the correct mathematical setting for linear
time invariant delay differential equations was an infinitedimensional space and not the finite-dimensional space where
the system was defined. This is a crucial observation because
a delay differential equation may be treated both as a process
in a finite-dimensional space and one in an infinite-dimensional space. Some properties of delay systems do not depend
on their infinite-dimensional character. For other properties,
this is prerequisite, and yet other properties, whose computational nature is finite-dimensional, can only be established by
considering the infinite-dimensional system. An example of
the first situation is given by the representation of solutions
of linear time invariant equations. These are obtained, as in
the case of ODE, by using a combination of linear algebra
and complex analysis. An infinite-dimensional property is the
notion of a solution to the initial value problem. A property
which is infinite-dimensional in nature, but sometimes computationally finite-dimensional, is the stability behavior of
the homogeneous time independent systems considered in
this article. The stability of these systems is determined by
the zeros of an entire analytic function, as in the ODE case.
However, the justification for this is based on the infinite-dimensional nature of these systems. This finite-infinite-dimensional duality is of critical importance in studying delay systems. The monograph by R. Bellman and K. Cooke (4)
develops many fundamental stability properties of delay sys-
tems using only the finite-dimensional approach. The monograph by J. K. Hale and S. M. Verduyn-Lunel (5) develops
both the coarse and fine structure of delay systems using the
powerful tools of infinite-dimensional analysis together with
the more mundane methods of linear algebra and complex
analysis.
There are two main categories of FDE considered in the
engineering literature, retarded functional differential equations (RFDE) and neutral functional differential equations
(NFDE). A delay system, written as a first order system, is a
RFDE if the derivative contains no delay terms, Eq. (1) is a
RFDE. If the derivative contains delay terms in a first-order
system, the equation is called a neutral functional differential
equation. Eq. (2) for d 0 is a NFDE. In engineering practice,
only a certain class of NFDE is considered; namely, D-stable
NFDE. A D-stable NFDE is one in which the difference equation associated with the derivative is uniformly exponentially
stable (u.e.s.) Eq. (2) is a D-stable NFDE if d 1 since the
respective difference equation,
y(t) dy(t 1) = 0
(3)
is u.e.s. If d 1, there is an unbounded solution of Eq. (3),

and so the equation is not stable. If d 1, then Eq. (3) is
stable, but not u.e.s. The definition of D-stability is due to M.
A. Cruz and J. K. Hale (6). They showed that the stability
properties of linear time invariant (LTI) RFDE and LTI Dstable NFDE are determined by the exponential solutions,
just as in the case of LTI ODE. D. Henry (7) proved that a
LTI NFDE cannot be uniformly exponentially stable unless it
is D-stable. Until the papers of Cruz, Hale and Henry, engineers routinely gave conditions for the stability of neutral systems without realizing that D-stability was an essential requirement.
PROPERTIES OF DELAY SYSTEMS
The study of delay systems is a natural extension of the theory of ODE, and we shall describe the basic properties of
these systems by comparing them with the analogous properties of ODE. Let Rn be Euclidean n-space.
Initial Conditions and Solutions
The initial condition for the solution of an ODE is given in
the finite dimensional phase space Rn. The initial condition
for the solution of an FDE or delay equation is an infinitedimension space, called a Banach space. The reason for this
is that the initial value of an FDE is a vector-valued function
on Rn defined over an interval [h, 0], h 0 representing the
delay. The point values of the solution evolve in Rn, which
also is called the phase space of the system. This is the dimension duality property of an FDE. The space of initial conditions is infinite-dimensional, and the phase space is finitedimensional.
The notion of the solution of an FDE is weaker than for an
ODE. This is because not all initial functions result in differentiable solutions. However for LTI FDE the Laplace transform provides a convenient alternative definition. The formal
Laplace transform of an LTI FDE does not explicitly contain
the derivative of the FDE, only its initial value. This is also
true for ODE. Thus, if the formal Laplace transform of an
DELAY SYSTEMS
FDE is actually the Laplace transform of a vector-valued

function in Rn, we call this function a solution to the equation.
It is shown in Ref. 7 that this is indeed the case for LTI FDE.
In the case where the FDE is nonlinear or linear with time
dependent coefficients, the definition of a solution is more
complex. In the applications to control problems in electrical
engineering, LTI FDE occur most frequently.
Solutions of nonhomogeneous LTI FDE have a variation of
parameters representation. This is given by the convolution
of an n n matrix-valued function, S(t), and the forcing function. The matrix function, S(t), is the inverse Laplace transform of the matrix version of the homogeneous system where
the initial value is the zero matrix when t 0 and the identity matrix at t 0. This matrix-valued function is the analogue of the fundamental matrix of an LTI ODE and as a
consequence is called the fundamental matrix of the system.
However, it is not a matrix exponential function as it would
be in the ODE case. Time varying nonhomogeneous linear
FDE also have their solutions represented by fundamental
matrices. However their computation is more difficult. One
numerical procedure used to compute these matrices is
known as the method of steps (4). This method works for the
systems given by equations (1) and (2) if c 0 because the
time delays are discrete. The method consists of finding the
fundamental matrix of the system over the interval [0, h],
then using this information to compute the fundamental matrix over the interval [h, 2h], etc. However, in many control
problems, the indeterminate time behavior of the system is
desired, and the method of steps is unsuitable for this
purpose.
Stability
In the applications of delay systems to electrical engineering,
which is practically synonymous with control theory, linear
systems are used almost exclusively, and of these LTI systems predominate. For LTI systems, the Lyapunov stability
theory is superficially the same as in the ODE case. However,
in place of symmetric matrices, one uses symmetric functionals called Liapunov-Krasovskii functionals. The mathematical
structure of these functionals was described by Yu. M. Repin
(8) and N. N. Krasovskii (3). Their practical application to
stability is very limited for two reasons. They are difficult to
construct, even for very simple systems, and once a functional
has been constructed, it is not easy to determine positive or
negative definiteness. Their use in stability theory is usually
on an ad hoc basis. That is, one guesses the form of the
functional, applies it to the system, and hopes it will have the
required positivity or negativity when applied to differentiable trajectories of the system. There is also an offshoot of the
Lyapunov theory, called the method of Razumikhin (9). This
method uses Lyapunov functions in place of functionals to determine stability. Its application requires that the FDE satisfy certain side conditions which are not always met in practice. However, when applicable, the method of Razumikhin is
preferable to the standard Lyapunov method. On the other
hand, unlike the Lyapunov method, the Razumikhin method
does not have converse theorems on stability.
The application of Lyapunov functionals or Razumikhin
functions to LTI FDE is a means to the end of locating the
eigenvalues of the system. The eigenvalues of an LTI FDE
are the zeros of an associated entire analytic function and
141
determine, as in the ODE case, the qualitative behavior of the

system. For LTI RFDE or LTI D-stable NFDE, if these zeros
lie in the open left half of the complex plane, the system is
u.e.s. Determining this condition or its absence is much more
difficult than for LTI ODE, since no simple criterion is available, such as the Routh-Hurwitz criterion.
There is a bounded input-bounded output (BIBO) criteria
for linear homogeneous FDE which applies to both time invariant and time dependent systems, provided the coefficients
of the time dependent system are uniformly bounded on the
real line. This is known as the Perron condition (10). This
condition states that if the nonhomogeneous version of such
a system with zero initial conditions has bounded solutions
for all bounded forcing terms, the system is u.e.s.
Control and Stabilization
There are several versions of the Pontryagin Maximum Principle for delay systems, and the most popular method of solving optimal control problems for delay systems is Bellmans
Method of Dynamic Programming (11,12). However, even for
ODE systems, these theories are rarely used in applications
to electrical engineering. Their main drawback is that they
are nearly impossible to apply to practical, nonlinear delay
systems. Moreover, most electrical engineering control problems are design problems, not optimal control problems. The
two major exceptions to this statement are Linear Quadratic
Regulator (LQR) problems and H-optimization problems.
The solutions of these problems result in linear feedback controls which stabilize a system. However, neither method requires the Pontryagin Maximum Principle or Dynamic Programming.
The main emphasis in engineering control is stabilization,
particularly for linear systems. One popular method of stabilization for single input-single output systems with one time
delay is the use of a Smith Predictor. This is a stabilization
procedure which uses a proportionalintegralderivative
(PID) feedback. This is a standard method and is discussed
in ADAPTIVE CONTROL and CLASSICAL DESIGN METHODS FOR CONTINUOUS TIME SYSTEMS. In this article, we concentrate on the
other two major methods of feedback stabilization for delay
system, LQR- and H-optimization.
The LQR method attempts to minimize an integral called
the cost over the positive real axis. The integrand is quadratic
and positive semidefinite in the space variable and quadratic
and positive definite in the control variable. If this optimization is possible for all initial values of the systems, the optimal control is a feedback control which stabilizes the system.
H-optimization optimizes an LTI FDE control problem with
an unknown disturbance. Here, the cost is positive and quadratic in the space and control variables as in the LQR problem but quadratic and negative definite in the disturbance
variable. For a fixed admissible control function u, one attempts to maximize the cost in terms of the disturbance, then
to minimize the resulting functional with respect to u. This is
called a min-max problem. The optimal solution, if it exists,
leads to a feedback control which is u.e.s. in the so-called
worst case disturbance.
For ODE, the LQR problem is solved using routine numerical packages. The numerical treatment of H-optimization for
ODE is much more difficult than LQR-optimization, but more
robust with respect to uncertainties in the system dynamics.
142
DELAY SYSTEMS
Theoretically, both methods may be used for LTI FDE and

the properties of the resulting feedback controls are known.
However, the practical implementation of either method is a
project for future investigation (13).
In terms of the operator-valued complex matrix S(), the

formal Laplace transform of Eqs. (4) and (5) is

r

x(,
) = S() (0)
D j (h j )
+ S()
ANALYTIC PROPERTIES OF DELAY SYSTEMS

We introduce some additional notation which will be used in
the remainder of this article. Let R (, ), R [0, ),
Z be the complex plane, Zn be complex n-space, I be the ndimensional identity matrix, cT be the transpose of an n-column vector c, c be the complex conjugate of an n-column vector, BT be the transpose of an n m matrix B, c cTc be
the length of an n-vector c, and det(A) be the determinant of
a square matrix A.
The set of all continuous n-vector functions from a closed
interval [h, 0] into Zn is denoted by C(h). If is in C(h), then
sup(t):h t 0.
If x(t) is an n-vector function defined on [h, ), then, for
t 0, xt x(t ):h 0, and xt() x(t ), h
0.
If x(t) is a vector or matrix function on Zn, then
L (x(t))() = x()
r

h j j=1
(A j + D j )e(+hj) ( ) d + S()B
u()
(7)
For any in C(h), the inverse Laplace transform exists

and is called a solution to the initial-value problem in Eqs.
(4) and (5) (14). If C(h) has a derivative, then the inverse
Laplace transform of Eq. (7) is

x(t, , u) = S(t) (0)
+
r

j=1
r

D j (h j )
j=1
0
h j
)) d (8)
S(t h j )(A j ( ) + D j (
S(t )Bu( ) dv
+
0
where
x(t)et dt
S(t) = L 1 (S())(t)
is the Laplace transform of x(t). The inverse Laplace transform of any vector or matrix valued function, provided it exists, is denoted by L 1(S())(t).
The delay systems most commonly encountered in electrical engineering are LTI systems with discrete time delays.
The fundamental properties of these systems serve as a paradigm for most other systems one encounters. These systems
which include RFDE and NFDE often have their dynamics
described by

r

d
x(t)
D j x(t h j )
dt
j=1
= A0 x(t) +
j=1
(9)
The function S(t) formally satisfies Eq. (4) with the initial
matrix S(0) I, S() 0 for 0 and is referred to as the
fundamental matrix solution of Eq. (4).
If the matrices in Eq. (4) are time varying, there is a representation of the solution similar to Eq. (8), but it is not obtained by using the Laplace transform. The matrix S(t) is replaced by a matrix S(t, ), where S(, ) I, and S(t, ) 0 if
t. The matrix function S(t, ) is formally a matrix solution
of Eq. (4) for t (5).
Stability
r

A j x(t h j ) + Bu(t),
t0
(4)
The difference equation associated with the NFDE in Eq. (8)

is
j=1
x(t)
and initial values
r

D j x(t h j ) = 0
(10)
j=1
x(t) = (t), h t 0,
C(h)
(5)
The condition for Eq. (4) to be D-stable is that the equation

In Eq. (4), the matrices Dj, Aj, and A0 are n n-matrices
with real entries; the matrix B is an n m-matrix with real
entries and 0 hj h, 1 j r. The m-vector u(t) is called
the control.
A solution x(t, , u) of Eq. (4), Eq. (5) is a function which
satisfies Eq. (5), is continuously differentiable for t 0, and
has a right hand derivative at t 0 which satisfies Eq. (4).
We can use the Laplace transform to obtain the existence of
and a specific representation for a solution. Let
!
1

r
r

h j
h j
Dje
A0
Aje
S() = I
j=1
j=1
(6)
det I
r

Dje
h j
=0
(11)
j=1
has all of its solutions in Re for some 0. If we seek

solutions of Eq. (10) of the form et for some nonzero complex
n-vector , then must be an eigenvalue of the matrix in Eq.
(11) and must be a corresponding eigenvector. For this reason, we say that satisfying Eq. (11) are eigenvalues of Eq.
(10). We remark that if Eq. (10) is D-stable at one collection
of the delays hj, 1 j r, then it is D-stable for all other
values of the delays (5).
The stability behavior of the homogeneous version of (4)(5), (i.e., when u(t) 0) is completely determined by the
DELAY SYSTEMS
eigenvalues of the system. For the same reason as indicated

for Eq. (10), these are the zeros of the entire analytic function
det I
r

D j eh j
A0
j=1
r

=0
d
[x(t) x(t 1)] = x(t)
dt
(13)
d
d2
[x(t) 2x(t 1) + x(t 2)] + 2 [x(t) x(t 1)] + x(t) = 0
dt 2
dt
(14)
The eigenvalues of both systems are solutions of the equation
(1 e) 1 0 which has all its solutions in Re 0.
Equation (13) has all its solutions tending to zero as t tends
to infinity, but it is not u.e.s. Equation (14) has solutions
which tend to infinity as t tends to infinity.
The stability or instability of LTI FDE can be determined
by LyapunovKrasovskii functions and sometimes by Razumikhin functions. However as was mentioned above, these
are difficult to find for all but the simplest systems and even
then are usually selected on an ad hoc basis. To illustrate
this, consider the scalar system
a > 0, a > |b| > 0, r > 0
(15)
For a constant c to be determined, choose the Lyapunov

Krasovskii functional
1
V () = ((0))2 + c
2
(( )) d
2
Thus, by Theorem 4.2, Chapter 5, in Ref. 5, the system is

u.e.s. for all r 0. In this case, the Razumikhin method
yielded in a much more straightforward way the same result
as above using the LyapunovKrasovskii functional.
There is yet another way to determine u.e.s. of LTI FDE.
This is to treat the delay terms as parameters in a family of
LTI FDE, which reduce to an ODE when all the delays are
zero. If the ODE is u.e.s., one tries to estimate the size of the
delays that the family can tolerate and yet remain u.e.s. This
is possible since for LTI RFDE or LTI D-stable NFDE, the
maximal exponential rate of expansion or contraction of a system depends continuously on the delay parameters (16). This
condition is an easy consequence of the Hurwitz Theorem in
Complex Analysis (17) since the maximal rate is determined
by an eigenvalue for which the real part can be chosen to be
continuous in the delay parameters. To illustrate this method,
consider Eq. (15). When r 0, the system is u.e.s. We try to
find the smallest positive value r for which the system has a
nontrivial periodic solution of period 2/w. This value, if it
exists, satisfies the equation
iw + a + b(cos i sin ) = 0,
= wr, w > 0
Since a b, for fixed w, this equation has no solution for real

. Thus, we conclude that the system is u.e.s. for all values
of r.
There is an important class of nonlinear FDE whose stability is determined by frequency domain methods. A typical example of such a system is one whose dynamics are described
by the equations
x(t)
= A0 x(t) + A1 x(t h) + b f ( ),
= cT x
(17)
where b is an n-vector, and f is a scalar function satisfying

the sector condition
a1 2 f ( ) a2 2 ,
0
r
dV
(x(t)) (a |b|)(x(t)) 0
dt
(12)
j=1
If the system is D-stable, then it is u.e.s. if and only if all

solutions of Eq. (12) satisfy Re 0 (5).
As mentioned above, before the papers of Cruz and Hale
(6) and Henry (7), engineers often assumed a system was
u.e.s. if solutions of Eq. (12) satisfied Re 0. If the D-stability condition is not satisfied, showing that the solutions of Eq.
(12) satisfy Re 0 may be insufficient to determine stability
or instability as the following two examples show (15). Consider the scalar systems
x(t)
= ax(t) + bx(t r),
If we consider only those solutions x(t) that satisfy the relation x(t) x(t r), then

A j eh j
143
0 < a1 < a2 < a
(18)
(16)
Along differentiable trajectories of Eq. (15)
The following theorem holds (18)

Theorem. Assume that the system
y(t)
= A0 y(t) + A1 x(t h)
(19)
K(iw) = cT (iwI A0 A1 eiwh )1 b
(20)
dV
(xt ) = (a + c)(x(t)2 + bx(t r)x(t) c(x(t r))2
dt
is u.e.s. and let
This functional will be negative on C(r) if we choose c a/2.
Therefore, by Theorem 2.1, Chapter 5, in Ref. 5, the region of
u.e.s. contains the set of coefficients a, b with a b. Notice
that this simple choice for the LyapunovKrasovskii functional yielded the stability region which is completely independent of the size of the delay.
Now, consider the Razumikhin function for Eq. (15) given
by V(x(t)) x2 /2. Along differentiable trajectories of
Eq. (15),
dV
(x(t)) = x(t)(ax(t) + bx(t r))
dt
If there exists a q 0 such that for all w in R,

Re(1 + iwq)K(iw)
1
0
a
(21)
then each solution of Eq. (17) tends to zero as t tends to infinity.

The above theorem is also true for D-stable systems of the
types in Eqs. (4) and (5) where bf() replaces Bu. This theo-
144
DELAY SYSTEMS
rem is one of a class of theorems which are known collectively

as Popov-type theorems. An interesting corollary to the above
theorem is that condition [Eq. (21)] guarantees the existence
of a Lyapunov functional for the system [Eq. (17)] which has
a particular structure.
Corollary. If Eq. (20) is satisfied, then there exists a Lyapunov functional on C(h) of the type
V (, ) = Q(, ) +
f (s) ds,
>0
(22)
such that Q is bilinear on C(h), Q(, ) 0 if 0 and,

along differentiable trajectories of Eq. (17), dV(xt, xt)/dt 0.
The proof the above corollary is a simple extension of the
same result for ODE given on p. 169 of Ref. 10. The converse
is also true; that is, if the corollary is satisfied, then so is the
theorem. The Lyapunov approach is in general not feasible,
whereas the frequency domain or Popov approach is easily
checked, especially by modern computing packages.
As was mentioned above, there is a necessary and sufficient condition for determining the u.e.s. of time varying
FDE. This is the Perron condition or bounded-input, boundedoutput criterion. We give an analytic form of this condition
for the system whose dynamic is described by the equation
x(t)
= A0 (t)x(t) + A1 (t)x(t h) + f (t)
(23)
but remark that the basic condition holds for any linear FDE
whose coefficient matrices are uniformly bounded.
We assume that the square matrices A0(t) and A1(t) have
all their entries uniformly bounded on R. It is known (5) that
the solutions of Eq. (23) with initial conditions zero in C(h)
may be represented in the form

x(t, t0 , f ) =
S(t, ) f ( ) d
(24)
t0
where S(t, ) 0 if t, S(, ) I and, for t ,

d
(S(t, )) = A0 S(t, ) + A1 S(t h, )
dt
(25)
Theorem. A necessary and sufficient condition for the homogeneous version of Eq. (23) to be u.e.s. is that for all f which
are uniformly bounded on R, the vector function in Eq. (24)
satisfies an inequality of the form x(t, t0, f) Mf, where Mf
is finite and depends only on f (10).
Although the Perron condition or BIBO condition may be
theoretically difficult to verify, a modified form often is used
in control engineering. The linear system is subjected to periodic forcing functions at a variety of frequencies and with uniformly bounded gains. If the outputs are uniformly bounded
over long time periods, the system is considered u.e.s.
Control and Stabilization
Controllability for an FDE system is function space control;
that is, one seeks to control a given initial point in C(h) to a
given terminal point in a finite time. There is also the notion

of -controllability, that is, control from a given point to an ball of another given point. This latter form of control is more
realistic for FDE systems, but in practice, neither form is
much used in engineering design. A simple example may indicate the reason. Consider the scalar system
x = ax(t) + bx(t h) + u(t)
(26)
where h 0 and b 0. For C(h) and 0, suppose

that one desires to find a u(t) with u(t) 1 which drives the
solution of Eq. (26) with initial value to the zero function in
some finite time T. The Laplace transform of the resulting
motion is given by
1
x(,
, u) =
a beh

(0) +
be
h
( +h)
( ) d + u()
(27)
Since both xt(, u) and u(t) are identically zero after time T,
the functions x(, , u) and u() in Eq. (27) must be entire
analytic functions (19). This means that u() must be chosen
so that the numerator in Eq. (27) is zero when the denominator is zero. But the zeros of a beh are infinite in number and can at best be approximated.
There are several versions of the Pontryagin Maximum
Principle for FDE control problems, and the theoretical
method used to solve control problems is the Method of Dynamic Programming. From an engineering point of view,
these are only of academic interest. Comprehensive reference
sources for this area of control are Refs. 12 and 20.
For multiple-input-multiple-output LTI ODE, there are
three basic methods of feedback stabilization. These are pole
placement, linear quadratic regulator (LQR)-optimization,
and H-optimization pole placement. The latter has the simplest numerical structure but has less eclat than the other
two methods. Pole placement methods are possible for LTI
FDE. In practice the best one can hope for are constant gain
feedbacks which guarantee a given decay rate. However,
LQR-optimization and H-optimization have in theory been
completely extended from LTI ODE to LTI FDE. There are at
least two ways to look at these extensions. One way relates
to the specific delay structure and minimizes the use of Banach space theory. The other embeds LTI delay systems into
a general class of LTI infinite-dimensional systems known as
Pritchard-Salamon systems (P-S systems) and makes extensive use of the theory of Sobelev spaces (21). Here, we confine
ourselves to the first approach.
A typical LQR-optimization problem is the following. For a
given positive definite matrix W and a given C(h), choose
the control function u(t) to minimize the functional
F (u) =
[xT (t)Wx(t) + uT (t)u(t)] dt
(28)
subject to the constraint that x(t) is the solution with initial

value of the equation
x(t)
= A0 x(t) + A1 x(t h) + Bu(t)
(29)
If the minimum for all in C(h) is finite, then there is a

bounded linear mapping K from C(h) into Zn such the optimal
DELAY SYSTEMS
u is given by a feedback u(t) BTKxt. Moreover, if q(t) Kxt,

then
q(t)
= Wx(t) AT0 q(t) AT1 q(t + h)
in (0, ) (21). The optimal solution is a linear feedback

control, K, which maps C(h) into Zn. The optimal u and d
satisfy
(30)
u(t) = BT Kxt , d(t) =
The eigenvalues of the feedback system

x(t)
= A0 x(t) + A1 x(t h) BBT Kxt
(31)

BBT
=0
I + AT0 + AT1 eh
q(t)
= Wx(t) AT0 q(t) AT1 q(t + h)
(32)

xT (t)Wx(t) + uT (t)u(t)
(33)

1 T
d
(t)d(t)
dt
2
(34)
subject to the constraint

(35)
where A0, A1, B, and W are the matrices in Eqs. (28) and
(29), and L in an n r-matrix. It is assumed that the
systems
x(t)
= A0 x(t) + A1 x(t h) + Bu(t)
(36)
x(t)
= A0 x(t) + A1 x(t h) + Ld(t)
(37)
and
are stabilizable. The object is to find, if possible, the

min max F (u, d)
u
(38)
and to show that it is nonegative for all initial values in

C(h) of the solution of Eq. (35). In this problem, the constant
plays a critical role. There exist for which there is either
no optimum or for which the optimum is negative for at least
one in C(h). However, there is a smallest 0 0 for which
Eq. (38) has a nonegative solution for all in C(h) and all
(41)
has all solution converging to the zero vector as t tends to

infinity. The system [Eq. (35)] with feedbacks [Eq. (39)] is
u.e.s.
There are many variants of the above problem. For example, one could attempt to optimize the functional
C(u, d) =
0

1
xT (t h)x(t h) + uT (t)u(t) 2 d T (t) d(t) dt
The basic structure of the optimizable problem is the same.

The critical problem is to find efficient numerical methods.
Since this problem has not been adequately solved for LQRoptimization, it will be much more difficult for H-optimization.
Fine Structure
The solutions of the homogeneous version of Eqs. (4) and (5)
generate a C0-semigroup on C(h). The domain of the infinitesimal generator of this semigroup are those points in C(h)
which are continuously differential and satisfy the
condition
(0)
x(t)
= A0 x(t) + A1 x(t h) + Ld(t) + Bu(t)
(40)
If 0 (dT()d())d , then the system

x(t)
= A0 x(t) + A1 x(t h) + BBT Kxt + Ld(t)
An example of H-optimization is to find the min-max of

the following system. Consider the functional
F (u, d) =
(39)
There are variants of the optimization problem in Eq. (28).

One is to optimize Eq. (28) where W is only positive semidefinite. This condition requires some additional assumptions
on the system [Eq. (29)], which are technical but do not alter
the basic structure of the solution. The fundamental assumption needed to solve the LQR-problem is that system [Eq.
(29)] can be stabilizable. This appears to be putting the cart
before the horse, and in some sense, it is. For example, an
LQR problem for the ODE system x Ax Bu, where A is
an n n matrix, is solvable if rank [B, AB, , An1B] n.
There is no such simple condition for LTI FDE. For instance,
the n-dimensional system [Eq. (29)] for B b an n-vector is
stabilizable if, for Re 0,
rank[I A0 A1 eh , b] = n
1 T
L Kxt
2
If q(t) Kxt, then
are the solutions with Re 0 of the equation

I A0 A1 eh
det
W
145
r

j=1
D j (h
j ) A0 (0)
r

A j (h j ) = 0
(42)
j=1
If all Dj 0, Eq. (42) is an RFDE. If the solutions of Eq. (11)

lie in Re 0 for some 0, then the system is a Dstable NFDE. For RFDE, the spectra of the semigroup at t
1 is the origin plus eigenvalues e where is an eigenvalue
which is a solution of the characteristic equation [Eq. (12)]
with all Dj 0 (5). For D-stable NFDE, the spectra of the
semigroup at t 1 has the essential spectrum with moduli
1 plus eigenvalues e, where is an eigenvalue which is a
solution of the characteristic equation [Eq. (11)] (5). An LTI
RFDE has only a finite number of its eigenvalues in any right
half plane, whereas a D-stable LTI NFDE has a vertical strip
in Re 0 which contains an infinite number of its eigenvalues. The solutions of LTI RFDE become more differentiable
as time increases. They pick up one derivative for each interval of length h. For example, even if in C(h) is not continuously differentiable over [h, 0], the solution xt() will be ktimes differentiable for t kh. LTI NFDE retain the smoothness of their initial conditions. For these reasons, LTI RFDE
have been compared to LTI parabolic partial differential
equations (PDE), and LTI D-stable NFDE have been compared to LTI hyperbolic PDE. However, the fine structures of
146
DELAY SYSTEMS
LTI RFDE and LTI parabolic PDE are dissimilar. For example, parabolic PDE generate analytic semigroups, and RFDE
generate C0-semigroups which are compact for t h but are
not analytic. One basic reason for this difference is that the
eigenvalues of LTI RFDE do not belong to a sector in the complex plane. On the other hand, some LTI hyperbolic PDE and
LTI D-stable NFDE have identical mathematical structures.
In particular, the equations for the dynamics of transmission
lines described by the telegraph equation can be transformed
to D-stable NFDE (5). An example which illustrates the similarity between some hyperbolic PDE and D-stable NFDE, are
the PDE system
wtt = wxx 2awt a2 w = 0,
w(0, t) = 0,
0 < x < 1, t > 0
wx (1, t) = Kwt (1, t)
(43)
(44)
where a 0 and K 0 are constant, and the D-stable

NFDE

1 K 2a
a
d
x(t)
e x(t 2) =
[x(t) + e2a x(t 2)]
dt
1+K
1+k
(45)
These systems have the same spectrum (22).
Small Solutions
Linear homogeneous FDE may have small solutions; that is,
nontrivial solutions which decay faster than any exponential
function. A characterization of the set of small solutions for
D-stable LTI NFDE and LTI RFDE is contained in Ref. 5. A
remarkable property of any D-stable LTI NFDE and LTI
RFDE is that there is a 0 such that any small solution is
identically zero in C(h) after time . An example of a system
with small solutions is
x = y(t 1),
y = x(t)
(46)
All solutions of Eq. (46) whose initial functions satisfy x(0)

0 and y(t) 0 for 1 t 0 are small solutions which vanish
in C(1) for t 1 [(5), p. 74].
Linear periodic systems also have small solutions, but
these are not necessarily zero after a finite time (5). An example is the system
x(t)
=
1
2
+ sin 2t x(t 1)
[see e.g. (5) p. 250].

Stability
As mentioned above, N. N. Krasovskii extended the Second
Method of Lyapunov to FDE. An example of this extension is
the following theorem (5).
Theorem. Let f be a continuously differentiable mapping
from C(h) into Rn with f(0) 0 and consider the RFDE
x(t)
= f (xt )
(47)
Let V be a continuous mapping from C(h) into R which satisfies the following two conditions:
(i) There is a nonnegative continuous function a(r) with

a(r) as r such that, for all in C(h),
a(|(0)|) V ()
(48)
(ii) There is a nonnegative continuous function b(r) such

that
lim sup
h0
1
[V (xh )() V ()] b(|(0)|)
h
(49)
Then the solution xt 0 is stable and every solution of Eq.

(47) is bounded. If b(r) is positive definite every solution of
Eq. (47) tends to zero as t tends to infinity.
Similar results exist concerning stability and instability
for autonomous and nonautonomous FDE. If the function V is
continuously differentiable, then the computation of the left
side of relation [Eq. (49)] may be performed on solutions with
smooth initial functions. The corresponding differential inequality gives estimates on these smooth solutions. Since the
initial data of these smooth solutions are dense in the space
C(h), one obtains estimates on all solutions. In this sense,
there is no essential difference in the method than for ODE.
A complete description for the Lyapunov method is given in
Ref. 5, Chapter 5.
The stability of linear homogeneous periodic RFDE and
linear homogeneous periodic D-stable NFDE can be determined by examining their solutions after any integer multiple
of their period which is larger than the delay. This results in
a bounded linear mapping, U, from C(h) into itself. The eigenvalues of U, called the characteristic multipliers of the system, determine the stability behavior of the system. If all of
the multipliers lie inside the unit circle in the complex plane,
then the system is u.e.s. If some are outside the unit circle,
the system is unstable. If the multipliers lie inside or on the
unit circle, the geometric multiplicity of those on the unit circle determines stability or instability of the system.
Feedback Stabilization
LTI FDE of the type given in Eqs. (4) and (5) are particular
examples of PritchardSalamon control systems. Their acronym is PS system (21). They are the largest class of
infinite-dimensional control systems to which the theory of
finite-dimensional LTI control theory can be most easily
extended. The most diverse class of PS systems are those
described by LTI FDE whose spaces of initial conditions
are Hilbert spaces and whose solutions evolve in their space
of initial conditions.
LQR- and H-stabilization are in theory completely developed for PS systems, but in a very abstract setting (21). In
the case of FDE, this setting requires the use of the infinitesimal generator of the associated semigroup, which is an unbounded linear operator. This operator is not as easy to manipulate as the Laplace transform of the solution of the
system and is the main reason why LQR- and H-stabilization
is theoretically possible but computationally difficult for these
systems. To illustrate the difficulty, consider the following
LQR problem for the finite-dimensional system
x(t)
= Ax(t) + Bu(t)
(50)
DELAY SYSTEMS
where the function to be optimized is given by Eq. (28). It is

known that the optimal solutions have controls u(t)
BTKx(t), where K is a unique positive definite n n matrix
with the property that, for any n-vector, x0 in Zn, the analytic
vector-valued function
I A
W
BBT
I + AT
1
x0
Kx0

(51)
in Z2n has no poles in the right half complex plane. Since the
poles of the matrix function in Eq. (51) are symmetric with
respect to the imaginary axis, the matrix K is uniquely determined once the solutions of
det
I A
W
BBT
I + AT
=0
(52)
147
ation of solutions, continuous dependence on data, and parameters, etc. for these systems are similar to the corresponding ones for delay systems in Rn. The major exception to this
statement occurs for properties which depend on the compactness of closed bounded sets in Rn or Zn. These systems are
often encountered in models in population ecology, genetic repression, control theory, climatology, coupled oscillators, age
dependent populations, etc. (29,30).
TIME DELAYS IN CONTROL SYSTEMS
Time delays are sometimes desired in the design of control
systems. For example in self-tuning control, one encounters
systems of the form
y(t) + a1 y(t 1)a2 y(t 2) + ana y(t na)
= b1 w(t 1) + b2 u(t 2) + + bub u(t nb)
are known. This method of finding the feedback is known as

spectral factorization. If the matrices A, B, and W in Eqs. (52)
and (50) are replaced by linear operators , , and W , where
is unbounded, a PS LQR problem will symbolically be represented by an expression of the form in Eq. (51), and a spectral factorization exists for such a system (21). However, how
does one in practice carry it out? This is the crux of the computational difficulties for LQR- and H-optimization in
PS systems. On the other hand, the LQR-optimization described by Eqs. (28) and (29) has the eigenvalues of the feedback system given by the solutions in Re 0 of Eq. (32). The
methods used to obtain this result were system specific (23);
that is, they depended on the explicit structure of the delay
system and not its abstract representation, and in this instance, yielded more information. The same is true of the frequency domain criterion [Eq. (21)] used in the Popov problem
described by Eq. (17). This problem has a PS setting (24).
However, in this setting, one has to unravel the simple Popov
criterion. Another instance of this is the Perron condition.
This condition exists for the evolution of the solutions of the
system in Eq. (23) in a Banach space setting, but in practice,
one examines the output of a system in Rn when the forcing
function is an n-vector not the output of an infinite-dimensional vector.
where the u-terms are the controls. These are known as

DARMA (deterministic autoregressive and moving average)
systems (31). They have their own methodology which is described in ADAPTIVE CONTROL in this encyclopedia. Our interest
here is in systems where unpredicted delays appear in the
controls, particularly feedback stabilized controls.
If the control system is finite-dimensional of the type
x(t)
= Ax(t) + Bu(t)
and is stabilized by a feedback of the form
u(t) = Rx(t)
wtt = wxx ,
ut duxx =
+ u(x, t 1)(1u (x, t))
(53)
(25), which is a nonlinear diffusion equation with a time delay. Extensions of time delay systems to PDE and abstract
Banach spaces may be found in Refs. 2629.
Time independent versions of these systems are often of
the form
x(t)
= Ax(t) + f (xt )
0 < x < 1, t > 0

wx (1, t) = u(t)
(57)
(58)
If, in Eq. (58), the control is feedback and given by
Delays may appear in PDE as well as ODE. For example,
(56)
then a small time delay in the control will not destroy the
stabilization. Of course, the word small depends on the particular system. However, if system [Eq. (55)] is infinite-dimensional, it may be unable to tolerate any delays, particularly if
it is an abstract representation of a boundary stabilized hyperbolic PDE. The simplest example of such a system is given
by
w(0, t) = 0,
EXTENSIONS TO INFINITE DIMENSIONAL PHASE SPACES
(55)
(54)
where A generates a C0-semigroup in a Banach space, X, and

f is a continuous mapping from the Banach C :[h, 0]
X is continuous. The proofs of existence, uniqueness, continu-
u(t) = wt (1, t)
(59)
then all of the solutions of the resulting feedback system are

identically zero after time t 2. However, if
u(t) = wt (1, t h),
h>0
(60)
then the system is unstableso much so that the following

result holds (32).
Theorem. Given any 0 there exists hn 0 as n
and n in Z, Ren such that the system [Eqs. (57) and
(58)] with the feedback
u(t) = wt (1, t hn )
(61)
148
DELAY SYSTEMS
has solutions
w(x, t) = e n t sinh nt
(62)
Systems of the type [Eqs. (57) and (58)] are often

approximated by finite-dimensional oscillatory systems of the
form
x + Ax = bu
(63)
where A is a positive definite n n matrix, with eigenvalues

0 12 n2, and b is an n-vector, which is not an eigenvector of A. Suppose the system [Eq. (63)] is stabilized by a
feedback
u(t) = cT1 x(t) + cT2 x(t)
(64)
This could be accomplished by pole placement, LQR-optimization, or H-optimization.

Theorem. The maximum time delay which the system
x(t)
+ Ax(t) = (cT1 x(t h) + cT2 x(t
h)) b
(65)
can tolerate and remain u.e.s. is in the interval

0<h<
2
n
(66)
This result implies that large-dimensional Galerkin approximations to systems of the type in Eqs. (57) and (58) become unstable for small time delays. Thus, there is a tradeoff
between the dimension of the approximation and the tolerance for delays.
The above example illustrates one of the important differences between LTI evolutionary systems whose initial space
is infinite dimensional and those for which this space is finite
dimensional. It is instructive to make some general remarks
about why such a situation might occur. Suppose that is a
parameter varying in a subset S of a Banach space, and
T(t), t 0 is a C0-semigroup of linear transformations on a
Banach space X which is continuous in ; that is, T(t)x is
continuous in (, t, x). Let r be the radius of the spectrum of
T(1). The asymptotic behavior of the semigroup is determined
by the spectrum (T(1)) of T(1). If (T(1)) is inside the unit
circle in the complex plane, then each orbit T(t), t 0 will
approach zero exponentially and uniformly. If there is a point
in (T(1)) outside the unit circle, then there is an unbounded
orbit, and we have instability. A fundamental role in the
study of stability and the preservation of stability under perturbations in the parameter is the behavior of the essential
spectrum e(T(1)) of T(1). Let re r(e(T(1))) denote the radius of the essential spectrum of T(1). If it is known that
re 1, then the stability or instability of 0 is determined by
eigenvalues of (T(1)). Furthermore, if 0 is unstable, then it
is due to only a finite number of eigenvalues; that is, the instability occurs in a finite dimensional subspace of X. If the
latter situation occurs, then it is natural to expect that the
stabilization of the system could be accomplished by using a
finite dimensional control. However, if it is required that the
stabilization be insensitive to small changes in the parame-
ter, then this may not be the case. If re 1, then the instability of the system is of such a nature that it cannot be controlled by a finite dimensional control.
In general, eigenvalues of T(1) can be chosen to be continuous functions of . On the other hand, the function re may
not be continuous in . The function re will be continuous in
at a point 0 if it is known that T(1) T0(1) is compact.
This condition is not necessary, but it is sufficient. If this difference is only bounded, then there is the possibility of a large
shift in re if we vary . For example, if re0 1, and the perturbation is only bounded, it is possible to have re 1 for a
sequence of j 0, and the semigroup Tj(t) will be unstable
for each j.
Let us interpret these remarks in terms of the examples
that we have been discussing above. For finite dimensional
problems, the semigroup is compact and thus the asymptotic
behavior of orbits is determined by the eigenvalues. Since the
semigroup for a LTI RFDE is compact for t h, the continuous spectrum always is the point zero, and so the asymptotic
behavior is again determined by dominant eigenvalues (finite
in number), and these are generally continuous in parameters.
For D-stable NFDE, the essential spectrum lies in the unit
circle for all values of the delay. Therefore, the introduction
of small delays in the control function does not disturb the
stabilization property of the feedback control.
It can happen that the solutions of the difference equation associated with the difference operator D for an NFDE
has all solutions approaching zero exponentially and uniformly for a particular value of the delays, and a small
change in the delays leads to exponential instability. If this
is the case, then the asymptotic behavior of solutions of
NFDE subjected to small variations in the delays is not
determined by the eigenvalues of the semigroup, but by the
essential spectrum, which in turn is determined by the
eigenvalues of the difference equations associated to the
difference operator D.
In the example [Eqs. (57) and (59)], the natural space of
initial data is HB1 (0, 1) L2(0, 1), where B represents the homogeneous boundary conditions w 0 at x 0, wx 0 at
x 1. In this case, the boundary control wt is bounded but
not compact. If this control is implemented with a delay, then
the radius of the essential spectrum is increased considerably
and, in fact, leads to instability.
It is possible to consider a more physical version of Eqs.
(5759), for which the boundary control problem is insensitive
to small delays in the time at which it is implemented. Consider the equation
wtt wxx cwxxt = 0,
0 < x < 1, t > 0
(67)
with the boundary conditions

w(0, t) = 0,
wx (1, t) + cwxt (1, t) = kwt (1, t h)
(68)
where h 0, c 0, k 0 are constants. In any space for

which one can define a C0-semigroup for Eqs. (67) and (68),
the control function is compact. Furthermore, the radius of
the essential spectrum is determined by the same problem
with k 0 and is given by e(1/c) 1. Therefore, stability is
preserved with small perturbations in the delay.
For further discussion of this topic, see Refs. 3237.
DEMODULATORS
BIBLIOGRAPHY
1. N. Minorsky, Self-excited oscillations in dynamical systems possessing retarded action, J. Appl. Mech., 9: 6571, 1942.
2. A. D. Mishkis, General theory of differential equations with a
retarded argument, Amer. Math. Soc. Trans., No. 55, 1951.
3. N. N. Krasovskii, Stability of Motion, Moscow, 1959, Translation:
Palo Alto: Stanford Univ. Press, 1963.
149
27. G. F. Webb, Functional differential equations and nonlinear

semigroups in Lp-spaces, J. Differ. Equ., 20: 7189, 1976.
28. R. Datko, Representation of solutions and stability of linear differential-difference equations in a Banach space, J. Differ. Equ.,
29: 105166, 1978.
29. J. Wu, Theory and applications of partial functional differential
equations, Appl. Math. Sci. 119: 50102, 1996.
4. R. Bellman and K. Cooke, Differential Difference Equations, New

York: Academic, 1963.
30. G. F. Webb, Theory of Nonlinear Age-Dependent Populations, New

York: Dekker, 1985.
strom and B. Wittenmark, Adaptive Control, Reading, MA:
31. J. F. A
5. J. K. Hale and S. Verduyn-Lunel, Introduction to functional differential equations, Appl. Math. Sci., Vol. 99, New York: SpringerVerlag, 1993.
32. R. Datko, Two examples of ill-posedness with respect to time delays revisited, IEEE Trans. Autom. Control, 42: 511, 515, 1997.
6. M. A. Cruz and J. K. Hale, Asymptotic behavior of neutral functional differential equations, Arch. Ration. Mech. Anal., 34: 331
353, 1969.
7. D. Henry, Linear autonomous neutral functional differential
equations, J. Differ. Equ., 15: 106128, 1974.
8. Yu. M. Repin, Quadratic Lyapunov functionals for systems with
delay (in Russian), Prikl. Mat. Mekh., 29: 564566, 1965.
9. B. S. Razumikhin, Applications of Lyapunovs method to problems in the stability of systems with a delay (in Russian), Autom.
Telemekh., 21: 740749, 1960.
10. A. Halanay, Differential Equations, New York: Academic, 1966.
11. E. A. Andreeva, V. B. Kolmanovskii, and P. E. Shaikhet, Control
of Systems with After Effect, Moscow: Nauka, 1992.
12. X. Li and J. Yong, Optimal Control Theory for Infinite Dimensional
Systems, Boston: Birkhauser, 1995.
13. C. Foias, A. Tannenbaum, and G. Zames, Weighted sensitivity
minimization for delay systems, IEEE Trans. Autom. Control, 31:
763766, 1986.
Addison-Wesley, 1989.
33. J. K. Hale, Effects of delays on dynamics, in A. Granas and J.

Frigon (eds.), Topological Methods in Differential Equations and
Inclusions, Dordrecht, The Netherlands: Kluwer, 1995, pp.
191238.
34. K. B. Hannsgen, Y. Renardy, and R. L. Wheeler, Effectiveness
and robustness with respect to time delays of boundary feedback
stabilization in one-dimensional viscoelasticity, J. Control Optim.,
26: 12001234, 1988.
35. W. Desch and R. L. Wheeler, Destabilization due to delay in one
dimensional feedback, Int. Ser. Numer. Math., 91: 6183, 1989.
36. H. Logemann, R. Rebarber, and G. Weiss, Conditions for robustness and nonrobustness of the stability of feedback systems
with respect to small delays in the feedback loop, J. Control Optim., 34: 572600, 1996.
37. J. K. Hale, Effects of delays on stability and control, CDSNS Tech.
Rep., Atlanta: Georgia Institute of Technology, 1997.
R. DATKO
Georgetown University
14. J. K. Hale and K. R. Meyer, A class of functional differential

equations of neutral type, Mem. Amer. Math. Soc., 76: 1976.
J. K. HALE
Georgia Institute of Technology
15. R. Datko, An example of an unstable neutral differential equation, Int. J. Control, 38: 263267, 1983.
16. K. Cooke and J. Ferreira, Stability conditions for linear retarded
functional differential equations, J. Math. Anal. Appl., 96: 480
504, 1983.
17. S. Saks and A. Zygmund, Analytic Functions, Warsaw: Polish Scientific Publishers, 1971.
18. V. M. Popov and A. Halanay, On the stability of nonlinear control
systems with time-lag (in Russian), Autom. Telemekh., 23: 31
38, 1962.
19. G. Doetsch, Handbuch der Laplace-Transformation, Birkhauser:
Basel, 1956, Vol. 3.
20. R. Gabasov and F. M. Kirillova, The Maximum Principle in the
Theory of Optimal Control (in Russian), Mrnsk: Minsk Science
and Technology, 1974.
21. B. von Keulen, H-Control for Distributed Parameter Systems: A
State-Space Approach, Birkhauser: Basel, 1993.
22. R. Datko, Two questions concerning the boundary control of certain elastic systems, J. Differ. Equ., 92: 2744, 1991.
23. R. Datko, The point spectrum of some LQR problems. Appl. Math.
Optim., 31: 8599, 1995.
24. J. Louis and D. Wexler, The Hilbert space regulator problem and
operator Riccati equation under stabilization, Ann. Soci. Sci.
Brux., 105: 137165, 1991.
25. K. Yoshida, The Hopf bifurcation and its stability for semilinear
diffusion equation with time delay arising in ecology, Hiroshima
Math. J., 12: 321348, 1982.
26. W. E. Fitzgibbon, Semilinear functional differential equations in
Banach space, J. Differ. Equ., 29: 114, 1978.
DELTA-SIGMA MODULATORS. See SIGMA-DELTA MODULATION.
DIGITAL CONTROL
445
DIGITAL CONTROL
The revolutionary advances in computer technology today
have made it possible to replace conventional controllers with
digital computers. Digital control thus refers to the control
scheme in which the controller is a digital device, generally a
digital computer. This means that we can make use of a much
more advanced control logic and versatility than those made
possible with conventional analog controllers. On the other
hand, we also need an interface that connects a computer
with real plants. In particular,
Measurement is made at discrete instants in time
Data must be spatially discretized to allow digital data
handling
In other words, digital controllers can handle data that are
discretized both in time and space. The former discretization
is usually referred to as sampling and the latter quantization.
These two features place digital control systems outside the
scope of the usual linear, time-invariant control systems.
(There is also the problem of saturation effect when controllers have a fixed word length. But this problem is much less
studied in the context of digital control.)
To see the situation more concretely, consider the unityfeedback digital control system shown in Fig. 1. Here r is the
reference signal, y the system output, and e the error signal.
These are continuous-time signals. The error e(t) goes
through the sampler (or an A/D converter) S . This sampler
r+
ed
e
S
ud
C(z)
y
H
C(z)
Figure 1. A unity-feedback digital control system consisting of a continuous-time plant P(s), discrete-time controller C(z), sampler S and
a hold device H .
446
DIGITAL CONTROL
Rf
;
;
; ;
u[t]
2R
4R
8R
Figure 2. Quantization converts the slanted straight line (thin) to

the piecewise step zigzag function (thick).
Figure 4. A D/A converter is constructed with an operational amplifier, switching, and resistors.
reads out the values of e(t) at every time step h called the
sampling period, and produces a discrete-time signal ed[k],
k 0, 1, 2, . . .. In this process, a quantization error (due to
round-off) as shown in Fig. 2 occurs. The sampling operator
S acts on a continuous-time signal w(t), t 0 as
In the process above, a quantization error occurs in the

A/D conversion. This is a round-off error that occurs when
we convert analog values to digital data (often with a fixed
wordlength), as shown in Fig. 2. This introduces a nonlinearity into the system although other system components may
be linear. A possible effect is that the closed-loop system may
exhibit typical nonlinear behavior, such as limit cycles. Such
phenomena are, however, much less studied compared to the
effect arising from data sampling in time, and one usually
assumes that sufficient spatial resolution is guaranteed so
that the effect of quantization is negligible.
The term digital control is thus used almost synonymously
with sampled-data control (that is, the control scheme where
measurement and control actions occur intermittently with a
fixed period) and quantization effects are ignored. Usually
one considers single-rate sampled-data control systems where
sampling and hold actions occur periodically with a fixed period in a synchronized way. In practice, however, there are
varied situations in which different sampling rates are employed at distributed control stations. Such a situation leads
to multirate sampled-data control systems. However, for the
sake of simplicity this article deas with single-rate systems.
S (w)[k] := w(kh),
k = 0, 1, 2, . . .
(The quantization effect is omitted here.) The discretized signal is then processed by a discrete-time controller C(z) and
becomes a control input ud. This signal then goes through another interface H called a hold device or a D/A converter to
become a continuous-time signal. A typical example is the
zero-order hold, where H simply keeps the value of a discretetime signal w[k] as a constant until the next sampling time:
(H (w[k]))(t) := w[k],
for kh t < (k + 1) h
A typical sample-hold action [with C(z) identity] is shown

in Fig. 3. A simple D/A converter can be constructed with
operational amplifiers, resistors, and switching devices as depicted in Fig. 4. Because this construction requires high precision in resistors, more elaborate circuitry is adopted in
practice.
There are other types of hold devices, for example, a firstorder hold for various reasons. In this article, however, we
confine ourselves to the zero-order hold above.
z-TRANSFORM
We start with a fundamental description of systems and se
denote a sequence with values in some
quences. Let w[k]k0
vector space X. Typically, X is the n-dimensional vector space
n, but we will later encounter an example where X is not
finite-dimensional. The z-transform of w w[k]k0

is defined
to be the formal sum (mathematically, this is called a formal
power series):
Z [w](z) :=
w[k]zk
k=0
Figure 3. A simple sample-hold combination maps a continuoustime signal to a piecewise step function.
(z). The negative

with indeterminate z. It is also denoted as w
powers of z is in accord with the usual convention. Here z is
just a formal variable, and z-transform at this stage simply
gives a convenient way of coding sequences via the correspondence t } zt.
DIGITAL CONTROL
It can be readily verified that the z-transform Z [w u] of

the discrete convolution
(w u)[k] :=
k
w[k j]u[ j]
j=0
is given by the product Z [w]Z [u] of the z-transforms of the

two sequences, i.e.,
y(t) = Cx(t)
As a special case, the multiplication by z1 yields the time
shift (delay): w[k]k0

w[k 1]k1
. Similarly, the multipli
cation by z yields the time-advance operator: w[k]k0
w[k 1]k0.
The z-transformation plays the role of the Laplace transformation in the continuous-time case. As with Laplace transforms, it is useful to consider the substitution of a complex
number to the variable z. For example, the geometric se
quence kvk0
has the z-transform
k zk v =
k=0
zv
z

x[k + 1] = x((k + 1)h) = e
Ah
0
h
= e Ah x[k] +
0
e A( ) Bd ud [k] =: Ad x[k] + Bd u[k]

(5)
In other words, the behavior of P(s) at the sampling instants

can be described by a time-invariant discrete-time system
(Ad, Bd, Cd). This is the formula due to Kalman and Bertram
(1). Its transfer function
Pd (z) := C(zI e Ah )1
There is a way to compute Z [x](z) from its Laplace transform

(see Theorem below). Note also that the z-transform of an exponential function et is z/(z eh):
(2)
Let us now give a system description. Suppose that a discrete-time system

x[k + 1] = Ax[k] + Bu[k]
y[k] = Cx[k] + Du[k]
e A(h ) Bud [k] d
y[k] = y(kh) = Cx[k] =: Cd x[k]
x(kh)zk
z
z eh
x[k] +
k=0
Z [et ](z) =
(4)
The first objective is to give a description of this plant at sampling instants t kh, k 0, 1, 2, . . .. By the zero-order hold,
the input to the plant for kh t (k 1)h is the constant
ud[k]. Suppose that the state of the plant at t kh is x[k]
x(kh). Then by integrating Eq. (4) from kh to (k 1)h, we
obtain

(1)
We can consider this as a function with complex variable z.

The sequence kv tends to zero if and only if 1; this is
equivalent to its z-transform being analytic in z : z 1.
For a fixed sampling period h, the z-transform of a continuous-time signal x(t) is understood to be the z-transform of its
sampled sequence:
Z [x](z) :=
ues on or outside the unit circle z : z 1. This is equivalent

to the transfer function being analytic in z : z 1, provided
that there are no hidden poles of (zI A)1 cancelled by the
numerator.
Let us now give a sample-point description of the continuous-time plant P(s). Let (A, B, C) be its (minimal) realization.
For brevity, we assume that the direct feedthrough term of
P(s) is zero. This means
x(t)
= Ax(t) + Bu(t)
Z [w u] = Z [w]Z [u]
447
(3)
is given. Taking z-transforms of sequences x[k], u[k],

y[k] and using the fact that the multiplication by z induces
the time-advance operator, we see that
zx = Ax + Bu
y = Cy + Du
Solving this, we have
y = C(zI A)1 x0 + [D + C(zI A)1 B]u

where x0 is the initial state at time 0. The second term D
C(zI A)1B is the transfer function of this system. It is
(asymptotically) stable if and only if zI A has no eigenval-
eA Bd
0
is called the pulse transfer function of P(s). Then the composite

transfer function of the closed-loop system, when confined to
sampled instants, is given by Pd(z)C(z)(I Pd(z)C(z))1. We
then have the following result.
Theorem 1. The behavior of the sampled-data system in Fig.
1 at sampled instants can be described by a time-invariant,
discrete-time equation. To be more precise, let (A0, B0, C0, D0)
and (A, B, C) be the minimal realizations of C(z) and P(s),
respectively. Then the behavior of the closed-loop system at
sampled instants can be represented by system matrices

!

A0
B0
B0Cd
,
, [0 Cd ]
BdC0 Ad Bd D0Cd
Bd DD0
Example 1. Let P(s) 1/s2. This has a realization

0 1
0
dx
=
x+
u
dt
0 0
1
y = [1
0] x
Equation (5) is then computed easily as

1 h
h2 /2
x[k + 1] =
x[k] +
ud [k]
0 1
h
y[k] = [1
0] x[k]
448
DIGITAL CONTROL
Much of the classical theory for sampled-data control is

devoted to the computation of the pulse transfer function from
a given continuous-time transfer function. Note that this precedes the advent of the modern state space theory, and elaborate formulas in the z and Laplace transform domains have
been found. For example, the following theorem is well
known (2,3):
x(t), its modified z-transform Z [x](z, m), 0 m 1 is defined

as
Theorem 2. Let P(s) be a rational function such that it is

analytic for s : s R for some R 0 and sP(s) as s
for some real . Then
Theorem 4. Assuming the same conditions as in Theorem 2,

the following formulas for the modified z-transform holds:
1
P(s)
ds
2 j 1 ehs z1

P(s)
Res
=
1 ehs z1
poles of P(s)
Z [x](z, m) :=
x(kh + mh)zk
(9)
k=0
As in Theorem 2, the following result holds.

P(s)emhs
1
ds
2 j C 1 emhs z1

P(s)emhs
Res
=
1 emhs z1
poles of P(s)
Z [P](z, m) =
Z [P](z) =
(6)
where denotes a contour that travels from c j to c

j [c: abscissa of convergence of P(s); the coordinate to the
right of which the Laplace integral defining P(s) converges]
and goes around a semcircle on the left half-plane that encircles all poles of P(s).
The modified z-transform has a close connection with lifted

transfer functions in the modern approach (see the section
entitled Modern Approach).
CLASSICAL DESIGN METHODS AND THEIR LIMITATIONS
Once the sample-point behavior is specified by the procedure above, it is easy to give a description of the intersample
behavior of the output or the state. Suppose, for example, that
the state of the plant P(s) takes values x[k] at t kh, k 0,
1, 2, . . .. By integrating Eq. (4) from t kh to t kh
(0 h), we get

x(kh + ) = e
x[k] +
e
0
A( )
Bud [k] d
(7)
and
y(kh + ) = Cx(kh + )
(8)
This shows that if x[k] and ud[k] tend to zero as k ,

then the intersampling behavior x(kh ), 0 h also
tends to zero uniformly for as k . This is because
the right-hand side of Eq. (7) is just the multiplication and
convolution of known continuous functions with x[k] and
ud[k] over a finite interval. Therefore, the stability of a sampled-data system can be determined solely by its sample-point
behavior. We summarize this observation in the form of a
theorem:
Theorem 3. The closed-loop system in Fig. 1 is stable if the
discrete-time closed-loop system consisting of C(z) and Pd(z) is
stable. Therefore, to stabilize the plant P(s) in the sampleddata setting Fig. 1, it is necessary and sufficient that Pd(z) be
stabilized by C(z).
This result gives a foundation for the classical treatment
of sampled-data systems. To design (or at least to stabilize) a
sampled-data system, one can equivalently stabilize the pulse
transfer function Pd(z) derived from P(s). This led to the classical design procedure based on pulse transfer functions.
Equations (7) and (8) are closely related to the notion of
the modified z-transform. For a continuous-time function
We briefly review the classical design methods and their limitations. The first is a design method based on the continuoustime design.
Continuous-Time Design
A simple, straightforward method is to employ a continuoustime design, obtain a continuous-time controller, and then
convert the controller to a discrete-time system via some kind
of discretization. Let Cc(s) be a continuous-time controller.
Typical discretization methods are the following:
Use the Tustin (bilinear) transformation:
C(z) = Cc
2
z1
h z+1
Employ the backward difference (z 1)/hz for approximating the differential operator s.
Approximate Cc(s) by using the sample/hold equivalent
Eq. (5).
Among these, the first method is most commonly used.
It is well known that the Tustin transformation preserves
stability: if Cc(s) is a stable transfer function (in the sense of
continuous-time systems), then the transformed function
Cc(2(z 1)/h(z 1)) gives a stable (in the sense of discretetime systems) discrete-time transfer function. Although this
is a great advantage in signal processing, care must be exercised in control system design, because this property does not
guarantee the closed-loop stability. In fact, as h becomes
larger, there is even a case in which the closed-loop stability
is violated (see the example in the section entitled H Design). This is because the original continuous-time design
does not usually take account of the sampling period. To take
care of this, one has to pay more attention to various robustness properties, such as gain and phase margins, and so
DIGITAL CONTROL
on. To discuss such properties, frequency domain considerations are highly desirable.
However, the notion of frequency response is not readily
available. To see the situation, let C(z) be a discrete-time
transfer function. Suppose that a sinusoid ejt is applied after
sampling. Then the actual input to C(z) is ekjhk0

with ztransform z/(z ejh) given by Eq. (2). The steady-state response of C(z) against this input is then given by
. It appears that we can discuss the frequency

ekjhC(ejh)k0
domain properties via C(ejh). For example, one might attempt
to employ phase lead/lag compensation based on this quantity. However, due to sampling, this frequency response does
not fully represent the nature of continuous-time inputs. For
example, not only ejt but also ej(2n/h)t, n 1, 2, . . . give
. The response is
exactly the same sampled values ekjhk0
then governed by the same C(ejh). This means that sampling
does not have enough resolution to distinguish all these sinusoids, and the notion of phase, which depends on the response
against sinusoids, is unclear in such a sampled-data control
setup. Another way of seeing this is to note that ejh ejh
and hence C(ej(2/h)h) C(ejh). This means that beyond the
frequency /h, the same gain characteristic repeats periodically and C(ejh) cannot be treated as the same frequency response concept as in the continuous-time case. This is related
to the notion of aliasing, which we examine in more detail in
the section Modern Approach.
It may be still possible to execute an elaborate continuoustime design that also works well in the sampled-data setting
by looking more closely into the nature of the Tustin transformation. However, in such a method, a systematic design
method such as H design theory is difficult to apply. Furthermore, one needs a more concrete understanding of the phenomena above, and this is much better done in the modern
approach treated in the subsequent sections.
Discrete-Time Design
Yet another classical approach is based on the pulse transfer
function Pd(z). As far as stability is concerned, one can deal
only with Pd(z). It was also recognized that sampled-data control can achieve performance that is not possible with linear,
time-invariant, continuous-time controller. For example, the
so-called deadbeat control achieves the property that the output (or state) settles exactly to zero after a finite time period.
This is done by placing all poles of the closed-loop system to
zero; the output, at least at sampled instants, then becomes
zero after a finite timea performance not possible with continuous-time controllers.
It should be, however, noted that such a classical treatment also shares the weakness of the classical transfer function approach. Namely, it did not take account of hidden
polezero configurations. In particular, it was observed that
merely settling the output might sometimes induce very large
intersample ripples.
It was Kalman and Bertram (1) who introduced the state
space approach for sampled-data systems. As we have already
seen, the sample-time inputoutput relation is described by

x[k + 1] = Ad x[k] + Bd ud [k] = e Ah x[k] +
449
Let us consider the stabilization by state feedback. If we employ a sampled state feedback
ud [k] = Kx[k]
then by Theorem 3 the stability of the closed-loop system is
determined by the spectrum of
A d Bd K
(10)
Thus this is a purely discrete-time pole-placement problem.

Furthermore, if we can set the eigenvalues of Eq. (10) all to
zero, then x[k] becomes zero after a finitely number of steps
if there is no external input. Also with Eqs. (7) and (8), not
only x[k] but also the intersampling trajectory will settle to
zero. This clearly shows the advantage of the state space theory, which was introduced around that time.
The problem is thus reduced to the pole allocation for the
discrete-time system (Ad, Bd) and the feasibility of this is reduced to the problem of determining the controllability and
stabilizability of this pair. Naturally, we may as well assume
that the continuous-time plant (A, B, C) is stabilizable or controllable. Otherwise, it is not possible to stabilize the plant
even with continuous-time controllers.
For brevity, let us consider controllability. The following
result is well known (4):
Theorem 5. Let 1, . . ., n be eigenvalues of A. Suppose that
for no pair i, j (i j), i j is an integer multiple of 2/h.
Then (Ad, Bd) is controllable if and only if (A, B) is controllable.
The proof is an easy consequence of the fact that the eigenvalues of Ad eAh are e1, . . ., en. This is a variant of the spectral mapping theorem, but we skip the details here (4,5).
By the discussions above, it may appear that sampled-data
control systems can be safely designed via discrete-time design methods. Note that, at least for the deadbeat control via
state space, we can also settle the intersample behavior identically to zero after a finite number of steps. However, this is
valid only for regulation problems, and the issue of the intersample behavior for tracking (servo control) problems, where
exogenous signals are present, is quite different. To see this,
consider the example depicted in Fig. 5. Here the continuoustime plant is 1/(s2 1) whose natural frequency is 1 rad/s.
On the other hand, the tracking signal is sin (1 2/0.1)t
where the sampling period h is 0.1 s. It so happens that, at
sampling instants t kh, k 0, 1, 2, . . ., this signal is identical with sin t because (2/0.1)kh 2k. Therefore, for the
discrete-time controller the tracking signal is no different
sin(1 + 20 )t
r+
ed
e
S
e2h1
z e2h
ud
H
s2
1
+1
h
0
e A Bd ud [k]
Figure 5. A unity feedback system with tracking reference signal

sin(1 20)t.
450
DIGITAL CONTROL
strict sense, if we take the intersample behavior into account.

This is closely related to the issue of the notion of aliasing
effects and Shannons sampling theorem. We briefly review
these in the next section.
0.5
SAMPLING THEOREM
0
0.5
1
43
43.2
43.4
43.6
43.8
44
Time (sec)
Figure 6. The simulation of Figure 5 shows that the input sin(1
20)t does not yield a sinusoid at the same frequency, and large intersample ripples result.
from sin t. The simulation result is shown in Fig. 6. The plant

output is shown by the solid curve while the reference input
is shown by the dashed curve. The output tracks sin t rather
than sin(1 2/0.1)t, and there is a large amount of intersample ripples due to the difference between sin(1 2/0.1)t
and sin t.
This example shows the following:
There can be large intersample ripples for sampled-data
systems.
Such ripples are difficult to characterize via the discretetime framework as described above.
The ripples do not appear to be stationary.
The observations above indicate that the discrete-time model
Eq. (5) is generally not appropriate for describing sampleddata systems when there are nontrivial intersample ripples.
What is indicated here is that we need a framework that can
give a description for the continuous-time behavior of a sampled-data system.
Suppose that we wish to describe a frequency response. Let
sin t be an input applied to the sampled-data system shown
in Fig. 1. For linear, time-invariant, stable continuous-time
systems, it is well known that a single sinusoid yields another
sinusoid in the steady-state output, with exactly the same frequency, possibly with gain and phase shifts. To be more precise, let G(s) be the transfer function of such a system. It is
well known that the steady-state output is
G( j) sin t
That is, as t , the output asymptotically approaches
G( j) sin t.
Such a separation principle does not hold for sampled-data
systems. In fact, the example shown in Fig. 6 shows a counterexample: the steady-state output against sin(1 2/0.1)t
is sin tsinusoid, but with a different frequency.
One of the reasons for such a phenomenon is that sampleddata systems are no longer time-invariant systems in a very
Let f(t) be a given continuous-time signal on (, ). To

make the sampling well defined, we assume that f is a contin
uous function. The sampled sequence is f(kh)k
. As it is,
this is just a sequence defined on the set of integers. The
question here is how we should represent this sequence in the
continuous-time domain.
Recall that the z-transform of f(kh)k

is
f (kh)zk
k=
We have extended the definition in a natural way to the negative ks. We also recall that the multiplication by z1 is the
right shift operator. Since the kth signal value f(kh) is placed
at t kh, this right shift corresponds to the right shift by
time length h in the continuous-time domain. It is also well
known that in the Laplace transform domain the right shift
operator by h is represented by the multiplication by ehs.
Therefore, it is natural to represent the Laplace transform of
the sequence f(kh)k

by
f (kh)e khs
k=
The inverse Laplace transform of this is the train of impulses

(Delta functions) multiplied by f(kh) at the kth step:
f (kh)(t kh)
(11)
k=
Observe that this is formally a multiplication of f(t) with the

train of impulses
(t kh)
(12)
k=
and thus it is called the impulse modulation of f(t).

The question that concerns us here is the following: Sup
pose that we are given a sampled sequence f(kh)k
, or
khs
k f(kh)e
just as well. How much can we recover the
original signal f(t) out of this piece of data?
If we impose no condition on f(t), then the solution is
clearly nonunique. There is infinite freedom in the intersampling periods while passing through f(kh), k . A typical
solution is obtained by assuming that f(t) is band-limited;
that is, its Fourier transform is zero outside a bounded interval. This is the content of the following Shannon sampling
theorem.
Theorem 6. Let f be a continuous function that is Fourier
transformable. Suppose that its Fourier transform is identi-
DIGITAL CONTROL
cally zero outside the interval (/h , /h ) for some

0. Then
f (t) =
f (nh)
n=
sin (t/h n)
(t/h n)
(t nh)
n=
It follows that
f (nh)(t nh)
2
h

n=

=F
f (t)
n=
1
f()
2
1
f()
h
2n
h
n=
1
2n
f
h n=
h
In other words, we get infinitely many copies of the shifted

image of f() as shown in Fig. 7. This is because a sinusoid
sin t behaves precisely the same as sin( 2m/h)t at sampled points t nh, n 0, 1, 2, . . .. Such higher frequency
signals that arise from sampling are called alias components.
It is clearly not possible to recover the original signal f(t) from
such data contaminated by aliasing. In particular, there is in
general an overlapping of f( 2n/h). (The period s :
2/h of these spectra is called the sampling frequency and its
half /h the Nyquist frequency.)
However, it is possible to recover f(t) if such an overlapping
does not occur. Indeed, it will be clear from Fig. 7 that if the
original spectrum f is zero outside the interval (/h, /h),
then there is no overlapping among those copies. The bandlimited hypothesis that f is zero outside (/h , /h )
guarantees this. To eliminate all unnecessary alias components, multiply the function

1 (|| /h)
() :=
(15)
0 (|| > /h)
f( )
f( 2 /h)
/h

2n/h)
f(
n=
f (nh)(t nh)
n=
f (nh)sinc(t nh)
n=
sinc t =
2n
h
where sinc t : hF []. This function is easily computed as
2n
2
t
h n=
h
= h F []
()
(14)
(t nh)

1
2n
=
f t
h n=
h
f (t) = F [f()]
=F
n=
to n f( 2n/h). Then only the spectrum f() in the

fundamental frequency range (/h, /h) remains. Applying
the inverse Fourier transform, we obtain
(13)
We now briefly indicate the outline of a proof.

As noted above, Eq. (11) is obtained by multiplying f(t) to
the train of impulses [Eq. (12)]. Hence its Fourier transform
is just the convolution of the respective Fourier transforms
(6). For the Fourier transform of Eq. (12), the following Poisson summation formula is well known (6):
451
/h
Figure 7. The spectrum of f repeats periodically with period 2/h.
sin t/h
t/h
This readily implies Eq. (13).

This result also clarifes the meaning of Theorem 5. When
there is a pair of eigenvalues that differ only by an integer
multiple of 2/h, the corresponding two modes cannot be distinguished because they yield the same eigenvalues when discretized.
Some remarks are in order. Although Eq. (13) certainly
gives a well-defined reconstruction formula, it is crucially
based on the assumption that the original signal f(t) is band
limited. This assumption, which appears quite innocent, is
seldom satisfied in practice. In fact, if f is band limited, it
must be an entire function; that is, it is analytic on the whole
complex plane. We can hardly expect real signals to be analytic functions. Therefore, the assumption for Eq. (13) can be
satisfied only in an approximate sense. The second drawback
is that Eq. (13) is not causal. In other words, it makes use of
future sampled values f(nh) to reconstruct the current value
f(t). It is therefore not physically realizable. To remedy this,
one should be content with approximation, and a large portion of digital signal processing is devoted to the various solutions of this problem.
Theorem 6 also yields the following observations:
By sampling, intersampling information is generally lost.
In particular, sinusoids sin( 2n/h)t, n 0, 1, 2,
. . . cannot be mutually distinguished.
However, this is about the maximum uncertainty introduced by sampling. After sampling, all the components
that arise in the output are combinations of all such alias
components sin( 2n/h)tn

.
The last statement still needs to be clarified. The basic
idea is the following: When a sinusoid sin t is sampled, it is
converted to a modulated train of impulses as shown in
Eq. (12). In other words, infinitely many alias components
sin( 2n/h)t are excited by sampling. To avoid an undesirable effect arising from such aliased components, it is generally necessary to place an analog low-pass filter (usually
called an anti-aliasing filter) in front of the sampler. Since
this cannot cancel the alias components completely, how
much such alias components affect the overall performance is
a concern. Such a question has been studied in the literature
(2,3,7). However, its general structure is better understood in
452
DIGITAL CONTROL
the scope of the modern approach, which we describe in the

subsequent sections.
t kh. As in Eq. (5), the state x[k 1] at time (k 1)h is

given by
MODERN APPROACH
x[k + 1] = e
Ah
e A(h ) Bu[k]( ) d
x[k] +
(17)
We now turn our attention to the foundation of the modern

treatment of sampled-data systems. From what we have presented up to this section, it is clear that the fundamental difficulty in sampled-data control systems lies in the fact that
they involve two different time sets: one is discrete (arising
from the digital controller) and the other is continuous (arising from the continuous-time plant). This difficulty has been
successfully circumvented in the modern approach.
A New Model with Intersample BehaviorLifting
While it is possible to recover intersample behavior via the
modified z-transform, it implicitly assumes sampling inputs
in its formulation. It is therefore not adequate for describing
correspondence from the exogenous continuous-time inputs to
continuous-time outputs.
A new solution was introduced in 19901991 (812). The
new idea, currently called lifting, makes it possible to describe sampled-data systems via a time-invariant, discretetime model while maintaining the intersample behavior.
The idea is very simple. Let f(t) be a given continuous-time
signal. Sampling surely results in a loss of intersample information. Then, instead of sampling f(t), we will represent it as
a sequence of functions. Namely, we set up the correspondence
(Fig. 8):
L : f { f [k]( )}
k=0,
f [k]( ) = f (kh + ),
The difference here is that the lifted input u[k]( ) need not
be a constant on (kh, (k 1)h), and the right-hand side integral gives an operator
L2 [0, h) Rn : u()
e A(h ) Bu( ) d
While state-transition is described in the discrete timing as

above, the system keeps producing an output. If we consider
lifting of x(t), it is easily seen to be described by
x[k]( ) = e A x[k] +
e A( ) Bu[k]( ) d
Then the lifted output y[k]( ) is given by

y[k]( ) = Ce A x[k] +
Ce A( ) Bu[k]( ) d
Observe that Eqs. (17) and (18) take the form

x[k + 1] = A x[k] + Bu[k]
y[k] = C x[k] + Du[k]
0 <h
This idea makes it possible to view (time-invariant or even

periodically time-varying) continuous-time systems as linear,
time-invariant discrete-time systems. (Basically the same
idea that converts periodically time-varying discrete-time systems to time-invariant systems were encountered and rediscovered many times in the literature. It appears to date back
at least to Ref. 13. Such a discrete-time lifting is also frequently used in signal processing, especially in multirate signal processing, and is called blocking.)
The basic idea is the following: Let
x(t)
= Ax(t) + Bu(t)
(16)
y(t) = Cx(t)
be a given continuous-time plant. Then we lift the input u(t)

to obtain u[k]( ). We consider that this lifted input is applied
at the timing t kh (h is a prespecified sampling rate), and
observe how it affects the system. Let x[k] be the state at time
f(t)
and the operators A , B , C , D do not depend on k. In other

words, it is possible to describe this continuous-time system
with discrete-timing, once we adopt the lifting point of view.
To be more precise, the operators A , B , C , D are defined as
follows:
A :
Rn Rn : x e Ah x
B:
L2 [0, h) Rn : u
e A(h ) Bu( ) d
C :
Rn L2 [0, h) : x Ce A x
D:
L2 [0, h) L2 [0, h) : u
(19)
Ce A( ) Bu( ) d
Thus the continuous-time plant in Eq. (16) can be described by a time-invariant discrete-time model. Once this is
done, it is entirely routine to connect this expression with a
discrete-time controller, and hence sampled-data systems (for
example, see Fig. 1) can be fully described by time-invariant
discrete-time equations, this time without sacrificing the intersampling information. We will also denote this overall
equation abstractly as
x[k + 1] = A x[k] + Bu[k]
2h
3h
4h
(18)
0 h
0 h
0 h
0 h
y[k] = C x[k] + Du[k]
(20)
Figure 8. Lifting maps a continuous-time signal to a discrete-time

signal with function components.
Since this is purely a discrete-time system except that the

inputoutput spaces are infinite-dimensional (L2 spaces), all
the formal developments presented earlier carries over to the
DIGITAL CONTROL
present situation without any change. For example, the transfer function is defined as
G(z) := D + C (zI A )1 B
When an input u(z) is applied to system in Eq. (20), its zeroinitial state output is given by G(z)u(z). Note also that operator A in Eq. (19) is a matrix, and hence A in Eq. (20) is also
a matrix. This means that the stability of Eq. (20) can be
tested by the poles of G(z). It is stable if G(z) is analytic for
z : z 1.
Steady State and Frequency Response
The time-invariance of the lifted model Eq. (20) naturally
yields a definition of stead-state and frequency responses.
Let G(z) be a stable transfer function as defined above.
Take a sinusoid ejt as an input. Its lifted image is
{e jkh e j }
k=0
According to Eq. (1), the z-transform of this sequence is
Since ej(ns)t is lifted to be ejkhej(ns)k0

, such aliasing
high-frequency signals are in the form ejkhv(). Thus the
definition above takes all the aliasing components into account, and takes the largest magnitude of enlargement
among them.
Frequency Response via Sequence Spaces

The observation above clearly shows that the gain of the frequency response takes all aliasing effects into account. It is
however unclear that aliasing exhausts all the freedom in
v L2[0, h). In other words, is it true that if we consider the
largest gain imposed by considering all aliased components,
does this give the same gain as Eq. (23)?
This is indeed true; its proof is based on the following
lemma which guarantees that the family ejn / hn

(n
2
ns) forms an orthonormal basis of L [0, h), and hence
any v L2[0, h) can be expanded into a series of aliased signals ejn, n 0, 1, 2, . . ..
Lemma 1. Fix any [0, s). Then every L2[0, h) can
be expanded in terms of ejnn

as
ze j
z e jh
( ) =
Since G(z) is stable, expand it in a neighborhood of z ejh:
G(z) = G(e jh ) + (z e jh )G(z)
G(z)
ze
zG(e )e
=
z e jh
z e jh
jh
an e j n
(24)
n=
with
(z) that is also analytic in z 1 (by the stability

with some G
of G). It follows that
j
453
+ G(z)e
The second term on the right tends to zero as k by the

, and hence the output approaches
analyticity of G
jh j
jh
zG(e )e /(z e ). Therefore, the lifted output y[k]( ) asymptotically approaches
an =
1
h
1
(
jn )
h
e j n ( ) d =
(25)
where denotes the Laplace transform of when extended

to L2[0, ) as 0 outside [0, h). Furthermore, the L2 norm
is given by
2 = h
|an |2
(26)
n=
y[k]( ) = (e jh )k G(e jh )[e j ]( )
(21)
as k .
Unless ejh 1, the asymptotic response above is really not
in steady state. However, its modulus G(ejh)ej remains invariant at each sampling time. The change at each step is a
phase shift induced by the multiplication by ejh. This explains
why the ripples in Fig. 6 look similar but not really the same
in different sampling periods.
This observation motivates the following definition:
Definition 1 Let G(z) be the transfer function of the lifted
system as above. The frequency response operator is the operator
G(e
jh
) : L [0, h) L [0, h)
2
sup
vL 2 [0, h)
G(e jh )v
v
jkh
G(e
jh
)[e

v
e j

(27)
j

]v
Expand G(ejh)[ej] in terms of ejn to get

G(e jh )[e j
] =
g
n e j n
n=
Substituting this into Eq. (27), we obtain

(23)
The maximum G(e ) over [0, s) is the H norm of G(z).

jh

e jkh G(e jh )[v] = e jkh G(e jh )
(22)
regarded as a function of [0, s) (s : 2/h). Its gain at

is defined to be
G(e jh ) =
Let us apply this result to the frequency response defined
by Eq. (23). Expand v L2[0, h) as v() vej. Note

jkh jn
jnt
that e e k0 is the lifted image of e , and hence
is vejt. By Eq. (21), the asymptotic reejkhv()k0

sponse of G(z) against this input is given by
e jkh G(e jh )[v] = e jkh
= n=
g
n e j n v
454
DIGITAL CONTROL
Since ej(ns)h ejh, this is the kth step response of
= n=
g
n e j n t v
where t kh . Interchanging the order of summation, this

is equal to
n=
g
n ()v
e j n t
(28)
This means that the response against vejt is again expressible as an infinite sum of all such aliased signals. It
should be intuitively clear that the largest gain among them
again gives the gain of the frequency response, when such
signals are equipped with norm (nv2)1/2. This isometric correspondence is guaranteed by the Parseval identity Eq. (26).
This is the viewpoint adopted in Refs. 14 and 15 to discuss
the frequency response of sampled-data systems; see also Ref.
16. It is also closer to the classical treatment based on the
impulse modulation (3,7).
where R (I D *D ). The important point to be noted

here is that all the operators appearing here are actually matrices. For example, by checking the domain and range
spaces, we easily see that B R1
B * is a linear operator from
n into itself, i.e., a matrix. Therefore, in principle, one can
solve the singular value Eq. (29) by finding a nontrivial solution for Eq. (30) (provided R is invertible) (17,18).
H /H 2 CONTROL PROBLEMS
A significant consequence of the modern approach to sampleddata control is that various robust control problems such as
H /H2 control problems are now completely solved. The problem was initiated by Chen and Francis (19) and later solved
in Refs. 9, 10, and 2022 in more complete forms; see Ref. 5
for the pertinent historical accounts.
To state the problem more precisely, let us introduce the
notion of generalized plants. Suppose that a continuous-time
plant is given in the following form:
xc (t) = Axc (t) + B1 w(t) + B2 u(t)

z(t) = C1 xc (t) + D11 w(t) + D12 u(t)
Gain Computation
The gain function G(ejh) is given as the operator norm at each
frequency, and its computation is primarily an infinite-dimensional problem. However, for most of the practical purposes,
it can be computed as the maximal singular value (17).
Our problem is thus reduced to that of solving the singular
value equation
[ 2 I G G(e jh )]w = 0
y(t) = C2 xc (t)
Here w is the exogenous input, u(t) is the control input, y(t)
is the measured output, and z(t) is the controlled output. The
controller is of the following form:
xd [k + 1] = Ad xd [k] + Bd S y[k]
v[k] = Cd xd [k] + Dd S y[k]
(29)
This is still an infinite-dimensional equation. However, since

A , B , C , D are finite-rank operators, we can reduce this to
a finite-dimensional rank condition. Note that, by lifting, a
realization of G(z) can be written in the form
x[k + 1] = A x[k] + Bw[k]
u[k]( ) = H( )v[k]
where H() is a suitable hold function. This is shown in Fig.
9. The objective here is to design or characterize a controller
that achieves a prescribed performance level > 0 in such
a way that
Tzw <
y[k] = C x[k] + Dw[k]
(31)
Its adjoint can then be easily derived as

p[k] = A p[k + 1] + C v[k]
e[k] = B p[k + 1] + D v[k]

Taking the z transforms of both sides, setting z ejh, and
substituting v y and e 2w, we get
jh
w
A
C1
C2
B1 B2
D11 D12
0
0
x = A x + Bw
p = e jh A p + C (C x + Dw)
( 2 D D)w = e jh B p + D C x
H( )
Solving these, we obtain

jh
I
0
BR1
B
A + C DR1
B
A + BR1
D C
1
C (I + DR D )C
Ad
Cd
!
0
x
=0
I
p
(30)
Bd
Dd
Figure 9. Generalized plant construction of a sampled-feedback system where z denotes the controlled output, y is the measured output,
w is the exogenous input, and u is the control input.
DIGITAL CONTROL
Initial condition results ( = 0.1)

2
1.5
1
Amplitude
where Tzw denotes the closed-loop transfer function from w to

z. This is the H control problem for sampled-data systems.
The H2 control problem is obtained by replacing the H norm
above by the H2 norm.
The difficulty here is that both w and z are continuoustime variables, and hence their lifted variables are infinitedimensional. A remarkable fact here is that the H problem
(and H2 problem as well) [Eq. (31)] can be equivalently transformed to the H problem for a finite-dimensional discretetime system. While we skip the details here [see the references above and (5)], we remark that this norm-equivalent
discrete-time system is entirely different from the one given
in the section on Discrete-Time Design in that it fully takes
intersampling behavior into account. The difference will be
exhibited by the design examples in the next section.
455
0.5
0
0.5
1
1.5
SOME EXAMPLES
2
To see the power of the modern design methods, let us consider two design examples. We start with the H design.
10
15
20
25
30
Time (s)
Figure 11. Time responses for h 0.1 by sampled-data (solid) and
continuous-time (dash) H designs do not show much difference.
H Design
Consider the unstable second-order plant
P(s) := C p (sI A p )1 B p =
1
s2 0.1s + 1
Figure 11 shows the impulse responses of the designed

closed loop for the sampling period h 0.1. The solid curve
represents the response for the sampled-data design and the
dashed curve shows that for the continuous-time design with
Tustin transformation. They do not present much difference
at this stage. However, when we increase the sampling period
(i.e., decrease the sampling rate) to h 0.55 (Fig. 12), the
continuous-time design is already very close to the stability
margin. In the conventional design, one may conclude that
this sampling period is already too long, and the whole configuration is not feasible for sampled-data implementation.
But quite contrary to such an intuition, the sampled-data H
design can tolerate such a long sampling period. The crucial
with weight matrices

Q1/2 = 1
R1/2 = 0.01 E = 0.01 N = 0.01
and the antialiasing filter

Faa (s) :=
1
hs + 1
depicted in Fig. 10.

We here compare the following two design results:
The direct sampled-data H design
The continuous-time H design with Tustin transformation
Initial condition results ( = 0.55)

2
In the continuous-time design, the antialiasing filter is bypassed. On the other hand, it is inserted in the sampled-data
design to make the total design well posed.
1.5
Q1/2
E
zc
wc
R1/2
N
Amplitude
1
0.5
0
0.5
1
1.5
Faa
Cp
(sI Ap)1
Bp
10
15
20
25
30
Time (s)
Figure 10. Generalized plant for sampled-data and continuous-time
H design.
Figure 12. Time responses for h 0.55 exhibit a clear difference

between sampled-data (solid) and continuous-time (dash) H designs.
456
DIGITAL CONTROL
Frequencygain response
50
40
30
0
Gain (dB)
20
10
0
10
30
101
100
Frequency (rad/s)
101
Figure 13. Frequency response plots for h 0.55 support the observation in Fig. 11.
difference here is that the sampled-data design incorporates

the sampling period in the design procedure, whereas the continuous-time design does not. This gap becomes even clearer
when we compare two designs via their frequency responses
(Fig. 13). Whereas the sampled-data design exhibits a rather
mild curve, the continuous-time design shows a very sharp
peak at around 1.5 rad/s. Observe also that this frequency
agrees precisely with the period of oscillation in the impulse
response (Fig. 13).
H 2 Design
In the case of continuous-time design, slower sampling rates
yield problems. For the sample-point discretization, fast sampling rates can induce very wild responses.
5
0
5
Gain (dB)
Time
20
10
15
20
25
30
35
0.01
0.1
Figure 15. Time responses for sampled-data (solid) and discretetime (dash) design show a clear difference.
Consider a simple second-order plant P(s) 1/(s2 2s 1).

For h 0.2, we execute
Sampled-data (continuous-time based) H2 design
Discrete-time H2 design
Figures 14 and 15 show the frequency and time responses
of the closed-loop systems, respectively. In Fig. 14, the solid
(thick) curve shows the response of the sampled-design,
whereas the dotted (thin) curve shows the discrete-time frequency response when the designed controller K is connected
with the discretized plant Gd (i.e., a purely discrete-time frequency response). At a first glance, it appears that the discretized design performs better, but actually it performs
poorer when we compute the real (continuous-time) frequency
response of G connected with Kd. The dashed curve shows this
frequency response; it is similar to the discrete-time frequency response in the low-frequency range but exhibits a
very sharp peak at the Nyquist frequency (/h 15.7 rad/s,
i.e., 1/2h 2.5 Hz).
In fact, the impulse responses in Fig. 15 exhibit a clear
difference between them. The solid curve shows the sampleddata design, and the dashed curve the discrete-time one. The
latter shows an oscillatory response. Also, both responses decay to zero very rapidly at sampled instants. The difference
is that the latter exhibits very large ripples, with periods of
approximately 0.4 s. This corresponds to 1/0.4 Hz, which is
the same as (2)/0.4 /h rad/s, i.e., the Nyquist frequency.
This is precisely captured in the modern (lifted) frequency response in Fig. 14.
It is worth noting that when h is smaller, the response for
the discrete-time design becomes even more oscillatory, and
shows a very high peak in the frequency response. The details
may be found in Ref. 23.
10
Frequency (rad/s)
Figure 14. Frequency response plots show the difference between
the sampled-data (solid), discrete-time (dashed) difference; the thin
dotted curve shows the frequency response with intersample behavior ignored.
BIBLIOGRAPHICAL NOTES
For classical treatments of sampled-data control, it is instructive to consult Refs. 2, 3, and 7. The textbooks (24,25) cover
both classical and modern aspects of digital control. For dis-
DIGITAL CONTROL
crete-time design methods and other related topics, the

reader is also referred to the handbook (26). For Shannons
sampling theorem, consult Ref. 27 for various extensions and
some historical accounts.
As noted in the main text, discrete-time lifting has been
introduced and rediscovered by several authors (see, for example, Refs. 13, 28, and 29). Aside from the H design, the
H2 control problem has also been studied extensively (3033).
L1-norm problems are studied in Refs. 3436. In relation to
the H control problem, various robust stability problems
have been studied (see, for example, Refs. 15 and 37).
The treatment of frequency response given here is based
on Refs. 16 and 17. Some computational aspects are discussed
in Ref. 38. For other approaches not dealt with here, see also
Refs. 39 and 40.
BIBLIOGRAPHY
19.
20.
21.
22.
23.
24.
1. R. E. Kalman and J. E. Bertram, A unified approach to the theory

of sampling systems, J. Franklin Inst., 267: 405436, 1959.
25.
2. E. I. Jury, Sampled-Data Control Systems, New York: Wiley,

1958.
26.
3. J. R. Ragazzini and G. F. Franklin, Sampled-Data Control Systems, New York: McGraw-Hill, 1958.
27.
4. R. E. Kalman, Y. C. Ho, and K. Narendra, Controllability of linear dynamical systems, Contrib. Differ. Equations, 1: 189213,
1963.
28.
5. T. Chen and B. A. Francis, Optimal Sampled-Data Control Systems, New York: Springer, 1995.
29.
6. A. H. Zemanian, Distribution Theory and Transform Analysis,

New York: Dover, 1987.
30.
7. J. T. Tou, Digital and Sampled-Data Control Systems, New York:

McGraw-Hill, 1959.
31.
8. B. Bamieh et al., A lifting technique for linear periodic systems

with applications to sampled-data control systems, Syst. Control
Lett., 17: 7988, 1991.
9. G. Tadmor, Optimal H sampled-data control in continuous time
systems, Proc. ACC, 1991, pp. 16581663.
32.
33.
10. H. T. Toivonen, Sampled-data control of continuous-time systems

with an H optimality criterion, Automatica, 28: 4554, 1992.
34.
11. Y. Yamamoto, New approach to sampled-data systems: A function space method, Proc. 29th CDC, 1990, pp. 18821887.
35.
12. Y. Yamamoto, A function space approach to sampled-data control

systems and tracking problems, IEEE Trans. Autom. Control, AC39: 703712, 1994.
36.
13. B. Friedland, Sampled-data control systems containing periodically time varying members, Proc. 1st IFAC Congr., 1961, pp.
361367.
37.
14. M. Araki, Y. Ito, and T. Hagiwara, Frequency response of sampled-data systems, Automatica, 32: 483497, 1996.
38.
15. G. Dullerud and K. Glover, Robust stabilization of sampled-data

systems to structured LTI perturbations, IEEE Trans. Autom.
Control, AC-38: 14971508, 1993.
16. Y. Yamamoto and M. Araki, Frequency responses for sampleddata systemstheir equivalence and relationships, Linear Algebra Its Appl., 205206: 13191339, 1994.
17. Y. Yamamoto and P. P. Khargonekar, Frequency response of
sampled-data systems, IEEE Trans. Autom. Control, AC-41: 166
176, 1996.
18. Y. Yamamoto, On the state space and frequency domain charac-
39.
40.
457
terization of H-norm of sampled-data systems, Syst. Control

Lett., 21: 163172, 1993.
T. Chen and B. A. Francis, On the L 2-induced norm of a sampleddata system, Syst. Control Lett., 15: 211219, 1990.
B. Bamieh and J. B. Pearson, A general framework for linear
periodic systems with applications to H sampled-data control,
IEEE Trans. Autom. Control, AC-37: 418435, 1992.
P. T. Kabamba and S. Hara, Worst case analysis and design of
sampled data control systems, IEEE Trans. Auto. Control, AC38: 13371357, 1993.
N. Sivashankar and P. P. Khargonekar, Characterization and
computation of the L 2-induced norm of sampled-data systems,
SIAM J. Control. Optim., 32: 11281150, 1994.
S. Hara, Y. Yamamoto, and H. Fujioka, Modern and classical
analysis/synthesis methods in sampled-data controla brief
overview with numerical examples, Proc. 35th IEEE CDC, 1996,
pp. 12511256.
strom and B. Wittenmark, Computer Controlled Systems
K. J. A
Theory and Design, Upper Saddle River, NJ: Prentice-Hall, 1996,
3rd ed.
D. Williamson, Digital Control and Implementation, New York:
W. S. Levine (ed.), The Control Handbook, Boca Raton, FL: CRC
Press, 1996.
A. I. Zayed, Advances in Shannons Sampling Theory, Boca Raton,
FL: CRC Press, 1993.
J. H. Davis, Stability conditions deried from spectral theory: Discrete systems with periodic feedback, SIAM J. Control, 10: 1
13, 1972.
P. P. Khargonekar, K. Poolla, and A. Tannenbaum, Robust control of linear time-invariant plants using periodic compensation,
T. Chen and B. A. Francis, H 2-optimal sampled-data control,
P. P. Khargonekar and N. Sivashankar, H 2 optimal control for
sampled-data systems, Syst. Control Lett., 17: 425436, 1991.
B. Bamieh and J. B. Pearson, The H 2 problem for sampled-data
systems, Syst. Control Lett., 19: 112, 1992.
S. Hara, H. Fujioka, and P. T. Kabamba, A hybrid state-space
approach to sampled-data feedback control, Linear Algebra Its
Appl., 205206: 675712, 1994.
G. Dullerud and B. A. Francis, L 1 performance in sampled-data
N. Sivashankar and P. P. Khargonekar, Induced norms for sampled-data systems, Automatica, 28: 12671272, 1992.
B. Bamieh, M. A. Dahleh, and J. B. Pearson, Minimization of the
L-induced norm for sampled-data systems, IEEE Trans. Autom.
Control, AC-38: 717732, 1993.
N. Sivashankar and P. P. Khargonekar, Robust stability and performance analysis of sampled-data systems, IEEE Trans. Autom.
Control, AC-38: 5869, 1993.
S. Hara et al., Computational aspects of gain-frequency response
for sampled-data systems, Proc. 34th IEEE CDC, 1995, pp. 1784
1789.
A. Feuer and G. C. Goodwin, Generalized sample hold functionsfrequency domain analysis of robustness, sensitivity and
intersample difficulties, IEEE Trans. Autom. Control, AC-39:
10421047, 1994.
J. S. Freudenberg, R. H. Middleton, and J. H. Braslavsky, Inherent design limitations for linear sampled-data feedback systems,
Int. J. Control, 61: 13871421, 1995.
YUTAKA YAMAMOTO
Kyoto University
458
DIGITAL FILTERS
DIGITAL CONTROL SYSTEMS DESIGN. See DISCRETE TIME SYSTEMS DESIGN METHODS.
602
DISCRETE EVENT DYNAMICAL SYSTEMS

In recent decades, many modern, large-scale, human-made
systems (e.g., exible manufacturing systems, computer and
communication networks, air and highway trafc networks,
and the military C3I/logistic systems) have been emerging.
These systems are called discrete event systems because of the
discrete nature of the events. Research indicates that these
human-made systems possess many properties that are similar to those of natural physical systems. In particular, the
evolution of these human-made systems demonstrates some
dynamic features; exploring these dynamic properties may
lead to new perspectives concerning the behavior of discrete
event systems. The increasing need for analyzing, controlling,
and optimizing discrete event systems has initiated a new research area, the dynamics of discrete event systems. To emphasize their dynamic nature, these systems are often referred to as discrete event dynamic systems (DEDS) in the
literature (1).
This article reviews the fundamental theories and applications of DEDS. Since the dynamic behavior is closely related
to time, we shall not discuss untimed models such as the
automata-based model (2); these models are mainly used to
study the logical behavior of a discrete event system.
In 1988, the report of the panel of the IEEE Control Systems Society noted, Discrete event dynamic systems exist in
many technological applications, but there are no models of
discrete event systems that are mathematically as concise or
computationally as feasible as are differential equations for
continuous variable dynamical systems. There is no
agreement as to which is the best model, particularly for the
purpose of control (3). This statement is still true today.
However, after the hard work of many researchers in the recent years, there are some relatively mature theories and
many successful application examples.
Most frequently used models for analyzing DEDS are
queueing systems and Petri nets. Queueing models are usually used to analyze the performance (in most cases, steady
state performance) of DEDS, and Petri nets provide a graphical illustration of the evolution of the system behavior and
are particularly useful in analyzing behaviors comprising concurrency, synchronization, and resource sharing (4). Other
models for DEDS include the more general but less structural
models such as Markov processes and generalized semiMarkov processes (GSMP), and the max-plus algebra that is
particularly suitable for modeling DEDS with deterministic
event lifetimes that exhibit a periodic behavior.
One main theory that employs a dynamic point of view to

study system behavior is the perturbation analysis (PA). The
objective of PA is to obtain performance sensitivities with respect to system parameters by analyzing a single sample path
of a discrete event system (59). The sample path, which describes how the DEDS evolves, can be obtained by observing
the operation of a real system or by simulation. The technique
is in the same spirit of the linearization of nonlinear continuous variable dynamic systems (6).
The sample path based approach of PA motivates the research of on-line performance optimization of DEDS. Recent
study shows that PA of discrete parameters (parameters that
jump among discrete values) is closely related to the Markov
decision problem (MDP) in optimization. The PA-based online optimization technique has been successfully applied to a
number of practical engineering problems.
The following section briey reviews some basic DEDS
models. The next section introduces PA in some details. The
nal section 4 discusses the application of PA in on-line optimization and points out its relations with the Markov decision problem.
DEDS MODELS
Queueing Systems
The simplest queueing system is the M/M/1 queue, where
customers arrive at a single server according to a Poisson process with rate and the service time for each customer is
exponentially distributed with mean 1/, . The steady
state probability of n, the number of customers in the queue,
is
p(n) = n (1 )
, n = 0, 1, . . .
From this, the average number of customers in the queue is

n =
server j with probability qi, j and leaves the network with probM
ability qi,0. We have j0 qi, j 1, i 1, 2, , M. The service
time of server i is exponentially distributed with mean si
1/i, i 1, 2, , M.
The system state is n (n1, n2, , nM), where ni is the
number of customers in server i. Let i be the arrival rate of
the customers to server i. Then
i = 0,i +
The more general model is a queueing network that consists

of a number of service stations. Customers in a network may
belong to different classes, meaning that they may have different routing mechanisms and different service time distributions. A queueing network may belong to one of three
types: open, closed, or mixed. In an open network, customers
arrive at the network from outside and eventually leave the
network; in a closed network, customers circulate among stations and no customer arrives or leaves the network; A mixed
network is open for some classes of customers and closed for
others.
An open Jackson network consists of M single-server stations and N single-class customers. Each server has an buffer
with an innite capacity and the service discipline is rstcome-rst-served. Customers arrive at server i according to a
Poisson process with (external) rate 0,i, i 1, 2, , M.
After receiving the service at server i, a customer enters
M
j q j,i
i = 1, 2, . . ., M
j=1
It is known that the steady state distribution is
p(n) = p(n1 , n2 , . . ., nM ) =
M
p(nk )
k=1
with
n
p(nk ) = (1 k )k k
k =
k
k
k = 1, 2, . . ., M
A load-independent closed Jackson (Gordon-Newell) network is similar to the open Jackson network described above,
except that there are N customers circulating among servers
M
according to the routing probabilities qi, j, k1 qi,k 1, i 1,
M
2, , M. We have k1 nk N. We consider a more general
case: the service requirement of each customer is exponential
with a mean 1; the service rates, however, depend on the
number of customers in the server. Let i,ni be the service rate
of server i when there are ni customers in the server, 0
i,ni , ni 1, 2, , N, i 1, 2, , M. We call this a
load-dependent network. In a load-independent network,
i,ni i for all ni, i 1, 2, , M.
The state of such a network is n (n1, n2, , nM). We
use ni, j (n1, , ni 1, , nj 1, , nM), ni 0, to
denote a neighboring state of n. Let

(nk ) =
and the average time that a customer stays in the queue is

T=
603
1 if nk > 0
0 if nk = 0
and let
(n) =
M
(nk )k,n
k=1
Then the ow balance equation for the steady state probability p(n) is
(n)p(n) =
M
M

i=1 j=1
(n j )i,n
i +1
qi, j p(n j,i )
Let yi 0, i 1, 2, , M, be the visit ratio to server i,

that is, a solution (within a multiplicative constant) to the
equation
yi =
M

j=1
q j,i y j
j = 1, 2, . . ., M
604
Let Ai(0) 1, i 1, 2 , M, and
Ai (k) =
b1
k

i, j
b2
Mach. 2
Mach. 1
i = 1, 2, . . ., M
b3
j=1
Exit
and for every n 1, 2, , N and M 1, 2, , M, let
Gm (n) =
m

n 1 ++n m =n i=1
yn i
Ai (ni )
Figure 1. A reentrant line.
Then we have
n
p(n) =
M

yi i
1
GM (N) i=1 Ai (ni )
This equation is often referred to as a product-form solution.

For load-independent networks, i,ni i, i 1, 2, ,
M. The product-form solution becomes
Gm (n) =
m

n 1 ++n m =n i=1
xi i
and
p(n) =
M

1
n
x i
GM (N) i=1 i
(1)
where xi yi /i yisi, i 1, 2, , M.
There are a number of numerical methods for calculating
p(n) and the steady state performance, among them are the
convolution algorithm and the mean value analysis (7); in addition, analytical expressions exist for the normalizing constant GM(N) (10). For more about queueing theory, see, for
example, Refs. 11 and 12.
One typical example of the queueing model is the resource
sharing problem. Consider the case where M resources are
shared by N users and each resource can be held by only one
user at any time. Every time a user grasps resource i, it holds
the resource for a random time with si, A user, after the completion of its usage of resource i, requests the hold of resource
j with a probability qi, j. This problem can be modeled exactly
as a closed queueing network with N customers and M
servers. This model can be successfully used in analyzing the
performance of packet switches, where the users are the
head-of-line packets and the resources are the channels, and
the performance of data-base systems, where the users are
programs and the resources are data records.
Petri Nets
Many DEDS consist of components [e.g., central processing
units (CPUs), disks, memories, and peripheral devices in computer systems; and machines, pallets, tools, and control units
in manufacturing systems] that are shared by many users
and exhibit concurrency. This feature makes Petri nets a suitable model for DEDS.
In a graphical representation, the structure of a Petri net
is dened by three sets: a set of places P p1, p2, , pn,
a set of transitions T t1, t2, , tm, and a set of arcs. A
place is represented by a circle and a transition by a bar. An
arc is represented by an arrow from a transition to a place or
a place to a transition. A place is an input (output) to a transition if an arc exists from the place (transition) to the transition (place).
The dynamic feature of a Petri net is represented by tokens, which are assigned to the places. Tokens move from
place to place during the execution of a Petri net. Tokens are
drawn as small dots inside the circle representing the places.
The marking of a Petri net is a vector M (m1, m2, ,
mn), where mi is the number of tokens in place pi, i 1, 2,
, n. A marking corresponds to a state of the system. A
Petri net executes by ring transitions. A transition is enabled if its every input place contains at least one token.
When a transition is enabled, it may re immediately or after
a ring delay, which can be a random number. The ring delay is used to model the service times. When a transition res,
one token is removed from each input place and one token
added to each output place. Thus, the number of tokens in a
place and in the system my change during the execution. In
addition to the arcs described above, another type of arc,
called the inhibitor arc, is often used to model the priority
among services. An inhibitor arc is drawn from a place to a
transition, with a small circle at its end. When an inhibitor
arc is used, if there is at least one token in the place, the
transition cannot re.
As an example, we consider the reentrant line (13) shown
in Fig. 1. The system consists of two machines and three buffers. Work pieces arrive at buffer b1 with rate and get service
from machine 1 with rate 1; after the service in b1, the piece
moves to buffer b2 and gets service at machine 2 with rate
2; after the service in b2, the piece reenters machine 1 at
buffer b3 and receives service with rate 3. Machine 1 can
serve one piece at a time, and pieces in b3 have a nonpreemptive higher priority than those in b1.
The Petri net model for the system is shown in Fig. 2. In
the gure, places bi, i 1, 2, 3, represent the buffers, and
transitions pi, i 1, 2, 3, represent the service processes of
the pieces in the three buffers. If there is a token in places
mi, i 1, 2, then machine i is available; if there is a token in
si, i 1, 3, then the work piece in buffer bi is under service.
It is clear that machine 1 is shared by the pieces in both b1
and b3, and the inhibitor arc from b3 to p4 models the priority.
(For more about Petri net theory and applications, see Refs.
4 and 1416).
The state process of a queueing system or a Petri net can
be modeled as a Markov process, or more generally, as a generalized semi-Markov process. In this sense, both Markov processes and GSMPs are more general than queueing systems
and Petri nets; however, these general models do not enjoy
the structural property that queueing systems and Petri nets
possess. In fact, the GSMP model is a formal description of
the evolution mechanism of a queueing system. Readers are
referred to Refs. 5 and 8 for a discussion of GSMP.
b1
in
p1
b2
We can prove that for all i 1
p2
K
s1
(2i)
(2i+1)
p4
m2
p5
3
Figure 2. The Petri net model for the reentrant line.
The Max-Plus Algebra

With the max-plus algebra proposed in Ref. 17, many DEDS
can be modeled as linear systems. In the max-plus algebra,
we dene the operation multiplication on two real numbers
a and b as a b a b and the addition as a b max
a, b. It is easy to verify that these two operations indeed
dene an algebra. We give a simple example to illustrate how
the max-plus algebra can be used to analyze the periodic behavior of a DEDS. Consider a single queue with deterministic
interarrival time a and service time s. Let ak and dk be the
arrival and departure times of the kth customer, respectively,
k 1, 2, , and a1 0, d0 0. Then for k 1, 2, we
have
ak+1 = a + ak
dk = max{ak + s, dk1 + s}
In the max-plus algebra, this can be written as a linear equation
xk+1 = Axk
k = 1, 2, . . .
where
a
A=
d
H
d
xk+1 = Ak xk
(2)
where Ak A(k1) A, k 1, and A1 A.

An interesting property of a matrix under the max-plus
algebra is the periodic property. This can be illustrated by an
example. Consider

5
3
=4
2
1

1
=4K
2
Perturbation analysis of DEDS is a multidisciplinary research area developed since early 1980s, with the initial
work of Ho et al. (19). PA provides the sensitivities of
performance measures with respect to system parameters
by analyzing a single sample path of a DEDS. This area is
promising because of its practical usefulness. First, compared with the standard procedure, which uses the difference of performance measures with two slightly different
values for every parameter, this technique saves a great
amount of computation in simulation, because PA algorithms can provide the derivatives with respect to all the
parameters by using only a single simulation run. In addition, PA estimates are more accurate than those obtained
from nite differences because the latter may encounter
numerical problems caused by dividing two small numbers.
Second, PA can be applied to on-line performance optimization of real-world systems by observing a sample path of
an existing system; for these systems, changing the values
of their parameters may not be feasible.
Cao (20) observed that the simple PA algorithms based on
a single sample path, called innitesimal perturbation analysis (IPA), in fact yield sample derivatives of the performance;
although these sample derivatives are unbiased or strongly
consistent for many systems, this is not the case for many
others. This insight has set up two fundamental research directions: to establish IPA theory, including the proof of convergence of IPA algorithms, and to develop new algorithms
for systems where IPA does not work well. After the hard
work of many researchers in more than one decade, the theory for IPA is relatively mature, and many results have been
obtained for problems where IPA does not provide accurate
estimates.
Infinitesimal Perturbation Analysis
with H being a large positive number. Thus,

1
M=
3
1
2
PERTURBATION ANALYSIS
p3
ak
xk =
dk1
1
0
b3
Exit
0
=
3

2
=
1
M is said to be of order 2 periodic. Cohen et al. (17) proved

that all matrices possess such periodicity. Therefore, Eq. (2)
can be used to study the periodic behavior of a DEDS. For
more discussions, see Ref. 18.
m1
s3
605
Let be a parameter of a stochastic discrete event system;

the underlying probability space is denoted as (, F , P ). Let
(), , be a random vector that determines all the
randomness of the system. For example, for a closed queueing
network, may include all the uniformly distributed random
variables on [0, 1) that determine the customers service times
and their transition destinations (say, in a simulation). Thus,
a sample path of a DEDS depends on and ; such a sample
path is denoted as (, ).
Let T0 0, T1, , Tl, be the sequence of the state
transition instants. We consider a sample path of the system
606
i,1
in a nite period [0, TL). The performance measured on this

sample path is denoted as L(, ). Let
L ( ) = E[L (, )]
i,1 i,2
(3)
t1
and
t2
Figure 3. The perturbation in a busy period.
( ) = lim L (, )
L
w.p.1.
(4)
where E denotes the expectation with respect to the probability measure P . We assume that both the mean and limit in
Eq. (3) and Eq. (4) exist. Thus L(, ) is an unbiased estimate
of L() and an strongly consistent estimate of (), respectively.
The goal of perturbation analysis is to obtain the performance derivative with respect to by analyzing a single sample path (, ). That is, we want to derive a quantity based on
a sample path (, ) and use it as an estimate of L()/ or
()/.
Given a single sample path, the realization of the random
vector is xed. Therefore, we x and consider L(, ) as a
function of . This function is called a sample performance
function. Now, we consider the following question: given a
sample path (, ), can we determine the sample path (
, ) with the same and / 1? If we can, then we can
get the performance for the perturbed system, L( , ),
and furthermore, the derivative of the sample performance
function:
( + , ) L ( ), )
(, ) = lim L
0
L

(5)
This is called a sample derivative.

It seems reasonable to choose the sample derivative /
L(, ) as an estimate for L()/ or ()/. This estimate
is called the innitesimal perturbation analysis (IPA) estimate. We require that the estimate be unbiased or strongly
consistent; that is, either

E
[L (, )] =
E[L (, )]
(6)
or

lim
[L (, )] =
lim [L (, )]
All the papers published in the area of IPA deal with these
two basic issues. Roughly speaking, the interchangeability requires that the sample performance function L(, ) be continuous with respect to . General conditions can be found in
Refs. 20 and 8.
The algorithms for obtaining sample derivatives are called
IPA algorithms. Given a nite-length sample path (, ), we
rst, by applying IPA algorithms, ctitiously construct a sample path for the DEDS with a slightly changed parameter and
the same random vector, ( , ), called a perturbed sample
path. The derivative of the performance with respect to can
be obtained by comparing these two sample paths, the original one and the perturbed one.
The principles used in IPA to determine the perturbed
path are very simple. We take closed queueing networks as
an example. The basic idea is that a change in parameter
(say, a mean service time) will induce changes of the service
completion times, and a change of a customers service completion time will affect the other customers service completion times. IPA rules describe how these changes can be determined.
Figure 3 illustrates a busy period of a server, say server i,
in a queueing network. Let Fi(s, ) be its service time distribution. The service time of its kth customer is
si,k = Fi1 (i,k , ) = sup{s : F (s, ) i,k },
where i,k, k 1, 2, , are uniformly distributed random
variables on [0, 1). With the same i,k, in the perturbed sys(i,k, ). Thus, the
tem, the service time changes to F 1
i
service time increases by
i,k = Fi1 (i,k , + ) Fi1 (i,k , )

Fi1 (i,k , )

=

=F (s , )
i,k
(7)
In Eq. (5), the same random variable is used for both

L( , ) and L(, ); this corresponds to the simulation
technique common random variable in estimating the difference between two random functions. This technique usually
leads to small variances. Equations (6) and (7) are referred to
as the interchangeability in the literature (20). From the
above discussion, the two basic issues for IPA are
1. To develop a simple algorithm that determines the sample derivative of Eq. (5) by analyzing a single sample
path of a discrete event system; and
2. To prove that the sample derivative is unbiased and/or
strongly consistent, that is, the interchangeability of
Eq. (6) and/or Eq. (7) holds.
(8)
i,k
Equation (8) is called the perturbation generation rule, and

i,k is called the perturbation generated in the kth customers
service time. If the service time is exponentially distributed
with its mean changed from si to si si, then Eq. (8) becomes
i,k =
si
s
si i,k
(9)
The delay of a servers service completion time is called the

perturbation of the server, or the perturbation of the customer being served. In Fig. 3, the perturbation of the rst
customer is i,1 (si,1 /si) si. Because of this perturbation, the
service starting time of the next customer is delayed by the
same amount. Furthermore, the service time of the second
customer increases by i,2 (si,2/ si) si, and thus the perturbation of the second customer is i,1 i,2 (see Fig. 3). In general, the service completion time of the kth customer in a
t2
t3
Server 1
Server 2
t0
t1
t4
Figure 4. Propagation of a single perturbation.
busy period will be delayed by j1 i, j, with i, j being determined by Eq. (8) or Eq. (9). This can be summarized as follows: a perturbation of a customer will be propagated to the
next customer in the same busy period; the perturbation of a
customer equals the perturbation generated in its service period plus the perturbation propagated from the preceding customer.
If a perturbation at the end of a busy period is smaller
than the length of the idle period following the busy period,
the perturbation will not affect (i.e., will not be propagated
to) the next busy period, because the arrival time of the next
busy period depends on another servers service completion
time.
A perturbation at one server may affect other servers
through idle periods. To see how servers may affect each
other, we study the evolution of a single perturbation. In Fig.
4, at t1, server 1 has a perturbation , and before t1, server 2
is idle. At t1, a customer arrives from server 1 to server 2.
Because server 1s service completion time is delayed by ,
server 2s service starting time will also be delayed by ; and
as a result, its service completion time will also be delayed by
the same amount. We say the perturbation is propagated
from server 1 to server 2 through an idle period (server 2 has
the same perturbation as server 1 after t1). At t3, this perturbation is propagated back to server 1.
In summary, if a perturbation is smaller than the lengths
of idle periods (we say that the original and the perturbed
paths are similar), then the evolution of this perturbation on
the sample path can be determined by the following IPA perturbation propagation rules:
k
1. A perturbation of a customer at a server will be propagated to the next customer at the server until it meets
an idle period, and
2. If, after an idle period, a server receives a customer
from another server, then after this idle period the former will have the same perturbation as the latter (the
perturbation is propagated from the latter to the
former).
The perturbation generation rule describes how perturbations are generated because of a change in the value of a parameter; perturbation propagation rules describe how these
perturbations evolve along a sample path after being generated. Combining these rules together, we can determine the
perturbed path.
To apply the propagation rules, the size of the perturbation
at the end of each busy period should not be larger than the
length of the idle period that follows, and the size of the perturbation of a customer that terminates an idle period should
607
not be larger than the length of the idle period; otherwise the
idle period in the original sample path will disappear in the
perturbed one and the simple propagation rules illustrated by
Fig. 4 no longer hold. It is easy to see that for any nitelength sample path (, ), we can always (with probability
one) choose a that is small enough (the size depends on )
such that the perturbations of all customers in the nite sample path are smaller than the shortest length of all the idle
periods in the sample path. This explains the word innitesimal in IPA. Therefore, we can always use IPA propagation
rules to get the perturbed sample path and the sample derivative.
The perturbed sample path is completely determined by
the perturbations of the servers. Given a sample path of a
single-class closed Jackson network of M servers with sm, m
1, 2, , M, being the mean service times, the perturbations of the perturbed system with si changed to si si (with
i xed) can be determined by the following algorithm.
IPA Algorithm for Closed Jackson Networks
0. Create a vector v (v1, v2, , vM); set its initial value
v (0, 0, , 0)
1. At the kth, k 1, 2, , service completion time of
server i, set vi : vi si,k
2. If on the sample path, a customer from server j terminates an idle period of server l, then set vl : vj.
Note that for simplicity in the algorithm we add si,k, instead
of (si /si) si,k, to the perturbation vector. Thus, the perturbation of server m, m 1, 2, , M, is (si /si) vm, with vm being
determined by the algorithm. We shall see that the term
si /si is eventually cancelled in Eq. (11).
The sample derivative can be obtained from these perturbations. Let the sample performance measure be
L( f ) =
1
L
TL
f [N(t)] dt
(10)
where N(t) is the state process and f is a function dened on

the state space. The steady state performance measure is
( f ) = lim L( f )
L
w.p.1
If f I 1 for all n, then we have L(I) TL /L and (I) 1/ ,

where is the throughout of the system. The sample derivative of L(f) can be easily obtained by adding performance calculations to the basic IPA algorithm as follows.
0. Set v (0, 0, , 0), H 0, and H 0
1,2. Same as steps 1 and 2 in the basic IPA algorithm
3. At server ms service completion time, denoted as Tl,
set H H f[N(Tl)](Tl Tl1) and H H
f[N(Tl)] f[N(Tl)]vm.
In the algorithm, H records the value of the integral in Eq.
(10), and
1 si
(H)
L si
608
represents the difference of L(f) between the original and the

perturbed paths. At the end of the sample path, we have
L( f ) (si , i ) =
H
L
and
L( f ) (si , i ) =
si H
si L
Thus, we can calculate the sample elasticity as follows
L( f ) (si , i )
si
H
=
(f)
si
H
L (si , )
(11)
It has been shown that Eq. (11) is strongly consistent (9,6),

that is,
L( f ) (si , i )
si
si ( f )
=
L ( f ) (s , )
si
( f ) si
i
L
lim
w.p.1
Similar algorithms and convergence results have been obtained for open networks, networks with general service time
distributions, and networks in which the service rates depend
on the system states (6). Glasserman (8) studied various IPA
algorithms and their unbiasedness and strong consistency in
the GSMP framework.
Extensions of IPA
For a sample derivative (i.e., the IPA estimate) to be unbiased
and strongly consistent, usually requires that the sample
function be continuous. This request, however, is not always
satised. A typical example illustrating the failure of IPA is
the two-server multiclass queueing network discussed in Ref.
(21); later, Heidelberger et al. (22) discussed in detail a few
extensions of the example.
In the past decade, many methods have been proposed to
extend IPA to a wider class of problems. Each of these
methods has some success on some problems at the cost of
increasing analytical difculty and computational complexity.
We shall review only briey the basic concepts of these
methods.
Smoothed Perturbation Analysis. The idea of smoothed perturbation analysis (SPA) is to average out the discontinuity
over a set of sample paths before taking the derivative and
expectation. To illustrate the idea, we rst write the expected
value of L(, ) as
L ( ) = E[L (, )] = E{E[L (, ) | Z ]}
where Z represents some random events in (, F , P ). Let
L(, Z ) E[L(, )Z ], then
L ( ) = E[L (, Z )]
and Eq. (6) becomes

[L (, Z )] =
E[L (, Z )]
E
(12)
SPA attempts to nd a suitable Z so that the average performance function L(, Z is smooth enough and the interchangeability of Eq. (12) holds, even if Eq. (6) does not. If this
is the case and the derivative of the conditional mean L(,
Z ) can be calculated, then / L(, Z ) can be used as an
unbiased estimate of / L().
The method was rst proposed in Ref. (23); a recent book
Ref. (5) contains a detailed discussion about it. The main issues associated with this method are that it may not be easy
to calculate / E[L(, Z )] and that the computation effort
required may be large.
Finite Perturbation Analysis. The sample derivative does not
contain any information about the jumps in the performance
function. This is because as goes to zero, the event sequence of any perturbed sample path is the same as that of
the original path (two paths are similar). Thus, with IPA, we
do not study the possibility that because of a parameter
change , two events may change their order. In nite perturbation analysis (FPA), a xed size of is assumed. For
any xed , the event sequence in the perturbed path may
be different from that in the original path. FPA develops some
rules that determine the perturbation when the event order
changes. The FPA algorithm is more complicated than IPA,
and it is usually approximate since only order changes between adjacent events are taken into account (9).
Sample Path Constructability Techniques. Given the nature
of IPA, it cannot be applied to sensitivities with respect to
changes of a xed size or changes in discrete parameters. Motivated by the principles of IPA, we ask the following question: Given a sample path of discrete event system under parameter , it is possible to construct a sample path of the
same system under a different parameter ? This problem is
formulated as the sample path constructability (7). Normally,
such constructability requires that the sets of events and
states of the sample path to be constructed (with parameter
) belong to the sets of events and states of the original sample path. For example, one may construct a sample path for
an M/M/1/K 1 queue (where K 1 denotes the buffer size)
from a sample path of an M/M/1/K queue. Ref. 24 shows that
for some systems with additional computation such sample
path construction can be done even if some states in the sample path to be constructed do not appear in the original sample path.
Techniques in this class include augmented system analysis
(7,25), extended perturbation analysis (26), and the standard
clock approach (27).
Structural Infinitesimal Perturbation Analysis. Structural innitesimal perturbation analysis (SIPA) was developed to address the problem of estimating the performance sensitivity
with respect to a class of parameters such as the transition
probabilities in Markov chains. At each state transition, in
addition to the simulation of the original sample path, an extra simulation is performed to obtain a quantity needed to get
the performance sensitivity. It has been shown that the extra
simulation requires bounded computational effort, and that
in some cases the method can be efcient (28). It is interesting
to note that this approach can be explained by using the concept of realization discussed in the next subsection.
Rare Perturbation Analysis. Bremaud (29) studies the performance sensitivity with respect to the rate of a point process
and proposes the method of rare perturbation analysis (RPA).
The basic idea is that the perturbed Poisson process with
rate with 0 is the superposition of the original
Poisson process with rate and an additional Poisson process
with rate . Thus, in a nite interval, the difference between
the perturbed path and the original one is rare. The performance derivative is then obtained by studying the effect of
these rare but big (meaning nite) perturbations on the system performance. The case 0 is called the positive
RPA.
When 0, the perturbed Poisson process with rate
can be constructed by thinning the original Poisson process with the thinning probability / . That is, some arrival
points in the original process will be taken away. The performance derivative is then obtained by studying the effect of
the removal of these rare arrival points. This is called the
negative RPA. Others in this direction include Refs. 30
and 31.
Estimation of Second Order Derivatives. The single path
based approach can also be used to estimate the second order
derivatives of the performance of a DEDS by calculating the
conditional expectations. See Ref. 32 for GI/G/1 queues and
Ref. 33 for Jackson networks.
Others. In addition to the above direct extensions of IPA,
it also motivated the study of a number of other topics, such
as the Maclaurin series expansion of the performance of some
queueing systems (35), the rational approximation approach
for performance analysis (36), and the analysis of performance discontinuity (37).
Finally, besides the PA method, there is another approach,
called the likelihood ratio (LR) method (3840), that can be
applied to obtain estimates of performance derivatives. The
method is based on the importance sampling technique in
simulation. Compared with IPA, the LR method may be applied to more systems but the variances of the LR estimates
are usually larger than those of IPA.
Perturbation Realization
One important concept regarding the sensitivity of steady
state performance of a DEDS is the perturbation realization. The main quantity related to this concept is called
the realization factor. This concept may provide a uniform
framework for IPA and non-IPA methods. The main idea
is: The realization factor measures the nal effect of a
single perturbation on the performance measure of a DEDS;
the sensitivity of the performance measure with respect to
a parameter can be decomposed into a sum of the nal
effects of all the single perturbations induced by the parameter change.
Perturbation Realization For Closed Jackson Networks. Suppose that at time t 0, the network state is n and server i
obtains a small perturbation , which is the only perturbation
generated on the sample path. This perturbation will be propagated through the sample path according to the IPA propagation rules and will affect system performance. The realiza-
609
tion factor of a perturbation of server i at t 0 with state n,

denoted as c(f)(n, i), is dened as

c
(f)
(n, i) = lim E
L
1

TL
TL
f [N (t)] dt

f [N(t)] dt
(13)
where TL and N(t) represents the quantities in the perturbed path.
A perturbation is said to be realized if at some time Tl
all the servers have the same perturbation ; it is said to be
lost if at some time Tl no server has any perturbation. It was
proved that in an irreducible closed network a perturbation
will be either realized or lost with probability one. The probability that a perturbation is realized is called the realization
probability.
Suppose that a perturbation is realized or lost at TL*. L*
depends on the sample path, that is, . If the perturbation is
lost, then f[N(t)] f[N(t)], for all t TL*; if it is realized,
then f[N(t)] f[N(t )] for all t TL*. Therefore, from the
Markov property, Eq. (13) becomes

c
(f)
n, i) = E
(n
1

TL
TL
f [N (t)] dt

f [N(t)] dt
(14)
where L* is a random number, which is nite with probability one.
Realization factors can be uniquely determined by a set of
linear equations (6). The steady state performance sensitivity
can be obtained by

si ( f )
n )c( f ) (n
n, i)
=
p(n
(I
)
si
all n
(15)
where I(n) 1 for all n and p(n) is the steady state probability of n.
A close examination reveals that the IPA algorithm provides a simple way for estimating the quantity alln
p(n)c(f)(n, i) on a single sample path. The theory has been extended to more general networks, including open networks,
state-dependent networks, and networks with generally distributed service times (6).
Perturbation Realization for Markov Processes. Consider an
irreducible and aperiodic Markov chain X Xn; n 0 on a
nite state space E 1, 2, , M with transition probabilM
ity matrix P [pij]M
i1j1. Let (1, 2, , M) be the vector
representing its steady state probabilities, and f [f(1), f(2),
, f(M)]T be the performance vector, where T represents
transpose and f is a column vector. The performance measure
is dened as its expected value with respect to :
= E ( f ) =
M
i f (i) = f.
(16)
i=1
Assume that P changes to P P Q, with 0 being a

small real number and Qe 0, e (1, 1, , 1)T. The performance measure will change to . We want to estimate the derivative of in the direction of Q, dened as
610
/Q lim0 / . It is well known that IPA does not work

for this problem.
In this system, a perturbation means that the system is
perturbed from one state i to another state j. For example,
consider the case where qki , qkj , and qkl 0 for all
l i, j. Suppose that in the original sample path the system
is in state k and jumps to state i, then in the perturbed path,
it may jump to state j instead. Thus, we study two independent Markov chains X Xn; n 0 and X n; n 0 with
X0 i and X 0 j; both of them have the same transition
matrix P. The realization factor is dened as (34):

di j = E
[ f (Xn ) f (Xn )]|X0 = i, X0 = j
n=0
i, j = 1, 2, . . ., M
(17)
Thus, dij represents the long term effect of a change from i to

j on the system performance. Equation (17) is similar to Eq.
(13).
If P is irreducibile, then with probability one the two sample paths of X and X will merge together. That is, there is a
random number L* such that X L * XL* for the rst time.
Therefore, from the Markov property, Eq. (17) becomes

L
1

di j = E
[ f (Xn ) f (Xn )]|X0 = i, X0 = j
n=0
discrete event systems, analytical approach does not usually

exist for parametric optimization problems. One has to resort
to simulation or experimental approaches, where the derivative estimates obtained by perturbation analysis can play an
important role.
There are two major algorithms used in stochastic optimization: KieferWolfowitz (KW) and RobbinsMonro (RM).
Both are essentially the hill-climbing type of algorithm. The
KW algorithm employs the performance difference as an estimate of the gradient. With PA, we can obtain, based on a
single sample path of a DEDS, the estimates of the gradients.
Thus, the RM algorithm, which is known to be faster than the
KW algorithm, can be used.
Suppose that we want to minimize a performance measure
(), where (1, 2, , M) is a vector of parameters. In
the RM algorithm using PA derivative, the (n 1)th value of
the parameter , n1, is determined by (see, e.g., Ref. 41)
n+1 = n n
n
( )
(22)
where
n
( ) =

n n
n
( ),
( ), ,
( )
1
2
M
g(i) can be estimated on a single sample path by gn(i)

n
El0[f(Xl)]X0 i. There are a few other methods for estimating g and D by using a single sample path.
The potential g satises the Poisson equation
is the estimate of the gradient of the performance function

() at n with each component being the PA estimate, and
n, n 1, 2, , are the step sizes. It usually requires that
n1 n and n1 n2 .
Many results have been obtained in this direction. For example, Ref. 41 studied the optimization of J() T() C()
for a single server queues, where T() is the mean system
time, C() a cost function, and the mean service time. It was
proved that under some mild conditions, the RobbinsMonro
type of algoritm (22) converges even if we update using the
IPA gradient estimate at any random times (e.g., at every
customer arrival time). Other works include Refs. 42, 43, 44,
and 45.
The optimization procedures using perturbation analysis
have been applied to a number of real-world problems. Successful examples include the bandwidth allocation problem in
communications, (46,47), and optimization of manufacturing
systems (4852).
For performance optimization over discrete parameters, for
example, in problems of choosing the best transition matrix,
we may use the approach of realization matrix and potentials
discussed in the last section. It is interesting to note that in
this context, PA is equivalent to the Markov decision process
(MDP) approach. To see this, let be vector of the steady
state probability for the Markov chain with transition matrix
P. From the Poisson equation [Eq. (21)], it is easy to prove
(I P + e )g = f
= Qg
i, j = 1, 2, . . ., M
(18)
which is similar to Eq. (14).

The matrix D [dij] is called a realization matrix, which
satises the Lyapunov equation
D PDPT = F
where F ef T feT, and e (1, 1, , 1)T is a column vector
all of whose components are ones. The performance derivative
is
= QDT T
Q
(19)
Since D is skew-symmetric, that is, DT D, we can write

D egT geT, where g [g(1), g(2), , g(M)]T is called a
potential vector. We have
= Qg.
Q
(20)
(21)
Thus, perturbation realization in a Markov process relates

closely to Markov potential theory and Poisson equations.
APPLICATIONS: ON-LINE OPTIMIZATION
A direct application of perturbation analysis is in the area
of stochastic optimization. Because of the complexity of most
(23)
The right-hand side of Eq. (23) is the same as that of Eq. (20)
except is replaced by . In policy iteration of an MDP problem, we choose the P corresponding to the largest Qg
(P P)g (component-wise) as the next policy. This corresponds to choosing the largest /Q in PA, because all the
components of and are positive. Therefore, the policy iteration procedure in MDP in fact chooses the steepest direction
of the performance measure obtained by PA as the policy in

the next iteration. Thus, in this setting, PA is simply a single
sample-path-based implementation of MDP. Further research
is needed in this direction. Another on-going research related
to DEDS optimization is the ordinal optimization technique
(53), whose main idea is that by softening the goal of optimization and by comparing different schemes ordinally instead
of obtaining the exact performance values, we can dramatically reduce the demand in the accuracy of the estimates.
BIBLIOGRAPHY
1. Y. C. Ho (ed.), Dynamics of Discrete Event Systems, Proc. IEEE,
77: 1232, 1989.
2. P. J. Ramadge and W. M. Wonham, Supervisory control of a class
of discrete event processes, SIAM J. Control Optim., 25: 206
230, 1987.
3. W. H. Fleming, (chair), Future directions in control theoryA
mathematical perspective, Report of the Panel on Future Directions in Control Theory, 1988.
4. R. David and H. Alla, Petri nets for modeling of dynamic systemsA survey, Automatica, 30: 175202, 1994.
5. M. C. Fu and J. Q. Hu, Conditional Monte Carlo: Gradient Estimation and Optimization Applications, Norwell, MA: Kluwer Academic Publishers, 1997.
6. X. R. Cao, Realization Probabilities: the Dynamic of Queueing Systems, New York: Springer-Verlag, 1994.
7. C. G. Cassandras, Discrete Event Systems: Modeling and Performance Analysis, Aksen Associates, Inc., 1993.
8. P. Glasserman, Gradient Estimation Via Perturbation Analysis,
Norwell, MA: Kluwer Academic Publishers, 1991.
9. Y. C. Ho and X. R. Cao, Perturbation Analysis of Discrete-Event
Dynamic Systems, Norwell, MA: Kluwer Academic Publishers,
1991.
10. P. G. Harrison, On normalizing constants in queueing networks,
Oper. Res. 33: 464468, 1985.
11. L. Kleinrock, Queueing Systems, vols. I, II, New York: Wiley,
1975.
12. J. Walrand, An Introduction to Queueing Networks, Englewood
Cliffs, NJ: Prentice Hall, 1988.
13. S. H. Lu and P. R. Kumar, Distributed scheduling based on due
dates and buffer priorities, IEEE Trans. Autom. Control, AC-36:
14061416, 1991.
14. J. L. Peterson, Petri Net Theory and the Modeling of Systems,
Englewood Cliffs, NJ: Prentice Hall, 1981.
15. C. Reutenauer, The Mathematics of Petri Nets, London: PrenticeHall, 1990.
16. M. C. Zhou and F. DiCesare, Petri Net Synthesis for Discrete Event
Control of Manufacturing Systems, Norwell, MA: Kluwer Academic Publisher, 1993.
17. G. Cohen et al., A linear system theoretical view of discrete event
processes and its use for performance evaluation in manufacturing, IEEE Trans. Autom. Control, AC-30: 210220, 1985.
18. F. Baccelli et al., Synchronization and Linearity, New York: Wiley, 1992.
19. Y. C. Ho, A. Eyler, and T. T. Chien, A gradient technique for
general buffer storage design in a serial production line, Int. J.
Production Res., 17: 557580, 1979.
20. X. R. Cao, Convergence of parameter sensitivity estimates in a
stochastic experiment, IEEE Trans. Autom. Control, AC-30: 834
843, 1985.
611
21. X. R. Cao, First-order perturbation analysis of a single multiclass finite source queue, Performance Evaluation, 7: 3141,
1987.
22. P. Heidelberger et al., Convergence properties of infinitesimal
perturbation analysis estimates, Management Science, 34: 1281
1302, 1988.
23. W. B. Gong and Y. C. Ho, Smoothed perturbation analysis of discrete event dynamic systems, IEEE Trans. Autom. Control, 32:
858866, 1987.
24. Y. Park and E. K. P. Chong, Distributed inversion in timed discrete event systems, Discrete Event Dynamic Systems: Theory and
Applications, 5: 219241, 1995.
25. A. A. Gaivoronski, L. Y. Shi, and R. S. Sreenivas, Augmented
infinitesimal perturbation analysis: An alternate explanation,
Discrete Event Dynamic Systems: Theory and Applications, 2: 121
138, 1992.
26. Y. C. Ho and S. Li, Extensions of perturbation analysis of discrete
event dynamic systems, IEEE Trans. Autom. Control, 33: 427
438, 1988.
27. P. Vakili, Using a standard clock technique for efficient simulation, Oper. Res. Lett., 11: 445452, 1991.
28. L. Y. Dai and Y. C. Ho, Structural infinitesimal perturbation
analysis (SIPA) for derivative estimation of discrete event dynamic systems, IEEE Trans. Autom. Control, 40: 11541166,
1995.
29. P. Bremaud, Maximal coupling and rare perturbation sensitivity
analysis, Queueing Systems: Theory and Applications, 10: 249
270, 1992.
30. F. Baccelli, M. Klein, and S. Zuyev, Perturbation analysis of functionals of random measures, Adv. in Appl. Prob., 27: 306325,
1995.
31. F. J. Vazquez-Abad and K. Davis, Efficient implementation of the
phantom RPA method, with an application to a priority queueing
system, Adv. in Appl. Prob., submitted, 1995.
32. M. A. Zazanis and R. Suri, Perturbation analysis of the GI/G/1
queue, Queueing Systems: Theory and Applications, 18: 199
248, 1994.
33. B. Gang, C. Cassandras, and M. A. Zazanis, First and second
derivative estimators for closed Jackson-like queueing networks
using perturbation analysis techniques, Discrete Event Dynamic
Systems: Theory and Applications, 7: 2968, 1997.
34. X. R. Cao and H. F. Chen, Perturbation realization, potentials,
and sensitivity analysis of Markov processes, IEEE Trans. Autom.
Control, 42: 13821393, 1997.
35. W. B. Gong and S. Nananukul, Rational interpolation for rare
event probabilities, Stochastic Networks: Stability and Rare Event,
New York: Springer, 1996.
36. W. B. Gong and J. Q. Hu, On the MacLauring Series of GI/G/1
Queue, J. Appl. Prob., 29: 176184, 1991.
37. X. R. Cao, W. G. Gong, and Y. Wardi, Ill-conditioned performance
functions of queueing systems, IEEE Trans. Autom. Control, 40:
10741079, 1995.
38. P. W. Glynn, Likelihood ratio gradient estimation: An overview,
Proc. Winter Simulation Conf., 366375, 1987.
39. M. I. Reiman and A. Weiss, Sensitivity analysis via likelihood
ratio, Oper. Res., 37: 830, 844, 1989.
40. R. Y. Rubinstein, Sensitivity analysis and performance extrapolation for computer simulation models, Oper. Res., 37: 7281,
1989.
41. E. K. P. Chong and P. J. Ramadge, Optimization of queues using
infinitesimal perturbation analysis-based algorithms with general update times, SIAM J. Control Optim., 31: 698732, 1993.
42. Y. C. Ho and X. R. Cao, Perturbation analysis and optimization
of queueing networks, J. Optim. Theory Appl., 40: 559582, 1983.
612
DISCRETE EVENT SYSTEMS
43. C. G. Cassandras and S. G. Strickland, On-line sensitivity analysis of Markov chains, IEEE Trans. Autom. Control, 34: 7686,
1989.
44. R. Suri and Y. T. Leung, Single run optimization of discrete event
simulationsan empirical study using the M/M/1 queue, IIE
Trans., 21: 3549, 1989.
45. Q. Y. Tang and H. F. Chen, Convergence of perturbation analysis
based optimization algorithm with fixed number of customers period, Discrete Event Dynamic Systems: Theory and Applications,
4: 359375, 1994.
46. C. A. Brooks and P. Varaiya, Using perturbation analysis to solve
the capacity and flow assignment problem for general and ATM
networks, IEEE Globcom, 1994.
47. N. Xiao, F. F. Wu, and S. M. Lun, Dynamic bandwidth allocation
using infinitesimal perturbation analysis, IEEE Infocom, 383
389, 1994.
48. M. Caramanis and G. Liberopoulos, Perturbation analysis for the
design of flexible manufacturing system flow controllers, Oper.
Res., 40: 11071125, 1992.
49. A. Haurie, P. LEcuyer, and C. van Delft, Convergence of stochastic approximation coupled with perturbation analysis in a class
of manufacturing flow control models, Discrete Event Dynamic
Systems: Theory and Applications, 4: 87111, 1994.
50. H. M. Yan and X. Y. Zhou, Finding optimal number of Kanbans
in a manufacturing system via perturbation analysis, Lecture
Notes in Control and Information Sciences, 199, Springer-Verlag,
572578, 1994.
51. H. M. Yan, G. Yin, and S. X. C. Lou, Using stochastic optimization to determine threshold values for the control of unreliable
manufacturing systems, J. Optim. Theory Appl., 83: 511539,
1994.
52. N. Miyoshi and T. Hasegawa, On-line derivative estimation for
the multiclass single-server priority queue using perturbation
analysis, IEEE Trans. Autom. Control, 41: 300305, 1996.
53. Y. C. Ho, Soft Optimization for Hard Problems, Computerized
lecture via private communication, 1996.
XI-REN CAO
The Hong Kong University of
Science and Technology
480
FILTERING AND ESTIMATION, NONLINEAR

To have a historical perspective of the advent of nonlinear
filtering and estimation, initially the development of linear
filtering and estimation is described. The first studies of linear filtering or linear estimation for stochastic processes were
made by Kolmogorov (1,2), Krein (3,4) and Wiener (5). The
research of Kolmogorov and Krein and the research of Wiener
were done independently. Kolmogorov, who was motivated by
Wold (6), gave a solution to the prediction problem for discrete-time stochastic processes. Since Kolmogorov and Krein
were not motivated for their work by any specific applications,
the formulae for the optimum predictor did not play a special
role. However, Wiener was motivated for his work during
World War II by the analysis of anti-aircraft fire-control problems from ships. He solved the continuous-time linear prediction problem and derived an explicit formula for the optimum
predictor. He also solved the filtering problem of estimating a
stochastic signal process that is corrupted by an additive
noise process. In this latter case Wiener expressed the solution in terms of an integral equation, the WienerHopf equation (7). Wiener had obtained this equation in his work on
potential theory a number of years earlier. This relation alludes to the probabilistic interpretation of potential theory using Brownian motion (8). Wieners book (5) contains a number
of elementary, explicitly solvable examples.
The sum of the signal process and the additive noise process is called the observation process. The prediction problem
is to estimate the signal process at some future time usually
on an ongoing basis from the observation process. The filtering problem is to estimate the signal process at the same
time. The smoothing problem has a couple of variants: (1)
Given the observation process in a fixed time interval, estimate the signal process at each element in the time interval,
and (2) estimate the signal process at a time that is a fixed lag
behind the observation process. The approach of Kolmogorov,
Krein, and Wiener to these problems assumed that the stochastic processes are (wide-sense) stationary and that the infinite past of the observation process is available. Both of
these assumptions are not physically reasonable, so there was
a need to relax these assumptions.
In the late 1950s, control and system theory were undergoing a significant change from the frequency-domain approach
to the state-space approach. Transfer function descriptions of
linear systems were replaced by ordinary differential equation descriptions of linear systems. This state-space approach
provided an impetus to reexamine the linear filtering problem. Using this approach the signal process is modeled as the
solution of a linear differential equation with a Gaussian
white noise input so the signal process is GaussMarkov. The
differential of the observations process is a linear transformation of the signal process plus Gaussian white noise. This filtering model does not require the infinite past of the observations. The signal and the observation processes can evolve
from some fixed time with a Gaussian random variable as the
initial condition for the differential equation that describes
the signal process. The processes are not required to be stationary; and, in fact, the coefficients of the differential equation for the signal process and the linear transformation for
the signal in the observation equation can be time-varying.
While it is not required that the ordinary differential equation for the signal process be stable, which is implicit in the
description for stationary processes, it is necessary to be able
to model the signal process as the solution of an ordinary differential equation with a white noise input. In general a stationary Gaussian process may not have such a model.
With the success of these linear filtering results that were
developed particularly by Kalman (9) for discrete-time processes and by Kalman-Bucy (10) for continuous-time processes, an interest developed in trying to solve a filtering
problem where the signal is a solution to a nonlinear differential equation with a white noise input. It is natural to call
such a problem a nonlinear filtering problem. The precise description of such a problem required the introduction of a significant amount of the modern theory of stochastic processes.
The major technique for describing the signal process is the
theory of stochastic differential equations that was initiated
by K. Ito (11).
The Gaussian white noise processes that appear as inputs
in the nonlinear differential equations require more sophisticated mathematical methods than do the inputs to linear differential equations. This occurs because the linear transformations of white noise have one natural interpretation but
the nonlinear transformations of white noise have no single
natural interpretation.
Interestingly, it was Wiener (12) who first constructed the
basic sample path property of the integral of Gaussian white
noise that is called the Wiener process or Brownian motion
and which provided the basis for the interpretations of nonlinear transformations of white noise. Many important properties of Brownian motion were determined by P. Levy (13). The
solution of a stochastic differential equation (a nonlinear differential equation with a white noise input) required the theory of stochastic integrals (14), which depends on some martingale theory (15) associated with Brownian motion.
No one definition of stochastic integrals arises naturally
from the Riemann sum approximations to the stochastic integral. This phenomenon occurs because Brownian motion does
not have bounded variation. The definition of K. Ito (14) is
the most satisfying probabilistically because it preserves the
martingale property of Brownian motion and Wiener integrals (stochastic integrals with deterministic integrands).
However, the calculus associated with the Ito definition of stochastic integral is somewhat unusual. The FiskStratonovich
definition of stochastic integral (16,17) preserves the usual
calculus properties, but the family of integrable functions is
significantly smaller. An uncountable family of distinct definitions of stochastic integrals can be easily exhibited (18).
This choice or ambiguity in the definition of a stochastic integral has played an important role in nonlinear filtering because initially some nonlinear filtering solutions were given
without specifying the interpretation or the definition of the
stochastic integrals. This ambiguity often arose by a formal
passage to the limit from discrete time.
In general, to compute conditional statistics of the state
process given the observation process, it is necessary to compute the conditional density of the state process given the observation process. For linear filtering the signal and the observation processes are Gaussian, so the conditional density
is determined by the conditional mean and the conditional
covariance. The conditional covariance is not random, so it
does not depend on the observation process. The conditional
mean can be shown to satisfy a stochastic differential equation that models the signal process and that has the observations as the input. These two conditional statistics (i.e., function of the observations are called sufficient conditional
statistics (19) because the conditional density can be recovered from them. For nonlinear filtering the solution does not
simplify so easily. In general there is no finite family of sufficient conditional statistics for a nonlinear filtering problem.
The conditional density can be shown to satisfy a nonlinear
stochastic partial differential equation (20,21). This equation
is especially difficult to solve because it is a stochastic partial
differential equaton and it is nonlinear. Even approximations
are difficult to obtain. The conditional density can be expressed using Bayes formula (22,23), so that it has the same
form as the Bayes formula in elementary probability though
it requires function space integrals. The numerator in the
Bayes formula expression for the conditional density is called
the unnormalized conditional density. This unnormalized conditional density satisfies a stochastic partial differential equation that is linear. It is called the DuncanMortensenZakai
(DMZ) equation of nonlinear filtering (2426).
In nonlinear filtering, the question of finite dimensional
filters describes the problem of finding finite dimensional solutions to the DMZ equation or to finite families of conditional
statistics. A basic approach to this question on the existence
or the nonexistence of finite-dimensional filters is the estimation algebra (27,28,28a), which is a Lie algebra of differential
operators that is generated by the differential operators in
the DMZ equation. Some families of nonlinear filtering problems have been given that exhibit finite-dimensional filters
(e.g., see Ref. 29).
481
Some methods from algebraic geometry have been used to

give necessary and sufficient conditions on the coefficients of
the stochastic equations for small state-space dimension, so
that the nonlinear filtering problem has finite-dimensional
filters.
Since the results for the existence of finite-dimensional filters for nonlinear filtering problems are generally negative,
many approximation methods have been developed for the
numerical solution of the DMZ equation or the equations for
some associated conditional statistics. In their study of Wiener space (the Banach space of continuous functions with the
uniform topology and the Wiener measure), Cameron and
Martin (30) showed that any square integrable functional on
Wiener space could be represented as an infinite series of
products of Hermite polynomials (Wick polynomials). K. Ito
(31) refined this expression by using an infinite series of multiple Wiener integrals. The relation between these two representations is associated with including or excluding the diagonal in multiple integration. This relation carries over to
Stratonovich integrals, and the explicit relation between
these two stochastic integrals in this case is given by the Hu
Meyer formula (32).
For the solution of the linear filtering problem it was well
known from the early work that the observation process in
the optimal filter appears with a linear transformation of the
estimate as a difference and that this difference is a process
that is white with respect to the observation process. Since a
family of square integrable zero mean random variables generates a vector space with an inner product that is the expectation of a product of two of the random variables, the (linear)
filtering problem can be posed as a projection problem in a
vector space (Hilbert space). The occurrence of a process that
is white with respect to the observations is natural from a
GramSchmidt orthogonalization procedure and projections.
This process has been historically called the innovation process (6). For Wiener filtering, this innovations approach was
introduced in the engineering literature by Bode and Shannon (6a). For linear filtering it is straightforward to verify
that the observation process and the innovation process are
equivalent (that is, there is a bijection between them) by
showing that a linear operator is invertible.
For nonlinear filtering there is still an innovation process.
It is more subtle to verify that the observation process and
the innovation process are equivalent (33). Thus the nonlinear filtering solution has a vector space interpretation via orthogonalization and projections as for the linear filtering solution. However, this is not surprising because in both cases
there is a family of (square integrable) random variables and
the conditional expectation is a projection operator. This occurrence of the innovation process can be obtained by an absolute continuity of measures (34). In information theory, the
mutual information for a signal and a signal plus noise can
be computed similarly (35).
The expression for the conditional probability or the conditional density, given the past of the observations as a ratio of
expectations, has a natural interpretation as a Bayes formula
(22,23) that naturally generalizes the well-known Bayes formula of elementary probability.
The stochastic partial differential equations for the conditional probability density or the unnormalized conditional
probability density are obtained by the change of variables
formula of K. Ito (36).
482
The fact that the sample paths of Brownian motion (the

formal integral of Gaussian white noise) do not have bounded
variation has important implications concerning robustness
questions. Wong and Zakai (37) showed that if Brownian motion in a stochastic differential equation is replaced by a sequence of piecewise smooth processes that converge (uniformly) to Brownian motion, then the corresponding sequence
of solutions of the ordinary differential equations obtained
from the stochastic differential equation by replacing the
Brownian motion by the piecewise smooth processes do not
converge to the solution of the stochastic differential equation
for many nonlinear stochastic differential equations. This result of Wong and Zakai has important implications for nonlinear filtering. If nonlinear filters are constructed from time discretizations of the processes and a formal passage to the limit
is made, then it may not be clear about the interpretation of
the resulting solution. This question is closely related to the
choice of the definition of stochastic integrals and the unusual
calculus that is associated with K. Itos definition. In the early
development of nonlinear filtering theory, solutions were
given that did not address the question of the definition of
stochastic integrals. Generally speaking, formal passages to
the limit from time discretizations require the Stratonovich
definition of stochastic integrals because these integrals satisfy the usual properties of calculus.
One classical example of the use of nonlinear filtering in
communication theory is the analysis of the phase-lock loop
problem. This problem arises in the extraction of the signal
in frequency modulation (FM) transmission. The process that
is received by the demodulator is a sum of a frequency modulated sinusoid and white Gaussian noise. The phase-lock demodulator is a suboptimal nonlinear filter whose performance
is often quite good and which is used extensively in FM radio
demodulation circuits.
If the state of an nth-order linear stochastic system is reconstructed from samples of the output by well-known numerical differentiation schemes, then even in the limit as the
sampling becomes arbitrarily fine, well-known computations
such as quadratic variation do not converge to the desired
results (8). This phenomenon did not occur for linear stochastic differential equations in the approach in Ref. 37.
spectively. The following assumptions are made on the coefficients of the stochastic differential equations (1) and (2) that
describe the signal or state process and the observation process, respectively.
The stochastic processes are defined on a fixed probability
space (, F , P) with a filtration (F t, t 0). Often the space
can be realized as a family of continuous functions. The
-algebras are assumed to be complete with respect to the
probability measure P.
NONLINEAR FILTERING PROBLEM

FORMULATION AND MAIN RESULTS
A transition probability measure or a transition probability function for a Markov process is a function P(s, x; t, ) for
s [0, t), x d, and Bd the Borel -algebra on d that
satisfies the following:
In this section a nonlinear filtering problem is formulated

mathematically and many of the main results of nonlinear
filtering are described.
A basic nonlinear filtering problem is described by two stochastic processes: (X(t), t 0), which is called the signal or
state process; and (Y(t), t 0), which is called the observation
process. These two processes satisfy the following stochastic
differential equations:
dX (t) = a(t, X (t)) dt + b(t, X (t)) dB(t)
dY (T ) = h(t, X (t),Y (t)) dt + g(t, Y (t)) dB(t)
(1)
A1. The drift vector a(t, x) in Eq. (1) is continuous in t and

globally Lipschitz continuous in x. The vector a(t, x)
is continuous in t and globally Lipschitz continuous
in x.
A2. The diffusion matrix b(t, x) in Eq. (1) is Holder continuous in t, globally Lipschitz continuous in x, and globally bounded. The symmetric matrix c bTb is strictly
positive definite uniformly in (t, x). The terms
ci j (t, x)
xi
2 ci j (t, x)
xi x j
i, j {1, . . ., n}
are continuous in t, globally Lipschitz continuous in

x, and globally bounded.
A3. The drift vector h(t, x, y) and the diffusion matrix g(t,
y) in Eq. (2) are continuous in x. The symmetric matrix f gTg is strictly positive definite uniformly in
(t, x); that is, fx, x cx2, where c 0.
The global Lipschitz conditions on the coefficients of the
stochastic differential equations in the space variables x and
y ensure the existence and the uniqueness of the strong (i.e.,
sample path) solutions of these equations. The additional
smoothness properties are used to verify properties of the
transition density of the Markov process (X(t), Y(t), t 0).
Since the soluton of Eq. (1) is generated by x0 and (B(t), t
0), the process (X(t), t 0) is independent of the Brownian
(t), t 0) and the process (Z(t), t 0) where
motion (B
dZ(t) = g(t, Z(t))dB(t)
(3)
1. P(s, x; t, ) is a probability measure on (d, Bd) for all

s [0, t).
2. P(s, ; t, ) is Bd-measurable for all s [0, t) and
Bd.
3. If s [0, t), u t and Bd, then

P(s, x; u, ) =
P(t, y; u, )P(s, x; t, dy)
(4)
(2)
where t 0, X(0) x0, Y(0) 0, X(t) n, Y(t) m, a:

n n, b: n L (n, n), h: n m
(t), t 0) are
m, g: m L (m, m), (B(t), t 0), and (B
independent, standard Brownian motions in n and m, re-
With the notion of transition probability measure (function), a Markov process can be defined.
Definition. Let P(s, x; t, ) be a transition probability measure and be a probability measure on (d, Bd). A probability
measure P on ((d)t, B(d) ) is called a Markov process with

transition function P(s, x; t, ) and initial distribution if
t
P(X (0) ) = ( ),
L=
and for each s [0, t) and Bd

P(x(t) | (X (u), 0 u s)) = P(s, X (s); t, )
(5)
The random variable X(t) is the evaluation of an element

on (d)t at t . Usually the Markov process is identified
as (X(t), t 0) and the Markov property (5) is described as
P(X (t) |X (u), 0 u s) = P(s, X (s); t, )
(6)
If P(s, x; t, ) P(t s, x, ), then the Markov process is said

to be (time) homogeneous.
If (X(t), t 0) is a homogeneous Markov process, then
there is a semigroup of operators (Pt, t 0) acting on the
bounded Borel measurable functions (39), which is given by

(Pt )(x) = Ex [ (X (t))] = (y)P(0, x; t, dy)
(7)
Consider the restriction of (Pt, t 0) to the bounded, continuous functions that vanish at infinity which is a Banach space
in the uniform topology. If (X(t), t 0) is a Markov process
that is the solution of the stochastic differential equation
dX (t) = a(X (t)) dt + b(X (t)) dB(t)
where a( ) and b( ) satisfy a global Lipschitz condition, then
the semigroup (Pt, t 0) has an infinitesimal generator that
is easily computed from Itos formula (36); that is,
Pff
L f = lim t
t0
t

f : lim
t0
d

i=1
ai

Pt f f
exists
t
2
1
+
ci j
xi
2
xi x j
(9)
where c bTb.
An analogous result holds if the Markov process is not homogeneous, so that
dX (t) = a(t, X (t)) dt + b(t, X (t)) dB(t)
where the theory of two-parameter semigroups is used so that
Ps,t f (x) = EX (s)=x [ f (X (t))]
and
dPs,t f
= Lf
dt
ai
1
2
+
ci j
xi
2
xi x j
(10)
In the filtering solution the formal adjoint of L, L*, appears;

that is,
L =

1 2
(ai ) +
(c )
xi
2
xi x j i j
(11)
The operator L is often called backward operator, and L* is

called the forward operator.
The stochastic integrals that occur in the solution of Eq.
(1) are interpreted using the definition of K. Ito (14). For a
smooth integrand the integral is the limit of Riemann sums
where the integrand is evaluated at the left endpoint assuming that the integrand is suitably measurable. This definition
preserves the martingale property of Brownian motion.
Let (M(t), t 0) be an Ito process or a semimartingale,
that is
dM(t) = (t) dt + (t) dB(t)
(12)
dM(t) = dA(t) + dN(t)
(13)
or
where (A(t), t ) is a process of bounded variation and (N(t),

t 0) is a martingale. The FiskStratonovich or Stratonovich
integral (16,17) of a suitable stochastic process ((t), t 0) is
denoted as

(s) dM(s)
(14)
(8)
It is straightforward to verify that
L=
for
f D(L) =
where
483
This integral is defined from the limit of finite sums that are
formed from partitions as in RiemannStieltjes integration
where the function is evaluated at the midpoint of each interval formed from the partition. Recall that the Ito integral
is formed by evaluating the integrand at the left endpoint of
each of the subintervals formed from the partition (40).
For the linear filtering problem of GaussMarkov processes it is elementary to show that the conditional probability density is Gaussian so that only the conditional mean and
the conditional covariance have to be determined. Furthermore, the estimate of the state, given the observations that
minimizes the variance, is the conditional mean. Thus one
approach to the nonlinear filtering problem is to obtain a stochastic equation for the conditional mean or for other conditional statistics. The difficulty with this approach is that typically no finite family of equations for the conditional statistics
is closed; that is, any finite family of equations depends on
other conditional statistics.
The conditional probability density of the state X(t), given
the observations (Y(u), 0 u t), is the density for the conditional probability that represents all of the probabilistic information about X(t) from the observations (Y(u), 0 u t). A
conditional statistic can be computed by integrating the conditional density with a suitable function of the state.
To obtain a useful expression for the conditional probability measure, it is necessary to use a result for the absolute
484
continuity of measures on the Borel -algebra of the space of

continuous functions with the uniform topology. These results
center around the absolute continuity for Wiener measure,
the measure for Brownian motion. The first systematic investigation of Wiener measure in this context was done by Cameron and Martin (30), who initiated a calculus for Wiener
measure. Subsequently, much work was done on general
Gaussian measures (e.g., see Ref. 41).
For Wiener measure and some related measures, a more
general probabilistic approach was given by Skorokhod
(42,43) and Girsanov (44). The following result is a version of
Girsanovs result.
Theorem. Let (, F , P) be a probability space with a filtration (F t, t [0, T]). Let ((t), t [0, T]) be an n-valued
process that is adapted to (F t, t [0, T]) and let (B(t), t [0,
T]) be an n-valued standard Brownian motion. Assume that
E[M(T )] = 1
A result for the absolute continuity of measures is given

that follows from the result of Girsanov (44). For convenience
it is stated that X V, though it easily follows that XY
VZ and in fact there is mutual absolute continuity.
Theorem. Let V and X be the probability measures on (,
F ) for the process (V(t), t [0, T]) and (X(t), t [0, T]), respectively, that are solutions of Eqs. (18) and (1). Then X is
absolutely continuous with respect to V, denoted X V,
and
dX
= (T )
dV
where is given by Eq. (19).
Corollary. Let (V(t), t [0, T]) satisfy (18) on (, F , V).
Then
(15)
B(t)
= B(t)
b1 (s, B(s))a(s, B(s)) ds
where

M(t) = exp
(s), dB(s)
0
1
2

|(s)|2 ds
is a Brownian motion on (, F , X).

(16)
Then the process (Y(t), t [0, T]) given by

Y (t) = B(t)
(s) ds
(17)
is a standard Brownian motion for the probability P, where

dP M(T) dP.
Let Z be the probability measure on the Borel -algebra
of m-valued continuous functions for the process (Z(t), t 0)
that is the solution of Eq. (3). Let (V(t), t 0) be the process
that is the solution of
dV (t) = b(t,V (t)) dB(t)
V (0) = x0
(18)
Let XY be the measure on the Borel -algebra of nm-valued

continuous functions for the process (X(t), Y(t), t 0) that
satisfy Eqs. (1) and (2). It follows from Girsanovs Theorem
above that XY V Z. The RadonNikodym derivative
(t) (t)(t) E[dXY/d(VZ) F t] is

(t) = exp
1

c1 (s, X (s))a(s, X (s)), a(s, X (s)) ds
1
2
(19)

(t) = exp
c1 (s, X (s))a(s, X (s)), dX (s)
0
t
Proposition. For t 0 the conditional probability measure

of X(t), given (Y(u), 0 u t), is given by
P(, t | x0 , Yu , 0 u t) =
E Z [1 t t ]
E Z [t t ]
(21)
for (X(t)), the -algebra generated by X(t).

The absolute continuity of measures and the associated
RadonNikodym derivatives are important objects even in elementary probability and statistics. In this latter context
there is usually a finite family of random variables that have
a joint density with respect to Lebesgue measure. The likelihood function in statistical tests is an example of a computation of a RadonNikodym derivative.
The conditional probability measure Eq. (21) is specialized
to the conditional probability distribution and a density is
given for this function. The conditional density is shown to
satisfy a stochastic partial differential equation (17, 20, 46).
Theorem. Let (X(t), Y(t), t 0) be the processes that satisfy
Eqs. (1) and (2). If A1A5 are satisfied, then
d p(t) = L p + f 1 (t, Y (t))(g(t) g(t)),

dY (t) g(t) p(t)
(22)
f 1 (s, Y (s))g(s, X (s), Y (s)), dY (s)
0
t

f 1 (s, Y (s))g(s, X (s),Y (s)), g(s, X (s), Y (s)) ds
It can be shown that the linear growth of the coefficients

ensures that there is absolute continuity, so that Eq. (15) is
satisfied (45).
The following result gives the conditional probability measure in function space (22,23).
(20)
To indicate the expectation with respect to one of the function
space measures, E is subscripted by the measurefor example, EX.
where
p(t) = p(X (t), t | x0 , Y (u), 0 u t)
(23)
g(t) = g(t, X (t), Y (t))
(24)
=
g(t)
E X [ (t)g(t)]
E X [ (t)]
(25)
485
Equation (22) is a nonlinear stochastic partial differential

equation. The nonlinearity occurs from the terms g(t)p(t), and
the partial differential operator L* is the forward differential
operator for the Markov process (X(t), t 0).
Often only some conditional statistics of the state given the
observations are desired for the nonlinear filtering problem
solution. However, such equations are usually coupled to an
infinite family of conditional statistics. The following theorem
describes a result for conditional statistics (47,48).
where pX is the transition density for the Markov process

(X(t), t 0).
Assume that A1A3 are satisfied. Then r satisfies the following linear stochastic partial differential equations:
Theorem. Let (X(t), Y(t), t 0) satisfy Eqs. (1) and (2). Assume that a(t, x) and b(t, x) in Eq. (1) are continuous in t and
globally Lipschitz continuous in x, h(t, x, y) is continuous in t
and globally Lipschitz continuous in x and y, and g(t, y) is
continuous in t and globally Lipschitz continuous in y and
f gTg is strictly positive definite uniformly in (t, y). If
C2(n, ) such that
q(t) = E X [ (t)]
E| (X (t))| dt <
(26)
|D (X (t)), X (t) |2 dt <
(27)
|D2 (X (t))X (t), X (t) |2 dt <
(28)
dr(X (t), t | x0 , Y (u), 0 u t)

= L r + f 1 (t, Y (t))g(t, X (t),Y (t)), dY (t) r
(32)
The normalization factor for r is

(33)
so that
p(x, t | x0 , Y (u), 0 u t) = r(t)q1 (t)
It is elementary to obtain a stochastic equation for q1(t)

using Itos formula; that is,
dq1 (t) = q2 (t) dq(t)

+ q3 (t) f 1 (t)E X [ (t)g(t)], E X [ (t)g(t)] dt
(34)
E
0
where
then the conditional expectation of (X(t)), given the observations (Y(u), u t),
dq(t) = f 1 (t) (t)g(t), dY (t)
(t) = E[ (X (t)) | x0 , Y (u), 0 u t]
To apply algebro-geometric methods to the nonlinear filtering problem the following form of the DMZ equation is
used
satisfies the stochastic equation
drt = [L (1/2)gt , gt ]rt dt + rt gtT dY (t)
d (t) = L (X (t)) dt + f 1 (t,Y (t)) g(t, X (t),Y (t))

X (t),Y (t)), dY (t) g(t,
X (t), Y (t)) dt
(X (t))g(t,
(29)
where L is the backward differential operator for the Markov
process (X(t), t 0) and is conditional expectation for
example,
(X (t)) =
E X [ (X (t)) (t)]
E X [ (t)]
(30)
The stochastic equation for the condition probability density is a nonlinear stochastic partial differential equation. The
stochastic equation for a conditional statistic is typically coupled to an infinite famiily of such equations. The conditional
density is more useful because it represents all of the probabilistic information about the state given the observations, but
it is a nonlinear equation. If the so-called unnormalized conditional density is used, then the stochastic partial differential
equation is linear. This unnormalized conditional density was
given by Duncan (24), Mortensen (25), and Zakai (26). The
equation is usually called the DuncanMortensenZakai
(DMZ) equation.
Theorem. Let (X(t), Y(t), t 0) be the processes that are the
solutions to Eqs. (1) and (2). Let r be given by
r(x, t | x0 , Y (u), 0 u t) = E X [ (t) | X (t) = x]pX (0, x0 ; t, x)
(31)
(35)
(36)
Recall that the symbol in Eq. (36) indicates the Stratonovich

integral. The reason that this form of the DMZ equation is
sometimes more useful is that it satisfies the usual rules of
calculus. Thus the Lie algebras can be computed in the same
way that would be used for smooth vector fields.
A more elementary nonlinear filtering problem than the
one for diffusion processes that is important for applications
is the case where the signal or state process is a finite-state
Markov process (in continuous time). The finite-state space
for the process significantly reduces the mathematical difficulties. Let S s1, . . ., sn be the state space for the finite
state Markov process and pi(t) P(X(t) si) and p(t)
[p1(t), . . ., pn(t)]T. It follows that
d
p(t)
= A p(t)
dt
(37)
where A is the intensity matrix or the transpose of the generator of the Markov process (X(t), t 0). The dependence of p
on the initial value X(0) has been suppressed for notational
convenience. By analogy with the DMZ equations (31) and
(36) in this case for the finite-state Markov process (49) it
follows that the unnormalized conditional density (t) satisfies

(t) = (0) +
A(s) ds +
0
B(s) dY (s)
0
(38)
486
or
(t) = (0) +
(A (1/2)B2 )(s) ds +
0
B(s) dY (s)
0
(39)
where B diag(s1, . . ., sn) and (t) [1(t), , n(t)]T. These

equations are a finite family of bilinear stochastic differential
equations for the unnormalized conditional probabilities. The
conditional expectation of the statistic :S , denoted
t(), is
t () =
n
i
i=1 (si ) (t)
n
i
i=1 (t)
(40)
A Lie algebra associated with the DMZ equation (36) plays

a basic role in determining the existence or the nonexistence
of finite-dimensional filters for conditional statistics of the signal (or state) process. To introduce Lie algebras, its definition
is given.
Definition. A Lie algebra V over a field k is a vector space
over k with a bilinear form []:V V V (the Lie bracket)
that satisfies for v1, v2, v3 V the following:
1. [v1, v2] [v2, v1],
2. [v1, [v2, v3]] [v2, [v3, v1]] [v3, [v1, v2]] 0.
A Lie subalgebra of a Lie algebra V is a linear subspace of
V that is a Lie algebra. If I, a subalgebra of V, is an ideal of
V, then the quotient algebra is V/I, a vector space with the
induced Lie bracket. A Lie algebra homomorphism :V1 V2
of the Lie algebras V1 and V2 is a linear map that commutes
with the bracket operations, ([u, v]) [(u), (v)].
The algebro-geometric methods for the nonlinear filtering
problem arose from the system theory for finite-dimensional,
nonlinear, affine, deterministic control systems. Consider a
deterministic control system of the form
m

dx
= f (x(t)) +
ui (t)gi (x(t))
dt
i=1
(41)
where x(t) M, a smooth d-dimensional manifold. A controllability property has local significance in analogy to its global
significance for linear control systems.
Definition. The controllability Lie algebra of Eq. (41) is the
Lie algebra L generated by f, g1, . . ., gm.L (x) is the linear
space of vectors in TxM, the tangent space of M at x, spanned
by the vector fields of L at x. The dimension of L (x) has implication for the local reachable set starting at x M.
Another basic notion in system theory is observability.
This condition implies that different statesthat is, different points in Mcan be distinguished using an appropriate
control.
Definition. Consider the control system (41). Let h
C(M, ) give the observation as
y(t) = h(x(t))
The system (41) and (42) is said to be observable under the

following condition: If x1, x2 M and x1 x2, then there is an
input (control) function u such that the outputs associated
with x1 and x2 are not identical.
(42)
By analogy to the structure theory of linear systems, there

is a state-space isomorphism theorem; that is, given two
systems of the form (41) and (42) on two analytic manifolds
such that the coefficients are complete analytic vector fields,
the systems are observable and dim L (x) is minimal, and the
two systems realize the same inputoutput map, then there
is an analytic map between the manifolds that preserves trajectories (50). Associated with the equivalence of systems of
the form (41) and (42) there is a realization of such systems
that is observable and dim L (x) n; that is, the Lie algebra
of vector fields evaluated at each point in the manifold has
maximal dimension.
In general, the DMZ equation (36) can be viewed as the
state equation of an infinite-dimensional system for (t) with
the input function from the observation Y and the output
t() (27,51). An investigation of the existence or the nonexistence of a finite dimensional realization of the inputoutput
map is the investigation of the existence of finite-dimensional filters.
A finite-dimensional (recursive) filter for t() is a stochastic equation
d(t) = a((t)) dt + b((t)) dY (s)
(43)
t () = ((t))
(44)
where (t) n.
The application of the Lie algebraic methods described
above and the use of nonlinear system theory presupposed a
finite-dimensional manifold. For the DMZ equation (36) the
solution evolves in an infinite-dimensional manifold. Thus it
is necessary to be precise when translating these finite-dimensional algebro-geometric results to the DMZ equation. If
this approach can be applied to the DMZ equation, then the
questions of the existence or the nonexistence of finite-dimensional stochastic equations for conditional statistics and the
equivalence of two nonlinear filtering problems can be resolved. Even for finite-state Markov processes it can be determined if some conditional statistics are the solutions of stochastic equations whose dimension is significantly smaller
than the number of states of the Markov process (52).
For the DMZ equation (36) by analogy with the finite-dimensional inputoutput systems (41) and (42), the Lie algebra generated by the operators L* (1/2)h, h and h,
acting on smooth (C) functions is called the estimation algebra associated with Eqs. (41) and (42) (53,54,54a).
To identify equivalent filtering problems it is important to
investigate transformations that induce isomorphic estimation algebras. A simple, important transformation is a change
of scale of the unnormalized conditional probability density
r( ). Let :n be a strictly positive, smooth function and
let r(t) r(t). This transformation acts on the generators of
the estimation algebra as
L 1
1
h, h
2
g1 ,
Thus the estimation algebras are formally isomorphic. Furthermore, a smooth homeomorphism of the state space induces an estimation algebra that is formally isomorphic to the
initial estimation algebra. The above two operations on the
estimation algebra have been called the estimation formal
equivalence group (55).
If for some distribution of X(0) a conditional statistic,
t(), can be described by a minimal finite-dimensional (recursive) filter of the form (43) and (44), then the Lie algebra
of this system should be a homomorphism of the estimation
algebra for this filtering problem. This property has been
called the homomorphism principle for the filtering problem
(56). This homomorphism principle can be a guide in the investigation of the existence of finite-dimensional filters.
A specific example of this homomorphism property occurs
when the estimation algebra is one of the Weyl algebras. The
Weyl algebra Wn is the algebra of polynomial differential operators over with operators x1, . . ., xn, /x1, . . ., /xn.
The Lie bracket is the usual commutator for differential operators. This Lie algebra has a one-dimensional center and the
quotient Wn / is simple; that is, it contains no nontrivial ideals. For the estimation algebra these two properties imply
that if Wn is the estimation algebra for a filtering problem,
then either the unnormalized conditional density can be computed by a finite-dimensional filter or no conditional statistic
can be computed by a finite-dimensional filter of the form (43)
and (44). More specifically, for n 0 there are no nonconstant
homomorphisms from Wn or Wn / to the Lie algebra of smooth
vector fields on a smooth manifold (57).
As an example of a Weyl algebra occurring as an estimation algebra, consider
dX (t) = dB(t)
(45)
dY (t) = X 3 (t) dt + dB(t)
(46)
It is straightforward to verify that the estimation algebra for

this filtering problem is the Weyl algebra W1. The homomorphism principle that has been described can be verified in
principle for this estimation algebra to show that there are
no nontrivial conditional statistics that can be computed with
finite-dimensional filters of the form (43) and (44) (58).
It is natural to consider the linear filtering problem using
the estimation algebra method. Consider the following scalar
model:
dX (t) = dB(t)
(47)
dY (t) = X (t) dt + dB(t)
(48)
The estimation algebra is a four-dimensional Lie algebra with

the basis
1
1 2
x2 , x, , 1
2 x2
2
x
This algebra is called the oscillator algebra in physics (27,28).
The oscillator algebra is the semidirect product of 1 and
the Heisenberg algebra that is generated by
1
1 2
x2 , x, and
2 x2
2
x
487
It can be verified that the Lie algebra of the linear filtering

equations of Kalman and Bucy is isomorphic to the oscillator
algebra.
A well-known example of a filtering problem given by
Benes (29) has a finite-dimensional filter and it is closely related to the linear filtering problem. Consider the scalar filtering problem
dX (t) = f (X (t)) dt + dB(t)
(49)
dY (t) = X (t) dt + dB(t)
(50)
where f satisfies the differential equation

df
+ f 2 (x) = ax2 + bx + c
dx
for some a, b, c . It is assumed that this Riccati equation
has a global solution, so that either a 0 or a b 0 and
c 0. The unnormalized conditional density can be computed
to verify that there is a ten-dimensional sufficient conditional
statistic. However, Benes (29) showed that a two-dimensional
filter provides a sufficient conditional statistic. The estimation algebra L for (49)(50) is generated by
1
1 2
f x2 , x
2
2 x
x
2
and is four-dimensional and solvable. The estimation algebra
L for (47)(48) arises for the algebra L by letting f 0. To
x
associate L with the estimation algebra L let F(x) f,
(x) exp(F(x)) and r(t, x) (x)r(t, x). Then the DMZ equation for (47) and (48) is transformed by the gauge transformation as

dr =

1 2
1
2
[(a + 1)x + bx + c] r + xr dY
2 x2
2
(51)
This has the same form as the DMZ equation for (49) and
(50). Thus the nonlinear filtering problem (49) and (50) is obtained from the linear filtering problem (47) and (48) by a
gauge transformation of the conditional density. Various
other examples of finite-dimensional filters are available (e.g.,
see Refs. 5961).
Ocone (62) showed that for a scalar filtering problem with
the observation equation of the form (50) the two examples
(47)(48) and (49)(50) are the only ones that give a finitedimensional estimation algebra. This result is given in the
following theorem.
Theorem. Let n m 1 in (49)(50) and let g 1 in the
observation equation (2). Then the dimension of the estimation algebra is finite only if:
1. h(x) ax and
df
+ f 2 = ax2 + bx + c
dx
or
2. h(x) ax2 x, a 0 and
df
+ f 2 (x) = h2 (x) + a(2ax + )2 + b + c(2ax + )1
dx
488
or
and
df
+ f 2 (x) = h2 (x) + ax2 + bx + c
dx
dR(t)
= B(t,Y )B(t, Y ) + A(t,Y )R(t)
+ R(t)A(t,Y
)
Some multidimensional results are discussed in the section

entitled Some Recent Areas of Nonlinear Filtering.
Another family of nonlinear filtering problems given by
Liptser and Shiryayev (63,64) that can be solved by the
Gaussian methods is the conditional linear models. Let (X(t),
Y(t), t 0) satisfy
dX (t) = [A(t,Y )X (t) + a(t,Y )] dt + B(t, Y ) dB(t)

+
d

[G j (t, Y )X (t) + g j (t, Y )] dY j (t)
G j (t, Y )R(t)G
(t, Y )
R(t)H(t,Y ) H(t, Y )R(t) dt

+
(56)

j
[G j (t, Y )R(t)
+ R(t)G
(t, Y ) ] dY j (t)
j
(52)
(0) R0.
where m
(0) m0 and R
(53)
SOME RECENT AREAS OF NONLINEAR FILTERING
j=1
dY (t) = [H(t, Y )X (t) + h(t, Y )] dt + dB(t)
where X(0) is a Gaussian random variable, Y(0) 0. The random variable X(t), given Y (t) (Y(u), u t), is conditionally
Gaussian. More precisely, it is assumed that (X(t), t 0) is
an n-valued (F t)-adapted process, (Y(t), t 0) is an m-val (t), t 0) are
ued (F t)-adapted process, (B(t), t 0) and (B
n
m
independent standard - and -valued Brownian motions,
respectively, in the filtered probability space (, F , (F t), P),
and X(0) is N(m0, R0). The functions A, a, B, Gj, gj, H, and h
are defined on C(, m) with values in a suitable Euclidean space, and they are progressively measurable. The
1
functions A2, a, B2, Gj2, H, and h are in Lloc
() for each
m
1
y C(, ). For each T 0, E (T) 1 where

T
H(s, Y )X (t) + h(s,Y ), dY (s)
(T ) = exp
0
1
2
(54)
|H(s, Y )X (s) + h(s,Y )| ds

2
Haussmann and Pardoux (65) proved the following result.

Theorem. Consider the filtering problem (52) and (53). For
each T 0 the conditional distribution of X(t), given Y (T)
(Y(u), u T), is Gaussian.
Furthermore, if it is assumed that a, gj, h, h gj, A,
B, Gj2, H2, h Gj, gj H, and H(t, Y)X(t) h(t, Y) are in
L2([0, T] ) for all T , then the following result for the
conditional mean m
(t) and the conditional covariance R(t) can
be verified (65)
Theorem. Consider the filtering problem (52) and (53). The
(t) satconditional mean m
(t) and the conditional covariance R
isfy the following equations:

dm(t)
= A(t,Y )m(t)
+ a(t,Y )
R(t)H(t,
Y ) [H(t, Y )m(t)
+ h(t, Y )]

j
+
G j (t, Y )R(t)H
(t,Y ) dt
A generalization of the filtering problem occurs when some or

all of the processes take values in an infinite-dimensional
space such as a Hilbert, Banach, or Frechet space. A basic
question is the existence of a probability measure on one of
these infinite-dimensional spaces. For example, for the existence of a zero mean Gaussian measure in a Hilbert space it
is necessary and sufficient that the covariance is nuclear
(trace class). The usual DaniellKolmogorov construction of a
probability measure from a projective family of measures on
finite-dimensional subspaces (finite dimensional distributions) does not guarantee a measurable space with a nice topology.
However, in some cases in infinite-dimensional spaces it
is possible to use a cylindrical noise (e.g., the covariance of
the Gaussian process is the identify) and have it regularized by the system so that the stochastic integral in the
variation of parameters formula is a nice process. To describe this approach consider a semilinear stochastic differential equation
sX (t) = AX (t) dt + f (X (t)) dt + Q1/2 dW (t)
(57)
where X(0), X(t) H, a separable, infinite-dimensional Hilbert space, and (W(t), t 0) is a standard cylindrical Wiener
process. A standard cylindrical Wiener process means that if
1, 2 H H*, 1, 2 0, and 1, 1 2, 2 1 where
, is the inner product in H, then (1, W(t), t 0) and
(2, W(t), t 0) are independent standard Wiener processes.
If A is the generator of an analytic semigroup (S(t), t 0)
and S(r)Q1/2 is HilbertSchmidt for each r 0 and

t
0
|S(r)Q1/2 |2L
2 (H )
dr <
(58)
where L2(H) is the norm for the HilbertSchmidt operators,

then the process (Z(t), t 0) where

(55)
Z(t) =
S(t r)Q1/2 dW (r)
(59)

j
+
[G j (t,Y )m(t)
+ g j (t,Y ) + R(t)H
(t, Y ) dY j (t)]
j
is an H-valued process that has a version with continuous

sample paths. Thus, the solution of Eq. (57) with some suit-
able assumptions on f (66) can be given by the mild solution

(67)
t
X (t) = S(t)X (0) +
S(t r) f (X (r)) dr
0
(60)
t
S(t r)Q1/2 dW (r)

0
This semigroup approach can be used to model stochastic

partial differential equations arising from elliptic operators
and delay-time ordinary differential equations. Some problems of ergodic control and stochastic adaptive control are described in Refs. 66 and 68.
For the stochastic partial differential equations it is natural to consider noise on the boundary of the domain or at discrete points in the domain. Furthermore, signal processes can
be considered to be on the boundary or at discrete points in
the domain. Some of the descriptions of these noise processes
can be found in Refs. 66 and 68.
For the nonlinear filtering problem it can be assumed that
the signal process is infinite-dimensional and that the observation process is finite-dimensional; or perhaps more interestingly it can be assumed that the signal process is finite-dimensional, occurring at distinct points of the domain or the
boundary, and that the observation process is infinite-dimensional in the domain.
Another nonlinear filtering formulation occurs when the
processes evolve on manifolds. This approach requires the
theory of stochastic integration in manifolds (69). Many wellknown manifolds arise naturally in the modeling of physical
systems such as spheres and positive definite matrices. To
justify the conditional mean as the minimum variance estimate and to compare the estimate and the signal, it is useful
to model the signal process as evolving in a linear space or a
family of linear spaces. The observation process can evolve in
a manifold and have the drift vector field depend on the signal
process, or the observation process can be the process in the
base of a vector bundle; for example, the tangent bundle and
the signal can evolve in the fibers of the vector bundle (70,71).
These formulations allow for some methods similar to filtering problems in linear spaces. An estimation problem in Lie
groups is solved in Ref. 72. The DMZ equation for a nonlinear
filtering problem in a manifold is given in Ref. 73. A description of the stochastic calculus on manifolds with applications
is given in Ref. 74.
The study of estimation algebras for nonlinear filtering
problems has been a recent active area for nonlinear filtering.
A number of questions naturally arise for estimation algebras. A fundamental question is the classification of finitedimensional estimation algebras. This classification would
clearly provide some important insight into the nonlinear filtering problem. This classification has been done for finitedimensional algebras of maximal rank that correspond to
state-space dimensions less than or equal to four (60,7577).
The result is described in the following theorem.
Theorem. Consider the filtering problem described by the
following stochastic differential equations:
dX (t) = f (X (t)) dt + g(X (t)) dV (t)
X (0) = x0
dY (t) = h(X (t)) dt + dW (t)
Y (0) = 0
(61)
(62)
489
Assume that n 4, where X(t) n and Y(t) . If E is the

finite-dimensional estimation algebra of maximal rank, then
the drift term f must be a linear vector field plus a gradient
vector field and E is a real vector space of dimension 2n 2.
Another basic question is to find necessary conditions for
finite-dimensional estimation algebras. It was conjectured by
Mitter (28) that the observation terms are polynomials of degree at most one. An important result related to this conjecture is the following result of Ocone (62,78) that describes
polynomials in the estimation algebra.
Theorem. Let E be a finite-dimensional estimation algebra.
If is a function in E , then is a polynomial of degree at
most two.
The following result of Chen and Yau (79) verifies the Mitter conjecture for a large family of estimation algebras.
Theorem. If E is a finite-dimensional estimation algebra of
maximal rank, then the polynomials in the drift of the observation equation (6) are degree-one polynomials.
Two basic approaches to the problem of finite-dimensional
filters for nonlinear filtering problems are the WeiNorman
approach and the symmetry (or invariance) group approach
(e.g., see Ref. 80). Wei and Norman (81) provided a global
representation of a solution of a linear differential equation
as a product of exponentials. The WeiNorman approach requires an extension of the WeiNorman results to semigroups. This has been done by introducing some function
spaces or using some results for the solutions of partial differential equations (82,83). This result is important for the construction of finite-dimensional filters from finite dimensional
estimation algebras (e.g., see Refs. 82 and 83).
Recall the problem of the existence of finite-dimensional
filters for a linear filtering problem with a non-Gaussian initial condition. The question of finite-dimensional filters for
nonlinear filtering problems can be formulated in different
ways. In one formulation the probability law of the initial condition is fixed. It has been shown (82) that a necessary condition for a finite-dimensional filter is the existence of a nontrivial homomorphism from the estimation algebra into the Lie
algebra of vector fields on a manifold.
Another formulation of the finite-dimensional filter problem is the requirement that a filter exist for all Dirac measures of the initial condition. It has been shown (82) that if a
finite-dimensional filter has a regularity property with respect to initial conditions and dynamics, then the estimation
algebra is finite-dimensional.
For linear filtering it is elementary to verify that minimizing a quadratic form which is the negative of the formal exponent in a likelihood function gives the solution of the linear
filtering problem. By analogy, an approach to the nonlinear
filtering problem based on minimizing a formal likelihood
function in function space was introduced in the 1960s
(84,85). This approach has been generalized and made rigorous by Fleming and Mitter (86) by relating a filtering problem
to a stochastic control problem. This method uses a logarithmic transformation.
490
For an example of this method of logarithmic transformation consider the following linear parabolic partial differential
equation:
1
tr a(x)pxx + g(x, t), px + V (x, t)p
2
p(x, 0) = p0 (x)
pt =
(63)
It is assumed that there is a C2,1 solution. If this solution is

positive, then S log p satisfies the nonlinear parabolic
equation:
St =
1
tr a(x)Sxx + H(x, t, Sx )
2
(64)
S(x, 0) = log p0 (x) = S0 (x)

H(x, t, Sx ) = g(x, t), Sx
(65)
1
a(x), Sx , Sx V (x, t)
2
(66)
This type of transformation is well known. For example, it

transforms the heat equation (g V 0) into Burgers
equation.
The nonlinear PDE (64) is the dynamic programming
(HamiltonJacobiBellman) equation for a stochastic control
problem. For example, let (X(t), t 0) satisfy
dX (t) = (g(X (t), t) + u(X (t), t)) dt + (X (t)) dB(t)
X (0) = x
and let the cost functional be

t
L(X (s), t s, u(s)) ds + S0 (X (t))
J(x, t, u) = Ex
(67)
(68)
where
L(x, t, u) =
1 1
a (x)u, u V (x, t)
2
(69)
With suitable assumptions on the family of admissible controls and conditions on the terms in the model it can be shown
from the Verification Theorem (87) that Eq. (63) is the dynamic programming equation for this stochastic control problem. This approach can provide a rigorous basis for the formal
maximization of a likelihood function in function space. See
Ref. 87a.
An approach to the robustness of the nonlinear filter
(87b,87c) is to obtain a so-called pathwise solution to the DuncanMortensenZakai (DMZ) equation by expressing the solution as an (observation) path dependent semigroup. The infinitesimal generator of this semigroup is the conjugation of
the generator of the signal process by the observation path
multiplied by the drift in the observation where the Stratonovich form of the DMZ equation is used. The fact that the
observation path appears explicitly rather than its differential implies the robustness of the solution of the DMZ
equation.
It is important to obtain estimation methods that are applicable to both stochastic disturbances (noise) and deterministic disturbances. For Brownian motion, a Hilbert (or Sobolev
space) of functions that are functions that are absolutely continuous and whose derivatives are square integrable having
probability zero often plays a more important role than the

Banach space of continuous functions that has probability
one. This Hilbert space (or Sobolev space) alludes to the fact
that there are some natural relations between stochastic and
deterministic disturbances. In recent years the study of risk
sensitive control problems has occupied an important place in
stochastic control. Risk sensitive control problems (i.e., control problems with an exponential cost) have been used with
the maximum likelihood methods in (85,87d) to obtain robust
nonlinear filters, that is, filters that are effective for square
integrable disturbances as well as Gaussian white noise (87e).
These ideas are related to the approach of H control as a robust control approach. The robust nonlinear filter can be naturally related to a robust nonlinear observer. For a number
of problems there is a natural relation between estimation for
deterministic and stochastic systems. For example, a
weighted least squares algorithm can be used for the identification of parameters for both deterministic and stochastic
systems.
Since it is usually not feasible to solve explicitly the stochastic equation for the conditional mean or the conditional
covariance for a nonlinear filtering problem it is important to
obtain lower and upper bounds on the filtering error. These
bounds enable an effective comparison of the suboptimal filters with the optimal filter. The bounds have typically been
obtained as the noise approaches zero (87f).
An important theoretical and practical problem in nonlinear filtering is the infinite time stability or continuity of the
filter with respect to the initial conditions and the parameters
of the filter. The problem of stability of the optimal nonlinear
filter with respect to initial conditions is investigated in (87g)
for two different cases. Stability of the Riccati equation for
linear filtering is used to obtain almost sure asymptotic stability for linear filters with possible non-Gaussian initial conditions. For signals that are ergodic diffusions it is shown that
the optimal filter is asymptotically stable in the sense of weak
convergence of measures for incorrect initial conditions. Another stability property that is important for the optimal filter
is asymptotic stability with respect to the parameters of the
filter.
Another important question in nonlinear filtering is to develop numerical methods for the DMZ equation. One numerical approach to the solution of the DMZ equation is to consider that it is a stochastic partial differential equation of a
special form and use numerical methods from PDE for the
numerical discretization of the problem (e.g., finite-difference
schemes). There has been some success with this approach
(8890), but it is limited to small space dimension and also
often to the small intervals of time.
Another approach is to use the Wiener chaos expansion
that is based on an orthogonal expansion of a square integrable functional on Wiener space (30,31,91). The solution, r,
of the DMZ equation is expressed in the following expansion
(92):
r(t, x) =
1
(t, x) (y)
!
(70)
where are Wick polynomials (special products of Hermite

polynomials) formed from Wiener integrals and are HermiteFourier coefficients in the orthogonal expansion. The
separation of x and y in the expansion (70) implies a splitting
in the computations, one that is associated with the Markov

process (X(t), t 0) and the other one that depends on the
observations (Y(t), t 0). The family of functions () can be
computed recursively from a family of equations of Kolmogorov type (92,93). The Wick polynomials () depend on the
observations and there are numerical methods to compute
them. For the numerical success, the Wick polynomials have
to be computed in special ways. A direct approach to the evaluation of the Wick polynomials and thereby the expansion
(70) is limited to short time intervals because the errors of
truncation of the infinite series increase rapidly, probabily exponentially.
The numerical methods for nonlinear filtering are still under active development. It seems that all of the methods are
limited to at most two or three space dimensions, and many
methods have significant restrictions on the time intervals
that are allowable for computations. However, these methods
have been demonstrated to perform significantly better than
more elementary methods such as the extended linear filter
which is commonly used. Many stochastic problems require
nonlinear filtering, so the filtering problems have to be addressed.
It should be clear that the area of nonlinear filtering is still
an active area for research. This research includes mathematical investigations using methods from probability and geometry and implementation investigations that use numerical
schemes for solving stochastic problems.
BIBLIOGRAPHY
1. A. N. Kolmogorov, Sur linterpolation et extrapolation des suites
stationnaires, C. R. Acad. Sci., 208: 2043, 1939.
2. A. N. Kolmogorov, Stationary sequences in Hilbert space (in Russian), Bull. Math. Univ. Moscow, 2: 6, 1941.
3. M. G. Krein, On a generalization of some investigations of G.
Szego, W. M. Smirnov, and A. N. Kolmogorov, Dokl. Adad. Nauk
SSSR, 46: 9194, 1945.
4. M. G. Krein, On a problem of extrapolation of A. N. Kolmogorov,
Dokl. Akad. Nauk SSSR, 46: 306309, 1945.
5. N. Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series, with Engineering Applications, New York: Technology Press and Wiley, 1949. (Originally issued in February
1942, as a classified National Defense Research Council Report)
6. H. Wold, A Study in the Analysis of Stationary Time Series, 2nd
ed., Uppsala, Sweden: Almqvist and Wiksell, 1938.
6a. H. W. Bode and C. E. Shannon, A simplified derivation of linear
least squares smoothing and prediction theory, Proc. IRE, 38:
417425, 1950.
491
14. K. Ito, Stochastic integral, Proc. Imp. Acad. Tokyo, 20: 519524,
1944.
15. P. A. Meyer, Probability and Potentials, Waltham, MA: Blaisdell, 1966.
16. D. L. Fisk, Quasi-martingales and stochastic integrals, Technical
Report 1, Dept. of Mathematics, Michigan State University, 1963.
17. R. L. Stratonovich, Conditional Markov process theory, Theory
Probability Appl. (USSR), 5: 156178, 1960.
18. E. J. McShane, Stochastic differential equations and models of
random processes, In Proceedings of the 6th Symposium on Mathematical, Statistics, and Probability, Berkeley, CA: University of
California Press, 1973.
19. S. S. Wilks, Mathematical Statistics, New York: Wiley, 1962.
20. H. Kushner, On the differential equations satisfied by conditional
densities of Markov processes, with applications, SIAM J. Control, 2: 106119, 1964.
21. R. L. Stratonovich, Application of the theory of Markov processes
for optimum filtration of signals, Radio Eng. Electron. Phys.
(URRS), 1: 119, 1960.
22. R. S. Bucy, Nonlinear filtering theory, IEEE Trans. Autom. Control, AC-10: 198, 1965.
23. G. Kallianpur and C. Striebel, Estimation of stochastic systems:
Arbitrary system process with additive white observation errors,
Ann. Math. Statist., 39: 785801, 1969.
24. T. E. Duncan, Probability Densities for Diffusion Processes with
Applications to Nonlinear Filtering Theory and Detection Theory,
Ph.D. Dissertation, Stanford University, Stanford, CA, June 1967.
25. R. E. Mortensen, Optimal Control of Continuous-Time Stochastic
Systems, Ph.D. dissertation, University of California, Berkeley,
CA, 1966.
26. M. Zakai, On the optimal filtering of diffusion processes, Z.
Wahrsch, Verw. Geb. 11: 230243, 1969.
27. R. Brockett, Remarks on finite dimensional nonlinear estimation,
Asterisque, 764: 4755, 1980.
28. S. K. Mitter, On the analogy between mathematical problems on
nonlinear filtering and quantum physics, Ric. Autom., 10 (2):
163216, 1980.
28a. S. K. Mitter, Filtering theory and quantum fields, Asterisque,
7576: 199205, 1980.
29. V. E. Benes, Exact finite dimensional filters for certain diffusions
with nonlinear drift, Stochastics, 5: 6592, 1981.
30. R. H. Cameron and W. T. Martin, The orthogonal development of
non-linear functionals in a series of FourierHermite functions,
Ann. Math., 48: 385392, 1947.
31. K. Ito, Multiple Wiener integral, J. Math. Soc. Japan, 3: 157
169, 1951.
32. Y. Hu and P. A. Meyer, Sur les integrales multiples de Stratonovich, Lecture Notes in Mathematics, No 1321, Berlin: SpringerVerlag, pp. 7281, 1988.
7. N. Wiener and E. Hopf, On a class of singular integral equations,

Proc. Prussian Acad., Math.-Phys. Ser., 696, 1931.
33. D. F. Allinger and S. K. Mitter, New results on the innovations

problem for nonlinear filtering, Stochastics, 4: 339348, 1981.
8. J. L. Doob, Wieners work in probability theory, Bull. Am. Math.

Soc., 72: 6972, 1966.
34. T. E. Duncan, Evaluation of likelihood functions, Inform. Control

13: 6274, 1968.
9. R. E. Kalman, A new approach to linear filtering and prediction

problems, Trans. ASME, Ser. D, J. Basic Eng., 82: 3445, 1960.
10. R. E. Kalman and R. S. Bucy, New results in linear filtering and
prediction theory, Trans. ASME, Ser. D, J. Basic Eng., 83: 95
107, 1961.
11. K. Ito, On stochastic differential equations, Mem. Am. Math. Soc.,
4: 1951.
12. N. Wiener, Differential space, J. Math. Phys., 2: 131174, 1923.
13. P. Levy, Processus Stochastiques et Mouvement Brownien, 2nd ed.,
Paris: Gauthier-Villars, 1964.
35. T. E. Duncan, On the calculation of mutual information, SIAM J.

Math., 19: 215220, 1970.
36. K. Ito, On a formula concerning stochastic differentials, Nagoya
Math. J., 3: 5565, 1951.
37. E. Wong and M. Zakai, On the relation between ordinary and
stochastic differential equations, Int. J. Eng. Sci., 3: 213229,
1965.
38. T. E. Duncan, P. Mandl, and B. Pasik-Duncan, Numerical differentiation and parameter estimation in higher order stochastic
systems, IEEE Trans. Autom. Control, 41: 522532, 1996.
492
39. A. Pazy, Semigroups of Linear Operators and Applications to Partial Differential Equations, New York: Springer-Verlag, 1983.
40. I. Karatzas and S. E. Shreve, Brownian Motion and Stochastic
Calculus, 2nd ed., New York: Springer-Verlag, 1991.
41. I. E. Segal, Tensor algebras over Hilbert spaces. I, Trans. Am.
Math. Soc., 81: 106134, 1956.
42. A. V. Skorokhod, On the differentiability of measures which correspond to stochastic processes. I: Processes with independent increments, Theory Probability Appl., 2: 407432, 1957.
43. A. V. Skorokhod, Studies in the Theory of Random Processes, Reading, MA: Addison-Wesley, 1965.
44. I. V. Girsanov, On transforming a certain class of stochastic processes by absolutely continuous substitution of measures, Theory
Probability Appl., 5: 285301, 1960.
45. T. E. Duncan and P. Varaiya, On the solutions of a stochastic
control system. II, SIAM J. Control, 13: 10771092, 1975.
46. A. N. Shiryayev, On stochastic equations in the theory of conditional Markov processes, Theor. Probability Appl., 11: 179184,
1966.
47. M. Fujisaki, G. Kallianpur, and H. Kunita, Stochastic differential
equations for non-linear filtering, Osaka J. Math., 9: 1940, 1972.
48. G. Kallianpur, Stochastic Filtering Theory, New York: SpringerVerlag, 1980.
49. W. M. Wonham, Some applications of stochastic differential equations to optimal nonlinear filtering, SIAM J. Control, 2: 347
369, 1965.
50. H. Sussmann, Existence and uniqueness of minimal realizations
of nonlinear systems, Math. Syst. Theory, 10: 263284, 1977.
51. R. Brockett and J. M. C. Clark, The geometry of the conditional
density equation, In O. Jacobs et al. (eds.), Analysis and Optimization of Stochastic Systems, New York: Academic Press, 1980,
pp. 299310.
52. S. I. Marcus, Low dimensional filters for a class of finite state
estimation problems with Poisson observations, Syst. Control
Lett., 1: 237241, 1982.
53. M. Davis and S. Marcus, An introduction to nonlinear filtering,
In M. Hazewinkel and J. Willems (eds.), Stochastic Systems: The
Mathematics of Filtering and Identification and Applications, Dordrecht, The Netherlands: D. Riedel, 1981, pp. 5375.
54. S. I. Marcus, Algebraic and geometric methods in nonlinear filtering, SIAM J. Control, 22: 817844, 1984.
54a. S. K. Mitter, Geometric theory of nonlinear filtering, Outils et
Mode`les Mathematiques pour lAutomatiques, 3: CNRS, 3760,
1983.
55. R. W. Brockett, Classification and equivalence in estimation theory, In Proceedings of the 18th IEEE Conference on Decision and
Control, Ft. Lauderdale, FL, 1979, pp. 172175.
56. R. Brockett, Nonlinear systems and nonlinear estimation theory,
Mathematics of Filtering and Identification and Applications, Dordrecht, The Netherlands: D. Riedel, pp. 442477, 1981.
57. M. Hazewinkel and S. Marcus, Some results and speculations on
the role of Lie algebras in filtering, In M. Hazewinkel and J. Willems (eds.), Stochastic Systems: The Mathematics of Filtering and
Identification and Applications, Dordrecht, The Netherlands: D.
Riedel, pp. 591604, 1981.
58. M. Hazewinkel, S. Marcus, and H. Sussmann, Nonexistence of
exact finite dimensional filters for conditional statistics of the cubic sensor problem, Systems Control Lett., 3: 331340, 1983.
59. D. Ocone, J. S. Baras, and S. I. Marcus, Explicit filters for diffusions with certain nonlinear drifts, Stochastics, 8: 116, 1982.
60. W. Wong, Theorems on the structure of finite dimensional estimation algebras, Systems Control Lett., 9: 117124, 1987.
61. S. S.-T. Yau, Finite dimensional filters with nonlinear drift. I: A

class of filters including both KalmanBucy filters and Benes filters, J. Math. Systems, Estim. Control 4: 181203, 1994.
62. D. Ocone, Topics in Nonlinear Filtering Theory, Ph.D. thesis,
Massachusetts Institute of Technology, Cambridge, MA, 1980.
63. R. Sh. Liptser and A. N. Shiryayev, Statistics of Random Processes. I, New York: Springer-Verlag, 1977.
64. R. Sh. Liptser and A. N. Shiryayev, Statistics of Random Processes. II, New York: Springer-Verlag, 1978.
65. U. Haussmann and E. Pardoux, A conditionally almost linear filtering problem with nongaussian initial condition, Stochastics,
23: 241275, 1988.
66. T. E. Duncan, B. Maslowski, and B. Pasik-Duncan, Ergodic
boundary/point control of stochastic semilinear systems, SIAM J.
Control Optim., 36: 1998.
67. G. Da Prato and J. Zabczyk, Stochastic Equations in Infinite Dimensions, New York: Cambridge University Press, 1992.
68. T. E. Duncan, B. Maslowski, and B. Pasik-Duncan, Adaptive
boundary control of linear distributed parameter systems described by analytic semigroups, Appl. Math. Optim., 33: 107
138, 1996.
69. T. E. Duncan, Stochastic integrals in Riemann manifolds, J. Multivariate Anal., 6: 397413, 1976.
70. T. E. Duncan, Estimation for jump processes in the tangent bundle of a Riemann manifold, Appl. Math. Optim., 4: 265274, 1978.
71. T. E. Duncan, Some filtering results on Riemann manifolds, Inform. Control, 35: 182195, 1977.
72. T. E. Duncan, An estimation problem in compact Lie groups,
Syst. Control Lett., 10: 257263, 1988.
73. T. E. Duncan, Stochastic filtering in manifolds, In Proceedings of
the International Federation on Autonomic Control World Congress, Elmsford, NY: Pergamon Press, 1981.
74. T. E. Duncan, An introduction to stochastic calculus in manifolds
with applications, In J. S. Baras and V. Mirelli (eds.), Recent Advances in Stochastic Calculus, New York: Springer-Verlag, pp.
105140, 1990.
75. J. Chen, S. S. Yau, and C. W. Leung, Finite dimensional filters
with nonlinear drift. VIII: Classification of finite dimensional estimation algebras of maximal rank with state space dimension
4, preprint.
76. R. Dong, L. Tam, W. Wong, and S. Yau, Structure and classification theorem of finite-dimensional exact estimation algebras,
SIAM J. Control Optim., 29: 866877, 1991.
77. L. Tam, W. Wong, and S. Yau, On a necessary and sufficient condition for finite dimensionality of estimation algebras, SIAM J.
Control Optim., 28: 173185, 1990.
78. D. Ocone, Finite dimensional Lie algebras in nonlinear filtering,
Mathematics of Filtering and Identification and Applications, Dordrecht, The Netherlands: D. Riedel, pp. 629636, 1981.
79. J. Chen and S. S. Yau, Finite dimensional filters with nonlinear
drift. VII: Mitter conjecture and structure of , preprint.
80. J. Baras, Group invariance methods in nonlinear filtering of diffusion processes, In M. Hazewinkel and J. Willems (eds.), Stochastic Systems: The Mathematics of Filtering and Identification
and Applications, Dordrecht, The Netherlands: D. Riedel, pp.
565572, 1981.
81. J. Wei and E. Norman, On the global representation of the solutions of linear differential equations as a product of exponentials,
Proc. Am. Math. Soc., 15: 327334, 1964.
82. M. Cohen de Lara, Finite-dimensional filters. Part I: The Wei
Norman techniques, SIAM J. Control Optim., 35: 9801001, 1997.
83. M. Cohen de Lara, Finite-dimensional filters. Part II: Invariance
group techniques, SIAM J. Control Optim., 35: 10021029, 1997.
FILTERING THEORY
84. A. E. Bryson and M. Frazier, Smoothing for linear and nonlinear
dynamic systems, in Proceedings of the Optimum System Synthesis
Conference, Wright-Patterson Air Force Base, Ohio, September,
1962, AST-TDR-63-119.
85. R. E. Mortensen, Maximum likelihood recursive nonlinear filtering, J. Optim. Theory Appl., 2: 386394), 1968.
86. W. H. Fleming and S. K. Mitter, Optimal control and nonlinear
filtering for nondegenerate diffusion processes, Stochastics, 8:
6377, 1982.
87. W. H. Fleming and R. W. Rishel, Deterministic and Stochastic
Optimal Control, New York: Springer-Verlag, 1975.
87a. S. K. Mitter, Lectures on nonlinear filtering and stochastic control, Lecture Notes in Math, 972: 170207, 1983.
87b. M. H. A. Davis, On a multiplicative functional transformation
arising in nonlinear filtering, Z. Wahr. Verw. Geb., 54: 125139,
1980.
87c. M. H. A. Davis, A pathwise solution of the equations of nonlinear filtering, Theory Prob. Appl., 27: 167175, 1982.
87d. O. Hijab, Minimum energy estimation, Ph.D. dissertation, Univ.
California, Berkeley, CA, 1980.
87e. W. H. Fleming and W. M. McEneaney, Risk sensitive and robust
nonlinear filtering, Proc. 36th IEEE Conf. Decision Cont., San
Diego, 10881093, 1997.
87f. B. Z. Bobrovsky and M. Zakai, Asymptotic bounds on the minimal
error of nonlinear filtering, Stochastic Systems: the mathematics of
filtering and identification and applications, M. Hazeainkel and J.
Willems (eds.), Dordrecht: Reidel, 1981, 573582.
87g. D. Ocone and E. Pardoux, Asymptotic stability of the optimal
filter with respect to the initial conditions, SIAM J. Con. Optim.,
34: 226243, 1996.
88. A. Bensoussan, R. Glowinski, and R. Rascanu, Approximations of
the Zakai equation by splitting up method, SIAM J. Control Optim., 28: 14201431, 1990.
89. R. J. Elliott and R. Glowinski, Approximations to solutions of the
Zakai filtering equation, Stochastic Anal. Appl., 7: 145168, 1989.
90. K. Ito, Approximation of the Zakai equation for nonlinear filtering, SIAM J. Control Optim., 34: 620634, 1996.
91. H. P. McKean, Geometry of differential space, Ann. Prob., 1: 197
206, 1973.
92. S. Lototsky, R. Mikulevicius, and B. L. Rozovskii, Nonlinear filtering revisited: A spectral approach, SIAM J. Control Optim.,
35: 435461, 1997.
93. A. Budhiraja and G. Kallianpur, Approximation to the solution
of the Zakai equation using multiple Wiener and Stratonovich
integral expansions, Stochastics and Stoch. Repts., 56: 271315,
1996.
94. W.-L. Chiou and S.-T. Yau, Finite-dimensional filters with nonlinear drift. II: Brocketts problem on classification of finite-dimensional estimation algebras, SIAM J. Control Optim., 32: 297
310, 1994.
TYRONE E. DUNCAN
University of Kansas
FILTERING, LINEAR. See CONVOLUTION.
493
GAIN SCHEDULING
MOTIVATION
A classical tradeoff in control design is model accuracy versus
model simplicity. While sophisticated models better represent
a physical systems behavior, the resulting analysis and control design are more involved.
One manifestation of this tradeoff is the use of nonlinear
versus linear models. Most physical systems exhibit nonlinear
behavior. Some common examples are saturations, rate limiters, hysteresis, and backlash. Predominantly nonlinear behavior may be found in robotic manipulator dynamics, aircraft or missile ight dynamics, undersea vehicle dynamics,
jet engine combustion dynamics, and satellite attitude dynamics. Although analysis and control design for nonlinear
systems remains an active topic of research, analysis of linear
systems is signicantly less complicated, and there is an
abundance of control design methodologies, ranging from
classical control to multivariable robust control.
One compromise is to linearize the system behavior, that
is, approximate the behavior of a nonlinear system near a
particular operating condition by a linear system. This simplication allows one to draw upon analysis and design methods
for linear systems. However, this simplication comes at the
cost of certain limitations:
The nonlinear system must be conned to operating near
the specied operating condition of the linearization.
The linearization analysis may give misleading or inconclusive results.
The linearization may ignore important nonlinear phenomena which dominate the system behavior.
Despite these limitations, linearization remains a widely used
method for control system analysis and design.
In many cases, conning a nonlinear system to operating
near a specied operating condition is too restrictive. One example is ight control. An aircraft typically experiences several ight conditions, including take-off, cruising at various
altitudes, specialized maneuvers, and landing. No single
linearization can adequately describe the aircraft dynamics at
all of these conditions. Another example is boiler-turbine control in power generation. Typical operating conditions include
power ramp up, steady power delivery at various levels, and
power ramp down. Again, no linearization can adequately describe the dynamics at all operating conditions.
sign several linear controllers, one for each operating

condition.
Each individual controller is expected to achieve good performance whenever the nonlinear plant is near the controllers associated operating condition. As the plant varies from
one operating condition to another, the gains of the individual
controllers are interpolated, or scheduled, to match the
changes in operating conditions. The nal result is a nonlinear controller which is constructed out of several local linear
controllers.
The implementation of a gain scheduled controller is depicted in Fig. 1. An auxiliary variable, usually called the
scheduling variable, is used to update the gains of the linear
controller. The scheduling variable should be a good indication of the current operating condition of the plant, and hence
should be correlated with the plant nonlinearities. The scheduling variable can be a combination of endogenous signals,
such as a plant measurements, or exogenous parameters
which reect environmental conditions.
One example is missile autopilot design. Useful scheduling
variables are the angle-of-attack and dynamic pressure, both
of which characterize the aerodynamic ight coefcients of
the missile. The angle-of-attack is the angle between the missile body and velocity vector and can be considered a state
variable, and hence endogenous to the missile dynamics. The
dynamic pressure, which is a function of missle velocity and
atmospheric pressure, is indicative of the environmental conditions. Atmospheric pressure is clearly an exogenous signal.
Since the dynamic pressure is also a function of missile velocity, it can be considered an endogenous variable. However,
the velocity variations in a simplied model are decoupled
from the attitude dynamics, and hence, dynamic pressure
may be modeled as an exogenous signal. This sort of ambiguity, namely that an exogenous signal is really an endogenous signal in a more sophisticated model, is common.
Gain scheduling has seen widespread industrial application. It is perhaps the most prevalent nonlinear method for
aircraft ight control and missle autopilot design. Other applications include power systems, process control, and automotive control. Despite its widespread usage, traditional gain
scheduling has been an ad hoc design approach accompanied
by heuristic guidelines.
WHAT IS GAIN SCHEDULING?
Scheduling
variables
Environmental
conditions
Reference
commands
Gain scheduling is an approach to overcome the local limitations associated with linearizations. The idea is simple and
intuitively appealing. Given a nonlinear plant with a wide
range of operating conditions, one can select several representative operating conditions within the operating regime, perform several linearizations of the nonlinear dynamics, and de-
Operating
conditions
Scheduled
controller
Nonlinear
plant
Regulated
outputs
Measured
outputs
Figure 1. Gain scheduled control implementation.
205
206
GAIN SCHEDULING
Some immediate drawbacks of gain scheduling are the following:

The design of several controllers at several linearization
points can be a tedious task.
Gain scheduled designs typically assume a xed operating condition, even though the operating condition is
varying.
Although the resulting controller is nonlinear, it is still
based on linearizations of plant dynamics, and hence
may neglect important nonlinear phenomena.
Let r(x) denote the residual error of approximation; that is,

r(x) = f (x) f (x0 ) + D f (x0 )(x x0 )
Then,
lim
xx o
It is easy to interpret these guidelines in terms of the

aforementioned drawbacks. Since gain scheduled designs assume a constant operating condition, slow variations among
operating conditions should be tolerable. Similarly, since gain
scheduling relies on a family of linearizations, the changes
within this family should be indicative of plant nonlinearities.
This article provides an overview of the gain scheduling
design procedure, discusses the theoretical foundations behind gain scheduling, as well as limitations of traditional gain
scheduling, and presents emerging techniques for gain scheduling which address these limitations.
r(x)
=0
|(x x0 )|
(5)
where v denotes the Euclidean norm of v R n,

|v| = (vT v)1/2
Two heuristic rules-of-thumb which guide successful gain

scheduled designs are the following:
The scheduling variable should vary slowly.
The scheduling variable should capture plant nonlinearities.
(4)
(6)
Example 1. Let f: R 2 R 2 be dened as

f (x) =

x1 |x1 | + x2
x21 x2
(7)
Then
2|x1 |
D f (x) =
2x1 x2
1
x21

(8)
Approximating f(x) near
x0 =

1
2
leads to
LINEARIZATION
Linearization of Functions
We begin by recalling some concepts from multivariable calculus. Let f: R n R p denote a multivariable function which
maps vectors in R n to vectors in R p. In terms of the individual components,
f 1 (x1 , . . ., xn )
..
f (x) =
.
f p (x1 , . . ., xn )
(1)
x1
.
.
D f (x) =
.
fp
x1
(x)
..
(x)
f1
(x)
xn
..
.
fp
(x)
xn
(2)
Now, let f: R n R m R p denote a multivariable function

which maps vectors in R n and R m together to vectors in R p.
In terms of the individual components,
f 1 (x1 , . . ., xn , u1 , . . ., um )
..
f (x, u) =
.
f p (x1 , . . ., xn , u1 , . . ., um )
(3)
(10)
In case f is differentiable, D1f(x, u) denotes the p n Jacobian matrix of partial derivatives with respect to the rst
variable,
For f continuously differentiable, i.e., if Df(x) has continuous elements, we may approximate f(x) by the truncated Taylors series
f (x) f (x0 ) + D f (x0 )(x x0 )
(9)
In case f is differentiable, Df(x) denotes the p n Jacobian

matrix of partial derivatives; i.e.,
f (x) f (x0 ) + D f (x0 )(x x0 )

3
2 1
x1 1
=
+
2
4 1
x2 2
(x, u)
x1
..
D1 f (x, u) =
.
fp
(x, u)
x1
..
f1
(x, u)
xn
..
fp
(x, u)
xn
(11)
GAIN SCHEDULING
and D2 f(x, u) denotes the p m Jacobian matrix of partial

derivatives with respect to the second variable,
1
(x, u)
u1
..
D2 f (x, u) =
.
fp
(x, u)
u1
..
f1
(x, u)
um
..
fp
(x, u)
um
(12)
f (x, u) f (x0 , u0 ) + D1 f (x0 , u0 )(x x0 ) + D2 (x0 , u0 )(u u0 )

(13)
Let
(14)
denote the approximation residual. Then as before,
r(x, u)
lim
=0
|x x0 |2 + |u u0 |2
xx 0
uu 0
(15)
(16)
x0 is stable if for any 0, there exists a 0 such that

(24)
Otherwise, x0 is unstable.
x0 is asymptotically stable if in addition to being stable,
(25)
In words, stability implies that the solution to Eq. (16)

stays near x0 whenever it starts sufciently close to x0,
whereas asymptotic stability implies that the solution also asymptotically approaches x0.
Theorem 4. Let f in Eq. (16) be continuously differentiable.
The equilibrium x0 is asymptotically stable if all of the eigenvalues of Df(x0) have negative real parts. It is unstable if
Df(x0) has at least one eigenvalue with a positive real part.
(17)
The reasoning for this terminology is that the initial condition x(0) x0 leads to the solution x(t) x0 for all time. So if
the solution starts at x0, it remains at x0, hence the term equilibrium.
In case f is continuously differentiable, we may rewrite
(16) as
(18)
where r(x) denotes the residual error in the approximation

(19)
Since x0 is both xed and an equilibrium,

d
dx
(x x0 ) =
dt
dt
(23)
Equation (22) is called the linearization of Eq. (16) about

the equilibrium x0. Intuitively, whenever x x0 is small, then
x in the linearization should be a good approximation of x
x0, and hence the linear dynamics of Eq. (22) should be a good
approximation of the nonlinear dynamics of Eq. (16).
It is often possible to make more denite statements about
nonlinear dynamics based on an analysis of the linearization,
in particular regarding the stability of the nonlinear system.
First, recall the following stability denitions.
Definition 2. The vector x0 is an equilibrium of (16) if
f (x) f (x0 ) + D f (x0 )(x x0 )
A = D f (x0 )
|x(0) x0 | < lim x(t) = x0
In the last section, we saw how to approximate the static behavior of a nonlinear function by using a truncated Taylors
series. We now show how similar tools can be used to approximate the dynamic behavior of a nonlinear system.
Consider an autonomous nonlinear system
x = f (x0 ) + D f (x0 )(x x0 ) + r(x)
(21)
Substituting these formulas into (18) and neglecting the residual term, r(x), leads to the approximate dynamics
.
x = Ax
(22)
|x(0) x0 | < |x(t) x0 |
Linearization of Autonomous Systems
f (x0 ) = 0
f (x0 ) = 0
Definition 3. Let x0 be an equilibrium of Eq. (16).
Equations (5) and (15) indicate that the approximations

are accurate up to the rst order.
x = f (x)
and
where
As before, if f is continuously differentiable, we may approximate f(x, u) by

r(x, u) = f (x, u) f (x0 , u0 ) + D1 f (x0 , u0 )(x x0 )

+ D2 (x0 , u0 )(u u0 )
207
(20)
Since the eigenvalues of Df(x0) determine the stability of

the linearization (22), Theorem 4 states that one can assess
the stability of a nonlinear system based on its linearization.
Example 5. The equations of motion for a pendulum of
length are
g
d
d2
sin( ) = 0
+c
dt 2
dt

(26)
where is the pendulum angle measure positive clockwise

with 0 being the upright position, c is a friction coefcient, and g is gravitational acceleration. In state space form,
these equations becomes

x2
d x1
(27)
= g
sin(x1 ) cx2
dt x2

where x1 , and x2 .
208
GAIN SCHEDULING
Linearizing about the upright equilibrium x0 0 leads to

0
1
x =
x
(28)
g/ c
The linearization has an eigenvalue with positive real part.
Hence, the upright equilibrium of the pendulum is unstable,
as expected.
Linearizing about the hanging equilibrium

x0 =
0
leads to

x =
0
g/

1
x
c
(29)
with the constant input u(t) u0 leads to the constant solution x(t) x0.
Proceeding as in the autonomous case, whenever f is continuously differentiable, we may rewrite (33) as
x = f (x0 , u0 ) + D1 f (x0 , u0 )(x x0 )
+ D2 f (x0 , u0 )(u u0 ) + r(x, u)]
(35)
where r(x, u) denotes the residual error in the approximation

f (x, u) f (x0 , u0 ) + D1 f (x0 , u0 )(x x0 )
+ D2 f (x0 , u0 )(u u0 )
(36)
Dropping this residual term and using that f(x0, u0) 0 leads
to the approximate linear dynamics
.
x = Ax + Bu
(37)
where
which is asymptotically stable. Therefore, the hanging equilibrium of the nonlinear pendulum is asymptotically stable,
as expected.
The above example demonstrates that different equilibrium points of the same nonlinear system can have different
stability conditions.
In some cases, using the linearization to assess stability
may be inconclusive.
A = D1 f (x0 , u0 ),
B = D2 f (x0 , u0 )
(38)
As before, Eq. (37) is called the linearization of the nonlinear

Eq. (33) about the equilibrium (x0, u0). The quantity x approximates x x0, whereas the quantity u equals u u0, exactly.
Definition 8. The equilibrium (x0, u0) of Eq. (33) is stabilized
by the state feedback u G(x) if
u0 G(x0)
x0 is a stable equilibrium of the closed loop dynamics
Example 6. Consider the scalar nonlinear systems

x = x3
(30)
x = x3
(31)
x = f (x, G(x))
(39)
and
The following is a direct consequence of Theorem 4.
It is easy to see that the equilibrium x0 0 is asymptotically

stable for the former system, while the same equilibrium is
unstable for the latter system.
Both systems have the same linearization at the equilibrium x0 0,
.
x = 0
(32)
Theorem 9. Let f in Eq. (33) be continuously differentiable,

and let Eq. (37) be the linearization of Eq. (33) about the equilibrium (x0, u0). Suppose the static linear feedback,
(note that x represents different quantities in the two linearizations). In this case, stability analysis of linearization is inconclusive; it does not indicate either the stability or instability of the equilibrium x0 0.
u = u0 K(x x0 )
Linearization of Systems with Controls

It is also possible to use linearization methods to synthesize
controllers for a nonlinear system.
Consider the controlled system
x = f (x, u)
(33)
Definition 7. The pair (x0, u0) is an equilibrium of Eq. (33) if

f (x0 , u0 ) = 0
(34)
The reasoning behind the term equilibrium is similar to

the autonomous case. The initial condition x(0) x0 along
u = K x
(40)
stabilizes the linearization of Eq. (37). Then the equilibrium

(x0, u0) of Eq. (33) is stabilized by the feedback
(41)
Theorem 9 states that we can construct stabilizing feedback for a nonlinear system by designing stabilizing feedback
for its linearization.
Example 10. Recall the simple pendulum example, but now
with a control torque input,

x2
d x1
(42)
= g
sin(x1 ) cx2 + u
dt x2

Linearizing about the upright equilibrium leads to

.
0
1
0
x =
x +
u
g/ c
1
(43)
GAIN SCHEDULING
The feedback
209
Then, the linearization about the equilibrium (x0, u0),

u = (k1
k2 )x
stabilizes the linearization for any k1 g/ and k2 c. From

Theorem 9, the feedback
u = u0 (k1
= (k1
k2 )(x x0 )
k2 )x
(45)
stabilizes the upright equilibrium, where we used that x0 0

and u0 0.
In some cases, analysis of the linearization does not aid in
the construction of stabilizing feedback.
Example 11. Consider the scalar nonlinear system
x = x + xu
.
x = Ax + Bx
(44)
where
A = D1 f (x0 , u0 ),
(46)
(47)
C = Dg(x0 )
(54)
Theorem 13. Let f in Eq. (33) and g in Eq. (49) be continuously differentiable, and let Eq. (53) be the linearization of
Eq. (33) about the equilibrium (x0, u0). Suppose the linear
feedback,
z = Az + By
(55)
u = Cz
stabilizes the linearization of Eq. (53). Then the equilibrium

(x0, u0) of Eq. (33) is stabilized by the output feedback
which is not stabilizable. However, the constant feedback

u 2 leads to the closed loop equations
x = x
B = D2 f (x0 , u0 ),
approximates the input-output behavior of Eq. (33) with measurement of Eq. (49). Here, x approximates x x0, y approximates y y0, and u exactly represents u u0.
Linearizing about the equilibrium (x0, u0) (0, 0) leads to

.
x = x
(53)
y = Cx
z = Az + B(y g(x0 ))
(56)
u = u0 + Cz
(48)
Example 14. Suppose we wish to control the simple pendulum under the output feedback
which is stable.
Now suppose that the state is not available for feedback.
Rather, the control is restricted to measurements
y = g(x)
(49)
By using a similar analysis, we can construct stabilizing output feedback for the nonlinear system based on stabilizing
output feedback for the linearization.
Definition 12. The equilibrium (x0, u0) of Eq. (33) is stabilized by the dynamic output feedback
z = F (z, y)
u = G(z)
y = (1 0)x
(57)
Linearizing about the upright equilibrium leads to
.
x =
0
g/

1
0
def
x +
u = Ax + Bu
c
1
y = (1 0)x = Cx
The observer based controller
z = (A BK HC)z + H y
(50)
(58)
def
u = Kz
(59)
stabilizes the linearization for appropriate gain matrices
if for some z0,
(z0, g(x0)) is an equilibrium of Eq. (50)

u0 G(z0)
(x0, z0) is an asymptotically stable equilibrium of the
closed loop dynamics
x = f (x, G(z))
z = F (z, g(x))
(51)
Let (x0, u0) be an equilibrium of Eq. (33), and dene y0

g(x0). In case g is continuously differentiable, we can approximate Eq. (49) as
y y0 + Dg(x0 )(x x0 )
(52)
K = (k1
k2 ),
h1
H=
h2

(60)
Since x0, u0, y0 0, the same controller (with input y and

output u) stabilizes the nonlinear pendulum.
BASIC GAIN SCHEDULING
Gain Scheduled Command Following
We now outline the basic procedure of gain scheduling in the
context of command following. The nonlinear plant of interest
is Eq. (33). The objective is to make the measured output Eq.
(49) approximately follow reference commands, r.
210
GAIN SCHEDULING
The primary motivation of gain scheduling is to address

local limitations associated with a control design based on a
single linearization. The main problem is that the performance and even stability of the closed loop system can deteriorate signicantly when the system is not operating in the
vicinity of the equilibrium.
Example 15. Consider the scalar system
x = x|x| + u
y=x
(61)
By linearizing about the equilibrium (x0, u0) (0, 0), we obtain the linear control u x r. For r 0, this control law
stabilizes the equilibrium (0, 0). The resulting closed loop system is
x = x|x| x
(62)
For x(0) 1, the solution asymptotically approaches 0. However, for x(0) 1, the solution diverges to innity.
Step 1: Construction of Linearization Family. Gain scheduling
attempts to overcome local limitations by considering a family
of linearizations, rather than a single linearization.
Definition 16. The functions (xeq( ), ueq( )) dene an equilibrium family for the nonlinear system Eq. (33) over the set
S if
f (xeq (s), ueq (s)) = 0
(63)
for all s S.
Example 17. Recall the controlled pendulum

x2
d x1
= g
sin(x1 ) cx2 + u
dt x2

An equilibrium family over the set S [, ] is

s
g
xeq (s) =
, ueq (s) = sin s

0
The associated linearization family is

0
1
0
x +
u
x = g
1
cos(s) c

(67)
(68)
(69)
Step 2: Fixed Operating Condition Designs. Let us select several operating conditions,
{s1 , s2 , . . ., sN } S
(70)
which characterize the variations within the operating envelope.

At the ith equilibrium, si, we can linearize the plant dynamics about the equilibrium (xeq(si), ueq(si)) and design a stabilizing linear controller to achieve approximate command following using any suitable linear design methodology. The result
is an indexed collection of controllers,
z = Ai z + Bi y + Li r
u = Ci z
(71)
where r denotes the reference command in local coordinates,
Associated with an equilibrium family are the output equilibrium values

def
yeq (s) = g(xeq (s))
(64)
The equilibrium family induces the following linearization

family for Eq. (33) with measurement Eq. (49),
.
x = A(s)x + B(s)u
y = C(s)x
(65)
where
(66)
C(s) = Dg(xeq (s))

The variable, s, which we will call the scheduling variable,
parameterizes a family of equilibrium points and plant linearizations. Typically, s can be a combination of both endogenous
and exogenous signals (recall discussion of missile autopilot
earlier). Any xed s will be called an operating condition, and
the set S denes the operation envelope, or range of operating conditions.
(72)
This step constitutes the core of gain scheduling, and accordingly, accounts for the bulk of the effort in a gain scheduled control design. Designing xed operating point controllers is especially tedious in the case of several design
operating conditions.
Step 3: Scheduling. The remaining step is to piece together
a global controller from the individual local controllers. As
the scheduling variable varies in time, the control gains are
updated to reect the current operating condition of the plant.
The resulting overall controller is
z = A(s)z + B(s)(y yeq (s)) + L(s)(r yeq (s)),
A(s) = D1 f (xeq (s), ueq (s)),

B(s) = D2 f (xeq (s), ueq (s)),
r = r yeq (si )
u = ueq (s) + C(s)z
(73)
The matrices A(s), B(s), C(s), and L(s) are functions of the
scheduling variable, as are the vectors yeq(s) and ueq(s). It is
important to note that the scheduling variable, s, which was
held constant during the design phase, is now time varying.
These matrix and vector functions are used to update the control parameters according to the variations in the scheduling
variable.
There are different options in how to schedule the controller parameters in Eq. (73).
GAIN SCHEDULING
Switched Scheduling. The operating envelope is divided

into disjoint regions, Ri, so that
S R1 . . . RN
(74)
and the controller matrices in (73) are scheduled according to
A1 , . s R 1 ;
..
, . . ., ueq (s)
A(s) =
A ,
.
s
R
N
N
ueq (s1 ),
s R1 ;
..
=
(75)
.
R
u (s ),
N
eq N
Continuous Scheduling. Any interpolation algorithm is
used to construct continuous matrices which interpolate
the design conditions so that
A(si ) = Ai , . . ., ueq (si ) = ueq (si )
(76)
which are globally stable, as opposed to the local stability of

Example 15.
Notice in this example that the linear control term (x
xeq(s)) has no effect on the closed loop equations. This is because of the smooth scheduling implementation with s x as
the scheduling variable. In this case, a desirable feedback
loop was eliminated in the scheduling implementation. It is
also possible to introduce undesirable feedback during implementation.
Theoretical Foundations
The gain scheduled controller is designed so that stability and
performance are achieved whenever the plant is in the vicinity one of the design operating conditions. Since the plant actually varies throughout the entire operating regime, an important question is to what degree the local properties of the
individual operating point designs carry over to the global
system.
The overall closed loop equations for a gain scheduled system are
x = f (x, u)
Some important points to consider are the following:

Gain scheduling is still based on linearizations, and
hence can ignore important nonlinear phenomena.
The xed operating point designs assume a constant
scheduling variable which is actually time varying.
Implementing the gain scheduled controller introduces
feedback loops which are not present in the xed operating point designs.
z = A(s)z + B(s)y + L(s)r

u = C(s)z + ueq (s)
ueq (s) = s|s|
(77)
leads to the linearization family

.
x = 2|s|x + u
(78)
Because of the simplicity of this system, we are able to design

controls for all s, rather than selected s. A suitable linear design for command following is
u = 3|s|x + (r x)
(79)
Implementing this design using smooth scheduling leads to

the gain scheduled control
u = ueq (s) + u
= ueq (s) 3|s|(x xeq (s)) + ((r xeq (s)) (x xeq s))
(80)
r = r yeq (s)
In general, the scheduling variable, s, can be written as
u = x|x| + (r x)
This feedback leads to the closed loop dynamics
x = x + r
LPV Systems. It is convenient to consider slow variation restriction in the context of linear parameter varying (LPV) systems. LPV systems are dened to be linear systems whose
dynamics depend on exogenous time varying parameters
which are unknown a priori, but can be measured upon operation of the control system.
An LPV system can be represented in state space form as
y = C( )x
(81)
(82)
(84)
for an appropriate function, . Clearly, the overall system is

nonlinear and hence, requires nonlinear methods for analysis.
An analysis of these equations (see Bibliography for
sources) leads to the conclusion that the overall gain scheduled system will exhibit similar stability and performance as
the local designs whenever (1) the scheduling variable, s,
changes sufciently slowly, and (2) the plant dynamics are
predominantly nonlinear in the scheduling variable.
The following sections provide some insight into these restrictions.
x = A( )x + B( )u
For the scheduling variable s x, the control becomes
(83)
y = g(x) yeq (s)
s = (x, r)
Example 18. Recall the system of Example 15. The equilibrium family
xeq (s) = s,
211
(85)
where is a time varying parameter. Typical assumptions on

are magnitude bounds; for example,
|| max
(86)
212
GAIN SCHEDULING
and rate bounds; for example,

| | max
By subtracting equation (91) from equation (89), we obtain

(87)
LPV systems form the underlying basis of gain scheduling.

It is convenient to associate the parameter with the scheduling variable and the LPV structure with the linearization
family, although this is not always the case as will be seen.
The following is a classical result from differential equations stated in an LPV context.
Theorem 2. If the equilibrium xo 0 of Eq. (85) is asymptotically stable for all constant , then it is asymptotically stable
for all time varying provided that max is sufciently small.
The relevance of Theorem 19 to gain scheduling is as follows. A closed loop LPV system is such that good stability and
performance is expected for xed values of the parameter/
scheduling variable. However, performance and even stability
can deteriorate in the presence of parameter time variations.
Theorem 19 provides a sufcient condition for the xed parameter properties to carry over to the varying parameter
setting.
Example 20. A classical example of instability from xed parameter stability is the time-varying oscillator,

0
1
x(t)
=
x(t)
(88)
(1 + (t)/2) 0.2
d
dt

s
0
= M(s)
+ B(u ueq (s))
v
v veq (s)
(92)
If veq(s) is differentiable,
d
veq (s) = Dveq (s)s
dt
= Dveq (s)M12 (s)(v veq (s)) + Dveq (s)B1 (u ueq (s))
(93)
where the matrices M12(s) and B1 are appropriate sub-matrices of M(s) and B.
Combining these equations leads to the alternate form of
Eq. (89),
d
dt

s
v veq (s)

0
M12 (s)
s
=
0 M22 (s) Dveq (s)M12 (s)
v veq (s)

B1
(u ueq (s))
+
B2 Dveq (s)B1
(94)
(95)
which can be written as
d
dt

s
s
= Anew (s)
+ Bnew (s)u
v
v
These equations can be viewed as a mass-spring-damper system with time-varying spring stiffness. For xed parameter
values, (t) o, the equilibrium xo 0 is asymptotically stable. However, for the parameter trajectory (t) cos(2t), it
becomes unstable. An intuitive explanation is that the stiffness variations are timed to pump energy into the oscillations.
where
Quasi-LPV Representation. It is also convenient to consider

the relationship between the scheduling variable and plant
nonlinearities in an LPV setting.
The relationship between LPV systems and gain scheduling is not limited to linearization families. Consider the following special nonlinear plant in which the scheduling variable is a subset of the state,

s
d s
= (s) + M(s)
+ Bu
(89)
dt v
v
The original equations now take a quasi-LPV form, where

the parameter is actually an endogenous variable. Note that
no linearization approximations were made to bring Eq. (89)
to the form Eq. (96).
This transformation shows that an underlying LPV structure exists, even without linearizations, in the extreme case
that the plant dynamics are nonlinear only in the scheduling
variable. Any additional nonlinearities not captured by the
scheduling variable enter as high order perturbations in Eq.
(96). This transformation then reveals the importance of the
scheduling variable to capture the plant nonlinearities.
These equations represent the extreme case where the nonlinearities are entirely captured in the scheduling variable, s.
Let (xeq(s), ueq(s)) be an equilibrium family, with

s
(90)
xeq (s) =
veq (s)
so that

s
0 = (s) + M(s)
+ Bueq (s)
veq (s)
v(t)
= v(t) veq (s(t)),
u(t)
= u(t) ueq (s(t))
(96)
(97)
Example 21. Consider the nonlinear system

x1 |x1 | + x2
x =
x21 x2 + u
(98)
and let s x1 be the scheduling variable. These equations

take the form of Eq. (89). The resulting equilibrium family is
(91)

s
xeq (s) =
,
s|s|
ueq (s) = s3 |s|
(99)
GAIN SCHEDULING
Performing the transformations described above leads to the

quasi-LPV form
d
dt

s
0
1
s
=
x2 (s|s|)
0 s2 2|s|
x2 (s|s|)

0
+
(u (s3 |s|)
1
In order to convexify the problem, consider the change in

variables
Q = P1 ,
(100)
Yi = Ki P1
Convex Optimization for LPV Systems

Because of the availability of numerically efcient methods
for large scale problems, convex optimization is an emerging
technique for gain scheduling design of for LPV and quasiLPV systems. As seen in the previous section, the development of methods for LPV systems is directly pertinent to gain
scheduling, since LPV systems form the underlying structure
of a gain scheduled design.
The main idea in convex optimization methods for LPV
systems is to combine stability and performance parameters
with controller parameters in a single convex optimization objective.
We will demonstrate these methods in the following simple
context. Consider the open loop LPV system
x = (A1 + (1 )A2 )x + Bu
The scheduling process is built into the construction of

the state feedback; it is not necessary to perform several
xed parameter designs.
Stability for arbitrarily fast parameter variations is assured.
Theorem 22 is only a sufcient condition for stability,
and hence may be conservative.
The method extends to more general control objectives,
other than stabilizing state feedback, including
(101)
(102)
We are interested in constructing stabilizing gain scheduled

state feedback. Let us impose the feedback structure
u = (K1 + (1 )K2 )x
(103)
which mimics the LPV variations of the system. The closed

loop dynamics are then
x = ( (A1 BK1 ) + (1 )(A2 BK2 ))x
disturbance rejection and command following

output feedback
rate constrained parameter variations
Extended/Pseudo-Linearization
The objective in extended and pseudo-linearization is to impose that the closed loop system has a linearization family
which is invariant in some desired sense.
Let us consider the special case of a tracking problem with
full state feedback for the nonlinear system Eq. (33). The objective is for the rst state, x1, to approximately track reference commands, r. Let the equilibrium family (xeq(s), ueq(s)) be
an such that
(104)
(1 0
A sufcient condition which guarantees the stability of

(104) is the following.
Theorem 22. The equilibrium xo 0 of Eq. (104) is asymptotically stable if there exists a positive denite matrix, P
PT 0, such that for both i 1 and i 2,
P(Ai BKi ) + (Ai BKi )T P < 0
(107)
Now the set of Q 0, Y1, and Y2 which satisfy Eq. (107) is

convex. This allows one to employ efcient convex feasibility
algorithms which either produce a feasible set of matrices or
determine denitely that no solution exists.
Some important points to consider are the following:
where the parameter is constrained by

0 1
(106)
Given Q 0 and Yi, one can solve for the original variables P
and Ki. With these variables, condition Eq. (105) is equivalent to
Ai Q BYi + QATi YiT BT < 0
ADVANCED METHODS FOR GAIN SCHEDULING
213
...
0)xeq (s) = s
(108)
Now consider the nonlinear feedback

u = G(x, r)
(109)
ueq (s) = G(xeq (s), s)
(110)
where
(105)
It is important to note that Theorem 22 only provides a

sufcient condition for stability. The main idea is that the
matrix P denes the Lyapunov function, V(x) xTPx, for the
closed loop system Eq. (104).
Our objective is to nd matrices K1, K2, and P which satisfy
Eq. (105). It can be shown that the set of matrices which satisfy Eq. (105) is not convex. This lack of convexity signicantly complicates any direct search process.
Then, a linearization family for the closed loop system

x = f (x, G(x, r))
(111)
is
x = (D1 f (xeq (s), G(xeq (s), s))

+ D2 f (xeq (s), G(xeq (s), s))D1 G(xeq (s), s))x
+ D2 f ((xeq (s), G(xeq (s), s))D2 G(xeq (s), s)r
(112)
214
GAIN SCHEDULING
One invariance objective is for the closed loop linearization

family to be constant; for example,
.
x = Adesx + Bdes r
H. K. Khalil and P. K. Kokotovic, On stability properties of nonlinear

systems with slowly varying inputs, IEEE Trans. Autom. Control,
36: 229, 1991.
(113)
M. Vidyasagar, Nonlinear Systems Analysis, Nonlinear Systems Analysis, Englewood Cliffs, NJ: Prentice-Hall, Inc., 1993.
One can state appropriate conditions involving partial differential constraint equations under which there exists any G
which achieves this objective.
The intention is that improved closed loop performance is
possible if the closed loop linearizations have some sort of invariance property. Such improved performance, if any, is difcult to quantify, but simulation studies indicate the potential benets.
A. Bacciotti, Local Stabilizability of Nonlinear Control Systems, Singapore: World Scientic Publishing Co., 1992.
Example 23. Consider the nonlinear system

x1
x2
=
x2
x1 x2 + x21 + u
The equilibrium family suggested by x1 is

s
xeq (s) =
, ueq (s) = s2
0
Overview of Gain-Scheduling and Its Theoretical Foundations

J. Wang and W. J. Rugh, On parameterized linear systems and linearization families for nonlinear systems, IEEE Trans. Circuits
Syst., 34: 650657, 1987.
J. S. Shamma and M. Athans, Analysis of nonlinear gain-scheduled
control systems, IEEE Trans. Autom. Control, 35: 898907, 1990.
(114)
W. J. Rugh, Analytical framework for gain-scheduling, IEEE Control

Syst. Magazine, 11: 7984, 1991.
J. S. Shamma and M. Athans, Gain scheduling: Potential hazards
and possible remedies, IEEE Control Syst. Magazine, 12: 101
107, 1992.
(115)
LPV System Analysis and Control
Suppose we want the closed loop linearization family to have

dynamics matrices

0
1
0
Ades =
,
Bdes =
(116)
1 2
1
J. S. Shamma and M. Athans, Guaranteed Properties of Gain Scheduled Control of Linear Parameter Varying Plants, Automatica, 27:
559564, 1991.
S. M. Shahruz and S. Behtash, Design of controllers for linear parameter-varying systems by the gain scheduling technique, J. Math.
Anal. Appl., 168: 125217, 1992.
A. Packard, Gain scheduling via linear fractional transformations,
An obvious selection for G(x, r) is

G(x, r) = x1 x2 x21 x1 2x2 + r
H. K. Khalil, Nonlinear Systems, 2nd Ed., New York: Macmillan,

1996.
(117)
Then setting u G(x, r) leads to the linear closed loop dynamics

0
1
0
x =
x+
r
(118)
1 2
1
The above choice for G achieves what is called feedback linearization, since the resulting closed loop dynamics are linear.
It is not necessary that feedback linearization is achieved.
For example, consider the modied dynamics

x1
x2 + x22 u
=
(119)
x1 x2 + x21 + u
x2
The resulting equilibrium families and linearization families
are the same. Furthermore, the same choice of G achieves the
desired invariance objective.
FOR FURTHER REFERENCE
Nonlinear Systems Analysis
M. Kelemen, A stability property, IEEE Trans. Autom. Control, 31:
766768, 1986.
D. A. Lawrence and W. J. Rugh, On a stability theorem for nonlinear
systems with slowly varying inputs, IEEE Trans. Autom. Control,
35: 860864, 1990.
P. Apkarian and P. Gahinet, A convex characterization of gain-scheduled H controllers, IEEE Trans. Autom. Control, 40: 853864,
1995.
P. Apkarian and R. J. Adams, Advanced gain-scheduling techniques
for uncertain systems, IEEE Trans. Control Syst. Technol., 60: 21
32, 1998.
Extended and Pseudo-Linearization
C. Reboulet and C. Champetier, A new method for linearizing nonlinear systems: the pseudolinearization, Int. J. Control, 40: 631
638, 1984.
W. T. Baumann and W. J. Rugh, Feedback control of nonlinear systems by extended linearization, IEEE Trans. Autom. Control, 31:
4046, 1986.
J. Wang and W. J. Rugh, Linearized model matching for single-input
nonlinear systems, IEEE Trans. Autom. Control, 33: 793796,
1988.
J. Huang and W. J. Rugh, On a nonlinear servomechanism problem,
Automatica, 26: 963972, 1990.
J. Huang and W. J. Rugh, Approximate noninteracting control with
stability for nonlinear systems, IEEE Trans. Autom. Control, 36:
295304, 1991.
D. A. Lawrence and W. J. Rugh, Input-output pseudolinearization for
nonlinear systems, IEEE Trans. Autom. Control, 39: 22072218,
1994.
I. Kaminer, A. M. Pascoal, P. P. Khargonekar, and E. Coleman, A
velocity algorithm for the implementation of gain-scheduled controllers, Automatica, 31: 11851192, 1995.
D. A. Lawrence and W. J. Rugh, Gain scheduling dynamic linear controllers for a nonlinear plant, Automatica, 31: 381390, 1995.
GALLIUM ARSENIDE FIELD EFFECT TRANSISTOR LOGIC CIRCUITS

Applications of Gain Scheduling
K. J. Astrom and B. Wittenmark, Adaptive Control, Chapter 9, Reading, MA: Addison-Wesley, 1989.
R. A. Nichols, R. T. Reichert, and W. J. Rugh, Gain scheduling for
H controllers: A ight control example, IEEE Trans. Control Syst.
Technol., 1: 6979, 1993.
J. S. Shamma and J. R. Cloutier, Gain-scheduled missile autopilot
design using linear parameter varying methods, J. Guidance, Control, Dynamics, 16: 256263, 1993.
J. Reeve and M. Sultan, Gain scheduling adaptive control strategies
for HVDC systems to accommodate large disturbances, IEEE
Trans. Power Syst., 9: 366372, 1994.
T. Meressi and B. Paden, Gain scheduled H controllers for a 2 link
exible manipulator, J. Guidance, Control, Dynamics, 17: 537
543, 1994.
P. Apkarian, P. Gahinet, and J. M. Biannic, Self-scheduled H-innity
control of a missile via LMIs. AIAA J. Guidance, Control, Dynamics, 18: 532538, 1995.
L. H. Carter and J. S. Shamma, Gain scheduled bank-to-turn autopilot design using linear parameter varying transformations, AIAA
J. Guidance, Control, Dynamics, 19: 10561063, 1996.
D. J. Leith and W. E. Leithead, Appropriate realization of gain scheduled controllers with application to wind turbine regulation, Int.
J. Control, 65: 223248, 1996.
R. D. Smith, W. F. Weldon, and A. E. Traver, Aerodynamic loading
and magnetic bearing controller robustness using a gain-scheduled Kalman lter, J. Eng. Gas Turbines Power, 118: 846842,
1996.
Alternative Perspectives on Gain-Scheduling
E. Gazi, W. D. Seider, and L. H. Ungar, Control of nonlinear processes using qualitative reasoning, Comput. Chem. Eng., 18: 189
193, 1994.
K. Pawelzik, J. Kohlmorgen, and K. R. Muller, Annealed competition
of experts for a segmentation and classication of switching dynamics, Neural Computat., 8: 340356, 1996.
S. C. Kramer and R. C. Martin, Direct optimization of gain scheduled
controllers via genetic algorithms, J. Guidance, Control, Dynamics,
19: 3846, 1996.
JEFF S. SHAMMA
The University of Texas at Austin
215

Copyright
H INFINITY CONTROL
This article describes an optimal multivariable control system design technique for achieving robustness under
external disturbances and model uncertainties. We show that many robust control problems can be formulated
as H norm optimization problems, and we describe analytically their solutions. We also give guidelines to the
choice of design parameters and insights to this optimal and robust control theory.
One of the motivations for the original introduction of H methods by Zames (1) was to bring plant
uncertainty, specified in the frequency domain, back to the center stage as it had been in classical control, in
contrast to analytic methods such as linear quadratic Gaussian (LQG) control. The H norm was found to be
appropriate for specifying both the level of plant uncertainty and the signal gain from disturbance inputs to
error outputs in the controlled system.
The standard H optimal control problem is concerned with the feedback system shown in Fig. 1(a)
where w represents an external disturbance, y is the measurement available to the controller, u is the output
from the controller, and z is an error signal that should be kept small. The transfer function matrix G represents
not only the conventional plant to be controlled but also any weighting functions included to specify the desired
performance, which will be discussed in more detail later. Suppose that G is partitioned consistent with the
inputs w and u and outputs z and y as
The closed loop transfer function from w to z, denoted by T zw , is defined to be the linear fractional transformation
(LFT) of G on K:
The H optimal control problem is then to design a stabilizing controller K, so as to minimize the H norm of
T zw , which is defined in the next section and is denoted by T zw . The H norm gives the maximum energy
gain, or sinusoidal gain of the system. This is in contrast to the H 2 norm T zw 2 , which for example gives the
variance of the output given white noise disturbances. The important property of the H norm comes from
the application of the small gain theorem, which states that if T zw , then the system in Fig. 1(b) will be
stable for all stable with < 1/. It is probably the case that this robust stability consequence was one
of the main motivations for the development of H methods. The synthesis of controllers that achieve an H
norm specification hence gives a well-defined practical and mathematical problem.
1
H INFINITY CONTROL
Fig. 1. Most control systems can be put in this unified framework where w represents an external disturbance, y is the
measurement available to the controller, u is the output from the controller, and z is an error signal that it is desired to
keep small.
H 2 and H Norms
We consider a q-input and p-output dynamical system with the matrix transfer function G(s). Let G(s) have
the following stabilizable and detectable state space realization:
We shall denote this state-space realization by
Many control design problems can be regarded as finding a suitable controller so that the undesirable responses
of the system are made small in some sense. There are obviously many ways to define the smallness for a given
control problem. Here we are mainly interested in one way of defining the smallness: the H norm. For
comparison we shall also mention another more classical way of defining the smallness in terms of the H 2
norm.
Let R H 2 denote the set of strictly proper and real rational stable transfer matrices. In terms of state-space
realizations, R H 2 is simply the set of finite dimensional systems with D = 0 and stable A. The H 2 norm of a
G(s) R H 2 is defined as
H INFINITY CONTROL
where trace(M), M, and i (M) denote, respectively, the trace, the complex conjugate transpose, and the ith
singular value of the matrix M. This norm can also be computed in the time domain as
where g(t) is the inverse Laplace transform of G(s). Thus the H 2 norm of a system is a measure of the total
energy of the system impulse response. It can be computed using state-space realization as
where Q and P are observability Gramian and controllability Gramian, which can be obtained from the following
Lyapunov equations
Let R H denote the set of proper (but not necessarily strictly proper) and real rational stable transfer
matrices. In terms of state-space realizations, R H includes all finite dimensional systems with stable A
matrices. The H norm of a transfer matrix G(s) R H is defined as
where (M)
denotes the largest singular value of a matrix M. When G(s) is a single input and single output
system, the H norm of the G(s) is simply the peak value on the Bode magnitude plot of the frequency response
G(j). It can also be regarded as the largest possible amplification factor of the systems steady state response
to sinusoidal excitations. For example, the steady state response of the system with respect to a sinusoidal
input u(t) = U sin(0 t + ) is
and thus the maximum possible amplification factor is
which is precisely the H norm of the transfer function. As an example, consider a standard second-order
system
Then max = n
1 22 and G = |G(jmax )| = 1/(2
1 2 ). If G(s) is the description of
a structure vibration, then max would be the most dangerous exciting frequency.
In the multiple input and multiple output (MIMO) case, the H norm of a transfer matrix G R H is
the peak value on the largest singular value Bode plot of the frequency response G(j). Analogous to the scalar
H INFINITY CONTROL
case, the H norm of G(s) can also be regarded as the largest possible amplification factor of the systems
steady state response to sinusoidal excitations in the following sense: Let the sinusoidal inputs be
Then the steady state response of the system can be written as
for some yi , i , i = 1, 2, . . ., p. Furthermore,
where is the Euclidean norm.

The H norm of a stable transfer matrix can also be thought of as the maximum amplification factor of
the input energy of the system at the output of the system. More precisely,
where denotes the time domain convolution and L 2 [0, ) denotes the space of all square integrable functions
with the norm defined as f 2 : =
. Thus it is important to make the H norm of all undesirable
transfer functions small in a feedback control system. That is one of the motivations for the development of
H control theory.
This discussion shows that the H norm of a transfer function can, in principle, be obtained either
graphically or experimentally. To get an estimate, set up a fine grid of frequency points, {1 , . . ., N }. Then an
estimate for G is
H INFINITY CONTROL
This value is usually read directly from a singular value Bode plot. The H norm can also be computed directly
using state-space representations. Let
Then
where
Hence, the H norm of a matrix transfer function can be calculated to the specified accuracy by a bisection
search.
Example. Consider a mass/spring/damper system as shown in Fig. 2. The dynamical system can be
described by the following differential equations:
Suppose that G(s) is the transfer matrix from (F 1 , F 2 ) to (x1 , x2 ) and suppose k1 = 1, k2 = 4, b1 = 0.2, b2 =
0.1, m1 = 1, and m2 = 2 with appropriate units. Then the H 2 norm of this transfer matrix is G(s)2 = 2.56,
whereas the H norm of this transfer matrix is G(s) = 11.47, which is shown as the peak of the largest
singular value Bode plot in Fig. 3. Since the peak is achieved at max = 0.8483, exciting the system using the
H INFINITY CONTROL
Fig. 2. A two mass/spring/damper system with the external forces F 1 and F 2 as inputs and the positions of the two masses
as outputs.
Fig. 3. The singular value Bode plot of the two mass/spring/damper system. The H norm G is the peak of the largest
singular value plot of G(j).
following sinusoidal input
gives the steady state response of the system as
This shows that the system response will be amplified 11.47 times for an input signal at the frequency max ,
which could be undesirable if F 1 and F 2 are disturbance force and x1 and x2 are the positions to be kept steady.
We will see later how to design an H feedback controller to suppress this kind of vibration.
We note that the state-space computational method is usually much more accurate than the graphical
method. Consider, for example, the standard second-order system again with n = 1 and = 0.01. By the
H INFINITY CONTROL
analytic formula or the state-space computational method, we get that the H norm is 50.0025. To estimate
the H norm graphically, we set up a frequency grid to compute the frequency response of G over a suitable
range of frequency. Take, for example, 200 points in the frequency range of [0.1, 10] uniformly on the log scale,
then we get an estimate of the norm 33.0743. This shows clearly that the graphical method may lead to a
wrong answer for a lightly damped system if the frequency grid is not sufficiently dense. Indeed, we would
get the H norm 43.5567, 48.1834, and 49.5608 from the graphical method if 400, 800, and 1600 frequency
points are used, respectively.
Weighted H Performance
We now consider how to formulate some performance objectives into mathematically tractable problems. It
is well known that the performance objectives of a feedback system can usually be specified in terms of
requirements on the sensitivity functions and/or complementary sensitivity functions or in terms of some other
closed-loop transfer functions. For instance, the performance criteria for a scalar system may be specified as
requiring
with S(j) = 1/(1 + P(j)K(j)) where P is the plant and K is the controller. However, it is much more convenient
to reflect the system performance objectives by choosing appropriate weighting functions. For example, this
performance objective can be written as
with
In order to use W e in control design, a rational transfer function W e (s) is usually used to approximate this
frequency response.
The advantage of using weighted performance specifications is obvious in multivariable system design.
First, some components of a vector signal are usually more important than others. Second, each component of
the signal may not be measured in the same units; for example, some components of the output error signal
may be measured in terms of length, and the others may be measured in terms of voltage. Weighting functions
are essential to make these components comparable. Also, we might be primarily interested in rejecting errors
in a certain frequency range (e.g., low frequencies); hence, some frequency-dependent weights must be chosen.
Consider a standard feedback diagram in Fig. 4. The weighting functions in Fig. 4 are chosen to reflect
the design objectives and knowledge on the disturbances and sensor noise. For example, W d and W i may be
chosen to reflect the frequency contents of the disturbances d and di . The weighting matrix W n is used to model
the frequency contents of the sensor noise, whereas W e may be used to reflect the requirements on the shape
of certain closed-loop transfer functions (e.g., the shape of the output sensitivity function). Similarly, W u may
be used to reflect some restrictions on the control or actuator signals, and the dashed precompensator W r is an
H INFINITY CONTROL
Fig. 4. Standard feedback configuration with disturbance weights and performance weights. W i , W d , and W n represent
the frequency contents of the input disturbance, output disturbance, and the sensor noise. W e represents the disturbance
rejection requirement, and W u puts the limit on the control effort. W r shapes the input signal.
optional element used to achieve deliberate command shaping or to represent a nonunity feedback system in
equivalent unity feedback form.
A typical control design may involve making the sensitivity function small over a suitable frequency
range while keeping the control effort within a reasonable limit. This may be mathematically formulated as
minimizing
subject to some restrictions on the control energy or control bandwidth:
Or more frequently, one may introduce a parameter and a mixed criterion
Note that can be absorbed into W u , so there is no loss of generality in assuming = 1. Finding a controller
so that the H norm of a certain closed-loop transfer function, such as the preceding one, is minimized is the
H control problem.
Similar H 2 norm minimization problems can be formulated if the disturbance is modeled as white noise
and the performance is measured in terms of output power.
H INFINITY CONTROL
Robust Stabilization
Another way that a weighted H norm minimization problem can arise naturally is when we consider robust
stability and robust performance of a closed-loop system with model uncertainties. For example, consider a
unity feedback system with a family of additively perturbed uncertain dynamical systems:
and assume that K stabilizes the nominal plant P. Then by the small gain theorem, the uncertain system is
stable for all admissible with < 1/ if and only if
Therefore, a related synthesis problem is to find a controller K so that this inequality holds.
be a normalized coprime factorization, that is,
1N
As another example, let P = M
Now consider a family of coprime factor perturbed uncertain systems:
A controller K stabilizing the nominal system P will robustly stabilize the family Fp if and only if
Similarly, many other robust stability problems can be formulated. It should be noted that the H 2 norm cannot
be used in the robust stability test because it does not satisfy a key multiplicative property (i.e., G1 G2 2
G1 2 G2 2 in general).
Selection of Weighting Functions

As we mentioned previously, a very important step in the H control design process is to choose the appropriate
weights. The appropriate choice of weights for a particular problem is not trivial. On many occasions, the
weights are chosen purely as a design parameter without any physical bases, so these weights may be treated
as tuning parameters that are chosen by the designer to achieve the best compromise between the conflicting
objectives. Hence, the selection of weighting functions for a specific design problem often involves ad hoc fixing,
iterations, and fine-tuning. It should be guided by the expected system inputs and the relative importance of
the outputs. It is very hard to give a general formula for the weighting functions that will work in every case.
Nevertheless, we shall try to give some guidelines in this section by looking at a typical single input single
output (SISO) problem.
10
H INFINITY CONTROL
Fig. 5. The desired shapes of S and KS and their upper bounds.
Consider the SISO feedback system shown in Fig. 4. Then the tracking error is
where S = (I + PK) 1 is the output sensitivity function and T = I S = PK(I + PK) 1 is the output
complementary sensitivity function. Note that tracking error is closely related to the low-frequency gain of S.
In particular, we must keep |S| small over the range of frequencies, typically low frequencies where r and d
are significant. For example, if we need the steady state error with respect to a step input to be no greater
than , then we need |S(0)| . Hence, the steady state tracking requirement can be fulfilled by constraining
the low-frequency gain of S. From classical control theory, we know that the dynamical quality of the system
time response can be quantified by rise time, settling time, and percent overshoot. Furthermore, the speed
of the system response is inversely proportional to the closed-loop bandwidth and the overshoot of the system
response increases with the resonant peak sensitivity defined as M s := S . Let b = min {:|S(j)| 1}.
Then we can regard b as the closed-loop bandwidth because, beyond this frequency, the closed-loop system
will not be able to track the reference and the disturbance will actually be amplified.
Now suppose that we are given the time domain performance specifications. Then we can determine the
corresponding requirements in frequency domain in terms of the low-frequency gain, the bandwidth b , and
the peak sensitivity M s . Hence, a good control design should result in a sensitivity function S satisfying all
these requirements as shown in Fig. 5. These requirements can be approximately represented as
If a steeper transition between low frequency and high frequency is desired, the weight W e can be modified as
follows:
for some integer k 1.
H INFINITY CONTROL
11
The selection of control weighting function W u follows similarly from the preceding discussion by considering the control signal equation
The magnitude of |KS| in the low-frequency range is essentially limited by the allowable cost of control effort
and saturation limit of the actuators; hence, in general the maximum gain M u of KS can be fairly large,
whereas the high-frequency gain is essentially limited by the controller bandwidth bc and the (sensor) noise
frequencies. Ideally, we would like to roll off as fast as possible beyond the desired control bandwidth so that
the high-frequency noises are attenuated as much as possible. Hence, a candidate weight W u would be
However, the standard H control design techniques cannot be applied directly to a problem with an improper
control weighting function. Hence, we shall introduce a faraway pole to make W u proper:
for a small 1 > 0 as shown in Fig. 5. Similarly, if a faster rolloff is desired, we may choose
for some integer k 1.

The weights for MIMO problems can be initially chosen as diagonal matrices with each diagonal term
chosen in the preceding form.
General Problem Formulation and Solutions

All the disturbance rejection problems and robust stabilization problems discussed in the previous sections can
be put in a unified framework of linear fractional transformation as shown in Fig. 1(a). For example,
12
H INFINITY CONTROL
and
For the following discussion, let us assume that state-space models of G and K are available and that their
realizations are assumed to be stabilizable and detectable. We say that a controller is admissible if it internally
stabilizes the system.
Optimal H 2 Control:
Find an admissible controller K(s) such that T zw 2 is minimized.
Optimal H Control:
Find an admissible controller K(s) such that T zw is minimized.
It should be noted that the optimal H controllers as defined here are generally not unique for MIMO systems. Furthermore, finding an optimal H controller is often both numerically and theoretically complicated.
This is certainly in contrast to the standard LQG or H 2 theory, in which the optimal controller is unique and can
be obtained by solving two Riccati equations without iterations. Knowing the achievable optimal (minimum)
H norm may be useful theoretically because it sets a limit on what we can achieve. In practice, however, it
is often not necessary and sometimes even undesirable to design an optimal controller, and it is usually much
cheaper to obtain controllers that are very close in the norm sense to the optimal ones, which will be called
suboptimal controllers. A suboptimal controller may also have other nice properties (e.g., lower bandwidth)
over the optimal ones.
Suboptimal H Control:
Given > 0, find an admissible controller K(s), if there are any, such that T zw < .
For these reasons mentioned, we shall focus our attention on the suboptimal H control. We shall assume
that the realization of the transfer matrix G takes the following form:
which is compatible with the dimensions of z(t) Rp 1 , y(t) Rp 2 , w(t) Rm1 , and u(t) Rm2 and the state x(t)
Rn . We make the following assumptions:
H INFINITY CONTROL
13
Assumption 1 is necessary for the existence of stabilizing controllers. Assumption 2 means that the penalty
on z = C1 x + D12 u includes a nonsingular penalty on the control u, that the exogenous signal w includes
both plant disturbance and sensor noise, and that the sensor noise weighting is nonsingular. Relaxation of
assumption 2 leads to singular control problems.
Assumptions 3 and 4 are made for a technical reason: together with assumption 1 they guarantee that the
two algebraic Riccati equations in the corresponding LQG or H 2 problem have the desired solutions. Dropping
assumptions 3 and 4 would make the solution very complicated.
Define
The following H 2 and H control results can be found in Refs. 2 and 3.

Theorem 1. There exists a unique optimal controller
that minimizes T zw 2 where
X 2 0 and Y 2 0 are the solutions to
such that both Ax B2 R 1 1 BT 2 X 2 and Ay Y 2 CT 2 R 1 2 C2 are stable. Moreover,
Theorem 2. Suppose G satisfies assumptions 14. Then there exists an admissible controller K such that
F
(G, K ) < (i.e., T zw < ) if and only if
(1) there exists an X 0 such that
and Ax + (B1 BT 1 / 2 B2 R 1 1 BT 2 )X is stable;

(2) there exists a Y 0 such that
and Ay + Y (CT 1 C1 / 2 CT 2 R 1 2 C2 ) is stable;

(3) (X Y ) < 2 where () denotes the spectral radius.
14
H INFINITY CONTROL
Furthermore, if these conditions are satisfied, all internally stabilizing controllers K (s) satisfying F
(G,
K ) < can be parameterized as
for any Q R H such that Q < where
The controller obtained by setting Q = 0
is called the central controller. Comparing the H central controller to the H 2 optimal controller, we can see
that the H central controller will approach the optimal H 2 controller as .
Example. Consider again the two mass/spring/damper system shown in Fig. 2. Assume that F1 is the
control force, F 2 is the disturbance force, and the measurements of x1 and x2 are corrupted by measurement
noise:
Our objective is to design a control law so that the effect of the disturbance force F 2 on the positions of the two
masses x1 and x2 are reduced in a frequency range 0 2. The problem can be set up as shown in Fig. 6,
where
H INFINITY CONTROL
15
Fig. 6. Rejecting the disturbance force F 2 of the two mass/spring/damper system by a feedback control of F 1 .
is the performance weight and W u is the control weight. In order to limit the control force, we shall choose
Now let u = F 1 ,
then the problem can be formulated in a LFT form with
where P1 and P2 denote the transfer matrices from F 1 and F 2 to
respectively. Let
that is, we want to reject only the effect of the disturbance force F 2 on the position x1 . Then the optimal H 2
performance is F
(G, K 2 )2 = 2.6584, and the H performance with the optimal H 2 controller is F
(G, K 2 )
= 2.6079, whereas the optimal H performance with an H controller is F
(G, K ) = 1.6101. This means
16
H INFINITY CONTROL
Fig. 7. The largest singular value Bode plot of the closed-loop two mass/spring/damper system with an H 2 controller and
an H controller.
that the effect of the disturbance force F 2 in the desired frequency range 0 2 will be effectively reduced
with the H controller K by 5/1.6101 = 3.1054 times at x1 . On the other hand, let
that is, we want to reject only the effect of the disturbance force F 2 on the position x2 . Then the optimal H 2
performance is F
(G, K 2 )
(G, K ) = 0.5189. This means
with the H controller K by 5/0.5189 = 9.6358 times at x2 . Finally, set
that is, we want to reject the effect of the disturbance force F 2 on both x1 and x2 . Then the optimal H 2
performance is F
(G, K 2 )
(G, K ) = 4.3611. This means
only with the H controller K by 5/4.3611 = 1.1465 times at both x1 and x2 . This result shows clearly that it
is very hard to reject the disturbance effect on both positions at the same time. The largest singular value Bode
plots of the closed-loop system are shown in Fig. 7. We note that the H controller typically gives a relatively
flat frequency response because it tries to minimize the peak of the frequency response. On the other hand, the
H 2 controller typically produces a frequency response that rolls off fast in the high-frequency range but with
a large peak in the low-frequency range.
H INFINITY CONTROL
17
H Filtering
In this section we shall illustrate how an H filtering problem can be converted to a special H control
problem. Suppose that a dynamic system is described by the following equations:
The filtering problem is to find an estimate z of z in some sense using the measurement of y. The restriction on
the filtering problem is that the filter has to be causal so that it can be realized (i.e., z must be generated by a
causal system acting on the measurements). We will further restrict our filter to be unbiased, that is, given T
> 0, the estimate z (t) = 0 t [0, T] if y(t) = 0, t [0, T]. Now we can state our H filtering problem.
H Filtering:
Given a > 0, find a causal filter F(s) R H if it exists such that
with z = F(s)y.
This H filtering problem can also be formulated in an LFT framework because
Hence, the filtering problem can be regarded as a special H problem. However, comparing this filtering
problem to the control problems, we can see that there is no internal stability requirement in the filtering
problem. Hence, the solution to this filtering problem can be obtained from the H solution in the last section
by setting B2 = 0 and dropping the internal stability requirement. Thus, a rational causal filter F(s) is given
by
Understanding H Control
Most existing derivations and proofs of the H control results given in Theorem 2 are mathematically quite
complex. Some algebraic derivations are simple but they provide no insight to the theory for control engineers.
In this section, we shall present an intuitive but nonrigorous derivation of the H results by using only
some basic system theoretic concept such as state feedback and state estimation. In fact, we shall construct
intuitively the output feedback H central controller by combining an H state feedback and an observer.
18
H INFINITY CONTROL
A key fact we shall use is the so-called bounded real lemma, which states that for a system z = G(s)w with
state space realization G(s) = C(sI A) 1 B H , G < , which is essentially equivalent to
if and only if there is an X = X

0 such that
and A + BB
X/ 2 is stable. Dually, there is a Y = Y
0 such that
and A + YC
C/ 2 is stable.
Note that the system has the following state space realization:
To keep the presentation simple, we shall make some additional assumptions: D 12 C1 = 0, B1 D 21 = 0, D 12 D12
= I, and D21 D 21 = I.
We shall first consider state feedback u = Fx. Then the closed-loop system becomes
By the bounded real lemma, T zw < implies that there exists an X = X

0 such that
which is equivalent, by completing the square with respect to F, to
Intuition suggests that we can take
which gives
This is exactly the X Riccati equation under the preceding simplified conditions. Hence, we can take F = F
and X = X .
H INFINITY CONTROL
19
Next, suppose that there is an output feedback stabilizing controller such that T zw < . Then x() =
0 because the closed-loop system is stable. Consequently, we have
Substituting x = Ax + B1 w + B2 u and z = C1 x + D12 u into the above integral and using the X equation, and
finally completing the squares with respect to u and w, we get
where v = u + B 2 X x = u F x and r = w 2 B 1 X x. Substituting w into the system equations, we have

the new system equations
Hence the original H control problem is equivalent to finding a controller so that T vr < or
Obviously, this also suggests intuitively that the state feedback control can be u = F x and a worst state
feedback disturbance would be w = 2 B 1 X x. Since full state is not available for feedback, we have to
implement the control law using estimated state:
where x is the estimate of x. A standard observer can be constructed from the new system equations as
where L is the observer gain to be determined. Let e := x x Then
20
H INFINITY CONTROL
Since it is assumed that T vr < , it follows from the dual version of the bounded real lemma that there
exists a Y 0 such that
The above equation can be written as
Again, intuition suggests that we can take
which gives
It is easy to verify that
where Y is as given in Theorem 2. Since Y 0, we must have
Hence the controller is given by
which is exactly the H central controller given in Theorem 2 under the simplified conditions.
We can see that the H central controller can be obtained by connecting a state feedback with a state
estimate under the worst state feedback disturbance.
H Loop Shaping
Consider the family of uncertain systems Fp again. It is now clear that finding a controller K such that
it robustly stabilizes the family Fp is a standard H norm minimization problem. Now suppose P has a
H INFINITY CONTROL
21
stabilizable and detectable state-space realization given by
and let Y 0 be the solution to
N)
given by
Then the left coprime factorization (M,
is a normalized left coprime factorization. Furthermore,
Define
Then bP,K > 0 implies that K also stabilizes robustly the following family of uncertain systems:
(1)
(2)
(3)
P = P + a such that P and P have the same number of unstable poles and a < bP,K .
P = (I + m )P such that P and P have the same number of unstable poles and m < bP,K .
P = (I + f ) 1 P such that P and P have the same number of unstable poles and f < bP,K .
These conclusions also hold when the roles of plant and controller are interchanged. The number bP,K can
also be related to the classical gain and phase margins of a SISO system:
Hence, bP,K is a good measure of a systems robustness. Define
22
H INFINITY CONTROL
Obviously, bopt is the largest admissible size of perturbation so that the system is stable. It follows from the
standard H solution that
where Q is the solution to the following Lyapunov equation
Moreover, for any > 1/bopt (P), a controller achieving bP,K > 1/ is given by
where
This stabilization solution can be used to devise an H loop-sharing design method. The objective of this
approach is to incorporate the simple performance/robustness trade-off obtained in the loop shaping with the
guaranteed stability properties of H design methods. Recall that good performance controller design requires
that
particularly in some low-frequency range where (PK) denotes the smallest singular value. And good robustness requires that
particularly in some high-frequency range.

The H loop-shaping design procedure is developed by McFarlane and Glover (4) and is stated in the next
section.
Loop-Shaping Design Procedure.

(1) Loop Shaping: The singular values of the nominal plant are shaped, using a precompensator W 1 and/or a
postcompensator W 2 , to give a desired open-loop shape. The nominal plant P and the shaping functions W 1 ,
W 2 are combined to form the shaped plant Ps , where Ps = W 2 PW 1 . We assume that W 1 and W 2 are such
that Ps contains no hidden modes.
(2) Robust Stabilization: (a) If bopt (P) 1 return to (1) and adjust W 1 and W 2 . (b) Select bopt (Ps ); then
synthesize a stabilizing controller K , which satisfies bPs,K .
(3) The final feedback controller K is then constructed by combining the H controller K with the shaping
functions W 1 and W 2 such that K = W 1 K W 2 .
H INFINITY CONTROL
23
A typical design works as follows: the designer inspects the open-loop singular values of the nominal plant,
and shapes these by pre- and/or postcompensation until nominal performance (and possibly robust stability)
specifications are met. (Recall that the open-loop shape is related to closed-loop objectives.) A feedback controller
K with associated stability margin (for the shaped plant) bopt (Ps ) is then synthesized. If bopt (Ps ) is small,
then the specified loop shape is incompatible with robust stability requirements, and should be adjusted
accordingly; then K is reevaluated.
Note that in contrast to the classical loop-sharing approach, the loop shaping here is done without explicit
regard for the nominal plant phase information. That is, closed-loop stability requirements are disregarded at
this stage. Also in contrast with conventional H design, the robust stabilization is done without frequency
weighting.
In fact, the preceding robust stabilization objective can also be interpreted as the more standard H
problem formulation of minimizing the H norm of the frequency weighted gain from disturbances on the
plant input and output to the controller input and output as follows:
This shows that the H loop-shaping design is equivalent to a standard H design with the shaping functions
as weighting functions.
Synthesis
As we discussed at the beginning of this article, F
(G, K) guarantees the robust stability of the uncertain
system shown in Fig. 1(b) for any (s) R H with < 1/. However, if a system is built from components,
which are themselves uncertain, then, in general, the uncertainty in the system level is structured, and this
robust stability guarantee may be overly conservative. Because the interconnection model G can always be
chosen so that (s) is block diagonal, and, by absorbing any weights, < 1. Thus we can assume that (s)
takes the form of
with i < 1 and j < 1. The robust stability analysis for systems with such structured uncertainty is
not as simple but can be formally characterized by using the structured singular value, see Ref. 5.
Define Cnn as
24
H INFINITY CONTROL
Then for M Cnn , the structured singular value of M, (M), is defined as
The itself is not easy to compute. But good bounds can be obtained efficiently. Let
Then for any and D D, D = D and
and the equality holds if 2S + F 3. This bound can be used frequency by frequency to determine the system
robust stability and performance with structured uncertainties. For example, the system in Fig. 1(b) is well
posed, internally stable for all stable (s) with (s0 ) , Re(s0 ) 0, and < 1/ if and only if
This result leads us to the following synthesis problem:
This synthesis problem is not yet fully solved in the general case. A reasonable approach is to obtain a solution
to an upper bound:
by iteratively solving for K and D. This is the so-called D-K iteration. The stable and minimum phase scaling
matrix D(s) is chosen such that D(s)(s) = (s)D(s). For a fixed scaling transfer matrix D, minK DF
(G,
K)D 1 is a standard H optimization problem because
For a given stabilizing controller K, infD,D 1 H DF

(G, K)D 1 is a standard convex optimization problem,
and it can be solved pointwise in the frequency domain:
Then a D(s) is found to approximate the magnitude frequency response D uniformly (usually by curve fitting).
D-K iterations proceed by performing this two-parameter minimization in sequential fashion: minimizing over
H INFINITY CONTROL
25
K with D(s) fixed, minimizing pointwise over D with K fixed, minimizing again over K, and then again over D,
and so on. With either K or D fixed, the global optimum in the other variable may be found using the and
H solutions. Although the joint optimization of D and K is generally not convex and the global convergence
is not guaranteed, many designs have shown that this approach works very well. In fact, this is probably the
most effective design methodology available today for dealing with such complicated problems.
Additional Applications
There are many additional extensions and development in the H control theory. Here are some of them:
H loop-shaping techniques using -gap metric, see Ref. 6.
Robust control design in the gap metric, see Robust control and Ref. 6.
Linear matrix inequality (LMI) approach to H control, see Convex optimization.
Time-varying and finite horizon H control and game theoretical approach to H control, see Refs. 7 and 8.
Operator theoretic approach to H control, see Ref. 9.
Chain-scattering approach to H control, see Ref. 10.
H control with pole placement, see Ref. 11.
H controller reduction, see Refs. 12 and 13.
Linear parameter varying H control, see Ref. 14.
Sampled-Data H control, see Ref. 15.
H control for infinite dimensional systems, see Refs. 16 and 17.
H control for nonlinear systems, see Ref. 18.
Software and applications, see Ref. 19.
A comprehensive treatment of H control theory can be found in Refs. 13 and 20.
BIBLIOGRAPHY
1. G. Zames, Feedback and optimal sensitivity: model reference transformations, multiplicative semi-norms, and approximate inverses, IEEE Trans. Autom. Control, 26: 301320, 1981.
2. K. Glover and J. Doyle, State-space formulae for all stabilizing controllers that satisfy an H norm bound and relations
to risk sensitivity, Systems and Control Letters, 11: 167172, 1988.
3. J. C. Doyle et al., State-space solutions to standard H 2 and H control problems, IEEE Trans. Autom. Control, 34(8):
831847, 1989.
4. D.C. McFarlane and K. Glover, A loop shaping design procedure using H synthesis, IEEE Trans. Autom. Control,
37(6): 759769, 1992.
5. A. Packard and J. C. Doyle, The complex structured singular value, Automatica, 29: 71109, 1993.
6. G. Vinnicombe, Frequency domain uncertainty and the graph topology, IEEE Trans. Autom. Control, 38(9): 13711383,
1993.
7. M. Green and D. J. N. Limebeer, Linear Robust Control, Englewood Cliffs, NJ: Prentice-Hall, 1995.
8. T. Basaer and P. Bernhard, H -Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach,
Systems and Control: Foundations and Applications, Boston: Birkhauser,

1991.
9. B. A. Francis, A Course in H Control Theory, Lecture Notes in Control and Information Sciences, Vol. 88, London:
Springer-Verlag, 1987.
1997.
10. H. Kimura, Chain-Scattering Approach to H -Control, Boston: Birkhauser,
26
H INFINITY CONTROL
11. M. Chilali and P. Gahinet, H design with pole placement constraints: An LMI approach, IEEE Trans. Autom. Control,
41(3): 358367, 1996.
12. P. J. Goddard and K. Glover, Performance preserving frequency weighted controller approximation: A coprime factorization approach, Proceedings of the 33nd Conference on Decision and Control, Orlando, FL, pp. 27202725, December
1994.
13. K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control, Upper Saddle River, NJ: Prentice-Hall, 1996.
14. A. Packard, Gain scheduling via linear fractional transformations, Systems and Control Lett., 22: 7992, 1994.
15. T. Chen and B. A. Francis, Optimal Sampled-Data Control Systems, London: Springer-Verlag, 1995.
16. R. F. Curtain and H. J. Zwart, An Introduction to Infinite-Dimensional Linear Systems Theory, New York: SpringerVerlag, 1995.
17. C. Foias, H. Ozbay,

and A. Tannenbaum, Robust Control of Infinite Dimensional Systems, Lecture Notes in Control and
Information Sciences, Vol. 209, London: Springer-Verlag, 1996.
18. A. van der Schaft, L2 -Gain and Passivity Techniques in Nonlinear Control, Lecture Notes in Control and Information
Sciences, Vol. 218, London: Springer-Verlag, 1996.
19. G. Balas et al., -Analysis and Synthesis Toolbox, Natick, MA: The MathWorks, Inc., 1995.
20. K. Zhou and J. C. Doyle, Essentials of Robust Control, Upper Saddle River, NJ: Prentice-Hall, 1998.
KEMIN ZHOU
Louisiana State University

Copyright
POSITION CONTROL
Position control has many applications, such as control of the elevator angle of a fighter, control of the
antenna angle in a satellite tracking system, and control of robot manipulators. Often, the essence of position
control is that the tracking error between the desired system output and the actual system output is used
to generate a suitable control input to drive the tracking error to zero. In other words, tracking control is an
important part of motion control, as it solves the problem of determining the control inputs necessary for a
system to track a desired trajectory and provides a way to achieve accurate performance.
In this article, we present adaptive position control of robot manipulators and teleoperation systems.
Robot manipulators are composed of links connected by joints. The joints may be electrically, hydraulically,
or pneumatically actuated. The number of joints determines the number of degrees of freedom (DOF) of the
manipulator. Position control of a robot manipulator involves control of the positions of the joints. Once given
a set of desired trajectories for all the joints, the controller is designed to track these trajectories so that the
end effector of the manipulator sweeps the desired positions in the workspace. The primary method of sensing
the positions is with position encoders located on the joints, either on the shaft of the motor that actuates the
joint or on the joint itself. At times, direct sensing of the end-effector position with the help of a camera is used
to improve the accuracy of the manipulator in tracking a desired trajectory.
A teleoperation system involves two distant yet coupled robots: a local master robot and a remote slave
robot. In teleoperation, the human operator controls the master robot. Motion commands are measured on the
master robot and transmitted to the slave robot, which executes these commands and is expected to track the
motion of the master robot. In addition, the contact force information sensed by the slave robot is reflected
to the master robot for force perception. Thus, the master acts as an position input device that generates a
desired trajectory. The goal of position or tracking control is to design the necessary control input that makes
the slave track the motion of the master. This control problem is an example of masterslave control.
The article is organized as follows: In the following section, we present robust adaptive control schemes
of robot manipulators. First, we present dynamic models of robot manipulators with time-varying parameters
or unmodeled dynamics. Second, we present the controller structure and adaptive law for the time-varying
parameter case and show the signal boundedness and the tracking performance of the robot system. Third, we
present and analyze stable adaptive control schemes for robot manipulators with unmodeled dynamics. Some
common topics of position control relevant to robot manipulators such as PD control, inverse dynamics, and
path or trajectory interpolation are discussed in the fourth subsection. In the third section, we present adaptive
control of teleoperation systems. Adaptive control schemes for teleoperation systems with unknown jumping
parameters and with parametrizable and unparametrizable smooth time-varying parameters are presented.
We also present some control issues relevant to teleoperation systems with communication time delays.
POSITION CONTROL
Adaptive Position Control of Manipulators

To make robot manipulators capable of handling large loads in the presence of uncertainty on the mass properties of the load or its exact position in the end effector, robust adaptive control designs for robot manipulators
have been developed. In Slotine and Li (1,2) an adaptive control scheme has been proposed for the motion control of robot manipulators, which guarantees global stability and asymptotic zero tracking error between the
actual joint trajectory and the desired one and needs only the measurements of the joint position and velocity.
This consists of a proportional derivative (PD) feedback part and a full dynamics feedforward compensation
part, with the unknown manipulator and payload parameters being estimated online. The algorithm is computationally simple, because of an effective exploitation of the particular structure of manipulator dynamics.
Various modified versions of this scheme have been shown to be applicable to robot systems with unmodeled
dynamics [Reed and Ioannou (3)], and joint flexibility [Spong (4)].
Recently, there has been considerable research interest in neural network control of robots, and satisfactory results have been obtained in solving some of the special issues associated with the problems of robot
control. In Lewis, Jagannathan, and Yeildirek (5), neural network controllers are designed for robot manipulators in a variety of applications, including position control, force control, parallel-link mechanisms, and
digital neural network control. These model-free controllers offer a powerful and robust alternative to adaptive
control.
In Ge et al. (6), a comprehensive study of robot dynamics, structured network models for robots, and
systematic approaches for neural-network-based adaptive controller design for rigid robots, flexible joint robots,
and robots in constraint motion are presented.
In this article, we will present a robust adaptive control scheme, based on the scheme developed by Slotine
and Li (1,2) with a modified controller structure and a modified adaptive law [Tao (7)], which ensures the signal
boundedness in the presence of time-variations in the manipulator parameters and a mean tracking error of
the order of the parameter variations, which are not required to be small. We will also show similar results
for a class of unmodeled dynamics. The allowance for the existence of possible large parameter variations and
unmodeled dynamics yields significant potentials for applications of the proposed robust adaptive manipulator
controller.
Manipulator Models and Parametrization. In this subsection, we first present the mathematical
models of robot manipulators with time-varying parameters or unmodeled dynamics, and their parametrized
forms, and then use a two-link planar manipulator to illustrate the manipulator modeling and parametrization.
Manipulator Models. To derive the dynamic equations of a n-link robot manipulator (see Fig. 1, which
shows an illustrative four-link manipulator) whose parameters may explicitly depend on time, we use the
EulerLagrange equations [Spong and Vidyasagar (8); Ortega and Spong (9)] of a mechanical system:
where q = (q1 ,. . ., qn )T is a set of position variables of n (n > 0) joints of the robot manipulator, u = (u1 ,. . ., un )T
is the applied joint torque, and L is the Lagrangian defined as L = K P, the difference between the kinetic
energy K and the potential energy P, in the form
POSITION CONTROL
Fig. 1.
Robot manipulator.
with D (q, t) Rnn being the symmetric and positive definite manipulator inertia matrix. For Eqs. (1), (2) to
represent the manipulator dynamics with time-varying parameters, the mass and the moment of inertia of
each link of the manipulator should not explicitly depend on q .
Letting dij be the ijth element of D (q, t) and (q, t) = P (q, t)/q, and substituting Eq. (2) in Eq. (1), we
obtain the manipulator dynamic equation:

where the kjth element of C(q, q , t Rnn is ckj = n i = 1 (dkj /qi + dki /qj dij /qk ) q i.
A key feature of the manipulator model (3) is that the inertia matrix D(q, t) and the potential energy P(q,
t) are explicitly time-dependent, which takes into account the effect of changes in environment of the robot
system or changes of the manipulator dynamics with time. Moreover, an important property of the manipulator
model (3) is
where M(q, q , t) = dD(q, t)/dt D(q, t)/t, whose ijth element is (dij /q)T q . When D(q, t) = D(q) does not
explicitly depend on t, that is, D(q, t)/t = 0, Eq. (4) becomes xT [dD(q)/dt 2C(q, q )]x = 0, which is well known
in the robotics literature.
A manipulator with unmodeled dynamics may be modeled as
POSITION CONTROL
where H i , i = 1, 2, 3, with appropriate dimensions, are linear operators with rational transfer matrices,
representing the unmodeled part of the robot dynamics, and g1 (q), g2 (q), g3 (u) are certain vector functions of
q , q, u. The functions D(q), C(q, q ), (q) have been defined above, and, for simplicity, they are assumed not
explicitly time-dependent in the unmodeled dynamics problem. The manipulator model (5) is generalized from
some practical robot systems [Reed and Ioannou (3); Ortega and Spong (9)].
Control Objective and Parametrization. Our control objective is, for a given reference signal qd (t), to
generate the applied torque u for the manipulator (3) or (5) with unknown parameters so that all signals in the
robot system are bounded and the joint position q tracks qd as closely as possible. To achieve such an objective
we first use the transformation technique developed in Slotine and Li (1,2) to parametrize the manipulator
model (3) or (5).
Let be any n n constant matrix whose eigenvalues have positive real parts; define
Clearly, it follows from Eq. (6) that
and s, v, v depend only on q, qd , q , q d , q d and not on the joint acceleration vector q (t).
Using Eq. (7), we express the manipulator model (3) as
where Y (q, qd , q , q d , q d , t) is an n r matrix of known functions for some r > 0, and (t) Rr contains
parameters, which may be time-varying. In Eq. (8), the regressor Y (q, qd , q , q d , q d , t) is bounded for bounded
q, qd , q , q d , q d
Let xt be the truncated x at time t. Denote by the Euclidean vector norm or the induced matrix norm,
and by (1 , 2 ) the L (L1 , L2 ) vector norm or the induced operator norm [Desoer and Vidyasagar (10)],
as the case may be.
We make the following assumptions about the manipulator model (8):
(t) for some constants 0 > 0, > 0;
(1) (t) 0 ,
(2) D(q, t)/t f (q) for some constant > 0 and known f (q) bounded for bounded q.
Similarly, for the manipulator model (5), we obtain
where Rr is a constant vector, and Y (q, qd , q , q d , q d ) is not explicitly time-dependent.

We make the following assumptions about the manipulator model (9):
POSITION CONTROL
Fig. 2.
Two-link planar manipulator.
(1) (g1 (q))t f 1 (q(t)), (g2 (q))t f 2 (q(t)) for some known f 1 (q), f 2 (q) that are bounded for bounded q , q,
and H 1 1 , H 2 2 for some constants 1 > 0, 2 > 0;
(2) (g3 (u))t ut , and H 3 i i , where H 3 i is the ith row of H 3 , i = 1,. . ., n.
We also make an assumption on the desired joint position vector qd (t):
(1) qd (t), q d (t), q d (t) are bounded.
Assumption (A1) requires only the boundedness of the manipulator parameters and their derivatives, not
the smallness of the time variations of the parameters. Smallness of the parameter variations is usually an
assumption for the design of adaptive control schemes for time-varying plants, but it is not needed here because
of the special structure of the robot manipulator dynamics. Assumption (A2) requires that D (q, t)/t satisfy
a certain relative boundedness condition. Assumption (A3) requires that the L gains of H 1 , H 2 be finite and
g1 (q), g2 (q) satisfy certain relative boundedness conditions. Assumption (A4) is similar to (A3), but i 0, i =
1,. . ., n, are to be specified for the robust stability of the adaptive robot system. We note that the bounds , 1 ,
2 are not needed for the adaptive controller design.
An Illustrative Example. In this sub-subsection, we consider a two-link planar manipulator [Spong and
Vidyasagar (8)], shown in Fig. 2 as an illustrative example for the robot system modeling and parametrization.
The manipulator configuration may be described as follows: there are two revolute joints with joint angles
q1 , q2 , and two links with masses M 1 , M 2 , lengths l1 , l2 , distances lc1 , lc2 from the joints to the mass centers,
and rotational inertias I1 , I2 . The inertia matrix D(q, t) has four elements: d11 = M 1 lc1 2 + M 2 (l1 2 lc2 2 + 2l1 lc2
cos q2 )+ I1 +l2 , d12 = d21 = M 2 (lc2 2 + l1 lc2 cos q2 ) + I2 , d22 = M 2 lc2 2 + I2 ; and the potential energy is P(q, t) =
(M 1 lc1 + M 2 l1 ) g sin q1 + M 2 lc2 g sin (q1 + q2 ), where g is the gravitational acceleration. The matrix C(q, q , t)
in Eq. (3) has four elements: c11 = h q 2 , c12 = (q1 + q 2 )h, c21 = q 1 h, c22 = 0, where h = M 2 l1 lc2 sin q2 .
POSITION CONTROL
The manipulator model without parameter variations and unmodeled dynamics is D(q) s + C(q, q )s = u
Y(q, qd , q , q d , q d ), where
When = (t) is time-varying, the manipulator model is Eq. (8) with Y(q, qd , q , q d , q d , t) and
Assuming that | i (t)| i , we obtain the bound in (A1) as

and the bounds , f (q) in (A2) as
= (1 2 + 2 2 + 43 2 + 64 2 + 5 2 + 46 2 )1/2 , f (q) = 1.
For the unmodeled dynamics problem, the manipulator model is Eq. (9). The bounds 1 , 2 , i in (A3) and
(A4) depend on the nature of the unmodeled dynamics.
Solution to the Parameter Variation Problem. In this subsection, we first present an adaptive control
scheme for robot manipulators modeled by Eq. (8) and then analyze the stability and tracking properties of the
proposed adaptive controller.
If the inertia D(q, t) = D(q) and the potential energy P(q, t) = P(q) are not explicitly time-dependent, that
is, D(q, t)/t = 0, and (t) = is constant in Eq. (8), then the adaptive control scheme proposed by Slotine
and Li (1,2),
guarantees that the closed-loop system is globally stable and convergent in the sense that q(t), q (t), (t) are
bounded, and limt e(t) = 0, as the positive definite function
has the property
(t) = sT (t)K D s(t) 0 [also see Spong et al. (11) for further analysis].
POSITION CONTROL
Fig. 3.
The switching modification.
When D(q, t), P(q, t) are both explicitly time-dependent, we have obtained the manipulator model as Eq.
(8) in which (t) is time-varying and the term D(q, t)/t q appears. If the parameters in D(q, t), P(q, t) were
known, then (t) and D(q, t)/t q could be calculated so that the control law u(t) = Y(q, qd , q , q d , q d , t)(t)
+ [D(q, t)/t] q K D s could be implemented, which guarantees global stability and asymptotic tracking. For
unknown D(q, t), P(q, t), next we present an adaptive control scheme that is robust with respect to the time
variation of (t) and [D(q, t)/t] q .
With Y(q, qd , q , q d , q d , t), s, K D , defined before, we propose the following feedback controller structure
for the manipulator (8):
and the following update law for (t):
where (t) as shown in Fig. 3 is the switching signal [Ioannou and Tsakalis (12)] using a priori knowledge of
the upper bound M on supt0 (t):
This adaptive control scheme has the following stability and tracking properties.
POSITION CONTROL
Theorem 1. All closed-loop signals are bounded, and the tracking error e(t) = q(t) qd (t) satisfies
for some constants 0 > 0, 0 > 0, and any t2 > t1 0. Moreover, e(t) L2 and limt e(t) = 0 in the absence of
parameter time variations, that is, when (t)

= 0, D(q, t)/t = 0.
Proof:. Consider the positive definite function
From Eqs. (4, 7, 8), (16, 17, 18) and from (A1), (A2), we have that
Since , , 0 are constants and (t) defined in Eq. (19) satisfies
it follows from the second inequality of (2.22) that (t) 0 for (t), s(t) outside a certain bounded set. Therefore
s(t) and (t) are bounded, which, in view of Eqs. (7), (16), implies that q(t), q (t), u(t) are also bounded.
Using the fact that (t)
(t)(t) 0 and the first inequality of Eq. (22), we obtain
POSITION CONTROL
for some constant k1 > 0. Since V(t) is bounded, from Eq. (24) we have
for some constants 1 > 0, 1 > 0 and any t2 > t1 0.

To show that Eq. (25) implies Eq. (20), let us consider the relation s(t) = e (t) + e(t), where is a stable
matrix [see Eq. (7)], and denote by Hthe linear operator or the impulse-response matrix from s(t) to e(t) as the
case may be, that is, e(t) = H[s](t) = 0 t H(t )s() d. It follows that

Since the operator H is exponentially stable and e(t) is bounded, both H2 and t1 t2 H (t t1 )e(t1 )2 dt are
finite for any t2 > t1 0. Hence from Eqs. (25) and (26) we prove Eq. (20).
When (t)
= 0 and D(q, t)/t = 0, that is, = = 0, it follows from Eq. (20) that e(t) L2 . This, together
with the boundedness of e (t) = s(t) e(t), proves limt e(t) = 0.
To implement the controller (16), we need the knowledge of f (q) to generate the bounding signal m(t) in Eq.
(17). A more sophisticated choice of f (q) admits a wider class of D(q, t)/t, but may make the implementation of
m(t) more complicated. We also note that the above design does not need the knowledge of the bounds , . For
a chosen f (q), different choices of k0 in generating m(t) may have different effects on the tracking performance,
while increasing k0 may reduce the effect of in the mean error (20). For the signal boundedness and the
mean tracking error (20), parameter variations characterized by and are not required to be small. This is
an important feature of the robot system. With q , q available for measurement, the manipulator mode (3) is
equivalent to the first-order model (8), for which the adaptive controller allows large parameter variations to
exist.
Solution to the Unmodeled Dynamics Problem. Consider the manipulator (9) with unmodeled
dynamics. If the terms H 1 [g1 (q)](t), H 2 [g2 (q)](t), H 3 [g3 (u)](t) were available for measurement and were
known, then the control law u(t) = Y(q, qd , q , q d , q d ) H 1 [g1 (q)](t) H 2 [g2 (q)](t) H 3 [g3 (u)](t) K D s(t) could
be implemented so that d/dt[sT (t)D(q)s(t)] = 2sT (t)K D s(t), showing the boundedness of s(t) and exponentially
fast tracking. However, to ensure the boundedness of u(t), one needs H 3 = maxi = 1,...,n i < 1.
To solve the adaptive control problem in which H 1 [g1 (q)](t), H 2 [g2 (q)](t), H 3 [g3 (u)(t), and are unknown,
with Y(q, qd , q , q d , q d ), K D , s(t) = (s1 (t),. . ., sn (t))T , (t), defined before, we propose the following feedback
10
POSITION CONTROL
controller structure for the manipulator (9):
and the following update law for (t):
The stability and tracking properties of this adaptive control scheme are:
Theorem 2. All closed-loop signals are bounded for any i [0, i ] i = 1,. . .,n, and the tracking error e(t)
satisfies
for some constants 0 > 0, 0 > 0. Moreover, e(t) L2 and limt e(t) = 0 in the absence of the unmodeled
dynamics, that is, when H 1 = 0, H 2 = 0, H 3 = 0.
POSITION CONTROL
11
Proof:. Considering V(t) defined in (15), from Eqs. (9), (27), (32) and (A3), (A4) we obtain
From Eq. (30), we see that
Hence it follows from Eqs. (34), (35) that, for 0 i i , i = 1,. . .,n, we have (t) 0 whenever (t) and s(t)
are outside a certain bounded set, that is, s(t), (t) are bounded, and so are q(t), q (t).
From Eqs. (27, 28, 29, 30), (A3), (A4), and the boundedness of s(t), q(t), q (t), (t), we have
for some constant k2 > 0, which, together with Eq. (31), implies that u(t) is bounded.
Using Eqs. (34), (35) and the fact that (t)
(t)(t) 0, we obtain
For = maxi = 1,...,n i , Eq. (37) implies that
for some constants 1 > 0, 1 > 0, and any t2 > t1 0, which implies Eq. (33).
12
POSITION CONTROL
When H 1 = 0, H 2 = 0, H 3 = 0, the expression (34) for
(t) becomes
Since sT (t)3 (t) 0, Eq. (39) shows that s(t) L2 . Hence, from Eq. (7), it follows that e(t) L2 and e (t) is bounded.
Therefore we have limt e(t) = 0.
We have thus proved the signal boundedness of the closed-loop system in the presence of H 1 [g1 (q)](t),
H 2 [g2 (q)](t), H 3 [g3 (q)](t). The gains of the linear operators H 1 , H 2 are assumed to be finite but not small. The
gain of H 3 is required to be small to ensure the boundedness of u(t).
The modifying term (t)(t) in Eq. (32) can be replaced by 0 (t), 0 > 0. The signal boundedness follows,
but Eq. (33) is changed to
This scheme cannot guarantee asymptotic tracking in the absence of the unmodeled dynamics, though the
scheme does not need the knowledge of the upper bound on .
The use of the bounding signals mi (t) defined in Eq. (28) is the key to ensuring signal boundedness in
the presence of the unmodeled dynamics satisfying (A3), (A4). To generate these signals, the knowledge of the
stability margin of the unmodeled dynamics is not needed. Alternative bounding signals may be used under
other assumptions for the unmodeled dynamics. For example, if 3 i i , where 3 i is the ith row of H 3 (s)(s
variable), i = 1,. . ., n, for some known constant a0 > 0, and (s+a0 ) 1 [g3 (u)])t
+ a0 ) (this s is the Laplace
1
(s+a0 ) [u])t , then we can choose m3 (t) = (s+a0 ) 1 [u])t and set 0 < i a0 . Another choice of m3 (t)
is the bounding signal [ /(s+0 )][u](t) when g3 (u) u and 3 i 1 i , where 3 i is the ith row of
H 3 (s 0 )s with the Laplace variable s, for some known constant 0 > 0. For this m3 (t), the condition on i is
< 0 and i > 0 for i = 1,. . .,n. We note that a similar L1 -norm condition can be established for
(1 ,. . .,m )T
H 1 and H 2 to design a robust adaptive controller.
When is known and H 1 [g1 (q)](t), H 2 [g2 (q)](t), H 3 [g3 (u)](t) are present, Eq. (27) with (t) = becomes a
robust nonadaptive controller, which results in e(t) converging exponentially to a residual set whose size is of
the order of 1 2 /k10 2 + 2 2 /k20 2 + .
Next, we present the robust adaptive control design assuming that the upper bounds on the gains of the
unmodeled dynamics H 1 , H 2 are known:
(1) (A3a)(g1 q ), g2 (q) are the same as in (A3), and H j i : ji for some known constants ji > 0,where H j i is
the ith row of H j , j = 1, 2, i = 1,. . ., n.
We propose to choose j (t) = (j1 (t),. . .,jn (t))T , j = 1,2,3, in Eq. (27) as
POSITION CONTROL
13
This scheme guarantees that all signals in the closed-loop system are bounded for 0 i 3i , i = 1,. . .,
n, and the tracking error e(t) satisfies
for some constants 0 > 0, 0 > 0, and any t2 > t1 0. Moreover, e(t) L2 and limt e(t) = 0 in the absence of
the unmodeled dynamics H 1 , H 2 , and H 3 .
We see from Eq. (43) that the mean tracking error explicitly depends only on the design parameter , not
on the bounds ji defined in (A3a). A smaller may result in a smaller mean tracking error (2.43). Hence, with
the knowledge of the unmodeled dynamics bounds, improvements of the tracking performance may be achieved
by using the control signals defined by Eq. (41).
Another interesting result is the adaptive controller with a so-called variable structure [Utkin (13)]:
letting ji 0 in (2.41), j = 1,2,3, i = 1,. . .,n, we obtain
It can be shown that for 0 i 3i , i = 1,. . .,n, all closed-loop signals are bounded and the tracking error
asymptotically converges to e(t) = 0 with possible chatterings. For a variable structure controller, (t) = 0 can
be used in the update law (32).
As a final remark, we note that the proposed designs can be combined to solve the problem in which both
the parameter variation and unmodeled dynamics are present.
Proportional Derivative Control, Inverse Dynamics, and Path Interpolation. In this subsection,
we discuss some of the general concepts related to the position control of robot manipulators.
Proportional Derivative Control. We first derive a PD control law for each joint of a manipulator based
on a single-input single-output (SISO) model. Coupling effects among the joints are regarded as disturbances.
Permanent-magnet dc motors along with gear reduction are commonly used in practice to actuate the joints of
the manipulator. For such dc-motor-actuated robotic manipulator, a simplified version of the dynamics of the
kth joint can be given as in Spong and Vidyasagar (8),
where J eff k = J mk + rk 2 dkk (q) is the effective joint inertia of the kth actuator (motor plus gear, J mk ) and the
manipulator link [dkk (q) is the kth diagonal element of D(q) in Eq. (3)], Beff k = Bmk + (K bk K mk /Rk ) is the
effective damping of the kth actuator (motor plus gear, Bmk ) with K bk the back emf constant, K mk the torque
constant, and Rk the armature resistance; mk is the kth motor (rotor) angular position; V ak is the armature
voltage of the kth motor; rk is the kth gear ratio; k = K mk /Rk ; and k is the actuator dynamics (3) specified
for the kth joint and is treated as a disturbance to simplify the problem, since in that case, we maintain the
14
POSITION CONTROL
linearity of (2.45). The last can be given as
where cijk = (dkj /qi + dki /qj dij /qk ) and k (q) = P(q)/qk with P(q) the potential energy.
The setpoint tracking problem is defined as the problem of tracking a constant step reference command
d = [d1 ,. . .,dn ] for n joints. This type of control is adequate for applications not involving very fast motion,
especially in robots with large gear reduction between the actuators and the links. A PD compensator for each
of the n joints can now be used to achieve setpoint tracking:
where K pk , K dk are the proportional and the derivative gains, respectively. The characteristic polynomial of the
closed-loop system is
indicating that the closed-loop system will be stable for all positive values of K pk and K dk and bounded
disturbances. The tracking error is given by
For a step reference input dk (s) = dk /s and a constant disturbance k (s) = k /s, the steady-state error is ekss
= rk k / k K pk [see Spong and Vidyasagar (8)]. Thus, the steady-state error due to a constant disturbance
is smaller for larger gear reduction and can be made arbitrarily small by making the position gain K pk large.
By using integral control as well (PID), we can achieve zero steady-state error while keeping gains small and
rejecting step disturbances. However, the PD or PID compensators perform poorly for position control when
there are large uncertainties in system parameters, or when varying disturbances and unmodeled dynamics
are present, as is common in applications. In such situations, the adaptive control designs presented in the
preceding two subsections perform much better than the PD or PID controller.
In the PD compensator presented above, the coupling effects among the joints were regarded as disturbances. In reality, the dynamic equations of a robot manipulator form a complex, nonlinear, and multivariable
system. Such a dc-motor-driven n-joint actuator may be represented in a matrix equation as
where D(q) is the time-invariant n n inertia matrix, C(q, q ) and (q) are the time-invariant versions of C(q,
q , t) and (q, t) in Eq. (3) respectively,
is a diagonal matrix with elements J mk /rk 2 , the input joint torque
has components uk = (K mk /rk Rk ) V ak , and B has elements k = Bmk + K bk Lmk /Rk (with Lmk the inductance),
for k = 1, . . ., n. An independent joint PD control scheme can be written for the system (50) as in Spong and
POSITION CONTROL
15
Vidyasagar (8):
where = qd q is the error between the desired and the actual joint displacements, and P , D are diagonal
matrices of positive proportional and derivative gains, respectively. In the absence of gravity [(q) = 0], the
PD control law (51) achieves asymptotic tracking of the desired joint positions. In presence of gravity, Eq. (51)
alone cannot guarantee asymptotic tracking and has to be modified as
to cancel the steady-state error due to the effect of the gravitational terms [see Spong and Vidyasagar (8)]. For
detailed analysis and performance study of PD controller for robot manipulators, please refer to Spong and
Vidyasagar (8).
Inverse Dynamics. Using inverse dynamics, a more complex nonlinear control technique can be implemented for trajectory tracking of rigid manipulators [Spong and Vidyasagar (8)]. Consider the system given by
Eq. (50) in a more simplified form,
where = D + , h = C q + q + . The idea of inverse dynamics is to seek a nonlinear feedback control

law u = (q, q ) which when substituted into Eq. (53) results in a linear closed-loop system. Since the inertia
matrix M is invertible, the control law
reduces the system q = V a with V a as the new input to the system, the armature voltages to be applied to to the
actuator motors. Thus, we have a double integrator system with n uncoupled double integrators. The nonlinear
control law (54) is called the inverse dynamics control and achieves a new linear and decoupled system, making
it possible to design V ak to control a simple linear second-order system and can be designed as
where 0 , 1 are diagonal matrices of position and velocity gains, respectively, and is the reference. The gains
could be chosen to get a joint response that is equal to the response of a critically damped linear second-order
system with desired natural frequencies for each of the desired speeds of the responses of the joints. The inverse
dynamics can be viewed as an input transformation that transforms the problem from one of choosing torque
input commands, which is difficult, to one of choosing acceleration input commands, which is easy. There are
many crucial issues of implementation and robustness that must be addressed to implement Eq. (54), and the
reader is referred to Spong and Vidyasagar (8).
Path Interpolation. The simplest type of robot motion is point-to-point motion. In this approach the robot
is commanded to go from an initial configuration to a final configuration without regard to the intermediate
path followed by the end effector. To understand the concept of configuration, it is helpful to review some
terminology used in Spong and Vidyasagar (8). Suppose a robot has n + 1 links numbered from 0 to n starting
16
POSITION CONTROL
Fig. 4.
Path interpolation: via points to plan motion around obstacles.
the base of the robot, which is taken as link 0. The joints are numbered 1 to n, and the ith joint is the point
in space where links i 1 and i are connected. The ith joint variable is denoted by qi . A coordinate frame is
attached rigidly to each link. We attach an inertial frame to the base and call it frame 0. Frames 1 to n are
chosen such that frame i is rigidly attached to link i. Now the configuration is given by the transformation
matrix that transforms the coordinates of a point from frame j to frame i and is denoted by T i j . For example, for
a seven-link robot manipulator, the initial and final configurations that are of interest in point-to-point motion
are the transformation matrices that transform the coordinates of frame 6 to frame 0; let them be denoted
by T 0 6 init and T 0 6 final. This type of motion is suitable for materials transfer jobs where the workspace is
clear of obstacles. Given the desired initial and final positions and orientation of the end effector, the inverse
kinematic solution is evaluated to find the required initial and final joint variables. Suppose, di j denotes the
position of frame j with respect to frame i, and Ri j denotes the orientation of frame j relative to frame i. For the
manipulator with seven links, the motion of the first three, joints is calculated by computing the joint variables
q1 , q2 , and q3 corresponding to d0 3 init and d0 3 final. The motion of the final three joint variables is found by
computing a set of Euler angles corresponding to R3 6 init and R3 6 final [Spong and Vidyasagar (8)]. For some
purposes, such as obstacle avoidance, the path of the end effector can be further constrained by the addition of
via points intermediate to the initial and the final configurations as shown in Fig. 4. Different techniques of
generating smooth trajectories in joint space, given the initial and final joint variables, are presented in Spong
and Vidyasagar (8).
Adaptive Control of Teleoperation Systems

A teleoperation system involves two distant yet coupled robots: a local master robot and a remote slave robot. An
ideal teleoperation is the one in which the impedance felt by the human operator is matched to the impedance of
the slave environment [Lawrence (14)]. The term transparency is used to describe such an ideal teleoperation.
For teleoperation systems with known and time-invariant dynamics, a transparency control scheme is proposed
in Lawrence (14), with a modification to handle communication time delays. For teleoperation systems with
unknown time-invariant dynamics, an adaptive control scheme based on Slotine and Li (1,2) algorithm is
presented in Hashtrudi-Zaad and Salcudean (15). Stability and signal boundedness of a similar adaptive control
system are investigated in Lee and Chung (16). Despite recent progresses in teleoperation, transparency issues
for teleoperation systems with unknown time-varying parameters, such as jumping and smoothly but rapidly
changing parameters, including control designs and transparency characterizations, remain open.
POSITION CONTROL
Fig. 5.
17
Structure of a teleoperation system.
In this article, we present new transparency concepts suitable for adaptive control of teleoperation systems
with time-varying parameters [Shi et al. (17)]. Adaptive control schemes for teleoperation systems with jumping
or rapidly time-varying parameters are developed [Shi et al. (17)]. The developed adaptive control schemes
lead to stable and transparent teleoperations in the presence of unknown constant or jumping or fast-varying
parameters. The teleoperation systems to be controlled are assumed to have no communication time delay. In
the first subsection, we present the new concepts of weak transparency, asymptotic weak transparency, and
approximate weak transparency, and formulate the transparency control problem for four types of teleoperation
systems with no communication time delay [Shi et al. (17)]. In the next subsection, we present adaptive
control schemes for teleoperation systems with unknown jumping parameters and with parametrizable and
unparametrizable smoothly time-varying parameters [Shi et al. (17)]. In the last subsection we present some
control issues relevant to teleoperation systems with communication time delays.
Teleoperation Systems. In this section, we present the general structure of a teleoperation system
and its dynamic description, introduce several new concepts for transparency of teleoperation systems, and
state the adaptive control objective with which the new transparency concepts are to be verified.
Dynamics of a Teleoperation System. A teleoperation system consists of five subsystems: the human
operator, the master robot, the communication channels, the slave robot, and the slave environment, as shown
in Fig. 5. The term teleoperator refers to the master and slave manipulators connected by the communication
channels. Bilateral teleoperation involves velocity and force information transfer between the master and the
slave. Communication time delays commonly exist in teleoperation systems due to the large distance and
restrictive data transfer. These delays are assumed to be absent in the following analysis, for confinement to
fundamentals and for simplicity of analysis. In Fig. 5, vh is the velocity of the human operators hand, vm is the
velocity of the master end effector, vs is the velocity of slave end effector during contact, F h is the force applied
by the human operator to the master robot, F e is the force exerted by the slave robot on its environment, and
F s is the coordinating torque. In the absence of communication time delay, vsd (t) = vm (t) and F md = F s (t). In the
presence of communication time delay T, vsd (t) = vm (t T) and F md = F s (t T). In the following analysis, no
communication time delay is assumed.
For analysis, a network representation of a teleoperation system is useful and Fig. 6 shows one commonly
used in the literature, in which the human operator and the slave environment are represented by one-port
networks, and the teleoperator by a two-port network. The blocks Zh , Zm , Zs , and Ze represent respectively
the dynamics of a human operator, a master robot, a slave robot, and the slave environment; signals m and s
denote control torques for master and slave robots; signals vh and ve refer to velocities of the human operators
hand and the slave environment. Note that vh equals vm , the velocity of master end effector, and ve equals
vs , the velocity of slave end effector during contact. The signals F h , F h , F e represent respectively the force
generated by the human operator, the force applied by the human operator to the master robot, and the force
exerted by the slave robot on its environment.
As in Hashtrudi-Zaad and Salcudean (15), Lee and Chung (16), and Raju et al. (18), we consider the
dynamics of the master and slave robots as
18
POSITION CONTROL
Fig. 6.
A two-port network for a teleoperation system.
where M, B, and K are inertia, damping, and stiffness parameters; the signal x is the position of end effector;
and the signal denotes the control torque with subscript m for the master and s for the slave. From Eq. (56),
we see that Zm (s) = M m s + Bm + K m /s.
Let Cm and Cs denote the master and slave feedback control, and Ci , i = 1, . . ., 4, represent the data
communication control for signals vm , F e , F h , and vs , respectively. Then the torques m and s have the following
descriptions:
where the minus sign indicates the feedback signals. We also assume the human operator and the slave
environment are passive, and as in Raju et al. (18), we use a generalized massdampingspring model to
describe the human operator and the slave environment,
where M, B, and K are the inertia, damping, and stiffness parameters with subscript h for the human operator
and e for the slave environment. Substituting Eq. (61) into Eq. (57), we get the slave system
where M = M s + M e , B = Bs + Be , and K = K s + K e .
Four types of teleoperation systems are usually met in applications: teleoperation systems (i) with known
time-invariant dynamics, (ii) with unknown constant environment, (iii) with jumping environment parameters,
and (iv) with smooth time-varying environment parameters. The transparency of adaptive teleoperation control
POSITION CONTROL
19
systems is of main interest in this article, for which we will introduce new concepts suitable for adaptive control
when the system parameters are unknown, for different cases of parameter uncertainties.
We first consider teleoperation systems with no communication time delay. The stability analysis is easy
and simple when communication delay is not involved. A closed-loop transfer function can be obtained for the
bilateral system, and the traditional tools such as the root locus technique and the RouthHurwitz stability
criterion can be used for stability analysis. It is reasonable to assume no communication delay to develop the
basic adaptive control techniques along with the stability analysis. In the last subsection below, we present
some of the control issues relevant to teleoperation with communication time delays.
Transparency of a Teleoperation System. The impedance transmitted to or felt by human operator,
Zt (see Fig. 6), is defined by F h = Zt vh , in the frequency domain.
Definition 1. [Lawrence (14)]. A teleoperation system is transparent if
This means that in a transparent teleoperation, the human operator feels as if he were manipulating the
slave environment directly. Note that when the slave robot is in contact with its environment, its velocity, vs
and the environment force F e are related by the impedance Ze as F e = Zs vs in the frequency domain. Since
vh = vm , if the slave exactly reproduces the motion of the master (i.e., vs = vm , and Zt = Ze , then F h = F e ,
that is, the master accurately feels the slave contact force. That is, for a transparent teleoperation, the velocity
tracking from the slave to the master leads to force tracking from the master to the slave.
Definition 2. A teleoperation system is weakly transparent if
The property (64) is called weak transparency because it only needs Zt = Ze for some specific operation
frequencies at which vs = vm .
Definition 3. A teleoperation system is asymptotic weakly transparent if
This weak transparency is ensured in adaptive teleoperation control systems with parametric uncertainties.
Definition 4. A teleoperation system is approximate weakly transparent if
for some constant c1 > 0, c2 > 0, > 0 and some design parameter > 0.
In this case, it is expected that the design parameter > 0 in the control system can be chosen to be large
so that the tracking error vs (t) vm (t) can be made small.
20
POSITION CONTROL
Definition 5. A teleoperation system is approximate weakly transparent in the mean if
for some constant 1 > 0 and 2 > 0, and any t2 > t1 0.

This weak transparency is ensured in adaptive teleoperation systems with both parametric and structural
uncertainties. In this case, it is expected that 1 = 0 for an adaptive control system, for some design parameter
> 0 that can be made small.
Control Objective. The control objective is to develop controllers Cm , Cs , and Ci , i = 1,. . ., 4, that ensure (i)
closed-loop signal boundedness, (ii) limt [vs (t) vm (t)] = 0 [vs (t) tracks vm (t) as closely as possible in the
sense (66) or (67)], and (iii) F h = F e for the slave environment with constant parameters, or with jumping
parameters, or with smoothly time-varying parameters, in the presence of parameter uncertainties. This is a
masterslave control problem.
Control Designs. In this subsection, we will first review two existing control designs: one for teleoperation systems with known constant parameters, and one for teleoperation systems with unknown constant
parameters. We will then present new adaptive control schemes for time-varying teleoperation systems with
jumping parameters or with smoothly time-varying parameters. The teleoperation systems in consideration
are assumed to have no communication time delay. For the new proposed control schemes, we will analyze the
system performance in terms of stability and transparency. The teleoperation system is said to be stable if the
state variables of the system are bounded at any time.
Design for System with Known Constant Parameters. As in Lawrence (14), the forces and velocities of
the teleoperator two-port as shown in Figure 6 are related by a hybrid matrix H:
where H ij , i, j = 1, 2, representing the inputoutput relation of the teleoperator two-port, are H 11 = (Zm + Cm )
D(Zs + Cs C3 C4 ) + C4 , H 12 = (Zm + Cm ) D(I C3 C2 ) C2 , H 21 = D(Zs + Cs C3 C4 ), H 22 = (I C3 C2 )
with D = (C1 + C3 Zm + C3 Cm ) 1 . Solving for F h , vh from Eq. (68), we get
To achieve Zt = Ze , it is needed that H 11 = H 22 = 0, H 12 = 1, H 21 = 1. Therefore, it can be derived

that C1 = Zs + Cs , C2 = 1, C3 = 1, C4 = (Zm + Cm ) with Cm , Cs stable. This control scheme achieves
transparency for teleoperation systems with known dynamics in the absence of communication time delay.
Sufficient conditions for stability of the teleoperation system are derived in Lawrence (14). A tradeoff between
stability and transparency is necessary [Lawrence (14)]. With the modified control schemes, good transparency
is achieved at lower frequency. However, Zt = Ze cannot be ensured for all frequencies.
In the next sub-subsections, we will consider the adaptive control problems when the slave systems
parameters are unknown, to show that new transparency concepts defined in Definitions 3.3, 3.4, 3.5 are
POSITION CONTROL
21
useful for adaptive teleoperation systems. To design an adaptive control law, we assume zero communication
time delay and
m [Lee and Chung (16)].
(1) The master position signal xm is bounded with bounded derivatives x m , 022
Design for System with Unknown Constant Parameters. In this section, the slave environment is assumed to have unknown constant parameters. Our control objective, as specified in the sub-subsection Transparency of a Teleoperation System above, is to achieve (i) closed-loop signal boundedness; (ii) limt [vs (t)
vm (t)] = 0, and (iii) F h = F e . The last two properties imply asymptotic weak transparency of the teleoperation
system. A control scheme based on Slotine and Lis (1) design is first applied to achieve signal boundedness
and velocity tracking. Force matching is then designed by using the relationship of (3.13) with H 11 = 0, H 12 =
1.
Adaptive Control Design. The slave system as defined in Eq. (62) is
where M = M s + M e , B = Bs + Be , K = K s + K e , and M s , Be , K e > 0 are unknown constants.

Let > 0 be a design parameter, and define the signals e(t) = xs (t) xm (t), v(t) = x m (t) e(t), s(t) = x s (t)
v. As in Slotine and Li (1), the control law is chosen as
where Y(v, v, xs ) = [v v xs ] is a vector of known signals, = [M B K]T is a vector of unknown parameters,

the estimate of , and K D > 0 is a design gain parameter. Choose the adaptive law as
where = T R33 is positive definite. Consider the Lyapunov function V =
(Ms2 +
), where
is
. Then it follows that = (K D + B)s 0, which implies that all the closed-loop signals are bounded,
and that e, e converge to zero asymptotically as the time t goes to [Slotine and Li (1)].
Transparency and Stability. With velocity tracking from the slave to the master, the force tracking from
the master to the slave will lead to a weak transparent teleoperation. Because F h is related to F e as F h = H 11
ve + H 12 (F e ), the condition for F h = F e is
2
Recall H 11 = (Zm + Cm )D(Zs + Cs C3 C4 ) + C4 , H 12 = (Zm + Cm )D(I C3 C2 ) C2 , and D = (C1 + C3 Zm +

C3 Cm ) 1 . The following design [Lee and Chung (16)] will satisfy the condition (71):
Thus far asymptotic velocity tracking and force tracking are ensured, which lead to asymptotic weak
transparency for a teleoperation system with unknown constant parameters. The developed teleoperation
system is stable because the master, the slave, and their controllers are passive, and the human operator and
the slave environment are also passive by assumption.
22
POSITION CONTROL
Design for System with Unknown Jumping Parameters. Parameter variations in this case are characterized by piecewise constant behavior. An example is the slave robot intermittently contacting with different
environments. Assume that parameters B and K in the slave system (69) are unknown and piecewise constant,
as modeled by
where Bi and K i , i = 1,. . ., l, are constants, which last for certain durations of time, and f i (t), i = 1,. . ., l, are
functions indicating which values of Bi and K i are taken by B(t) and K(t) at a given time. The indicator functions,
f i (t), are known functions that reveal the durations and the time instants of the parameter discontinuities, and
are defined as
The function f i (t) = 1 indicates the value Bi or K i is taken by the system parameter B(t) or K(t). In addition,
f i (t) indicates that only one value can be active at any time, that is, f i (t)f j (t) = 0 for i = j. With (t) = [M B(t)
= (t), so we cannot use (t) in Eq. (70) to ensure (t) 0 for any t
K(t)]T , it follows that (t) = (t) (t)
> 0.
In this sub-subsection, we first present an adaptive control scheme for the slave system (69) with unknown
jumping parameters described by (73, 74, 75).
Adaptive Control Design. We propose the following controller structure:
where v(t), s(t), Y( v , v, xs ), and K D are defined in the preceding sub-subsection, and
=[
T
estimate of = [M B(t) K(t)] , with (t) = 1 f 1 (t) + 2 f 2 (t) + . . . + l f l (t) and (t) = 1 f 1 (t) +
. . . + l f l (t). Substituting (3.21) into (3.14) reveals
where the estimate errors

function
M,
i
Bi , and
]T is the
2 f 2 (t) +
K i , i = 1,. . ., l. Let us choose the positive
POSITION CONTROL
where m , b i, k i > 0, i = 1,. . ., l. We choose the adaptive laws for
With this adaptive design, the derivative of Eq. (78),
i,
23
and K i as
(t), becomes
For stability analysis, we need the following lemma.

Lemma 1. [Tao (19)]. If f (t) L , f (t) L2 , then limt f (t) = 0.
The fact that (t) = (K D + B)s2 0 implies s L2 L , L , i L , i L , i = 1, 2,. . ., l. Since
s = e + e, we conclude that e, e L2 L . Hence xs , x s L , as from assumption (A1) we have xm , x m
L . From Eqs. (76) and (77) it follows that s , s L ; then e L . Applying Lemma 3.1, we conclude that the
position tracking error e(t) and velocity tracking error e (t) = vs (t) vm (t) go to zero as t goes to . In summary,
we have proven the following theorem.
Theorem 3. The adaptive controller (76) with the adaptive laws (79, 80, 81), applied to the system (69) with
jumping parameters (73, 74, 75), guarantees that all closed-loop signals are bounded and the tracking errors
e(t) and e (t) go to zero as t goes to .
Transparency and Stability. With velocity tracking, controllers that ensure force tracking will also lead
to asymptotic weak transparency of the teleoperation system. For such transparency, the force control (72) is
also a choice for a teleoperation system with jumping parameters. Because the parameter jumping is bounded,
the resulting jumping in acceleration and velocity is bounded as well. This will not change the passivity of the
slave system, because its elements are still passive. Hence the system stability is guaranteed with respect to
passivity.
Design for Smooth Time-Varying Parameters. Parameter variations in those systems are characterized
by continuous bounded functions with bounded derivatives. The slave system is represented by
where m(t) > 0, B(t) > 0, K(t) > 0 represent the time-varying mass, damping, and spring parameters. This
model follows from the EulerLagrange equation [Spong and Vidyasagar (8)] with kinetic energy K = x s M(t)
x s . Transparent teleoperation designs for known and unknown time-varying parameters are considered in this
section. To achieve weak transparency the key is velocity tracking and force tracking between slave and master
robots.
24
POSITION CONTROL
Control Design for Known Time-Varying Parameters. A control scheme that ensures asymptotically weak
transparency is proposed first for the teleoperation system with known time-varying slave system. This scheme
is then extended to the time-varying slave system with bounded parameter disturbances.
Design I for Known Time-Varying Parameters.Design I for Known Time-Varying Parameters
slave system (83) with known time-varying parameters, we propose the control scheme
For the
where (t) = [M(t) B(t) K(t)]T , Y( v , v, xs ) = [ v , v, xs ], v = x m (t) e(t), s = x s (t) v, and e(t) = xs (t) xm (t), as
in the sub-subsection Design for System with Unknown Constant Parameters above, and K D > 0 is a design
gain to be specified later. Substituting the controller (84) into the slave system (83) reveals
Define the positive function
The time derivative
To ensure
(t) of V(t) is
(t) 0, we choose K D to be such that
The result that (t) 0 implies that s L2 L . Since s = e + e, we conclude that e and e L2 L . Hence
xs , x s L . From Eqs. (84) and (85) we have s , s L , and therefore e L . Applying Lemma 1, we conclude
that the tracking errors e(t) and e (t) = vs (t) vm (t) go to zero as t goes to .
In summary, we have the following results:
Theorem 4. All signals in the closed-loop system with the time-varying model (83) and the controller (84)
where K D satisfies (3.32) are bounded, and the tracking errors e(t) and e (t) go to zero as t goes to .
Design II for Time-Varying Parameters with Unknown Disturbances.Design II for Time-Varying Parameters with Unknown Disturbances In this case, the system parameters (t) and M(t) satisfy the assumptions:
(1) The time-varying parameter vector (t) satisfies
POSITION CONTROL
25
for some known parameter 0 (t) R3 and some unknown but bounded disturbance (t) R3 such that
(t) < 1 for some constant 1 > 0.
(2) The time-varying parameter (t) satisfies
for some known function

for some constant 2 > 0.
0 (t),
and some unknown but bounded disturbance M (t) such that |M (t)| < 2
We propose the controller structure as
where K D > 0 is the design gain. We choose the positive function V(t) as defined in Eq. (86). Choose K D >
+
2 + k0 , for some design parameter k0 > 0. Then
V(t) is bounded. Since V =
(B + k0 )s2 + 1 2 /4K D + 2 2 /4K D , which implies that
M(t)s2 , we have
which implies
where 1 > 0 is a constant. We then have
and
where k1 > 0, k2 > 0, and 1 > 0 are constants, k0 is a design parameter that can be chosen to be large, and so
is K D > 0. Since s(t) = e (t) + e(t) where > 0 is a constant, we have
26
POSITION CONTROL
where c1 > 0, c2 > 0, d1 > 0, and d2 > 0 are some constants, and 1 =
be chosen to large so that the errors in Eqs. (93) and (94) are small.
In summary, we have the following results.
is a design parameter that can
Theorem 5. All signals in the time-varying system (83) with parameter disturbances (A2), (A3) and controller
(90) are bounded, and the tracking errors e(t), e (t) satisfy Eqs. (93) and (94), respectively. Moreover, e(t) L2 ,
e (t) L2 , and limt e(t) = 0, limt e (t) = 0 in the absence of parameter disturbances, that is, when = 0,
M = 0.
Adaptive Control for Unknown Time-Varying Parameters. Transparent teleoperations are designed for
two types of slave systems: those with unknown smooth time-varying (parametrizable) parameters, and those
with unknown and disturbed time-varying (unparametrizable) parameters. An adaptive control scheme is
proposed for the first type of system to achieve asymptotic weak transparency. With modification, this scheme
ensures approximate weak transparency in the mean for the second type of system.
Design I for Parametrizable Parameter Variations.Design I for Parametrizable Parameter Variations
present an adaptive control design for systems satisfying the following assumptions:
We
(1) The unknown time-varying parameter vector (t) satisfies
for some known function Y 0 (t) R3r and some unknown but constant parameter 0 Rr , for some r
1 [under this assumption, Y( v , v, xs )(t) = Y( v , v, xs )Y 0 (t)0 , so that Slotine and Lis (1) design in the
sub-subsection Design for System with Unknown Jumping Parameters above can be applied].
(2) The time-varying term
(t)( x s + v) can be expressed as
for some known function Z(xs , x s , xm , x m , t) R1r and some unknown but constant parameter 0 Rr, for
some r 1.
We propose the adaptive controller structure
where 0 , 0 are the estimates of 0 and 0 .

Including the controller (97) in the slave system (83) leads to
POSITION CONTROL
Define the parameter errors
0 ,
0 , and choose the positive function
where = T > 0 and = T > 0 are constant matrices of the appropriate dimensions. To ensure that
0, we choose the adaptive laws for
and
27
as
With this choice of 0 and 0 , we have = K D s2 0, which implies that s L2 L and 0 , 0 L . Since
s = e + e, we conclude that e, e L2 L . Hence xs , x s L . From Eq. (97) it follows that s , s L ; therefore,
e L . Applying Lemma 3.1, we conclude that the tracking errors e(t) and e (t) = vs (t) vm (t) go to zero as t
goes to .
Theorem 6. The adaptive controller (97) with the adaptation law (100) and (101) applied to the time-varying
system (83) guarantees that all closed-loop signals are bounded and the tracking error e(t) and e (t) go to zero
as t goes to .
Design II for Unparametrizable Parameter Variations.Design II for Unparametrizable Parameter Variations We assume the unparametric parameters having a parametric part and bounded disturbance part.
They satisfy the modified assumptions:
(1) The parameter (t) satisfies
where Y 0 (t) and 0 are the same as that defined in (A4), such that 0 < M 1 for some constant M 1 0, and
(t) < 1 for some constants 1 > 0.
(2) The term
(t)( x s + v) satisfies
where Z(xs , x s , xm , x m , t) and 0 are the same as that defined in (A5) such that |0 | < M 2 for some constant
M 2 > 0, and |(t)| < Y 1 (t)2 for some constant 2 > 0 and some known function Y 1 (t). Remark: One choice
of Y 1 (t) is Y 1 (t) = x s (t) + v(t).
28
POSITION CONTROL
We propose the controller structure as
and the adaptive law for 0 and 0 as
where , are switching signals defined as
for some constants 0 > 0 and 0 > 0.

The Lyapunov candidate function is same as defined in Eq. (99). Using the facts that
0 and
T
T
and 0 T 0 go unbounded if 0 (t) and 0 (t) go unbounded [see Ioannou and
0
0 0, that
0
Sun (20)], and that and 2 are finite [see assumptions (A4 ) and (A5 )], we have that V(t) is bounded, and
Since V(t) is bounded, from Eq. (109) we have
for some constants 0 , 0 > 0, and any t2 > t1 0. Because of the relation s(t) = e (t) + e(t) and > 0 is
constant, we can obtain
POSITION CONTROL
29
and
where 1 , 1 , 2 , and 2 are some positive constants [see Tao (7)]. In this case, the design parameter K D > 0
can be chosen to be large so that the mean errors are small in Eqs. (111) and (112).
Theorem 7. All signals in the closed-loop system (83) with unparametric time-varying parameters, and adaptive control law (104) and adaptive law (105, 106, 107, 108, 109, 110), are bounded, and the tracking errors
e(t), e (t) satisfy Eqs. (111) and (112), respectively. Moreover, e(t) L2 , e (t) L2 and limt e(t) = 0 limt e (t)
= 0 in the absence of parameter disturbances, that is, when = 0, = 0.
Transparency and Stability. For teleoperation systems with known time-varying parameters or parametric time-varying parameters, the adaptive control schemes (84) and (97) ensure velocity tracking from the
slave to the master. Therefore the force tracking design as in Eq. (72) will lead to asymptotic weak transparency
(Definition 3). For time-varying systems with bounded disturbances, an arbitrary small tracking error can be
obtained by increasing the design gain K D . By using the force tracking design in Eq. (72), approximate weak
transparency (Definition 4) or approximate weak transparency in the mean (Definition 5) is achieved. Stability
of the resulting releoperation system is ensured by the boundedness of all the closed-loop signals.
Teleoperation with Communication Time Delay. Communication time delay in a bilateral teleoperation system reduces system stability and performance. Delay on the order of a tenth of a second were shown
to destabilize the teleoperator. The stability problem becomes difficult when a communication time delay T is
present, because a time delay introduce a factor e sT into the system and hence makes the system infinitedimensional. In bilateral teleoperation, the force reflection from the slave, introduced for providing the feeling
of the remote task, has effects on the masters motion, which generates disturbances on the desired motion. The
communication delay may worsen the situation as well. With a time delay T, the conventional communication
law results in tracking of both position and force in the steady state. However, with the delay, the system is not
passive, and will probably never reach a steady state. A preliminary result on a modified control scheme that
provides improved tracking performance for a noncontact task in the presence of time delay and for arbitrary
master trajectories has been developed in Shi et al. (21).
In the previous research, the passivity formalism and network expression were used to investigate the
stability of a bilateral teleoperation system. In these methods, the human operator input is assumed to be
bounded, and the human operator and the environment are assumed to be passive. In the presence of communication time delay, and with the passivity assumptions about the operator and the environment, passivity of
the system depends on passivity of the communication block.
Two approaches can be used to produce a passive communication block. The first was developed by
Anderson and Spong (22) using scattering transformation theory. This solution uses the transmission line
equations as a basis for deriving a passive communication control law. By applying scattering theory, it is
shown how conventional approaches lead to infinite gain of a scattering operator at finite frequencies, and how
by implementing a set of time delay equations this instability can be overcome. The resulting system is then
passive for all time delays. The proposed control law maintains steady-state force and velocity tracking.
The second approach to produce a passive communication block is developed by Niemeyer and Slotine
(23). This approach uses an energy formulation to construct a teleoperation system that imitates physical
systems and obeys an energy conservation law. A wave variable is utilized to characterize time delay systems
and leads to a new configuration for force-reflecting teleoperation.
30
POSITION CONTROL
Since the dynamic control of the remote slave by an operator is severely restricted by the time delays in
the transmission, it is important to provide consistent dynamic performance locally at the remote site in the
face of uncertainties and varying operating conditions. With the development of high-speed and high-capacity
computer networks, it is possible to deliver teleoperation over a public computer network. The problem of
varying communication time delays arises in such a teleoperation system. Adaptivity of a teleoperation system
to uncertain time delay in also desirable. The related stability, tracking, and transparency problems of a
bilateral teleoperation system under uncertain environment but now with communication time delay are
important issues to be addressed. Adaptive control solutions proposed in the subsection Control Designs
need to be modified to provide adaptation mechanisms for adjusting the controller parameters to achieve
desired system performance, despite system uncertainties due to the unknown slave environment, and now in
the presence of communication time delays.
The passivity-based solution to the bilateral teleoperator time delay problem developed by Anderson and
Spong (22) is based on the result in circuit theory that a circuit consisting of passive elements only is passive
and therefore stable. However, if some elements in a circuit representing a teleoperation system are not passive,
one cannot use passive network theory to conclude the stability of the teleoperation system. On the other hand,
if the transfer function of a teleoperation system is positive real, then the system is passive. In Shi et al. (24),
a notion of positive realness has been used to investigate the passivity of the teleoperation system proposed
by Anderson and Spong (22). Shi et al. (24) have also proposed a modified control scheme that use the master
accelaration information (with delayed operation, which can be obtained from the velocity information) for
slave control and ensures that in the absence of slave environment torque the slave position tracks that of the
master asymptotically, that is, achieves improved tracking performance for the teleoperation system.
Summary
Position control for robot manipulators and teleoperation systems involves many dimensions of control theory,
such as controller design, robustness analysis, and adaptive designs, along with many practical applications.
Robust adaptive control schemes have been presented to handle situations in which the robot system has
bounded parameter variations or/and unmodeled dynamics of bounded gains. The distinct feature of the manipulator dynamics were used to define bounding signals in the controller structure, whose parameters are
updated from a robust adaptive law to ensure signal boundedness and tracking errors of the order of parameter
variations and unmodeled dynamics, which may not be small. Some common topics relevant to position control
of robot manipulators, such as PD control, inverse dynamics control, and path or trajectory interpolation, were
also discussed.
Adaptive motion control of teleoperation systems was addressed. Several new concepts of transparency
were defined for teleoperation control systems with unknown parameters. These new transparency concepts
are useful for developing adaptive control schemes for control of a teleoperation system with unknown constant
parameters, or with unknown jumping parameters, or with unknown smooth but large time-varying parameters. Weak transparency properties have been established for such adaptive teleoperation control systems.
Some important control issues for teleoperation systems with communication time delay were also discussed.
BIBLIOGRAPHY
1. J. Slotine W. Li On the adaptive control of robot manipulators, Int. J. Robotics Res., 6 (3): 4959, 1987.
2. J. Slotine W. Li Adaptive manipulator control: A case study, IEEE Trans. Automat. Control, AC-33: 9951003, 1988.
3. J. S. Reed P. A. Ioannou Instability analysis and robust adaptive control of robotic manipulators, IEEE Trans. Robotics
Automation, 5: 381386, 1989.
POSITION CONTROL
31
4. M. W. Spong Adaptive control of flexible joint manipulators, Systems Control Lett., 13: 1521, 1989.
5. F. L. Lewis S. Jagannathan A. Yeildirek Neural Network Control of Robot Manipulators and Nonlinear Systems, London:
Taylor & Francis, 1999.
6. S. S. Ge T. H. Lee C. J. Harris Adaptive Neural Network Control of Robotic Manipulators, River Edge, NJ: World
Scientific, 1998.
7. G. Tao On robust adaptive control of robot manipulators, Automatica, 28 (4): 803807, 1992.
8. M. W. Spong M. Vidyasagar Robot Dynamics and Control, New York: Wiley, 1989.
9. R. Ortega M. Spong Adaptive motion control of rigid robots: A tutorial, Automatica, 25 (6): 877888, 1989.
10. C. A. Desoer M. Vidyasagar Feedback Systems: InputOutput Properties, New York: Academic Press, 1975.
11. M. W. Spong R. Ortega R. Kelly Comments on Adaptive manipulator control: A case study, IEEE Trans. Automat.
Control, AC-35: 761762, 1990.
12. P. A. Ioannou K. Tsakalis A robust direct adaptive controller, IEEE Trans. Automat. Control, AC-31: 10331043, 1986.
13. V. Utkin Variable structure systems with sliding modes, IEEE Trans. Automat. Control, AC-22: 212222, 1977.
14. D. A. Lawrence Stability and transparency in bilateral teleoperation, IEEE Trans. Robotics Automation, 9 (5): 624637,
1993.
15. K. Hashtrudi-Zaad S. E. Salcudean Adaptive transparent impedance reflecting teleoperation, Proc. IEEE Int. Conf. on
Robotics and Automation, Minneapolis, 1996, pp. 13691374.
16. H. K. Lee M. J. Chung Adaptive controller of a masterslave system for transparent teleoperation, J. Robotic Systems,
15 (8): 465475, 1998.
17. M.-Y. Shi et al. Adaptive control of teleoperation systems, Int. J. Robotics Res. Preliminary version, Proc. 38th IEEE
Conf. on Decision and Control, Phoenix, AZ, 1999, pp. 791796.
18. G. J. Raju G. C. Verghese T. B. Sheridan Design issues in 2-port network models of bilateral remote manipulation, Proc.
1989 IEEE Int. Conf. on Robotics and Control, 1989, pp. 13161321.
19. G. Tao A simple alternative to the Barbalat

lemma, IEEE Trans. Automat. Control, 42: 698, 1997.
20. P. A. Ioannou J. Sun Robust Adaptive Control, Upper Saddle River, NJ: Prentice-Hall, 1996.
21. M.-Y. Shi et al. Control issues in teleoperations with communication time delays, Proc. 32nd Ann. Conf. on Information
Sciences and Systems, Princeton, NJ, 1998.
22. R. J. Anderson M. W. Spong Bilateral control of teleoperator with time delay, IEEE Trans. Automat. Control, 34:
494501, 1989.
23. G. Niemeyer J. E. Slotine Stable adaptive teleoperation, IEEE J. Ocean Eng. 16: 152162, 1991.
24. M.-Y. Shi G. Tao J. H. Downs Positive realness and tracking of teleoperation systems, Proc. 37th IEEE Conf. on Decision
and Control, Tampa, FL, 1998, pp. 25272532.
GANG TAO
AVINASH TAWARE
MINYAN SHI
University of Virginia
14
JET ENGINE CONTROL, IMPLEMENTATIONS
tion and deceleration schedules to provide transient limit protection. More advanced controls schedule variable engine geometry and augmentor fuel, provide fan and booster stall
protection, control variable parasitic engine flows, improve integrated engine-airframe performance, and provide engine
health monitoring and diagnostics.
It should be noted that only recently have electronic computers been used to implement engine controls. This is primarily due to the inherent need for safe operation and the
harsh temperature and vibration environment in which the
computer operates. Many engines in use today are controlled
by a hydromechanical controller commonly referred to as an
HMU. These are ingenious mechanical computers, which implement the desired control strategy in terms of cams and
mechanical integrators. Of necessity, the implemented control
strategies must be fairly simple. A drawing of a typical HMU
is shown in Fig. 1. More detailed discussions of their operation can be found in (4).
The changeover to electronic controllers began in the 1980s
as rugged integrated circuits became available and as the
need for improved performance led to increased functionality
and tighter control. Pratt and Whitney calls its controller a
Digital Engine Control (DEC), while General Electric calls it
a Full Authority Digital Electronic Control (FADEC). These
are highly customized computers, whose complexity depends
mainly on the number of sensor inputs and actuator outputs.
Such electronic controllers result in higher engine operating
efficiencies, by allowing tighter engine control through the
use of higher loop gains and improved strategies to reduce
transient overshoot or undershoot. It also allows implementation of control algorithms, which would be difficult to implement mechanically.
BASIC ENGINE TYPES

In one form or another, jet engines power all but the smallest
airplanes and helicopters. They produce propulsive thrust
from the thermal energy of jet fuel. Sir Frank Whittle (1) is
credited with developing the first jet engine during the 1930s.
It was similar to the turbosupercharger that had been developed in the 1920s, which also used a single-stage centrifugal
compressor, a combustor, and a single-stage turbine. Apparently unaware of Whittles work, a jet engine was also patented in Germany by Hans von Ohain and Max Hahn (2,3)
in 1936. The subsequent German development led to the
JUMO 109 engine, which had many design features of more
modern engines, such as a multistage axial compressor, turbine blade cooling, and a variable-area exhaust nozzle. Unfortunately, it was limited by available materials to an operating
life of about 10 hours.
Feedback control has been as essential part of a jet engine
from the beginning. Engines are most effective when they can
be operated at or near their mechanical or aerothermal limitations, such as rotor speeds, turbine temperatures, internal
pressures, and so on. Controlling at but not exceeding a limit
is a very important aspect of engine control, which must,
therefore, provide both regulation and limit management.
Minimum control requirements include a main fuel control
for setting and holding steady-state thrust, with fuel accelera-
Three basic types of jet engines are in current use:

1. Turbojets
2. Turbofan engines
3. Turboprop/turboshaft engines
The turbojet was the earliest form of jet engine, and is the
simplest of the three. Its major components include a compressor, combustor, turbine (which drives the compressor),
and exhaust nozzle. It produces a relatively high specific
thrust, defined as thrust per kilogram of airflow. It is the best
type of engine for high subsonic and supersonic flight speeds.
The turbofan uses a turbojet for its core and adds a fan in
front of the core compressor and a second power turbine behind the core turbine, to drive the fan, as shown in Fig. 2.
The flow capacity of the fan is designed to be substantially
larger than the compressor, so that the excess air can be bypassed around the core and exhausted through a separate
nozzle. The bypass approach reduces engine specific thrust,
but increases propulsion efficiency, thereby reducing fuel consumption and is the engine of choice for subsonic commercial
airplanes.
The turboprop or turboshaft engine includes the turbojet
core and power turbine, but has no fan. Its power turbine can
drive an external propeller or helicopter rotor through a gear
15
Forward
thrust
stop
Power
lever
stop
Reverse
thrust
stop
Forward
signal
Shut off
stop
Atmos
Ratio
lever
Overboard drain leakage

Comp inlet press.
Servo press.
Bypass to pump
interstage
Regulated press.
Water
drain
Feed-back lever
Accel.
limiting
cam
Press
ratio
reg.
Burnerpress sensor
position adj.
Figure 1. Hydromechanical controllerHamilton Standard JFC25 (4).
Atmospheric press.
Increasing R.P.M.
Burner press.
Speedsensing
servo
Speed
servo
stop
Flapper valve
Feedback
lever
stop
Droop
lever
Speed
servo
pos. adj.
Max ratio adj.
Speed cam
follower
Dec.
inc
Burnerpressure
output
Mult. lever
Balancing
nozzle
Burner-press.
limiting
valve
Water
drain
Throttle valve
position adj.
Sequencing
valve
Sequencing
valve cam
Throttle-valve
metered press.
Return to engine
pump inlet
(boost press.)
Droop
cam
Atmos
Fixed
orifice
Speed- Speedset lever set cam
Droop adj.
Min.
ratio
adj.
Droop cam
followers
Filter
Pump discharge press.
90 stop
Max
trim
Idle
trim
Shut off
lever
Thrust
reverser
inlet
signal
Shut off
lever stop
Detent
roller
Power
lever
Atmos
Reverse
signal
Thrustreversing
actuator
Burner-press.limiting-valve adj.
Fail-safe
orifice
Burner
press
Evacuated
bellows
N2 drive
Throttle valve
Min. flow adj.
Fixed orifice
Bypass to pump
interstage
Press-reg valve
Fixed orifice
Fail-safe orifice
Fuel-temp
compensator
Flapper valve
Press-reg
servo sensor
Filter relief valve

Press-reg valve adj.
Fuel inlet
from pump
Overboard drain
Speed-sensing governor
Speed-servo rate adj.
Coarse filter
Fine filter
Min press and

shut off valve
To engine nozzles
16
Figure 2. Pratt & Whitney PW4084 turbofan engine.
reduction unit. The rotor or propeller further increases total

engine air flow, decreases specific thrust, and increases propulsion efficiency. The turboshaft is the best type of powerplant for helicopters and small, lower speed aircraft.
Variations of the above basic engine types can include:
Dual rotor core engines containing two compressors and
two turbines
Turbofans with a booster compressor between the fan
and core compressor for supercharging the core
Mixed-flow turbofans which mix the bypass and core discharge flows and exhaust both through a single nozzle
Turbojet augmentors, turbofan fan burners, and mixed
flow augmentors for increasing engine thrust for better
takeoff, transonic acceleration, or combat capabilities
SIMPLIFIED ENGINE THEORY
The most common form of jet engine is the high bypass ratio
turbofan. It will be described in this article. More detailed
discussion on engine design and operation can be found in
(5,6). A turbofan engine bypasses a substantial fraction of the
inlet air around the hot section or core of the engine, in order
to achieve a high propulsive efficiency. A simplified diagram
of such an engine is shown in Fig. 3.
Engine
inlet
The numbers in Fig. 3 refer to standardized (5,7) engine

station locations:
0
1
2
25
3
4
45
5
8
9
Double-digit numbers 1218 are used for bypass flow stations

from fan tip entry (station 12) through the bypass duct to the
bypass nozzle (station 18).
Reciprocating automobile engines operate on a four-stroke
Otto cycle. Their internal combustion process achieves extremely high pressures through constant volume combustion,
which results in a high power per kilogram of air flow.
HPC
discharge
HPC
inlet
Figure 3. Engine station locations.
Freestream ambient air conditions

Inlet entry
Fan entry
High-pressure compressor entry
High-pressure compressor exit
Burner exit/high-pressure turbine entry
High-pressure turbine exit/low-pressure turbine
entry
Turbine exit
Nozzle throat
Exhaust nozzle exit
2.5
LPT
discharge
Combustor
discharge
Afterburner
inlet
Exhaust
Exhaust
nozzle
nozzle
inlet
discharge
Exhaust
nozzle
throat
Conversely, the jet engine operates on a continuous flow

Brayton cycle, which ideally involves isentropic compression
and constant pressure combustion. It operates at a substantially lower maximum temperature and pressure than the
Otto cycle. Figure 4 shows pressure-volume and temperatureentropy (TS) diagrams for the ideal Brayton cycle. It is the
basis of the turbojet engine and the core of the turbofan. It
contains the following thermodynamic processes:
02
23
34
45
59
Isentropic compression in the engine inlet

Isentropic compression in the engine fan, booster,
and compressor
Constant pressure heat release in the combustor
Isentropic expansion in the high-pressure and lowpressure turbines
Isentropic expansion to atmospheric pressure in the
exhaust nozzle
An isentropic process means that there is no temperature

raise and the process is reversible. Hence entropy is constant.
The actual cycle involves near isentropic compression and
expansion processes, a pressure loss in the combustor, and
an incomplete expansion to near-atmospheric pressure in the
exhaust system. Total temperatures and pressures, which include the effect of the air velocity, are used for all internal
engine conditions.
Alternative options, which include the turbojet, turboshaft,
and turbo-augmented cycles, will not be discussed, but can be
determined from similar techniques. The following sections
describe each of the turbofan engine processes in more detail.
Compression System
The turbofan compression system consists of a fan, a booster,
and one or more compressors. Operating characteristics of
each are a function of their inlet total temperature and pressure, rotational speed, and discharge pressure. It is convenient to normalize flow and speed with respect to inlet temperatures and pressures. The following corrected parameters
can be used to represent operating characteristics independent of the actual inlet temperature and pressure levels:
Corrected flow (Wa / a)

Corrected speed (N/ a)
Pressure ratio (Pb /Pa)
Adiabatic efficiency (c)
where W is the flow rate, is the inlet temperature divided

by the ambient standard sea level temperature, is the inlet
pressure divided by the ambient standard sea level pressure,
N is the compressor rotational speed, and c is the compressor
adiabatic efficiency. The subscripts a and b refer to inlet and
discharge conditions, respectively, for the fan, booster or compressor.
Operating characteristics of a typical compressor are
shown in terms of the above corrected parameters in the compressor map in Fig. 5. Given measured corrected speed and
pressure ratio, one can determine corrected flow and efficiency (not shown in Fig. 5). Exit temperature and required
work can then be obtained from:

Tb = Ta
Inlet Compression
Air flow is supplied to the engine by the inlet which compresses the inlet air. Assuming that the ambient air pressure, P0, and temperature, T0, is not moving and the inlet is
moving at the flight mach number, M, the total pressure and
temperature at the inlet is:

1/k
1
M2
P2 = r P0 1 +
2

1
M2
T2 = T0 1 +
2
(1)
(2)
where P is pressure, T is temperature, M is flight Mach number, is the specific heat ratio of air (constant pressure specific heat/constant volume specific heat), k is the ratio (
1)/, and r is the ram recovery (actual total pressure/ideal
total pressure) which is approximately 1 for subsonic flight.
Combustion
Compressor
Nozzle
2
Inlet

(3)
(4)
where HP is the work required to drive the compressor and

cp is specific heat of air at constant pressure. This work is
expended heating the air from Ta to Tb.
Compressor characteristics must be obtained from extensive testing of individual stages, the full compressor, and
sometimes the entire engine.
Stable compressor operation is limited to the region below
the compressor stall line shown in Fig. 5. Two modes of instability can occur: surge, which is a longitudinal flow oscillation
over the length of the compressor and turbine, and stall,
which is the lack of pressure rise between the compressor
blades. Often stall occurs at low rotor speeds and surge at
high rotor speeds. Both surge and stall generate violent axial
oscillations of the internal air column, which can cause sub-
4
Turbine
3
Turbine
(P /Pa )k 1
1+ b
c
HP = Wa c p (Tb Ta )
Combustion
4
Pressure
Pressure
17
Compressor
2
Inlet
Nozzle
Volume
Entropy
Figure 4. Pressure-volume and temperature-entropy (TS) diagrams for turbojet.
18

High
Surge line
@ Sea level
@ 35,000 ft
@ 50,000 ft
Acceleration line
Deceleration line
Pt3
Pt2
Low
Low
T2
where Wf is the fuel flow rate, Qf is the higher heating value

of the fuel, and f is the combustion efficiency. The subscript
4 refers to high-pressure turbine inlet conditions. Depending
on the combustor pressure, the combustor will operate only
in certain regions, as shown in Fig. 6. The high-temperature
blowout region is referred to as rich blowout and the low temperature region is referred to as lean blowout.
The total gas flow downstream of the combustor is the sum
of the air flow and the fuel flow. The specific heat of the gas
mixture can be obtained from the equation:
c pg =
c pa + f cpf
1+ f
(6)
Other mixture properties, such as enthalpy and entropy but

not can be obtained by a similar process.
A significant amount of air is required to cool the combustor liner, but it is returned to the gas stream prior to the
turbine.
Corrected flow function (WgaTa /Pa)

Corrected speed (Na / Ta)
Temperature ratio [(Ta Tb)/Ta]
Adiabatic efficiency (t)
2000
Effect of varying T3
1800
1600
1400
1200
ou
(5)
e
tr
gio
T3
ow
Fuel is burned in the combustor at a slight pressure drop,

and the resulting products of combustion are expanded in the
turbines. The required fuel flow can be obtained from:
The turbine expansion system provides the power to drive the

compression system. The high-pressure turbine drives the
compressor through the high-pressure shaft, and the lowpressure turbine drives the fan and booster through the lowpressure shaft. The operating characteristics of each turbine
are defined in terms of the following corrected parameters:
Bl
Combustion System
Turbine Expansion
T4 T3; Combustion total temperature rise (F)
stantial damage to both the compressor and the engine. Fan

stalls can be caused by operation with too small a fan duct
nozzle area, booster stalls by a throttle reduction to a lower
engine rotational speed, and compressor stalls by a rapid
throttle increase. The engine and control system must be designed to avoid surge/stall, with sufficient design margin to
offset the effects of altitude, increased tip clearances, component deterioration, and engine/airflow operation at high
angles of attack.
W4 c p (T4 T3 )
Q f f cpf T4
Corrected shaft
speed
High
Wa T2
Figure 5. Engine compressor map.
Wf =
Zone of poor
compressor
efficiency
N/ T2 Low
Steady-state
operating line
@ Sea level
@ 35,000 ft
@ 50,000 ft
N/ T2 High
Pressure ratio
Surge zone
1000
800
Operating area
600
400
200
Blowout region
0
10
20 30 40 50 60 70
Combustion total pressure
inches (HG ABS.)
Figure 6. Combustor blowout limits.
80
The subscripts a and b refer to inlet and discharge conditions,

respectively, for the high-pressure and low-pressure turbines.
Operating characteristics of a typical turbine are illustrated,
in terms of the above corrected parameters, in the turbine
map shown in Fig. 7. Given measured corrected speed and
temperature ratio, one can determine corrected flow and efficiency. Note that the turbine inlet flow is choked at a constant
value over a large part of the operating range.
The temperature ratio across the turbine can be obtained
from the required work of the compressor (for the high-pressure turbine) or the work of the fan and booster (for the lowpressure turbine):

Ta Tb
HP
=
(7)
Ta
Wga c pg Ta
nozzle can be used for exhaust flow, which is either subsonic

or sonic, and a converging-divering (C-D) nozzle would be required for supersonic flow. The throat or minimum area of the
nozzle will regulate the amount of flow that can be exhausted
through the nozzle. The required throat area for the core
stream can be obtained from:
A8 =
Total fan inlet air flow

Bypass ratio =
Core inlet air flow
8 N8 P8

8 = g/R

3
1.2
N8 = M8
1 + .2M82
(9)
(10)
(11)
The throat Mach number must be 1 if the nozzle total-tostatic pressure ratio is greater than the critical value of about
1.8. If the pressure ratio is less than critical, the throat Mach
number will be less than 1, and can be obtained from the
relationship:

M=
(8)
It represents a major turbofan engine design parameter. The

bypass duct operates with a slight pressure drop of about 5
percent of the fan discharge pressure and no loss in total temperature.
Wg8 T8
where 8 is the sonic flow function and N8 is the ratio of flow

to sonic flow:
Bypass Duct
A fraction of the fan discharge air is bypassed around the
engine core, and exhausted through either a separate bypass
nozzle, or mixed with the core stream and exhausted through
the core nozzle. In either case, the bypassed air improves the
propulsive efficiency of the engine, and makes it the preferred
approach for the worlds large commercial fleet. The bypass
ratio is defined as:
19

2
[(PT /PO )k 1]
1
(12)
The exit area will effect the thrust output of the engine. Similar equations can be used for the bypass or mixed flow
streams.
Augmentor
Exhaust System
The core and bypass streams are expanded through core and
bypass nozzles to the pressure of ambient air. A converging
.28
5.0
Nozzles choke
.26
4.0
.24
.85
.22
T = .84
.20
T4 T5
T4
.18
122
3.0
120
.16
T4
W
.14
P4
= 115
Military engines generally have an augmentor, either behind

the low-pressure turbine or in the bypass duct. Augmentors
are sometimes referred to as afterburners. They are used to
increase engine thrust for selected segments of the flight,
such as takeoff, climb, acceleration to supersonic speed, or
combat. Augmentation is a relatively inefficient approach for
generating thrust. This penalty can be minimized by maintaining the engine at its maximum nonaugmented setting,
thereby minimizing the thrust increment provided by the
augmentor.
Augmentation requires a variable exhaust nozzle. The reason can be seen from Eq. (9). For a fixed A8, an increase in
T8 must be offset by an increase in P8. Since combustion is
essentially a constant-pressure process, the increase in P8 results in an increase in turbine pressure, P5 and, hence, an
increase in P3, moving the engine operating line closer to
stall. From Eq. (7), the increase in P5 also produces less work
from the turbine, which will reduce core rotor speed. The control system will increase main fuel flow, to keep rotor speed
constant, resulting in increased temperature of the turbine.
Thus A8 must be opened to maintain a constant P8, to avoid
compressor stall and overtemperature of the turbine.
.12
110
.10
60
70
80
90
100
P4/P5 = 2.0
110
120
N/ T4
Figure 7. Turbine map.
130
140
Engine Trade-Offs
Engine operating characteristics are set predominantly by
four interacting design variables: (1) bypass ratio, (2) turbine
inlet temperature, (3) overall pressure ratio, and (4) fan pressure ratio. The best design choice will be dependent on the
20
intended application or mission, the level of technology available, the degree of subsequent growth capability required,
and expected competition from other engines.
Bypass ratio will have the most dominant effect on engine
performance. High bypass ratios of 4 to 8 are used for most
large commercial engines. Increased bypass ratio will improve
(decrease) specific fuel consumption (SFC) at cruise, and improve specific thrust (thrust per kilogram/second of air flow)
at takeoff. Ultra-high bypass ratios of 10 to 20 have been considered for improved cruise performance, but would require
an unducted fan (8) or reduction gears between the fan and
low-pressure turbine. Lower bypass ratios of 1 to 3 provide
improved thrust for flight Mach numbers of 1 to 2, and are
used for military fighters and bombers. A pure turbojet has a
zero bypass ratio, and would be used for supersonic transport.
High turbine inlet temperature leads to improved specific
thrust and a lighter engine, but requires more expensive turbine materials and a more complex turbine cooling system,
which reduces cruise performance. The proper balance will
depend on the relative importance of specific thrust, which
sets engine size, and weight and cruise performance, which
sets fuel requirements. Military applications will tend to demand higher temperatures, to achieve a lighter engine
weight, while commercial applications will place a stronger
emphasis on cruise performance.
Overall pressure ratio, which is the compressor discharge
pressure divided by fan inlet pressure, will affect both takeoff
and cruise performance. Optimum pressure ratio tends to increase will increased turbine inlet temperature, but decreases
as flight Mach number increases. Pressure ratios of 40 : 1 and
50 : 1 could be effective with modern temperatures at takeoff,
but should be limited to the 10 : 1 to 15 : 1 range at supersonic
speeds. Extremely high-pressure ratios would require the use
of high alloy steel or titanium at the rear of the compressor,
and cooled-cooling air for use in the high-pressure turbine.
High fan pressure ratio improves takeoff performance, but
increases exhaust noise. The achievable pressure ratio will be
dependent on the work available from the low-pressure turbine. Both will be dependent on turbine inlet temperature
and overall pressure ratio. Fan pressure ratios of 2 to 3 can
be achieved on low bypass military engines, but would be limited to the 1.6 to 2.0 regime for high bypass commercial engines. Mixed-flow turbofans would also require the use of fan
pressure ratios that produce duct pressure levels roughly
equal to the turbine discharge pressure, in order for mixing
to occur.
CONTROL REQUIREMENTS
The overall function of an engine controller is to provide
thrust in response to throttle position. It must achieve the
requested thrust with the lowest specific fuel consumption. It
must also insure that the following limits are not exceeded:
Maximum fan speed
Maximum compressor speed
Maximum turbine temperature
Fan stall
Compressor stall
Maximum compressor discharge pressure
Minimum compressor discharge pressure

Lean burner blowout
Rich burner blowout
For an aircraft turbine engine, it is necessary to achieve
maximum thrust with minimum engine weight. This means
that all components operate at mechanical or thermal limits
for at least one of the engines critical operating conditions.
At other operating conditions, operation at only one or more
of the above limits may be required. Figure 8 shows, various
typical limits as function of mach number and altitude. The
control must directly or indirectly control each limiting parameter and limit engine thrust, so that no limits are exceeded. Engine operation at maximum power will, consequently, require operation at one or more of the engine
operating limits. Part power engine operation should occur
below all limits and at the lowest specific fuel consumption
for the thrust requested.
SENSORS AND ACTUATORS
The pilot controls aircraft speed by setting the throttle position to a thrust which will permit operation at the desired
speed. One would like to run the engine directly to the required thrust, by modulating engine fuel flow until the required thrust has been achieved. Similarly, one would like to
operate at the fan and compressor stall limits and the turbine
inlet temperature limits. However, none of these parameters
can be measured directly in flight. Thrust can only be measured in a test cell, where one can establish a stationary reference. Likewise, stall margins are established in test rigs, by
actually stalling the fan or compressor. Thus one must select
practical measurements that are related to the ideal measurements. These practical measurements must also be relatively
immune to engine-to-engine variation due to manufacturing
tolerances and engine deterioration. Generally, Monte Carlo
Various structural
and overspeed
limitations
Primary
and augentor
A ignition and
blow out
l
t
i
Low
t
pressure
u
limit
d
e
Constant
T2 limit
T4 and T7
limits
separate
hard
augmentor
light-off
Minimum
burner
pressure
at low
power
Maximum
turbine
inlet
temperature
High pressure
limit
Blade loading
and burner case
limits
Mach number
Figure 8. Engine limits (19).
simulations are performed to establish the appropriate selection.

Various thrust setting parameters can be used to set
thrust indirectly. GE uses fan rotor speed, Pratt and Whitney
uses core engine pressure ratio (EPR), and Rolls Royce uses
an integrated engine pressure ratio (IEPR, which is a flowweighted average of the core and bypass duct pressure ratios).
Commonly selected basic sensors are:
N1
N2
EPR
Wf /P3
T45
Fan speed
Core speed
Engine pressure ratio P5 /P2
Fuel flow divided by compressor exit pressure
Low pressure turbine inlet temperature
Three of these are required for the direct determination of

the critical overspeed and overtemperature limits. Because of
the need to shield the thermocouples, the T45 measurement
has a slow response time, and is not particularly good for control purposes. Either EPR or Wf /P3 is used to control the engine. Wf /P3 is very commonly used and is a natural control
parameter. To see why, note that the flow through the turbine
is:
c
c
W4 = P4
= P3
T4
T4
(13)
since there is very little pressure drop across the burner.

Therefore
Wf
P3
Wf
W4
T4
(14)
Since the fuel air ratio, Wf /W4, is directly related to flame

temperature and, hence, turbine temperature T4, so is Wf /P3.
Thus Wf /P3 provides good control of T4, good reaction to stall,
and good reaction to burner blowout. An added advantage is
that Wf /P3 is relatively easy to sense.
Commercial engines will generally have variable compressor stator vanes, a fuel control valve, and a variable booster
bleed valve (if there is a booster). The variable stators are for
improving low-speed stall margin and high-speed performance in the core compressor. The booster bleed valve is used
to bleed booster air into the bypass duct, to improve booster
stall margin at part power and during throttle reductions.
Both are generally controlled open-loop. The only closed-loop
control in a commercial engine is the main fuel control. Military engines will generally add an afterburner for increasing
thrust during takeoff, transonic acceleration, and combat. The
afterburner will require both an augmentor fuel control and
a variable exhaust nozzle throat area (A8), to permit a wider
range of augmented thrust and to manage interactions with
the main fuel control.
ENGINE CONTROLS
Proportional and Integral Control
The simplest and most common type of control is based on a
proportional control. With minor variations, it is the basis of
21
all hydromechanical controls, and has been used for some of

the earliest electronic controls as well. More recent engines
with electronic controls use proportional plus integral control,
to minimize errors. The control can be subdivided into a
steady-state control and a control for transient operation. The
steady-state control, which maintains engine operation along
its steady-state operating line, will be discussed first. The
transient controls necessary to accelerate and decelerate the
engine while meeting stall, flameout, and temperature limitations, will then be added.
A block diagram of the single-input, single-output proportional plus integral control, which maintains the engine at a
desired operating point, is shown in Fig. 9. A fan speed demand schedule establishes the desired fan speed as a function
of inlet temperature and throttle angle. Fan speed error is
determined from the difference between the demand speed
and the actual fan speed. A delta Wf /P3 signal is obtained proportional to the fan speed error. This signal is multiplied by
the P3 sensor signal and the resulting change in fuel flow is
used to drive the fuel metering valve. The gain is scheduled
as a function of core speed, to meet the control requirements
over the entire flight envelop. In an electronic control, the desired fuel flow is computed directly, rather than as a ratio,
with P3. The control will provide the control response for
small increases or decreases in throttle angle. Controllers
based on EPR or IEPR substitute pressure ratio demand
schedules and error signals for the corresponding fan speed
parameters.
With proportional control there is always an error or droop
between the desired speed and the actual speed. As the load
on the engine increases, this error will increase. The loop gain
is normally made high, to minimize this droop. The use of
integral control eliminates this problem.
The throttle schedule is designed to provide thrust as a
linear function of the throttle angle. It also maintains the engine at the most efficient operating point possible. Engine
thrust can be shown to be a nonlinear function of engine
speed. The throttle schedule inverts this nonlinear relationship, to provide the desired speed as a function of throttle
angle.
Transient operation is illustrated in Fig. 10. It shows
Wf /P3 as a function of core speed for a variety of engine operating modes during engine accelerations and decelerations.
An acceleration will occur when the throttle is advanced to a
demand speed higher than the current speed. A min-select
strategy will then lead to the following series of processes:
The positive speed error will result in a proportional
Wf /P3 error, which will open the fuel valve, admitting
more fuel to the engine and result in an acceleration
from point 1 along the speed governor line to point 2,
where it will intersect the maximum fuel ratio line.
Min-select strategy will switch operation to the maximum ratio line, and it will continue to accelerate to point
3, where it will intersect the compressor surge limit line.
Min-select strategy will switch operation again, this time
to the surge limit line, where it will continue to accelerate at a slower pace to point 4.
At point 4, the min-select strategy will switch operation
back to the maximum fuel ratio line, and the engine will
continue to accelerate to point 5.
22

Scheduled fuel
Throttle
T2
Operating
schedule
P3
Desired fan
speed
Gain
Engine
Figure 9. Block diagram of proportional

controller.
Fan speed
At this point, operation will be switched back to the

speed governor line, leading to the steady-state operating
point at the demanded speed at point 6.
This final process will reduce Wf /P3, until the engine stabilizes at point 6.
An engine deceleration will occur when the throttle is retarded to a speed demand lower than the current speed. A
max-select strategy during the deceleration will then lead to
the following:
The negative speed error will lead to a proportional
Wf /P3 error, which closes the fuel valve, reduce fuel flow,
and cause a deceleration along the speed governor line to
point 7.
Max-select strategy will switch operation at point 7 to
the minimum fuel ratio line to point 8.
Max-select strategy will switch operation again at point
8 to the speed governor line, which will lead back to
steady-state operation at point 1.
Fuel flow will be increased until the engine stabilizes at
the demanded speed at point 1.
Note that each of the above lines in Fig. 10 are generally
functions of inlet pressure (P2) and possibly temperature (T2)
as well. To make the engine accelerate or decelerate rapidly
and remain as efficient as possible, the control is designed to

keep the engine close to or on each limit.
The above processes will occur during large increases or
decreases in throttle/speed demand. During small throttle increases, the minimum select process will transition engine operation from the 19 speed governor line directly to the 910
speed governor line. This will lead to stable operation at point
10. A small deceleration command at point 6 will result in a
similar transition at point 11 to stable operation at point 12.
Other limits can be handled in a similar fashion. The deceleration limit provides sufficient fuel to maintain combustion. Except for high flows, it can be independent of speed. An
idle speed controller is provided, to maintain minimum engine speed. It is a proportional controller similar to Fig. 9,
with no additional schedule and the reference speed is the
desired idle speed. Overtemperature and overspeed are also
handled as proportional controls, with the maximum temperature or speed as the reference. The overtemperature controller senses T45.
A block diagram of the complete control is shown in
Fig. 11.
Ndot Control
A variant of proportional control uses the derivative of rotor
speed (9,10) (Ndot or rotor acceleration), rather than rotor
speed, to control engine acceleration and deceleration. Direct
Wf /Ps3
14.7 psia [101.36 kPa]

6 psia [41.37 kPa]
2 psia [13.79 [kPa]
7
8
Surge
area
6
Max. ratio line
Rich blowout 5
(8-9)
area
(5-6)
Speed
(78)
(2-3)
3D cam
governor
3D cam
3D
(4-5)
cam
Speed
(6-7)
(9-10)
governor Governor
3D cam
droop
lines
Speed
3
(infinite number)
governor
2
Steady state
lines
(3-4) 4 12
Speed
governor
Speed reset
(11-12)
(10-11)
11 Min. flow stop
line
10
(1-2)
Min. ratio line
Speed
1 governor
0
Figure 10. Characteristic

schedules.
controller
Idle
N
Percent rotor speed
Nmax
2 psia
[13.79 kPa]
14.7 psia
9 [101.36 kPa]
100%
23
Fan speed
T2
Core speed
T2
Fan speed
controller
PS3
Deceleration
controller
Core speed
T2
Core speed
T2
Core speed
T2
Acceleration
controller
Idle
controller
Min
Max
Min
Engine
Max and min
fuel limits
Max
Operating
schedule
controller
Figure 11. Block diagram of complete

controller.
control of acceleration, rather than speed, allows tighter control of engine acceleration, thereby improving transient response and reducing mechanical stress. While rotor acceleration cannot be easily measured directly, a second-order filter
applied to speed can be used to give a good approximation.
The algorithm shown in Fig. 12 replaces that shown in Fig. 9.
The previous acceleration schedule is replaced with one that
directly specifies allowable engine acceleration. A lead lag
compensator is necessary to improve transient response. The
Ndot control will result in a more consistent acceleration between a cold engine that has not been run for at least 30
minutes, and a warm engine that is being reaccelerated in a
shorter time.
Fan Speed Control
Throttle position is converted to a specific fan speed requirement as a function of engine inlet temperature, T2, and inlet
pressure, P2. Fan speed is held to the desired speed by replacing the proportional control with an isochronous integral control. The bandwidth of the loop will determine the speed of
response and, hence, how tightly the fan speed is maintained.
The core speed floats, within limits, at a speed necessary to
provide the sufficient power to maintain fan speed.
The agility of a helicopter (11) depends on maintaining rotor speed and, hence, power turbine speed during maneuvers
such as a waveoff from an autorotational descent. The load
on the rotor is changed by the pilot, changing either the col-
Fan speed
controller
Wf /P3 demand
Integrator
T2
Ndot
schedule
lective or cyclic settings. System responsiveness is determined by the dynamics of the power turbine isochronous governor. Unfortunately, the helicopter rotor first torsional
resonance mode limits the bandwidth of the control. Notch
filters centered at the rotor resonance are used to allow high
crossover and, therefore, a more responsive system.
A further refinement of fan speed control is called Power
Management Control. This control integrates aircraft and engine requirements, to compute the necessary thrust levels to
maintain uniform aircraft speed. The controller uses information on the specific aircraft configuration, inlet pressure, temperature, Mach number, and engine bleeds, to calculate the
desired thrust and a reference fan speed or pressure ratio. In
order to compensate for engine-to-engine quality variation
and deterioration, the pilot then moves the throttle to match
the reference fan speed or pressure ratio. Aircraft speed is
maintained, during variations in inlet pressure and temperature, by closed-loop adjustment of these reference values.
Multivariable Control
As indicated previously, commercial engines are generally
controlled by a single, closed-loop main fuel control. The additional actuators have minimal interactions between each of
the open-loop controls and with the main fuel control. Consequently, single-input, single-output design techniques are adequate.
P3
Compensator
Engine
Min
select
2nd order
rate filter
Compressor speed
Figure 12. N-dot controller.
24
This is, however, not the case for military engines, which
generally have an afterburner augmentor, to increase engine
thrust for selected segments of the flight, such as takeoff,
climb, acceleration to supersonic speed, or combat. The augmentor fuel flow will determine augmentor temperature, T6,
which affects engine pressure ratio and turbine temperature.
However, the primary combustor fuel control also controls
these variables. Thus the augmentor fuel control loop and the
combustor fuel control strongly interact. Early design approaches used the concept of spectral separation, which made
the combustor fuel control loop an order of magnitude faster
than the augmentor control loop. More recent designs have
used a multivariable approach, to achieve two control loops of
approximately the same response time.
Another application of multivariable control is on a variable cycle engine (VCE), which has a variable area bypass
injector, or VABI. The VABI allows the bypass ratio to be
changed during flight. When closed, the engine behaves more
like a turbojet, providing more thrust during supersonic
flight. Opening the VABI makes the engine more like a turbofan, improving specific fuel consumption during cruise.
Much of the published literature on multivariable engine
control is focused on the regulator problem of maintaining the
engine near the desired operating trajectory. It is based on
linear models valid for small signal analysis, and avoids the
important problem of limit protection. The early 19741982
applications are summarized by Zeller, Lehtinen, and Merrill
(11). The earliest work was that of McMorran and MacFarlane (12,13), which was tested on a two-spool afterburning
engine. They used fuel flow to control compressor speed and
A8 to control fan speed due to the engine bypass duct.
The International Forum on Alternatives for Linear Multivariable Control (14), held in 1977, is of particular interest,
because it showed the use of several multivariable techniques
using a linear model of the F100 engine as a theme problem.
Four authors developed control strategies based on three different multivariable techniques: multivariate transfer functions (15,16), Inverse Nyquist Array (17), and Characteristic
Root Locus (18). Each strategy used three or four of the measured variables, fan speed, N1, compressor speed, N2, compressor exit speed, P3, exhaust pressure, P7, and turbine inlet
temperature, FTIT and controlled fuel flow, Wf , exhaust nozzle areas, A8, and compressor guide vanes, CIVV, or fan guide
vanes, RCVV. In all cases, decoupled control of each of the
measured variables was achieved.
One of the most complete early multivariable designs extensively tested was that by De Hoff and Hall (1922) for
the F100 engine, using extended linear quadratic regulator
techniques. This work went beyond the previous studies, with
ability to handle large power excursions without exceeding
engine or actuator limits and to operate over the entire engine
operating envelope. A block diagram of the control system is
shown in Fig. 13. The feedback law itself is an optimal regulator structure, with integral trims for steady-state accuracy
and engine limit tracking. Linear controllers were designed
at five operating points: sea level static, subsonic and supersonic. Dominant gain elements were determined by assessing
the closed-loop eigenvalue sensitivity to each gain element.
Over 50 percent of the controller gains were eliminated in
the final implementation, with little or no effect on system
performance. Important gain elements were fitted with univariate functions of fan inlet pressure and temperature and
core speed. Unique to this control design is the transition trajectory generator, whose purpose is to smooth rapid throttle
changes by providing a piecewise linear transition from the
current engine state to the requested state. The rate of transition depended on whether the engine is at low, medium, or
high power.
Takeoff thrust for the F100 is defined as the thrust obtained at maximum allowable turbine temperature. At altitude conditions, the minimum burner pressure defines engine
idle. Thus various physical limits must be held exactly at various flight conditions, as shown in Fig. 10. In theory, an optimal controller could have been designed for each limit at appropriate operating conditions. This would have required an
exponentially large number of gain matrices to cover all combinations. De Hoff and Hall (19) used an ad hoc approach to
solve this problem, by designing single-loop spectrally separated integral trims for each input, corresponding to each desired set point and number of unsaturated controls. The control was switched whenever an actuator saturated or an
engine limit was reached.
A very similar multivariable design was developed for a
GE23 variable cycle engine (23,24). A much larger number of
linear design points were needed, due to the greater changes
in engine configuration. Instead of using integral trim to handle engine limits, a model of the engine was incorporated into
the transition logic. This allowed the generated trajectories
always to satisfy engine limits. The problem with this approach is that it is an open-loop feedforward approach. How
well the engine limits are held depends on the accuracy of the
model and how well it can handle engine-to-engine variations.
One possible solution to handling engine limits is demonstrated by Adibhatla (25,26) for a short takeoff and vertical
landing airplane (STOVL) using an F110 engine. This is a
full-range multivariable design, which was pilot evaluated in
the NASA Ames fixed-base and vertical-motion simulators.
The primary objective of this engine control is to manage
thrust, through the aft nozzle during cruise, and through ejectors at the aircraft wing roots and ventral ports on the underside of the aircraft during transition and hover. An estimate
of thrust is determined based on fan speed, fan operating line,
fuel flow, and the three thrust port areas. During cruise a
2 2 controller regulates fan speed and aft thrust, using fuel
flow and aft nozzle area. During transition and hover, the
controller is expanded to a 4 4 controller, in order to additionally regulate the ejector and ventral thrust using the ejector and ventral area actuators. To prevent engine stall and
overtemperature, and to ensure sufficient pressure for customer bleed, three single-input, single-output regulators were
designed as limit regulators.
Hanus technique (27) for integrator antiwindup is used to
transition between these five regulators and to handle actuator limits. In this technique, an observer-based structure is
used, such that the normal controller is modified to assume
the form:
xc = Ac xc + Bc (ys p y) + L(sat(u) u)
u = Cc xc + Dc (ys p y)
(15)
where Ac Bc Cc Dc the state space description of the five controller dynamics with the desired set point ysp and the engine
output y, sat(u) is the actual bounded engine input and L is
the anti-windup gain. When a given regulator is controlling
25
Nominal input
Reference
point
schedules
Transition
control
+
LQR
State
Engine limit
protection
Actuators
Engine
CP
Integral
control
CI
Gain
schedules
Limit flags
CP
Sensors
and FITT
estimator
Figure 13. F100 multivariable controller (19).
the engine actuators, its elements of sat(u) u are zero. The

gain L forces the remaining sat(u) u elements to zero,
thereby tracking the ones currently applied to the engine.
This ensures a smooth transition from one regulator to another, and automatically handles the problem of engine
limits.
Recently, Kapoor and Teel (28) have extended Hanus technique by replacing L(sat(u) u) with an equivalent dynamic
system. This allows more design freedom and guarantees stability. An example is shown where their dynamic scheme
gives response very close to the unconstrained case, while using a static L is unstable.
An excellent summary of multivariable control development in the United Kingdom from 1985 to 1995 is given by
Dadd, Sutton, and Greig (29). A variety of control laws have
been tested on either the RB199 or Sprey Mk202 engines.
Each was distinguished primarily by the combination of sensor used, since there was very limited flexibility in terms of
available actuators. Again the Hanus technique (27) was
used to handle limit protection and controller selection. Of
specific interest is a fast response test on the Sprey Mk202
engine, using a 3 3 controller for most of the operating regime, and a 2 2 for low power settings where the high pressure inlet vanes are fully closed. The input output parameters
chosen for the 3 3 controller were:
Inputs
Outputs
Fuel flow
Inlet guide vanes
A8 nozzle areas
Fan speed
Compressor speed
Bypass duct Mach number
The 2 2 controller used fuel flow and nozzle area to control fan speed and bypass duct Mach number. Both the 2 2
and the 3 3 controllers were run simultaneously, with authority switched between them as the guide vanes either approached or moved off their limit.
During sea level static tests, it was found that this fast
response multivariable controller produced a 60 percent re-
duction in time to go from 50 percent to 100 percent thrust. It

was found that the specific fuel consumption was not affected
significantly, while the thermal cycle range of the turbines
was reduced. A second set of tests was performed, with increased integral gains to shift the closed-loop poles. This improved the robustness and disturbance rejection of the
control.
CONCLUDING REMARKS
This article has focused on the basic theory of engine operation and on control designs currently in commercial and military use. Single-input, single-output controls are used on commercial engines where the emphasis in on economical
operation. Multivariable controls provide enhanced performance for military engines. In both cases, the management of
limits is an essential part of the control.
Due to limited space, it has not been possible to discuss
the many ongoing advanced research areas in engine controls.
Of immediate interest to the authors is the work (30,31) using
a real-time model of the engine as an observer to provide estimates of unmeasurable variables, such as thrust and stall
margins. Of much longer-term potential is the work on active
control of stall (3234) and combustion (3537).
BIBLIOGRAPHY
1. G. G. Smith, Gas Turbines and Jet Propulsion, New York: Philosophical Library, 1955.
2. J. V. Casamassa and R. D. Bent, Jet Aircraft Power Systems, 3rd
ed., New York: McGraw-Hill, 1965.
3. E. W. Constant II, The Origins of the Turbojet Revolution, Baltimore, MD: John Hopkins University Press, 1980.
4. I. E. Treager, Aircraft Gas Turbine Engine Technology, 2nd ed.,
5. G. C. Oates, Aerothermodynamics of Gas Turbine and Rocket Propulsion, Revised and Enlarged, Washington, DC: American Institute of Aeronautics and Astronautics, 1988.
26
JET TRANSPORT MAINTENANCE
6. H. Cohen, G. F. C. Rogers, and H. I. H. Saravanamuttoo, Gas

Turbine Theory, 4th ed., Essex U.K.: Longman, 1996.
7. Aerospace Recommended Practise, ARP 755A, Society of Automotive Engineers, 1974.
8. K. Mordoff, General Electric flies first unducted fan demonstrator, Aviation Week & Space Technology, 125: 31, Aug. 25, 1986.
9. D. A. Parsons, N-dot schedules dynamic compensation system for
gas turbinesInputs sum of speed and rate of change of speed
of gas generator to schedule to output desired acceleration as
function of generator speed, U.S. Patent 5029441, 1991.
10. J. Zeller, B. Lehtinen, and W. Merrill, The Role of Modern Control Theory in the Design of Controls for Aircraft Turbine Engines, NASA Tech., Memo. 82815, Lewis Research Center, Cleveland, OH, 1982. (Also in AIAA Twentieth Aerospace Sciences
Conf., Orlando, FL, Jan. 1114, 1982.)
11. D. R. Gilmore, Jr., Two decades of T700 control system growth,
Proc. of the 4th CASI Symposium on Propulsion, May 5, 1992,
pp. 1525.
12. P. D. McMorran, Design of gas-turbine controller using inverse
Nyquist method, Proc. of IEE , 117, 1970.
13. A. G. J. MacFarland et al., Application of multivariable control
techniques to aircraft gas turbines, Conf. on Multivariable Control
Systems Design and Application, Sept. 13, 1971.
14. M. K. Sain, J. L. Peczkowski, and J. L. Melsa, eds. Alternatives
for Linear Multivariable Control with Turbofan Engine Theme
Problem, Nat. Eng. Consortium, Chicago, 1978.
15. L. G. Hofmann, G. L. Teper, and R. F. Whitbeck, Application of
Frequency Domain Multivariable Control Synthesis Techniques
to an Illustrative Problem in Jet Engine Control, 5170, in Reference 14.
16. J. L. Peczkowski and M. K. Sain, Linear multivariable synthesis
with transfer functions, pp. 7187, in Reference 14.
17. H. A. Spang, III, Insight into the application of the inverse Nyquist array method, pp. 138155, in Reference 14.
18. B. Kouvaritakis and J. M. Edmunds, The characteristic frequency
and characteristic gain design method for multivariable feedback
systems, pp. 229246, in Reference 14.
19. R. L. DeHoff et al., F100 Multivariable Control Synthesis Program, Vols. I and II, AFAPL-TR-77-35, June 1977.
20. C. J. Skira, R. L. DeHoff, and W. E. Hall, Jr., Design, Evaluation
and Test of an Electronic, Multivariable Control for the F100
Turbofan Engine, AGARD-PP-274, Advanced Control System for
Aircraft Power, Cologne, Germany, Oct. 1979.
21. R. L. DeHoff and W. E. Hall, Jr., Multivariable quadratic synthesis of an advanced turbofan engine controller, J. of Guidance and
Control, I: 136142, 1978.
22. R. L. DeHoff and W. E. Hall, Jr., Optimal control of turbine engines, J. of Dynamic Systems, Measurements and Controls, 101:
117126, 1979.
23. S. M. Rock and R. L. DeHoff, Variable Cycle Engine Multivariable Control Synthesis, AFAPL-TR-79-2043, 1979.
24. R. L. DeHoff and W. E. Hall, Jr., Multivariable control design
principles applied to a variable cycle turbofan engine, Proc. of the
Asilomer Conf. on Circuits, Systems, and Computers, Pacific Grove,
CA, Nov. 1978.
25. S. Adibhatla, Propulsion Control Law Design for the NASA
STOVL Controls Technology Program, AIAA, 93-4842, AIAA Int.
Powered Lift Conf., Santa Clara, CA, 1993.
26. S. Adibhatla et al., STOVL Controls Technology, Vol. I, Integrated Flight/Propulson Control Design, NASA Contractor Report 195361, 1994.
27. R. Hanus, M. Kinnaert, and J. L. Henrotte, Conditioning technique, A general anti-windup and bumpless transfer method, Automatica, 23: Nov. 1987.
28. N. Kapoor and A. Teel, A Dynamic Windup Compensation

Scheme Applied to a Turbofan Engine, Proc. 36th IEEE Conf. on
Decision and Control, San Diego, CA, 46894694, 1997.
29. G. J. Dadd, A. E. Sutton, and A. W. M. Greig, Multivariable control of military engines, Proc. Advanced Aero-Engine Concepts and
Controls, Defense Research Agency, Farnborough, England, June
1996. (AGARD Presentation, Seattle WA, Sept. 1995.)
30. W. J. Dwyer, Adaptive Model-Based Control Applied to a Turbofan Aircraft Engine, M.S. Thesis, Department of Mechanical
Eng., MIT, Cambridge, MA, June 1990.
31. S. Adibhatla and T. Lewis, Model-Based Intelligent Digital Engine Control, AIAA 97-3192, 33rd AIAA Joint Propulsion Conf. &
Exhibit, Seattle, WA, July, 1997.
32. A. H. Epstein, J. E. Ffowcs-Williams, and E. M. Greitzer, Active
suppression of aerodynamic instabilities in turbomachines, J.
Propulsion and Power, 5 (2): 204211, 1989.
33. I. J. Day, Active Suppression of Rotating Stall and Surge in Axial
Compressors, International Gas Turbine and Aeroengine Congress and Exposition, ASME Paper 91-GT-87, June, 1991.
34. K. M. Eveker et al., Integrated control of rotating stall and surge
in aeroengines, Proc. Sensing, Actuation, and Control in Aeropropulsion Conf., Orlando, FL, Apr. 1995. (SPIE Proceedings, Vol.
2494, 1995, pp. 2135.)
35. E. J. Gutmark et al., Active control in combustion systems with
vortices, Proc. 4th IEEE Conf. On Control Applications, Albany,
NY, Sept. 1995, pp. 679684.
36. D. St. John and S. Samuelson, Optimal, active control of oxides
of nitrogen (Nox) emissions from a natural gas-fired burner using
a simple genetic algorithm., Proc. 4th IEEE Conf. on Control Applications, Albany, NY, Sept. 1995, pp. 673678.
37. E. J. Gutmark et al., Compact waste incinerator based on vortex
combustion, Combustion Science and Tech., 121: 333349, 1996.
H. AUSTIN SPANG, III

GE Research and Development
Center
HAROLD BROWN
Gray Fox Technologies

Copyright
INPUT-OUTPUT STABILITY
The earliest mathematical studies of control systems focused solely on the input-output stability of systems
as described in the works of Black (1), Bode (2), and Nyquist (3). In fact, most of the classical control work
was limited to the input-output study of single-inputsingle-output linear and mostly time-invariant systems.
Notions of input-output stability for nonlinear systems were later advanced by Sandberg (4), Zames, (5,6)
Safonov (7), and others. The first book dedicated completely to the subject was by Willems (8) in 1971, followed
shortly by that of Desoer and Vidyasagar in 1975 (9). With the popularity of state-space methods, Lyapunovstability concepts became the preferred analysis and design tools for nonlinear systems until the 1980s when
researchers became interested again in the input-output behavior of systems. The relationships between the
input-output and Lyapunov-stability concepts were developed in Refs. 10 and 11 and culminated in the various
versions of the Kalman-Yakubovich-Popov (KYP) lemma (12). The current studies in input-output systems are
highly dynamic with the introduction of new concepts such as input-to-state stability (13,14,15), the interaction
with geometric nonlinear control (16), applications to robust control (16,17,18), research in the adaptive control
for linear (19) and nonlinear systems (17), the interest in various mechanical and electric systems (20), and
the publication of various theoretical and applied books (18,20,21). It is now clear that the two points of view
(state-space and input-output) are complementary and that many deep relationships between the various
stability concepts are yet to be explored.
In this article we concentrate our discussion on continuous-time systems and survey the classical as well
as the more recent results in the input-output approach. The major results included in this article are taken
mainly from Refs. 9,18,22,23,24, and the reader is referred to those books for most proofs. An excellent 1995
chapter on input-output stability from a distance-separation point of view appeared in Ref. 25. Related concepts
for discrete-time systems were presented in Ref. 9 and recently revived by Byrnes and co-workers (26,27), but
will not be discussed here. In addition, while we mention absolute stability as an important application area of
input-output concepts, we refer the reader to the article on Absolute Stability in this encyclopedia for details.
The article starts with a collection of basic definitions followed by the general results on the basic concepts
of input-output stability and results for testing input-output stability and its relationship with Lyapunov stability. Next, we discuss the stability of interconnected systems and present the small-gain and passivity results.
Related concepts such as absolute stability, dissipativity, and input-to-state and input-to-output stability are
then reviewed, followed by our conclusions. Various technical definitions are presented in the appendices. We
have attempted to include the main references on the subject of input-output stability, striving to be current
and relevant rather than encyclopedic.
Basic Concepts
The general ideas of input-output stability involve the relative size of signals as they are processed by
dynamical systems. We will thus begin by providing mathematical measures of the size of signals and of the
effect that a particular system has on that size.
1
In order to introduce the mathematical notions of input-output stability, we need some preliminary
definitions of signal spaces. (A detailed treatment of measure, signal spaces, and signal norms is beyond the
scope of this article. The reader is referred, for example, to Ref. 28.) The Lm p set, with p [1, +), consists of
all functions u: [0, +) m that are Lebesgue measurable [i.e., functions that are the limits (except for a set
of measure zero) of a sequence of piecewise constant functions], such that
where vq denotes the q-norm of the vector v = (v1 v2 vm )T , defined as
Remark 1. For the finite-dimensional set m all q-norms are equivalent, in the sense that for all k, h [1,
+] there exist positive constants c1 and c2 such that
for all x
[0, ).
. This is why, when defining the Lp m set, we do not need to specify q, as u(t)
for any fixed t
The Lm set consists of all the functions u: [0, +) m that are measurable and essentially bounded
on [0, +) (where essentially bounded means that the function is bounded except on a set of measure zero).
For a function belonging to Lm p , p [1, +), we introduce the norm
whereas for a function belonging to Lm we introduce the norm
Note that at the left-hand side of definitions (2) and (3), we should specify the dependence of the norm on q,
but we omit it to avoid cumbersome notation; the choice of q will be clear from the context.
For the Lm 2 set, the norm is usually defined by setting q = 2 in Eq. (2), obtaining
The Lm 2 set with the norm (4) is of particular interest, since in some contexts the norm (4) is proportional
to the energy of the signal u, as is the case, for instance, for which u(t) is the voltage across a resistance R.
Then the total energy delivered to the resistance would be found by integrating the instantaneous power
u(t)2 /R. As usual, in the following we shall denote the L1 p set, p [1, +), by Lp .
Remark 2. In contrast with finite-dimensional spaces (see Remark 1), for the infinite-dimensional set Lm (the
set of all measurable functions u: [0, ) m ) the p-norms may be nonequivalent. For instance,
u(t) = 1/(1+t) with 0 < < 1 belongs to L and to Lp with p < 1/, but does not belong to Lp for p 1/.
u(t) = sin(t) belongs to L but does not belong to Lp for any p [1, +).
In this article, we are interested in studying the stability properties of a generic operator G that maps a
signal space into another signal space ,
We shall define the operator G in Eq. (5) as being Lp -stable if for any u() Lm p the output y() belongs to
L p . To give a formal definition of Lp stability and to provide a broad definition of unstable systems, we first
need to define precisely the domain and the image of the operator G, which in general are not limited to Lm p
and Lr p , respectively (for example, we may be interested in considering unbounded inputs and/or outputs). For
this reason, we need to introduce some extended Ln p spaces. First, let us define the truncation uT (t) of a signal
u(t) in the following way:
r
The extended set Lm pe , with p [1, +], consists of all the functions u: [0, +)
such that
Introducing this extended space, we can treat unbounded signals. For instance, u(t) = t does not belong to Lp
for any p [1, +] but its truncation
belongs to Lp , with p [1, +], for every finite T. Therefore u(t) = t belongs to Lpe for every p
[1, + ].
Finally we end the section with the definition of causality of an operator G. When we later specify an
operator in terms of state-space differential equations, causality is an intrinsic property of the operator: the
output y at time t depends on the initial conditions and on the values of the input u up to time t. On the other
hand, when dealing with a generic operator as in Eq. (5), we need to enforce causality. An operator G: Lm pe
Lr pe is said to be causal if the value of the output at time t depends only on the values of the input up to time t
as defined next.
Definition 1 (causality). An operator G: Lm pe Lr pe is said to be causal if (G(u))T = (G(uT ))T T 0, u
Lm pe .
We are now ready to present the first input-output stability concept.
Lp Stability
Now we give the definition of Lp -stable systems that transform Lp input signals into Lp output signals.
Definition 2. An operator G: Lm pe Lr pe is
Lp -stable if
Finite-gain Lp -stable if there exist finite constants p and p such that
Finite-gain Lp -stable with zero bias if there exists a finite constant p such that
For finite-gain Lp -stable systems, the smallest scalar p for which there is a p such that relation (7) is
satisfied (when such p exist) is called the gain of the system. Similarly, for finite-gain Lp -stable systems with
zero bias, the smallest scalar p such that relation (8) is satisfied (when such p exist), is called the gain with
zero bias of the system.
Regarding finite-gain Lp stability for a causal operator G, we have the following result.
Lemma 1. (Ref. 29)
Let G: Lm pe Lr pe be a causal finite-gain Lp -stable operator with constants p and p . Then
Proof. Since G is causal (G(u))T = (G(uT ))T . Moreover if u Lm pe , then uT Lm p for all T 0; hence G(uT )
Lm p . Finally, for a generic function x, xT Lp xLp . Therefore the following inequalities hold:
For a causal, finite-gain Lp -stable operator with zero bias, it can be shown in the same way that
Remark 3. Recalling the definition of the Lm set with the norm (3), L stability is in fact what is normally
termed bounded-input, bounded-output (BIBO) or external stability. It is usually defined in reference to a system
specified through a state-space representation, and guarantees that if the input to the system is essentially
bounded, then its output is also essentially bounded.
Example: This example illustrates an L -stable and a finite-gain L -stable operator.
Let us consider the function g: defined by g(u) = uk with k (1, +). Correspondingly, define the
operator G: Le Le that assigns to every input u(t), t 0 the output g(u(t)). We will show that G is L
stable. Indeed set uL = c < +; then we have
Therefore G is a BIBO operator. It is not a finite-gain L -stable operator; however, since we cannot find
fixed scalars and such that ck c + holds for all c [0, +).
By similar arguments as above, it is easy to check that G associated with the function g(u) = uk with k
(0, 1) is a finite-gain L -stable operator.
Example: There are some special cases when the gain of an operator can be numerically or explicitly found;
one of these cases is the L2 gain for linear systems. Let us consider a linear time-varying (LTV) system in the
form (for the sake of simplicity, we assume no feedthrough term)
where A(t), B(t), and C(t) are piecewise continuous and bounded.
= A(t)x is exponentially stable (see Appendix 1 for the definition of
We assume that the unforced system
exponential stability and Appendix 2 for a necessary and sufficient condition for exponential stability of LTV
systems). Let G be the input-output operator mapping u to y; then the L2 gain of G (which is also called energy
gain and induced-operator norm of G) is defined by
For a given > 0, we find (see Ref. 30) that 2 if and only if there exists an > 0 such that the Riccati
differential equation
admits a positive definite solution P(t) for which a symmetric matrix-valued function P(t) is said to be positive
definite if there exists a positive such that xT P(t)x x2 for all x n and t 0. Therefore by conducting a
binary search over , the L2 gain of G can be computed up to the desired precision.
The time-invariant case is even simpler, since we only have to deal with the algebraic version of Eq. (11). In
denote, as usual, the transfer matrix

this case there is actually another way of computing the L2 norm; let G(s)
1
in fact
G(s) = C(sI A) B. It is possible to show that the L2 gain (10) is related to the transfer matrix G(s);
it is given by
where max (A) :=

is the maximum singular value of the matrix A, and A denotes the conjugate
.
transpose of a matrix A. The norm (12) is known in literature as the H norm of G (j) and is denoted by G
So far, we have considered operators whose domain is the whole space Lm p . The following example
motivates a local version of Definition 1, concerning input signals that lie in a subset of Lm p .
Example: Let us consider the function g: (1, 1) defined by g(u) = 1/(1u2 ). As in Example 1 define the
associated operator G: Le Le that assigns to every input u(t), t 0, the output g(u(t)). Since the function
is defined only when the input signal is such that
the G is not an L -stable operator according to Definition 1. However, let |u| c < 1; then
and so
which implies that G is an L -stable operator in a new sense made clear in the next definition.
The following definition is an extension of one originally presented in Ref. 31.
Definition 3. An operator G: Lm pe Lr pe is small-signal Lp -stable if there exists a positive constant r such
that Eq. (6) is satisfied for all u Lm p with supt u(t) < r. Similarly, G is a small-signal finite-gain Lp -stable
[operator small-signal finite-gain Lp -stable operator with zero bias] if there exists a positive constant r such that
inequality (7) [inequality (8)] is satisfied for all u Lm p with supt u(t) < r.
Remark 4. In Definition 3, we do not need to specify a particular norm to evaluate supt u(t), since we are
dealing with a norm in m and norms are equivalent in finite-dimensional spaces (see Remark 1). From Eq. (3)
it is clear that if u(t) is uniformly bounded then so is the signal norm u()L . This is not true in general as
it is easy to construct examples in which the signal norm u()Lp , with p [1, +), is arbitrarily large even if
u(t) is uniformly bounded.
Sufcient conditions for Lp stability. In the previous section we have provided the definition of Lp
stability for a given input-output operator. An important question then arises: How do we check whether an
input-output operator is Lp -stable or not?
To answer this question we should not focus, as done so far, on generic input-output operators; in this
section we assume that the operators under consideration are specified in terms of a state-space representation
of a dynamical system in the form
where x
,u
,y
, and
with D := {x n : x < r}, Du := {u m : u < ru }, where r and ru are positive numbers.
It is important to note that Eqs. (13a) define an input-output operator for any given initial state x0 . In other
words, system (13) defines an entire class of input-output operators, each obtained in relation to a particular x0 .
We should always distinguish between the concept of a system and that of an operator. In many textbooks, when
looking at the input-output behavior of a system, it is assumed that the initial condition is zero, and therefore
the system and the corresponding operator are the same object. As opposed to our treatment, the discussion of
Ref. 25 maintains the distinction between the input-output concepts and the state-space description.
In the remainder of this article, we assume that a state-space description of the dynamical system is given
unless otherwise specified. This will allow us to provide in this section a sufficient condition for Lp stability
and, at the same time, to establish the first connection between Lyapunov (internal) stability and Lp stability.
Assume that x = 0 is an equilibrium point for system (13a) with u = 0, that is,
We shall see that, if x = 0 is an exponentially stable equilibrium point (see Appendix A for the definition)
and some other additional technical assumptions hold, the corresponding input-output operator is Lp -stable
for any x0 as described in the following theorem.
Theorem 1. (Corollary 6.1 of Ref. 23)
Assume that
(1) x = 0 is an exponentially stable equilibrium point for system (13a) under the input u = 0;
(2) f is continuously differentiable and the Jacobian matrices f /x and f /u are bounded, uniformly in t
(3) h is of Lipschitz form with respect to x and u, that is, there exist positive constants 1 , 2 such that
for all (t, x, u) [0, +) D Du .

Then there exists a constant r0 > 0 such that for each x0 satisfying x0 < r0 the operator defined by system
(13) with initial condition x0 , is a small-signal finite-gain Lp -stable operator for each p [1, +].
If all the assumptions hold globally, with D = n and Du = m , then for each x0 n the operator defined
by system (13) with initial condition x0 is a finite-gain Lp -stable operator for each p [1, +].
Note that a linear system
always satisfies assumption ii and iii of Theorem 1 globally if A() and B() are continuously differentiable
(actually this hypothesis can be relaxed to piecewise continuity) and uniformly bounded and C() and D() are
uniformly bounded; moreover, the exponential stability of x = 0 of system (14a) with u = 0 is always global.
Therefore we can state the following corollary of Theorem 1.
Corollary 1. Consider the linear system
where A() and B() are continuously differentiable and uniformly bounded and C() and D() are uniformly
bounded. Assume that the equilibrium point x = 0 under u = 0 of Eq. (15a) is exponentially stable; then for each
x0 n the operator defined by system (15) with initial condition x0 is finite gain Lp -stable for each p [1, +].
Recall that the L2 gain of the operator associated with the linear system (14) for x0 = 0 can be computed
according to the procedure detailed in Remark 4. Finally, a sufficient condition for exponential stability is given
in Appendix 2.
Relations between Lyapunov stability and Lp stability. So far in this section we have shown the
following (Theorem 1):
This represents the first connection between Lyapunov and Lp stabilities.

The remainder of this section is devoted to find the reverse connection between Lyapunov and Lp stabilities. It is, however, difficult to find a general result in the spirit of Theorem 1. Following the guidelines of
Ref. 29 we shall restrict ourselves to time-invariant systems and focus on attractivity rather than exponential
stability. Roughly speaking, the next theorem will show the following result:
Theorem 2. Assume that

(1) system (13) is time invariant, reachable, and uniformly observable and
(2) the input-output operator defined by system (13) with initial condition x(0) = 0 is a small-signal L2 -stable
operator.
Then x = 0 is an attractive equilibrium point for system (13).
Moreover, if system (13) is globally reachable and the input-output operator is L2 -stable, x = 0 is a globally
attractive equilibrium point.
For the definitions of reachability, uniform observability, and attractivity see Appendix 1. A LTI system
in the form
is globally reachable if and only if it is reachable and is uniformly observable if and only if it is observable.
Moreover small-signal L2 stability implies L2 stability, and attractivity implies exponential stability. Therefore
we can derive the following corollary of Theorem 2.
Corollary 2. Assume that
(1) system (16) is reachable and observable and
(2) The input-output operator defined by system (16) with initial condition x(0) = 0 is L2 -stable.
Then x = 0 is an exponentially stable equilibrium point for system (16).
Interconnected Systems
One of the main applications of the formalism of input-output stability is the study of the stability of interconnected systems, without explicit knowledge of the internal dynamics of the composite subsystems. Let us
consider the feedback interconnection of Fig. 1, where G1 : Lm pe Lr pe and G2 : Lr pe Lm pe . Input-output
stability allows us to investigate how the signals propagate through this scheme. Before presenting the main
Fig. 1. Basic feedback system.
results, we need to introduce the concept of well-posedness of the feedback interconnection. Well-posedness
guarantees that for each choice of u1 Lm pe and u2 Lr pe there exist unique solutions e1 ,y2 Lm pe and e2 ,y1
Lr pe that satisfy the loop equations
Small-Gain Theorems. Theorem 3. Consider the feedback system of Fig. 1. Suppose that G1 : Lm pe
L pe and G2 : Lr pe Lm pe are causal finite-gain Lp -stable operators with constants p1 ,p1 and p2 ,p2 , respectively. Moreover, suppose that the feedback interconnection is well-posed. Then the feedback system of
Fig. 1 is a finite-gain Lp -stable system if
r
and
Proof. Consider inputs u1 Lm p and u2 Lr p . Since the closed-loop system is well-posed, there exist unique
solutions e1 ,e2 ,y1 ,y2 . With respect to e1 and e2 we have
10
Since G1 and G2 are causal finite-gain Lp -stable operators, we find that (see Lemma 1)
After some trivial manipulations, recalling that p1 p2 < 1 by assumption, Eqs. (18a) become
Since for a generic function x, xT Lp xLp , we have
Now, the right-hand sides of inequalities (20) are independent of T. Therefore it can be easily shown that e1
and e2 belong to Lm p and Lr p , respectively, and that
In a similar way it can be shown that y1 and y2 belong to Lr p and to Lm p , respectively, and that inequalities
(17c) and (17d) hold.
The work of Safonov (7) exploited the general input-output concepts in order to study the robustness
of closed-loop systems. His results and variations were later exploited in H robust control analysis and
design. The small-gain theorem is thus extremely useful in studying the robustness of a closed-loop system,
when a nominal system is subject to a perturbation as shown in Fig. 2. The next example shows one of these
applications.
11
Fig. 2. Feedback robustness loop.
Example: Let us consider the feedback scheme of Fig. 2 where G(s) and (s) are asymptotically stable
transfer matrices. The transfer matrix G(s) represents the nominal system, whereas (s) is a model of the
uncertainty. Let u1 and u2 belong to L2 . In Example 2 we have seen how to compute the L2 gains of G(s) and
(s); let
The small-gain theorem tells us that if
then the closed-loop system is a finite-gain L2 -stable system. In other words, it gives us an estimate of how
large the perturbation (s) can be, in terms of its H norm, preserving the closed-loop L2 stability.
There are various versions of the small-gain theorem, a sample of which is the incremental small-gain
theorem below, which needs a preliminary definition.
Definition 4. An operator G: Lm pe Lr pe is said to be an incrementally finite-gain stable operator if
(1) G(u) Lr p when u 0 and
(2) there exists a constant such that
for all T > 0 and for all u, v Lm pe .

Theorem 4. Consider the interconnected system of Fig. 1. Let both G1 and G2 be incrementally finite-gain stable
operators with respective gains 1 and 2 . Then, the feedback interconnection is well-posed and incrementally
finite-gain stable from u = [u1 u2 ] to y = [y1 y2 ] if
Passivity Theorems. One of the main related concepts to the input-output stability concepts discussed
so far is the concept of passive systems. In a way, while Lp stability deals with the effect the system has on the
size of signals, passivity results deal with the effect the system has on the energy of signals. We start with
few definitions and follow with the main results for interconnected systems.
12
Definition 5. We say that a system G: Lm 2e Lr 2e is

(1) Passive if there exists
such that for all T and all u Lm 2e
(2) Strictly passive if there exists > 0 and
such that for all T and all u Lm 2e
Theorem 5. Consider the interconnected system of Fig. 1. Assume that the systems G1 and G2 satisfy
The closed-loop system is L2 finite-gain stable if
Note that this theorem does not require both systems G1 and G2 to be passive, as long as one of the two
systems is passive enough. If on the other hand, one of the two systems is passive and the other is strictly
passive, the previous theorem simplifies to the following.
Theorem 6. Consider the interconnected system of Fig. 1, and let u2 = 0. Then, the closed-loop system is
finite-gain L2 stable if one of the following conditions holds:
G1 is passive and G2 is strictly passive

G1 is strictly passive and G2 is passive.
In the special case of affine in the control systems, the passivity of one system is equivalent to the
L2 -stability of a related system. This is detailed in the following theorem:
Theorem 7. (Proposition 3.2.12 of Ref. 18)
Let
and
where u = 12 (v z) and y = 12 (v + z). Then, system (22) is passive system (23) has L2 gain 1.
13
Fig. 3. The Lure problem.
Related Stability Concepts And Applications

In this section, we review various concepts that are related to input-output stability and discuss some of their
applications.
Dissipativity. In 1972, Willems (10,11) introduced the notion of dissipativity in an attempt to further
unify input-output and Lyapunov-stability concepts. The notion of dissipativity is a generalization of passivity
and captures the concept that a system will dissipate energy if the sum of the energy it stores and the energy
it dissipates to the environment is less than the total energy that the environment supplies to it. This is a
manifestation of the second law of thermodynamics and is the case of most physical systems that transform
some form of energy to another, but also lose some in the process. In the following, + = [0, ).
Definition 6. The system (13) is dissipative with respect to the supply rate w(u, y):
exists a storage function V: n + such that
for all u, all T 0, and all x(0)
r if and only if there
Note that passivity can actually be defined as a special case of dissipativity by letting w(u, y) = uT y
(therefore, the system is square and m = r). We can also define other types of passivity as follows: the system is
an input-strictly-passive (ISP) system if it is dissipative with supply rate w(u, y) = uT y u u2 , u > 0, and it
is an output-strictly-passive (OSP) system if it is dissipative with supply rate w(u, y) = uT y y y2 , y > 0. The
system is a state-strictly-passive (SSP) system if w(u, y) = uT y x (x), x > 0 and (x) is a positive-definite
function of x. Note that an OSP system is necessarily L2 stable (18,23). In addition, one can guarantee the L2
stability of a system by making sure it is dissipative with the particular supply rate w(u, y) = 12 2 u2 y2
for some positive , which then becomes an upper bound on the L2 gain of the system (18,23).
The Linear Case and the KYP Lemma. One of the more important applications of the input-output
approach is in the solution of the so-called Lure problem shown in Fig. 3. The details of this approach are
detailed in another chapter of this encyclopedia and only a few comments are included here for completeness.
The basic question asked by Lure is to find conditions in the single-inputsingle-output case on the linear
system G(s) such that when the nonlinear block (t, y) is static (i.e., a non-time-varying function of y only),
the closed-loop system is stable (32). Popov provided graphical, frequency-domain criterion for the absolute
stability problem when the nonlinear block (y) is time invariant (33,34). Yakubovich (35) and Kalman (36)
introduced different versions of the so-called positive-real or Kalman-Yakubovich-Popov (KYP) lemma to relate
Popovs criterion to the existence of a special Lyapunov function. This then provides another connection between
input-output stability concepts and Lyapunov concepts. Anderson then (12,22) extended the KYP lemma to the
multi-inputmulti-output case. The KYP lemma has found various applications in adaptive control (19) and
14
has recently been generalized to the case in which the linear block G(s) is replaced by a nonlinear but affine
nonlinear system = f (x) + g(x)u, y = h(x) (16,18).
In the linear case, passivity concepts may be related to the concept of positive-realness, already introduced
in the study of electrical networks (22). In fact, consider a stable, square, LTI system with minimal state-space
realization
where u, y
,x
, and let the transfer function be
Since the state-space realization is minimal, then (A, B) is controllable and (C, A) is observable. Recall that (A,
B) is controllable if and only if rank ( ) = n and (C, A) is observable if and only if rank ( ) = n where = [B
AB An 1 B] and T = [CT AT CT (An 1 )T CT ].
Definition 7. Let H(s) be a proper m m rational transfer matrix. Then
H(s) is positive real (PR) if

(1) No element of H(s) has a pole in Re[s] > 0
(2) Any pole of an element of H(s) on the j axis must be simple and the associated residue matrix is positive
semidefinite Hermitian, and
(3) For all real for which j is not a pole of an element of H(s), Z(j) + ZT (j) is positive semidefinite.
(4) H(s) is said to be strictly positive real (SPR) if H(s ) is PR for some > 0.
For variations on this definition, the reader should consult Ref. 37, where various PR concepts are
discussed.
Lemma 2. Let H(s) be an m m transfer matrix (25) where
(1) A is stable
(2) (A, B) is controllable and (C, A) is observable
Then, the transfer function H(s) is SPR if and only if there exist a positive definite symmetric matrix P,
matrices W and L, and a real > 0 such that
15
Note that if D = 0, that is, H(s) is strictly proper, then Eqs. (26a) simplify to the more familiar
Lemma 3. H(s) is PR if and only if it is dissipative with storage function V(x) = xT Px.
Hill and Moylan (38) and others expanded the dissipativity notions in order to explain the KYP lemmas.
The KYP lemmas have applications to adaptive control for linear systems (19) and a generalization to nonlinear
systems (17). Connections between passivity and stability are provided in the next lemma.
Lemma 4. (Lemma 10.6 in Ref. 23)
Given the autonomous system
then the following holds true.

(1) If Eq. (27) is passive with a positive-definite storage function V(x), then the origin of Eq. (27) with zero input
is stable.
(2) If the system is OSP, then it is a finite-gain L2 -stable system.
(3) If the system is OSP with a positive-definite storage function V(x) and zero-state observable (see Appendix
1), then the origin of Eq. (27) with zero input is asymptotically stable.
(4) If the system is SSP with a positive-definite storage function V(x), then the origin of Eq. (27) with zero input
Passication via Feedback. In recent years, the input-output approach has gained new footing as a
design tool for nonlinear control systems. One of the main applications of such an approach has been to use
feedback in order to render a closed-loop system passive or strictly passive (or the passification of an open-loop
system). The main motivation for such designs is of course that a passive system will tolerate large-magnitude
uncertainties as long as the uncertainties are passive (see Theorem 5). References 16 and 17 contain a large
number of results on the passification of nonlinear systems. Roughly speaking, all designs require that the
open-loop system be of minimum phase and of a relative degree one in order for it to be made passive using
static output feedback. Such concepts have been generalized to a large class of nonlinear systems. As mentioned
previously, and following the early concepts (9), there has been much recent work on the discrete-time versions
of the input-output stability concepts including the passification designs in Refs. 26 and 27.
Input-to-State and Input-to-Output Stability. In a series of papers (13,14,15), Sontag and co-workers
have advanced the notion of input-to-state stability to study the behavior of state-space systems when the
input is bounded. Roughly, the input-to-state stability concepts guarantee that the state x(t) is bounded for
any bounded input u(t), which may be an external disturbance or a tracking signal. This idea is in some
ways a more restrictive version of the input-output concepts unless y = x and is more tightly coupled to the
Lyapunov-stability concepts. In what follows, we deal with system (13), or with its autonomous version:
16
Definition 8. The system (13a) is said to be locally input-to-state stable (ISS) if there exists a class L function
, a class function (see Appendix 1 for the definitions of such functions), and positive constants k1 and k2
such that for any initial state x(t0 ) with x(t0 ) < k1 and any input u(t) with supt t0 u(t) < k2 , the solution
x(t) exists and
The system is said to be ISS stable if the preceding requirement holds globally (i.e., if D =
any bounded input u(t) and any initial condition x(t0 ).
and Du =
) for
Theorem 5.4. (Lemmas 5.4 and 5.5 (Ref. 23

Let f (t, x, u) be continuously differentiable and of global Lipschitz form in (x, u) uniformly in t. Then, if
the system (13a) has a globally exponentially stable equilibrium point at x = 0, it is ISS. In the case for which
the system is autonomous, f (x, u) in Eq. (28a) is continuously differentiable, and the origin is an asymptotically
stable equilibrium point of Eq. (28a), then Eq. (28a) is ISS.
Definition 9. The system (13) is locally input-to-output stable if there exists a class L function , a class
function , and positive constants k1 and k2 such that for any initial condition x(t0 ) such that x(t0 ) < k1 and
any input u(t) such that supt t0 u(t) < k2 , and for any t t0 0, the following holds true.
(1) The solution x(t) exists.
(2)
The system (13) is said to be input-to-output stable (IOS) if D =

initial state x(t0 ) and any bounded input u(t).
, Du =
, and Eq. (30) holds for any
Note that while this is similar to the Lp -stability concepts presented previously, it is actually more general
as the function need not be linear and the function need not be a constant.
Theorem 9. (Theorem 6.3 of Ref. 23)
Let f (t, x, u) be piecewise continuous in t and of local Lipschitz form in (x, u), and let h be piecewise
continuous in t and continuous in (x, u). Assume that the system (13) is ISS, and that there exists class
functions 1 and 2 such that
Then the system (13) is locally IOS. If all assumptions hold globally and Eq. (13a) is ISS, then it is IOS.
Conclusions
In this article we have attempted to summarize various concepts of input-output stability for nonlinear dynamical systems, focusing on the continuous-time case. We have presented the basic input-output concepts but also
17
some extensions and their applications to stability robustness analysis and design and to the adaptive control
of linear and nonlinear systems, as mentioned previously.
It is now clear that the connections between Lyapunov stability and input-output stability are strong and
may be exploited for further design. On the other hand, it is clear that the input-output approach remains a
versatile tool. This approach allows us to be able to determine the stability of the closed-loop system although
we have have little knowledge of the internal dynamics of the open-loop system and its uncertainties. This is
clearly an advantage when dealing with uncertain systems as the dynamics of the systems and its uncertainties
may be unknown. One of the limitations of the input-output approach, however, is that it remains limited to the
study of stability while other objectives such as the optimization of some performance indices remain beyond
the reach of these techniques. We end this article by mentioning that prior to its introduction to feedback
systems and control, the input-output approach was part of operator theory and functional analysis. Finally,
the input-output approach has been applied to various areas such as communications (39,40) and to the study
of neural network stability (41).
Appendix 1: Denitions
Definition 1.1 (Lipschitz functions). A function f (x) is said to be of local Lipschitz form on a domain D
if for each point x0 D there exist a neighborhood D0 of the point x0 and a constant L0 such that
If equation (32) holds for all x D with the same constant L, then f is said to be of Lipschitz form on D. If f is
of Lipschitz form on n than it is said to be of global Lipschitz form.
Definition 1.1 can be extended to the case of f (t, x) provided that the Lipschitz condition holds uniformly
on t for a given time interval.
Definition 1.2. (Function of class .
A continuous function : [0, +) [0, +) is said to be of class
Definition 1.3. (Function of class ).
A function : [0, +) [0, +) is said to be of class
if it is strictly increasing and (0) = 0.
if it is of class
and (s) as s .
Definition 1.4. (Function of class L).

A function : [0, +) [0, +) [0, +) is said to be of class L if for each fixed t the mapping (s, t)
is of class and for each fixed s the mapping (s, t) is decreasing with respect to t and (s, t) 0 as t .
Example:
The function (s): [0, +) s/(s+1) is of class since (s) = 1/(s+1)2 > 0, but it is not of class since
lims (s) = 1 < +.
The function (s): [0, +) s is of class since (s) = 1 > 0 and lims (s) = +.
The function (s, t): [0, +) [0, +) se t is of class L. Indeed it is strictly increasing in s, since
18
strictly decreasing in t, since
and (s, t) 0 as t .
Definition 1.5 (State transition matrix). Given the LTV system
with x n and A(t) piecewise continuous, the state transition matrix (c,c):
is defined as the unique solution of the matrix differential equation
nn
, (t, t0 ) (t, t0 ),
Consider the zero-input system
where f (): D
is of local Lipschitz form on D.
Definition 1.6 (Attractivity). Consider the zero-input system (1.2) and denote by s(t, x0 ) the solution starting
from x0 at time t = 0. Assume that x = 0 is an equilibrium point of system (1.2); then x = 0 is attractive if there
exists a domain Da D, 0 Da such that
The equilibrium point x = 0 is globally attractive if Da =
Definition 1.7 (Stability and Asymptotic Stability). Consider the zero-input system (1.2) and denote by
s(t, x0 ) the solution starting from x0 at time t = 0. Assume that x = 0 is an equilibrium point of system (1.2);
then x = 0 is
stable if, for each > 0, there exists a = () such that
unstable if it is not stable.

asymptotically stable if it is stable and can be chosen such that x = 0 is attractive on the domain D =
{x n , x < }.
globally asymptotically stable if it is stable and globally attractive.
Now consider the zero-input system
where f (,): [0, +) D
, D = {x
, x < r}, and t0 0.
19
Definition 1.8 (Exponential stability). Consider the zero-input system (1.3) and assume that x = 0 is an
equilibrium point. Then x = 0 is exponentially stable if there exist positive numbers K, , c such that
The equilibrium x = 0 is globally exponentially stable if the above condition is verified for any initial state.
Definition 1.9 (Zero-State Observability). The system (27) is said to be zero-state observable from the output
y if for all initial conditions, y(t) 0 x(t) 0. The system is zero-state detectable if for all initial conditions
y(t) 0 limt x(t) = 0.
Definition 1.10 (Reachability). Consider the system
and denote by s(t, x0 , u) the solution starting from x0 at time t = 0 under the input u. Then system (1.5) is said
to be reachable if there exists a class function and a set D := {x n : x < r}, such that for all x D there
exists a time t and an input u such that uL (x) and s(t, 0, u) = x. The system is said to be globally
reachable if all the assumptions hold for all x n .
Definition 1.11 (Uniform observability). Consider the system
and denote by s(t, x0 , u) the solution starting from x0 at time t = 0 under the input u. Then system (1.6) is said
to be uniformly observable if there exists a class function such that for all x,
Appendix 2: Sufcient Conditions For Exponential Stability

Theorem 2.1 (Sufficient condition for exponential stability). Let x = 0 be an equilibrium point of system
(1.3) and assume there exists a continuously differentiable Lyapunov function V(,): (t, x) [0, +) D
satisfying
for some positive constants k1 , k2 , k3 , and . Then the equilibrium x = 0 is exponentially stable. Moreover, if all
the assumptions hold globally, x = 0 is globally exponentially stable.
20
Theorem 2.2 (Exponential stability of linear systems). The equilibrium point x = 0 of the LTV system
is exponentially stable if and only if there exist positive constants and k such that
where (t, t0 ) is the state transition matrix of system (2.2).

Note that the concept of exponential stability is equivalent, for linear systems, to that of uniform asymptotic stability (see Ref. 23 for the definition). This equivalence is no longer true for nonlinear systems, where
the concept of exponential stability is stronger than that of uniform asymptotic stability. Finally, note that for
LTI systems a necessary and sufficient condition for exponential stability is the Hurwitz character of the A
matrix, that is, all its eigenvalues should have negative real part.
BIBLIOGRAPHY
1. H. S. Black Wave Translation System, US Patent No. 2,102,671, December 21, 1937.
2. H. W. Bode FeedbackThe History of an Idea, in R. Bellman and R. Kalaba (eds.), Selected Papers on Mathematical
Trends in Control Theory, pp. 106123. New York: Dover, 1964.
3. H. Nyquist Regeneration theory. Bell System Tech. J., 11: 126147, 1932.
4. I. W. Sandberg On the l2 -boundedness of solutions of nonlinear functional equations, Bell Sys. Tech. J., 43: 15811599,
1964.
5. G. Zames On the input-output stability of nonlinear time-varying feedback systems: Part i, IEEE Trans. Autom. Control,
11: 228238, 1966.
6. G. Zames On the input-output stability of nonlinear time-varying feedback systems: Part ii, IEEE Trans. Autom.
Control, 11: 465477, 1966.
7. M. Safonov Stability and Robustness of Multivariable Feedback Systems, Cambridge: MIT Press, 1980.
8. J. C. Willems The Analysis of Feedback Systems, Cambridge: MIT Press, 1971.
9. C. A. Desoer M. Vidyasagar Feedback Systems: Input Output Properties, New York: Academic Press, 1975.
10. J. C. Willems Dissipative dynamical systems: Part i: General theory, Arch. Ration. Mech. Anal., 45: 325351, 1972.
11. J. C. Willems Dissipative dynamical systems part ii: Linear systems with quadratic supply rate, Arch. Ration. Mech.
Anal., 45: 352393, 1972.
12. B. D. O. Anderson The small-gain theorem, the passivity theorem and their equivalence, J. Franklin Institute, 293(2):
105115, 1972.
13. E. Sontag On the input-to-state stability property, Eur. J. Control, 1: 2436, 1995.
14. E. Sontag Y. Wang Notions of input to output stability, Syst. Control Lett., 38: 235248, 1999.
15. E. Sontag Y. Wang Lyapunov characterizations of input to output stability, SIAM J. Control Optimization, 39, 226249,
2000.
16. C. I. Byrnes A. Isidori J. C. Willems Passivity, feedback equivalence, and the global stabilization of minimum phase
nonlinear systems, IEEE Trans. Autom. Control, 36: 12281240, 1991.
17. M. Krstc Kanellakopoulos P. V. Kokotivc Nonlinear and Adaptive Control Design, New York: Wiley-Interscience, 1995.
18. A. J. van der Schaft L2 -Gain and Passivity Techniques in Nonlinear Control, vol. 218, Lecture Notes in Control and
Information Sciences, Heidelberg: Springer-Verlag, 1996.
19. K. S. Narendra A. M. Annaswamy Stable Adaptive Systems, Englewood Cliffs, NJ: Prentice-Hall, 1985.
20. R. Ortega et al. Passivity-Based Control of Euler-Lagrange Systems: Mechanical, Electrical and Electromechanical
Applications, London: Springer-Verlag, 1998.
21
21. R. Lozano O. Egeland B. Maschke Dissipative Systems Analysis and Control: Theory and Applications, London: Springer,
2000.
22. B. D. O. Anderson S. Vongpanitlerd Network Analysis and Synthesis: A Modern Systems Theory Approach, Englewood
Ciffs, NJ: Prentice Hall, 1973.
23. H. K. Khalil Nonlinear Systems, 2d ed., Upper-Saddle River, NJ: Prentice Hall, 1996.
24. S. Sastry Nonlinear Systems: Analysis, Stability, and Control, New York: Springer, 1999.
25. A. R. Teel T. T. Georgiou L. Praly E. Sontag Input-output stability, in W. S. Levine (ed.), The Control Handbook, pp.
895908, Boca Raton FL: CRC Press, 1995.
26. C. I. Byrnes W. Lin Losslessness, feedback equivalence, and the global stabilization of discrete-time nonlinear systems,
IEEE Trans. Autom. Control, 39(1): 8398, 1994.
27. W. Lin C. I. Byrnes Passivity and absolute stabilization of a class of discrete-time nonlinear systems, Automatica, 31(2):
263267, 1995.
28. H. L. Royden Real Analysis, London: Macmillan (Collier-MacMillan Limited), 2d. ed., 1968.
29. M. Vidyasagar Nonlinear System Analysis, 2d ed., Upper-Saddle River, NJ: Prentice Hall, 1993.
30. G. Tadmor Input/output norms in general linear systems, Int. J. Control, 51(4): 911921, April 1990.
31. M. Vidyasagar A. Vannelli New relationships between input-output and Lyapunov stability, IEEE Trans. Autom.
Control, 27(2): 481483, 1982.
32. A. I. Lure Some Nonlinear Problems in the Theory of Automatic Control, London: H.M. Stationery Office, 1957.
33. V. M. Popov Absolute stability of nonlinear systems of automatic control, Autom. Remote Control, 22: 857875, 1962.
34. V. M. Popov Hyperstability of Control Systems, New York: Springer, 1973.
35. V. A. Yakubovich Solution of certain matrix inequalities in the stability theory of nonlinear control systems, Engl.
Translation Sov. Math. Dokl., 3: 620623, 1962.
36. R. E. Kalman Lyapunov functions for the problem of Lure in automatic control, Proc. Nat. Acad. Sci., 49: 201205,
1963.
37. J. T. Wen Time domain and frequency domain conditions for strict positive realness, IEEE Trans. Autom. Control, 33:
988992, 1988.
38. D. J. Hill P. Moylan Dissipative dynamical systems: Basic input-output and state properties, J. Franklin Inst., 5:
327357, 1980.
39. I. W. Sandberg On the properties of systems that distort signals, Bell Syst. Tech. J., 42: 20332047, 1963.
40. I. W. Sandberg Notes on PQ theorems, IEEE Trans. Circuits Syst. I: Fundam Theory Appl., 41: 303307, 1994.
41. A. Mayer-Base F. Ohl H. Scheich Asymptotic hyperstability for the Hopfield model of neural networks, in Proc. IEEE
Joint Int. Conf. Neural Networks, pp. 24362441, Orlando, FL, 1994.
CHAOUKI T. ABDALLAH
University of Tennessee
FRANCESCO AMATO
MARCO ARIOLA
Universita` degli Studi di Napoli Federico II
INTELLIGENT CONTROL
AUTONOMOUS SYSTEMS, CONTROL, ROBOTS,
VEHICLES
on intelligent control for high-autonomy systems. Hybrid

systems that contain both continuous and digital components are also briey discussed, as they are central in intelligent control.
FOUNDATIONS OF INTELLIGENT CONTROL
INTELLIGENT SYSTEMS, ROBOTS, VEHICLES,

MANUFACTURING
Intelligent control describes the discipline for which control methods are developed that attempt to emulate important characteristics of human intelligence. These characteristics include adaptation and learning, planning under large uncertainty, and coping with large amounts of
data. The main difculty in specifying exactly what is
meant by the term intelligent control stems from the fact
that there is no agreed-upon denition of human intelligence and intelligent behavior, and the centuries-old debate of what constitutes intelligence is still continuing,
nowadays among educators, psychologists, computer scientists, and engineers. Apparently the term intelligent control was coined in the 1970s by K. S. Fu.
There are a number of areas related to the area of intelligent control. Intelligent control is interdisciplinary as
it combines and extends theories and methods from areas such as control, computer science, and operations research. It uses theories from mathematics and seeks inspiration and ideas from biological systems. Intelligent
control methodologies are being applied to robotics and
automation, communications, manufacturing, and trafc
control, to mention but a few application areas. Neural
networks, fuzzy control, genetic algorithms, planning and
expert systems, intelligent agents, and hybrid systems are
all areas where related work is taking place. The areas
of computer science and in particular articial intelligence provide knowledge representation ideas, architectures, methodologies, and tools such as semantic networks,
frames, reasoning techniques, and computer languages.
Concepts and algorithms developed in the areas of adaptive
control and machine learning help intelligent controllers to
adapt and learn. Advances in sensors, actuators, computation technology, and communication networks help provide
the intelligent control hardware necessary for implementation and inuence the type of architecture used. For example, low-cost processing and communication devices together with low-cost sensors are making it possible to have
distributed, networked, embedded control systems in many
application areas.
In the following, fundamental ideas of intelligent control are emphasized rather than particular methodologies;
note that several related areas are described at length elsewhere in this encyclopedia. Fundamental ideas and characteristics of intelligent systems are introduced in the section on foundations of intelligent control, and a historical
perspective is given in the section on intelligent learning
control in which the role of machine learning is discussed.
The quest for machines that exhibit higher autonomy has
been the driving force in the development of control systems over the centuries, and this is discussed in the section
The term intelligent control has come to mean, particularly to those outside the control area, some form of control
using methodologies such as intelligent agents, genetic algorithms, or fuzzy and/or neural networks. Intelligent control, however, does not restrict itself only to those methodologies. The fact is that there are problems of control today
that cannot be formulated and studied in the conventional
differential or difference equation mathematical framework using conventional (or traditional) control methodologies; these methodologies were developed in the past
decades to control dynamical systems. To address these
problems in a systematic way, a number of methods have
been developed in recent years that are collectively known
as intelligent control methodologies. It is worth remembering at this point that intelligent control uses conventional
control methods to solve lower level control problems and
that conventional control is included in the area of intelligent control. In summary, intelligent control attempts to
build upon and enhance the conventional control methodologies to solve new challenging control problems.
Conventional and Intelligent Control
The word control in intelligent control has a different,
more general meaning than the word control in conventional control. First, the processes of interest are more general and may be described, for example, by discrete event
system models, differential or difference equation models,
or both. This has led to the development of theories for hybrid control systems, which study the control of continuousstate dynamic processes by discrete-state controllers. In
addition to the more general processes considered in intelligent control, the control objectives can also be more
general. For example, replace part A in satellite can be
the general task for the controller of a space robot arm;
this is then decomposed into a number of subtasks, several of which may include, for instance, follow a particular
trajectory, which may be a problem that can be solved by
conventional control methodologies. To attain such control
goals for complex systems over a period of time, the controller has to cope with signicant uncertainty that xed
feedback robust controllers or adaptive controllers cannot
deal with. Since the goals are to be attained under large
uncertainty, fault diagnosis, and control reconguration,
adaptation and learning are important considerations in
intelligent controllers. It is also clear that task planning is
an important area in intelligent control design. So the control problem in intelligent control is an enhanced version of
the problem in conventional control. It is much more ambitious and general. The area of intelligent control is interdisciplinary, and it attempts to combine and extend theories
and methods from areas such as control, computer science,
and operations research to attain demanding control goals
in complex systems.
Intelligent Systems, Robots, Vehicles, Manufacturing
Note that the theories and methodologies from the areas of operations research and computer science cannot,
in general, be used directly to solve control problems, as
they were developed to address different needs. The techniques from computer science and operations research are
primarily analysis tools developed for non-real-time systems, while in control, synthesis techniques to design realtime feedback control laws for dynamic systems are mainly
of interest. Recent work on real-time systems in computer
science has moved the research along lines that are of great
interest to intelligent control. In view of this discussion, it
should be clear that intelligent control research, which is
mainly driven by applications, has a very important and
challenging theoretical component. Signicant theoretical
strides must be made to address the open questions. The
problems are nontrivial, but the payoff is very high indeed.
As it was mentioned above, the word control is intelligent control has a more general meaning than in conventional control; in fact it is closer to the way the term control
is used in everyday language. Because intelligent control
addresses more general control problems that also include
the problems addressed by conventional control, it is rather
difcult to come up with meaningful benchmark examples.
Intelligent control can address control problems that cannot be formulated in the language of conventional control.
To illustrate, in a rolling steel mill, for example, conventional controllers may include the speed (revolutions per
minute) regulators of the steel rollers. In the intelligent
control framework one may include, in addition, fault diagnosis, alarm systems, and perhaps the problem of deciding on the set points of the regulators, which are based
on the sequence of orders processed, economic decisions,
maintenance schedules, availability of machines, etc. All
these factors have to be considered as they play a role in
controlling the whole production process, which is really
the overall goal.
Another difference between intelligent and conventional control is the separation between controller and the
system to be controlled. In conventional control the system
to be controlled, called the plant, typically is separate and
distinct from the controller. The controller is designed by
the control designer, while the plant is, in general, given
and cannot be changed. In intelligent control problems,
which are most often complex and challenging, there may
not be a clear separation of the plant and the controller; the
control laws may be embedded in and part of the system to
be controlled. This opens new opportunities and challenges
as it may be possible to affect the design of processes in a
more systematic way.
Areas relevant to intelligent control, in addition to
conventional control, include hybrid systems, planning
and knowledge-based systems, intelligent agents, machine
learning, search algorithms, fault diagnosis and control reconguration, and real-time and embedded systems, predicate logic, automata, Petri nets, neural networks, and fuzzy
logic genetic algorithms. In addition, in order to control
complex systems, one has to deal effectively with the computational complexity issue; this has been in the periphery
of the interests of the researchers in conventional control,
but it is clear that computational complexity is a central
issue whenever one attempts to control complex systems.
Intelligence and Intelligent Control

It is appropriate at this point to comment briey on the
meaning of the word intelligent in intelligent control. The
precise denition of intelligence has been eluding mankind
for thousands of years. More recently, this issue has been
addressed by disciplines such as psychology, philosophy, biology, and of course articial intelligence (AI). AI is dened
to be the study of mental faculties through the use of computational models. No consensus has yet emerged of what
constitutes intelligence. The controversy surrounding the
widely used IQ tests also points to the fact that we are far
away from having understood these issues. A term that has
been introduced more recently is that of computational intelligence to distinguish machine intelligence from human
intelligence. In this article we introduce and discuss several characterizations of intelligent systems that appear
to be useful when attempting to address complex control
problems.
Intelligent controllers can be seen as machines which
emulate human mental faculties such as adaptation and
learning, planning under large uncertainty, and coping
with large amounts of data in order to control complex processes effectively. This is the justication for the use of the
term intelligent in intelligent control, since these mental
faculties are considered to be important attributes of human intelligence. An alternative term, which is discussed
later in this article, is autonomous (intelligent) control; it
emphasizes the fact that an intelligent controller typically
aims to attain higher degrees of autonomy in accomplishing and even setting control goals, rather than stressing
the (intelligent) methodology that achieves those goals.
Dening Intelligent Control Systems
Intelligent systems can be characterized in a number of
ways and along a number of dimensions. There are certain
attributes of intelligent systems that are of particular interest in the control of systems; see Ref. 1. We begin with a
general characterization of intelligent systems: An intelligent system has the ability to act appropriately in an uncertain environment, where an appropriate action is that
which increases the probability of success, and success is
the achievement of behavioral subgoals that support the
systems ultimate goal. In order for a manmade intelligent
system to act appropriately, it may emulate functions of
living creatures and ultimately human mental facilities.
An intelligent system can be characterized along a number of dimensions. There are degrees or levels of intelligence
that can be measured along the various dimensions of intelligence. At a minimum, intelligence requires the ability
to sense the environment, to make decisions, and to control action. Higher levels of intelligence may include the
ability to recognize objects and events, to represent knowledge in a world model, and to reason about and plan for the
future. In advanced forms, intelligence provides the capacity to perceive and understand, to choose wisely, and to act
successfully under a large variety of circumstances so as
to survive and prosper in a complex and often hostile environment. Intelligence can be observed to grow and evolve,
both through growth in computational power and through
accumulation of knowledge of how to sense, decide, and act
in a complex and changing world.

This characterization of an intelligent system is rather
general. Accordingly, a great number of systems can be considered intelligent. In fact, according to this denition even
a thermostat may be considered to be an intelligent system,
although of a low level of intelligence. It is common, however, to call a system intelligent when in fact it has a rather
high level of intelligence. There exist a number of alternative but related denitions of intelligent systems that emphasize systems with high degrees of intelligence. For example, the following denition emphasizes the fact that the
system in question processes information, and it focuses on
manmade systems and intelligent machines: Machine intelligence is the process of analyzing, organizing, and converting data into knowledge; where (machine) knowledge
is dened to be the structured information acquired and
applied to remove ignorance or uncertainty about a specic task pertaining to the intelligent machine. This denition relates to the principle of increasing precision with
decreasing intelligence of Saridis.
Next, an intelligent system can be characterized by its
ability to assign subgoals and control actions dynamically
in an internal or autonomous fashion: Many adaptive or
learning control systems can be thought of as designing
a control law to meet well-dened control objectives. This
activity represents the systems attempt to organize or order its knowledge of its own dynamical behavior so as to
meet a control objective. The organization of knowledge can
be seen as one important attribute of intelligence. If this
organization is done autonomously by the system, then intelligence becomes a property of the system, rather than of
the systems designer. This implies that systems that autonomously (self)-organize controllers with respect to an
internally realized organizational principle are intelligent
control systems.
A procedural characterization of intelligent systems is
given as follows: Intelligence is a property of the system
that emerges when the procedures of focusing attention,
combinatorial search, and generalization are applied to the
input information in order to produce the output. One can
easily deduce that once a string of the procedures given
previously is dened, the other levels of resolution of the
structure of intelligence grow as a result of the recursion.
Having only one-level structure leads to a rudimentary intelligence that is implicit in the thermostat or to a variablestructure sliding mode controller.
Control and Intelligent Systems
The concepts of intelligence and control are closely related
and the term intelligent control has a unique and distinguishable meaning. An intelligent system must dene and
use goals. Control is then required to move the system to
these goals and to dene such goals. Consequently, any intelligent system will be a control system. Conversely, intelligence is necessary to provide desirable functioning of
systems under changing conditions, and it is necessary to
achieve a high degree of autonomous behavior in a control
system. Since control is an essential part of any intelligent
system, the term intelligent control systems is sometimes
used in engineering literature instead of intelligent systems
or intelligent machines. The term intelligent control system simply stresses the control aspect of the intelligent
system.
Characteristics or Dimensions of Intelligent Systems
There are several essential properties present in different degrees in intelligent systems. One can perceive them
as intelligent system characteristics or dimensions along
which different degrees or levels of intelligence can be measured. In the following we discuss three such characteristics that appear to be rather fundamental in intelligent
control systems.
Adaptation and Learning. The ability to adapt to changing conditions is necessary in an intelligent system. Although adaptation does not necessarily require the ability
to learn, for systems to be able to adapt to a wide variety of
unexpected changes, learning is essential. So the ability to
learn is an important characteristic of (highly) intelligent
systems.
Autonomy and Intelligence. Autonomy in setting and
achieving goals is an important characteristic of intelligent
control systems. When a system has the ability to act appropriately in an uncertain environment for extended periods of time without external intervention it is considered
to be highly autonomous. There are degrees of autonomy;
an adaptive control system can be considered as a system
of higher autonomy than a control system with xed controllers, as it can cope with greater uncertainty than a xed
feedback controller. Although for low autonomy no intelligence (or low intelligence) is necessary, for high degrees
of autonomy, intelligence in the system (or high degrees
of intelligence) is essential.
Structures and Hierarchies. In order to cope with complexity, an intelligent system must have an appropriate
functional architecture or structure for efcient analysis
and evaluation of control strategies. This structure should
provide a mechanism to build levels of abstraction (resolution, granularity) or at least some form of partial ordering so to reduce complexity. An approach to study intelligent machines involving entropy (of Saridis) emphasizes such efcient computational structures. Hierarchies
(that may be approximate, localized, or combined in heterarchies) that are able to adapt may serve as primary
vehicles for such structures to cope with complexity. The
term hierarchies refers to functional hierarchies or hierarchies of range and resolution along spatial or temporal
dimensions, and it does not necessarily imply hierarchical
hardware. Some of these structures may be hardwired in
part. To cope with changing circumstances the ability to
learn is essential so these structures can adapt to signicant, unanticipated changes.
In view of the preceding points, a working characterization of intelligent systems [or of (highly) intelligent (control) systems or machines] that captures the essential characteristics present in any such system follows: An intelligent system must be highly adaptable to signicant unanticipated changes, and so learning is essential. It must ex-
INTELLIGENT LEARNING CONTROL
Figure 1. Intelligent autonomous controller functional architecture. The three levels of a hierarchical intelligent control architecture are the execution level, the coordination level, and the management or organization level.
hibit a high degree of autonomy in dealing with changes. It

must be able to deal with signicant complexity, and this
leads to certain types of functional architectures such as
hierarchies or heterarchies.
Some Examples
Man-made systems that solve complex problems and incorporate some of the essential characteristics of intelligent control systems given previously do exist today. Here
are some examples from Ref. 1: A hierarchically intelligent
control system was designed and built at the Rensselaer
Polytechnic Institute laboratories to do truss construction
remotely in deep space for the NASA space station Freedom. This intelligent control system had a functional hierarchy that consisted of three levels: the lowest was the
execution level, the highest was the organization level,
and the middle was the coordination level (see Fig. 1 and
the section on intelligent autonomous control later in this
article); see Ref. 1. Intelligent control systems using the
real-time control system (RCS) implementations from the
National Institute for Standards and Technology (NIST):
robot vision-based object pursuit; robot deburring; include
an automated manufacturing research facility; robot machine loading or unloading for a milling workstation; multiple autonomous undersea vehicles; NASA space station
telerobotics and the Mars vehicle; army eld material handling robot; DARPA submarine automation; coal mine automation; and army unmanned land vehicles. Other examples of existing intelligent control systems include mobile
robots based on sulsumption and other architectures. See
Refs. 16.
For additional information and insight into the foundations of intelligent control, the interested reader may refer
to Refs. 112.
The term intelligent control was coined in the 1970s.

Earlier used terms included learning control and selforganizing control. A brief description of some of the early
developments in the area that is known today as intelligent
control is now given.
As discussed previously, learning is an important dimension or attribute of intelligent control. Highly autonomous behavior is a very desirable characteristic of advanced control systems, so they perform well under changing conditions in the plant and the environment (even
in the control goals), without external intervention (note
that intelligent autonomous control is discussed at length
later). This requires the ability to adapt to changes affecting, in a signicant manner, the operating region of the
system. Adaptive behavior of this type typically is not offered by conventional control systems. Additional decisionmaking abilities should be added to meet the increased
control requirements. The controllers capacity to learn
from past experience is an integral part of such highly
autonomous controllers. The goal of introducing learning
methods in control is to broaden the region of operability
of conventional control systems. Therefore the ability to
learn is one of the fundamental attributes of autonomous
intelligent behavior; see Refs. 1 and 2.
The ability of manmade systems to learn from experience and, based on that experience, improve their performance is the focus of machine learning. Learning can be
seen as the process whereby a system can alter its actions to perform a task more effectively due to increases
in knowledge related to the task. The actions that a system may take depend on the nature of the system. For example, a control system may change the type of controller
used, or vary the parameters of the controller, after learning that the current controller does not perform satisfactorily within a changing environment. Similarly, a robot
may need to change its visual representation of the surroundings after learning of new obstacles in the environment. The type of action taken by the machine is dependent upon the nature of the system and the type of learning
system implemented. The ability to learn entails such issues as knowledge acquisition, knowledge representation,
and some level of inference capability. Learning, considered fundamental to intelligent behavior, and in particular
the computer modeling of learning processes have been the
subject of research in the eld of machine learning since
the 1960s; see Refs. 19 and 14.
Learning and Adaptive Control
The problem of learning in automatic control systems has
been studied in the past, especially in the late 1960s, and it
has been the topic of numerous papers and books; see, for
example, Refs. 1519. References 15, 17, and 19 provide
surveys on the early learning techniques. All of these approaches involve a process of classication in which all or
part of the prior information required is unknown or incompletely known. The elements or patterns that are presented
to the control system are collected into groups that correspond to different pattern classes or regions; see Ref. 19.
Thus learning was viewed as the estimation or successive

approximation of the unknown quantities of a function; see
Ref. 15. The approaches developed for such learning problems can be separated into two categories: deterministic
and stochastic. Where can learning be used in the control
of systems? As it was already mentioned, learning plays an
essential role in the autonomous control of systems. There
are many areas in control in which learning can be used
to advantage and these needs can be briey classied as
follows: (1) Learning about the plant; that is, learning how
to incorporate changes and then how to derive new plant
models. (2) Learning about the environment; this can be
done using methods ranging from passive observation to
active experimentation. (3) Learning about the controller;
for example, learning how to adjust certain controller parameters to enhance performance. (4) Learning new design
goals and constraints. What is the relation between adaptive control and learning control? Learning is achieved, in a
certain sense, when an adaptive control algorithm is used
to adapt the controller parameters so that, for example, stability is maintained. In this case the system learns and the
knowledge acquired is the new values for the parameters.
Note, however, that if later the same changes occur again
and the system is described by exactly the same parameters identied earlier, the adaptive control algorithm still
needs to recalculate the controller and perhaps the plant
parameters since nothing was kept in memory. So, in that
sense the system has not learned. It has certainly learned
what to do when certain types of changes take place. In
particular, it has been told exactly what to do, that is, it
was given the adaptive algorithm, and this is knowledge
by rote learning. The knowledge represented by the new
values of the controller and the plant parameters and the
circumstances under which these values are appropriate
are not retained. So a useful rule of thumb is that for a
controller to be a learning controller, memory is required
so that past knowledge is stored in such a way that it can
be used to benet when a similar situation arises.
With respect to terminology it is perhaps benecial at
this point to bring in a bit of history. In the 1960s, adaptive control and learning received a lot of attention in the
control literature. It was not always clear, however, what
those terms meant. The comment by Y. Tsypkin, in Ref. 18
describes quite clearly the atmosphere of the period: It is
difcult to nd more fashionable and attractive terms in
the modern theory of automatic control than the terms of
adaptation and learning. At the same time, it is not simple
to nd any other concepts which are less complex and more
vague. Adaptation, learning, self-organizing systems, and
control were competing terms for similar research areas.
The term pattern recognition was appearing together with
adaptive, learning, and self-organizing systems in the control literature of that era. It is obvious that there was no
agreement as to the meaning of these terms and their relation. Pattern recognition is today a research discipline in
its own right, which develops and uses an array of methods
ranging from conventional algorithms to articial intelligence methods. The term self-organizing system is not being used as much today in the control literature. Adaptive
control has gained renewed popularity in the last decades
and mainly emphasizes studies in the convergence of adap-
tive algorithms and in the stability of adaptive systems;

the systems considered are primarily systems described
by differential (or difference) equations where the coefcients are (partially) unknown. In an attempt to enhance
the applicability of adaptive control methods, learning control has been reintroduced in the control literature; see, for
example, Refs. 1012 and 20 for learning methods in control with emphasis on neural networks.
INTELLIGENT CONTROL FOR HIGH-AUTONOMY
SYSTEMS
From a control systems point of view the use of intelligent control methods is a natural next step in the quest for
building systems with higher degrees of autonomy. These
ideas are discussed in the following.
In the design of controllers for complex dynamical systems there are needs today that cannot be successfully
addressed with the existing conventional control theory.
They mainly pertain to the area of uncertainty. Heuristic methods may be needed to tune the parameters of an
adaptive control law. New control laws to perform novelcontrol functions to meet new objectives should be designed
while the system is in operation. Learning from past experience and planning control actions may be necessary.
Failure detection and identication are needed. Such functions have been performed in the past by human operators. To increase the speed of response, to relieve operators from mundane tasks, to protect them from hazards, a
high degree of autonomy is desired. To achieve this, highlevel decision-making techniques for reasoning under uncertainty and taking actions must be utilized. These techniques, if used by humans, may be attributed to intelligent
behavior. Hence, one way to achieve a high degree of autonomy is to utilize high-level decision-making techniques,
intelligent methods, in the autonomous controller. Autonomy is the objective, and intelligent controllers are one way
to achieve it.
Evolution of Control Systems and the Quest for Higher
Autonomy
The rst feedback device on record was the water clock invented by Ktesibios, a Greek living in Alexandria, Egypt
around the 3rd century BC This was certainly a successful
device, as water clocks of similar design were still being
made in Baghdad when the Mongols captured that city in
1258 AD The rst mathematical model to describe plant behavior for control purposes is attributed to J. C. Maxwell,
of the Maxwell equations fame. In 1868 Maxwell used differential equations to explain instability problems encountered with James Watts yball governor; the governor was
introduced in 1769 to regulate the speed of steam engine
vehicles. When Maxwell used mathematical modeling and
methods to explain instability problems encountered with
Watts yball governor, it demonstrated the importance
and usefulness of mathematical models and methods in
understanding complex phenomena and signaled the beginning of mathematical system and control theory. It also
signaled the end of the era of intuitive invention. Control
theory made signicant strides in the past 120 years, with
the use of frequency domain methods and Laplace transforms in the 1930s and 1940s and the development of optimal control methods and state-space analysis in the 1950s
and 1960s. Optimal control in the 1950s and 1960s, followed by progress in stochastic, robust, adaptive, nonlinear hybrid, and networked control methods in the 1960s
to today, has made it possible to control signicantly more
complex dynamical systems than the original yball governor more accurately.
Conventional control systems are designed today using
mathematical models of physical systems. A mathematical model that captures the dynamical behavior of interest
is chosen and then control design techniques are applied,
aided by computer-aided design (CAD) packages, to design
the mathematical model of an appropriate controller. The
controller is then realized via hardware or software and it
is used to control the physical system. The procedure may
take several iterations. The mathematical model of the system must be simple enough so that it can be analyzed with
available mathematical techniques and accurate enough
to describe the important aspects of the relevant dynamical behavior. It approximates the behavior of a plant in the
neighborhood of an operating point.
The control method and the underlying mathematical
theory were developed to meet the ever-increasing control
needs of our technology. The need to achieve the demanding
control specications for increasingly complex dynamical
systems has been addressed by using more complex mathematical models such as nonlinear and stochastic ones,
and by developing more sophisticated design algorithms
for, say, optimal control. The use of highly complex mathematical models, however, can seriously inhibit our ability
to develop control algorithms. Fortunately, simpler plant
models, for example, linear models, can be used in the control design; this is possible because of the feedback used
in control that can tolerate signicant model uncertainties. When the xed feedback controllers are not adequate,
then adaptive controllers are used. Controllers can then
be designed to meet the specications around an operating
point where the linear model is valid, and then via a scheduler a controller emerges that can accomplish the control
objectives over the whole operating range. This is, for example, the method typically used for aircraft ight control
and it is a method used to design xed controllers for certain classes of nonlinear systems. Adaptive control in conventional control theory has a specic and rather narrow
meaning. In particular, it typically refers to adapting to
variations in the constant coefcients in the equations describing the linear plant; these new coefcient values are
identied and then used, directly or indirectly, to reassign
the values of the constant coefcients in the equations describing the linear controller. Adaptive controllers provide
for wider operating ranges than xed controllers and so
conventional adaptive control systems can be considered to
have higher degrees of autonomy than control systems employing xed feedback controllers.
Intelligent Control for High-Autonomy Systems
There are cases in which we need to increase the operating range of the system signicantly. We must be able
to deal effectively with signicant uncertainties in models of increasingly complex dynamical systems in addition
to increase the validity range of our control methods. We
need to cope with signicant unmodeled and unanticipated
changes in the plant, in the environment, and in the control
objectives. This will involve the use of intelligent decisionmaking processes to generate control actions so that a certain performance level is maintained even though there are
drastic changes in the operating conditions. I have found
useful to keep in mind an example that helps set goals for
the future and also teaches humility, as it shows how difcult, demanding, and complex autonomous systems can
be. Currently, if there is a problem on the space shuttle,
the problem is addressed by the large number of engineers
working in Houston Control, the ground station. When the
problem is solved the specic detailed instructions about
how to deal with the problem are sent to the shuttle. Imagine the time when we will need the tools and expertise of all
Houston Control engineers aboard the space shuttle or another space vehicle for extended space travel. What needs
to be achieved to accomplish this goal is certainly highly
challenging!
In view of the above it is quite clear that in the control
of systems there are requirements today that cannot be
successfully addressed with the existing conventional control theory. It should be pointed out that several functions
proposed in later sections, to be part of the high-autonomy
control system, have been performed in the past by separate systems; examples include fault trees in chemical process control for failure diagnosis and hazard analysis, and
control system design via expert systems.
An Intelligent Control Architecture for High-Autonomy
Systems
To illustrate the concepts and ideas involved and to provide
a more concrete framework to discuss the issues, a hierarchical functional architecture of an intelligent controller
that is used to attain high degrees of autonomy in future
space vehicles is briey outlined as an example; full details
can be found in Ref. 21. This hierarchical architecture has
three levels, the execution level, the coordination level, and
the management or organization level. The architecture
exhibits certain characteristics, which have been shown in
the literature to be necessary and desirable in autonomous
systems. Based on this architecture we identify the important fundamental issues and concepts that are needed for
an autonomous control theory.
Architecture Overview: Structure and Characteristics. The
overall functional architecture for an autonomous controller is given by the architectural schematic of the Fig. 1.
This is a functional architecture rather than a hardware
processing one; therefore it does not specify the arrangement and duties of the hardware used to implement the
functions described. Note that the processing architecture
also depends on the characteristics of the current processing technology; centralized or distributed processing may
be chosen for function implementation depending on available computer technology.
The architecture in Fig. 1 has three levels. At the lowest

level, the execution level, there is the interface to the vehicle and its environment (the process in the gure) via the
sensors and actuators. At the highest level, the management or organization level, there is the interface to the pilot
and crew, ground station, or onboard systems. The middle
level, called the coordination level, provides the link between the execution level and the management level. Note
that we follow the somewhat standard viewpoint that there
are three major levels in the hierarchy. It must be stressed
that the system may have more or fewer than three levels,
which, however, can be conceptually combined into three
levels. Some characteristics of the system that dictate the
actual number of levels are the extent to which the operator can intervene in the systems operations, the degree
of autonomy or level of intelligence in the various subsystems, and the hierarchical characteristics of the plant.
Note that the three levels shown in Fig. 1 are applicable to
most architectures of intelligent autonomous controllers,
by grouping together sublevels of the architecture if necessary. The lowest, execution level, involves conventional
control algorithms, while the highest, management and
organization level, involves only higher-level, intelligent,
decision-making methods. The coordination level provides
the interface between the actions of the other two levels
and it uses a combination of conventional and intelligent
decision-making methods.
The sensors and actuators are implemented mainly with
hardware. Software and perhaps hardware are used to implement the execution level. Mainly software is used for
both the coordination and management levels. There are
multiple copies of the control functions at each level, more
at the lower and fewer at the higher levels. Note that the
autonomous controller is only one of the autonomous systems on the space vehicle. It is responsible for all the functions related to the control of the physical system and allows for continuous on-line development of the autonomous
controller and provides for various phases of mission operations. The tier structure of the architecture allows us
to build on existing advanced control theory. Development
progresses, creating each time higher-level adaptation and
a new system that can be operated and tested independently. The autonomous controller performs many of the
functions currently performed by the pilot, crew, or ground
station. The pilot and crew are thus relieved from mundane
tasks and some of the ground-station functions are brought
aboard the vehicle. In this way the degree of autonomy of
the vehicle is increased.
Functional Operation. In Fig. 1, commands are issued
by higher levels to lower levels and response data ows
from lower lever upwards. However we allow the ow of
data within a level and so the functional architecture is
not a pure hierarchy but rather an heterarchy. Parameters
of subsystems can be altered by systems one level above
them in the hierarchy. There is a delegation and distribution of tasks from higher to lower levels and a layered distribution of decision-making authority. At each level, some
preprocessing occurs before information is sent to higher
levels. If requested, data can be passed from the lowest
subsystem to the highest, for example, for display. All sub-
systems provide status and health information to higher

levels. Human intervention is allowed even at the control
implementation supervisor level, with the commands, however, passed down from the upper levels of the hierarchy.
Here is a simple illustrative example to clarify the overall operation of the autonomous controller. Suppose that
the pilot desires to repair a satellite. After dialogue with the
management level via the interface, the task is rened to
repair satellite using robot A. This is a decision made using the capability-assessing, performance-monitoring, and
planning functions of the management level. The management level decides if the repair is possible under the current performance level of the system and in view of nearterm other planned functions. Using its planning capabilities, it then sends a sequence of subtasks to the coordination level sufcient to achieve the repair. This sequence
could be to order robot A to go to satellite at coordinates (x,
y, z); open repair hatch; repair. The coordination level,
using its planner, divides, say, the rst subtask, go to satellite at coordinates (x, y, z), into smaller subtasks: go from
start to (x1 , y1 , z1 ), then maneuver around obstacle, move
to (x2 , y2 , z2 ), . . . , arrive at the repair site and wait. The
other subtasks are divided in a similar manner. This information is passed to a control implementation supervisor
at the coordination level, which recognizes the task and
uses stored control laws to accomplish the objective. The
subtask go from start to (x1 , y1 , z1 ) can, for example, be
implemented using stored control algorithms to rst proceed forward 10 m, to the right 15 , etc. These control algorithms are executed in the controller at the execution
level utilizing sensor information; the control actions are
implemented via the actuators.
Characteristics of Hierarchical Intelligent Controllers for
High-Autonomy Systems
Based on the architecture previously described, important
fundamental concepts and characteristics that are needed
for an autonomous intelligent control theory are now identied. The fundamental issues that must be addressed for
a quantitative theory of autonomous intelligent control are
discussed.
There is a successive delegation of duties from the
higher to lower levels; consequently the number of distinct
tasks increases as we go down the hierarchy. Higher levels
are concerned with slower aspects of the systems behavior
and with its larger portions, or broader aspects. There is
then a smaller contextual horizon at lower levels, that is,
the control decisions are made by considering less information. Also notice that higher levels are concerned with
longer time horizons than lower levels. Due to the fact that
there is the need for high-level decision making abilities at
the higher levels in the hierarchy, the proposition has been
put forth that there is increasing intelligence as one moves
from the lower to the higher levels. This is reected in the
use of fewer conventional numeric-algorithmic methods at
higher levels as well as the use of more symbolic decisionmaking methods. This is the principle of increasing intelligence with decreasing precision of Saridis; see also Ref.
8 and the references therein. The decreasing precision is
reected by a decrease in time scale density, decrease in
bandwidth or system rate, and a decrease in the decision

(control action) rate. (These properties have been studied
for a class of hierarchical systems in Ref. 22.) All these
characteristics lead to a decrease in granularity of models
used, or equivalently, to an increase in model abstractness.
Model granularity also depends on the dexterity of the autonomous controller.
It is important at this point to discuss briey the dexterity of the controller. The execution level of a highly dexterous controller is very sophisticated and it can accomplish
complex control tasks. The coordination level can issue
commands such as move 15 cm to the right and grip standard, xed dimension cylinder in a dexterous controller, or
it can completely dictate each mode of each joint (in a manipulator) move joint 1 15 then move joint 5 3 , etc., in
a less dexterous one. The simplicity and level of abstractness of commands in an autonomous controller depend on
its dexterity. The more sophisticated the execution level is
the simpler are the commands that the control implementation supervisor needs to issue. Notice that a very dexterous
robot arm may itself have a number of autonomous functions. If two such dexterous arms were used to complete a
task that required the coordination of their actions, then
the arms would be considered to be two dexterous actuators and a new supervisory autonomous controller would
be placed on top for the supervision and coordination task.
In general, this can happen recursively, adding more intelligent autonomous controllers as the lower-level tasks,
accomplished by autonomous systems, need to be supervised.
There is an ongoing evolution of the intelligent functions
of an autonomous controller. It is interesting to observe the
following. Although there are characteristics that separate
intelligent from nonintelligent systems, as intelligent systems evolve, the distinction becomes less clear. Systems
that were originally considered to be intelligent evolve to
gain more characteristics of what is considered to be nonintelligent, numeric-algorithmic systems. An example is a
route planner. Although there are AI route-planning systems now, as problems like route planning become better
understood, more conventional numeric-algorithmic solutions are developed. The AI methods that are used in intelligent systems help us to understand complex problems so
we can organize and synthesize new approaches to problem
solving, in addition to being problem-solving techniques
themselves. AI techniques can be viewed as research vehicles for solving very complex problems. As the problem
solution develops, purely algorithmic approaches, which
have desirable implementation characteristics, substitute
AI techniques and play a greater role in the solution of the
problem. It is for this reason that we concentrate on achieving autonomy and not on whether the underlying system
can be considered intelligent.
if this description makes subsequent analysis too difcult

or too computationally complex to be useful. The complexity of the plant model needed in design depends on both
the complexity of the physical system and how demanding
the design specications are. There is a tradeoff between
model complexity and our ability to perform an analysis
of the system via the model. However, if the control performance specications are not too demanding, a more abstract, higher-level, model can be utilized, which will make
subsequent analysis simpler. This model intentionally ignores some of the system characteristics, specically those
that need not be considered in attempting to meet the particular performance specications; see also the discussion
on hybrid systems later in this article. For example, a simple temperature controller could ignore almost all dynamics of the house or the ofce and consider only a temperature threshold model of the system to switch the furnace
off or on.
Discrete event system (DES) models using nite automata, Petri nets, queuing network models, Markov
chains, etc., are quite useful for modeling the higher-level
decision-making processes in the intelligent autonomous
controller. The choice of whether to use such models will,
of course, depend on what properties of the autonomous
system need to be studied.
The quantitative, systematic techniques for modeling,
analysis, and design of control systems are of central and
utmost practical importance in conventional control theory. Similar techniques for intelligent autonomous controllers do not exist. This is mainly due to the hybrid
structure (nonuniform, nonhomogeneous nature) of the dynamical systems under consideration; they include both
continuous-state and discrete-state systems. Modeling techniques for intelligent autonomous systems must be able to
support a macroscopic view of the dynamical system; hence
it is necessary to represent both numeric and symbolic information. The nonuniform components of the intelligent
controller all take part in the generation of the low-level
control inputs to the dynamical system; therefore they all
must be considered in a complete analysis. Research could
begin by using different models for different components
of the intelligent autonomous controller, since much can
be attained by using the best available models for the various components of the architecture and joining them via
some appropriate interconnecting structure. For instance,
systems that are modeled with a logical discrete event system (DES) model at the higher levels and a difference or
differential equation at the lower level should be examined;
see the discussion on hybrid systems later in this article.
In any case, good understanding of hierarchical models is
necessary for the analysis and synthesis of intelligent autonomous controllers.
Models for Intelligent Controllers. In highly autonomous

control systems, the plant is normally so complex that it is
either impossible or inappropriate to describe it with conventional mathematical system models such as differential
or difference equations. Even though it might be possible
to describe some system accurately with highly complex
nonlinear differential equations, it may be inappropriate
Research Directions. One can roughly categorize research in the area of intelligent autonomous control into
two areas: conventional control theoretic research, which
addresses the control functions at the execution and coordination levels, and the modeling, analysis, and design
of higher-level decision-making systems found in the management and coordination levels.
It is important to note that in order to obtain a high

degree of autonomy it is necessary to adapt or learn.
Neural networks offer methodologies to perform learning
functions in the intelligent autonomous controller. In general, there are potential applications of neural networks
at all levels of hierarchical intelligent controllers that provide higher degrees of autonomy to systems. Neural networks are useful at the lowest execution levelwhere
the conventional control algorithms are implemented via
hardware and softwarethrough the coordination level,
to the highest management level, at which decisions are
being made based on possibly uncertain and/or incomplete information. One may point out that at the execution
levelconventional control levelneural network properties such as the ability for function approximation and the
potential for parallel implementation appear to be very important. In contrast, at higher levels abilities such as pattern classication and the ability to store information in,
say, an associative memory appear to be of signicant interest. Machine learning is of course important at all levels.
We stress that in control systems with high degrees of
autonomy we seek to widen the operating range of the system signicantly so that nontrivial failures and environmental changes can occur and performance will still be
maintained. All of the conventional control techniques are
useful in the development of autonomous controllers and
they are relevant to the study of autonomous control. It is
the case, however, that certain techniques are more suitable for interfacing to the autonomous controller and for
compensating for signicant system failures. For instance,
the area of restructurable or recongurable control systems
studies techniques to recongure controllers when signicant failures occur.
Conventional modeling, analysis, and design methods
should be used whenever applicable for the components of
the intelligent autonomous control system as well as fuzzy
controllers. For instance, they should be used at the execution level of many autonomous controllers. The symbolicnumeric interface is a very important issue; consequently it
should be included in any analysis. There is a need for systematically generating less detailed, more abstract models
from differential or difference equation models to be used
in higher levels of the autonomous controller; see the later
discussion on hybrid systems. Tools for the implementation
of this information extraction also need to be developed. In
this way conventional analysis can be used in conjunction
with the developed analysis methods to obtain an overall quantitative, systematic analysis paradigm for intelligent autonomous control systems. In short, we propose to
use hybrid modeling, analysis, and design techniques for
nonuniform systems. This approach is not unlike the approaches used in the study of any complex phenomena by
the scientic and engineering communities.
HYBRID SYSTEMS
Hybrid control systems contain two distinct types of systems, systems with continous dynamics and systems with
discrete dynamics, that interact with each other. Their
study is central in designing intelligent control systems
with a high degree of autonomy and it is essential in designing discrete event supervisory controllers for continuous systems; see Refs. 126.
Hybrid control systems typically arise when continuous
processes interact with, or are supervised by, sequential
machines. Examples of hybrid control systems are common in practice and are found in such applications as exible manufacturing, chemical process control, electric power
distribution, and computer communication networks. A
simple example of a hybrid control system is the heating
and cooling system of a typical home. The furnace and air
conditioner, along with the heat-ow characteristics of the
home, form a continuous-time system that is to be controlled. The thermostat is a simple discrete-event system
that basically handles the symbols {too hot, too cold} and
{normal}. The temperature of the room is translated into
these representations in the thermostat and the thermostats response is translated back to electrical currents that
control the furnace, air conditioner, blower, etc.
Since the continuous and discrete dynamics coexist and
interact with each other it is important to develop models that accurately describe the dynamic behavior of such
hybrid systems. In this way it is possible to develop control strategies that fully take into consideration the relation and interaction of the continuous and discrete parts
of the system. In the past, models for the continuous- and
discrete-event subsystems were developed separately; the
control law was then derived in a rather empirical fashion,
except in special cases such as the case of digital controllers
for linear time-invariant systems. The study of hybrid systems provides the backbone for the formulation and implementation of learning control policies. In such policies,
the control acquires knowledge (discrete data) to improve
the behavior of the system as it evolves in time. Hybrid
systems have become a distinctive area of study due to
opportunities to improve on traditional control and estimation technologies by providing computationally effective
methodologies for the implementation of digital programs
that design or modify the control law in response to sensordetected events, or as a result of adaptation and learning.
The interested reader should consult Refs. 25, 26 and especially the references therein.
Certain important issues in hybrid systems are now
briey discussed using a paradigm of a continuous system supervised by a DES controller from Refs. 23 and 24.
The hybrid control system of interest here consists of a
continuous-state system to be controlled, also called the
plant, and a discrete-state controller connected to the plant
via an interface; see Fig. 2.
The plant contains all continuous-state subsystems
of the hybrid control system, such as any conventional
continuous-state controllers that may have been developed
and a clock if time and synchronous operations are to be
modeled. The controller is an event-driven, asynchronous
DES, described by a nite state automaton or an ordinary
Petri net. The hybrid control system also contains an interface that provides the means for communication between
the continuous-state plant and the DES controller. The interface receives information from the plant in the form of
a measurement of a continuous variable z(t), such as the
continuous state, and issues a sequence of symbols {z(i)}
10
CONCLUDING REMARKS
Figure 2. Hybrid supervisory control architecture. The interface

receives continuous measurements z(t) and issues a sequence of
symbols {z(i)} which the DES controller processes to issue a sequence of control symbols {r (i)}. These are translated by the interface to (piecewise) continuous input commands r(t).
to the DES controller. It also receives a sequence of control

symbols {r(i)} from the controller and issues (piecewise)
continuous input commands r(t) to the plant.
The interface plays a key role in determining the dynamics and the behavior of the hybrid control system. Understanding how the interface affects the properties of the
hybrid system is one of the fundamental issues in the theory of hybrid control systems. The interface can be chosen
to be simply a partitioning of the state space; see Ref. 23. If
memory is necessary to derive an effective control, it is included in the DES controller and not in the interface. Also
the piecewise continuous command signal issued by the interface is simply a staircase signal, not unlike the output of
a zero-order hold in a digital control system. Including an
appropriate continuous system at (the input of) the plant,
signals such as ramps and sinusoids can be generated if
desired. So the simple interface is used without loss of generality. It allows analysis of the hybrid control system with
development of properties such as controllability, stability
and determinism, in addition to control design methodologies; see Refs. 23 and 24. In general, the design of the interface depends not only on the plant to be controlled, but
also on the control policies available, as well as on the control goals. Depending on the control goals, one may or may
not need, for example, detailed state information; this corresponds to small or large regions in the partition of the
measured signal space (or greater of lower granularity).
This is, of course, not surprising as it is rather well known
that to stabilize a system, for example, requires less detailed information about the systems dynamic behavior
than to do, say, tracking. The fewer the distinct regions in
the partitioned signal space, the simpler (fewer states) the
resulting DES plant model and the simpler the DES controller design. Since the systems to be controlled via hybrid
controllers are typically complex, it is important to make
every effort to use only the necessary information to attain
the control goals. This leads to simpler interfaces that issue only the necessary number of distinct symbols and to
simpler DES plant models and controllers. The question of
systematically determining the minimum amount of information needed from the plant in order to achieve specic
control goals via a number of specialized control policies is
an important question.
There may be the temptation to classify the area of intelligent autonomous systems as simply a collection of methods
and ideas already addressed elsewhere, the need only being some kind of intelligent assembly and integration of
known techniques. This is not true. The theory of control
systems is not covered by, say, the area of applied mathematics, because control has different needs and therefore
asks different questions. The problems of interest in intelligent systems require development of novel concepts, approaches, and methods in computer science, operations research control systems, to mention but a few. The marriage
of all these elds can only be benecial to all. Computer science and operation research methods are increasingly used
in control problems, while control system concepts such
as feedback and methods are providing the base for new
theories and methods in those areas. Intelligent control
for high degree of autonomy systems is a vibrant research
and applications area where developments are followed by
interdisciplinary research and advances in computational
networking, sensing, and artworking technologies.
BIBLIOGRAPHY
1. P. J. Antsaklis, Dening intelligent control, Report of the IEEE
Control Systems Society Task Force on Intelligent Control (P.
J. Antsaklis, Chair; J. Albus, K. Baheti, J. D. Birdwell, M.
Lemmon, M. Mataric, A. Meystel, K. Narendra, K. Passino, H.
Rauch, G. Saridis, H. Stephanou, P. Werbos, Members), IEEE
Control Syst. Mag., 14 (3): 45, 5866, 1994.
2. P. J. Antsaklis, K. M. Passino (eds.), An Introduction to Intelligent and Autonomous Control, Norwell, MA: Kluwer, 1993.
3. A. Meystel, Autonomous Mobile Robots, River Edge, NJ: World
Scientic, 1991.
4. K. P. Valavanis, G. N. Saridis, Intelligent Robotic System Theory: Design and Applications, Norwell, MA: Kluwer, 1992.
5. J. S. Albus and A. M. Meystel, Intelligent Systems:Architecture,
Design, Control, Wiley-Interscience, 2001.
6. R. A. Brooks Cambrian Intelligence: The Early History of the
New AI, The MIT Press, 1999.
7. J. S. Albus, Outline for a theory of intelligence, IEEE Trans.
Syst. Man Cybern., 21 (3): 432509, 1991.
8. G. N. Saridis, K. P. Valavanis, Analytical design of intelligent
machines, Automatica, 24 (2): 123133, 1988.
9. J. S. Sinha and A. M. Meystel, Engineering of Mind: An
Introduction to the Science of Intelligent Systems, WileyInterscience, 2001.
10. D. A. White, D. A. Sofge (eds.), Handbook of Intelligent Control Neural, Fuzzy, and Adaptive Approaches, New York: Van
Nostrand Reinhold, 1992.
11. M. M. Gupta, N. K. Sinha (eds.), Intelligent Control: Theory
and Practice, Piscataway, NJ: IEEE Press, 1994.
12. N. K. Sinha andM. M. Gupta (Eds), Soft Computing & Intelligent Systems: Theory and Applications, Academic Press, 2000.
13. R. S. Michalski, J. G. Carbonell, T. M. Mitchell, Machine
LearningAn Articial Intelligence Approach, Palo Alto, CA:
Tioga, 1983.See also R. S. Michalski andG. Tecuci (eds.), Ma-
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
chine LearningA Multistrategy Approach, San Francisco:

Morgan-Kaufmann, 1994, Vol. IV.
T. M. Mitchell, Machine Learning, WCB/McGraw-Hill, 1997.
K. S. Fu, Learning control systemsReview and outlook,
IEEE Trans. Autom. Control, 15: 210221, 1970.
J. M. Mendel, K. S. Fu, Adaptive, Learning and Pattern Recognition Systems, New York: Academic, 1970.
J. Sklansky, Learning systems for automatic control, IEEE
Y. A. Z. Tsypkin, Adaptation and Learning in Automatic Systems, New York: Academic, 1971.
Y. A. Z. Tsypkin, Self-learningWhat is it? IEEE Trans. Autom. Control, 13: 608612, 1968.
J. A. Farrell, M. M. Polycarpou, Adaptive Approximation Based
Control: Unifying Neural, Fuzzy and Traditional Adaptive Approximation Approaches, Wiley-Interscience, 2006.
P. J. Antsaklis, K. M. Passino, Introduction to Intelligent Control Systems with High Degree of Autonomy, inP. J. Antsaklis
andK. M. Passino (eds.), An Introduction to Intelligent and Autonomous Control, Norwell, MA: Kluwer, 1993.
K. M. Passino, P. J. Antsaklis, Event rates and aggregation in
hierarchical discrete event systems, J. Discrete Event Dynamic
Syst. Theory. Appl., 1 (3): 271287, 1992.
P. J. Antsaklis, J. A. Stiver, M. D. Lemmon, Hybrid System
Modeling and Autonomous Control Systems, inR. L. Grossman
et al. (eds.), Hybrid Systems, New York: Springer-Verlag, 1993,
366392.
J. A. Stiver, P. J. Antsaklis, M. D. Lemmon, A logical DES approach to the design of hybrid control systems, Math. Comput.
Model., 23 (11/12): 1996.
P. J. Antsaklis, A Brief Introduction to the Theory and Applications of Hybrid Systems, Proceedings of the IEEE, Special Issue on Hybrid Systems: Theory and Applications, P. J.
Antsaklis (Ed.), Vol. 88, No. 7,pp. 879887, July 2000.
P. J. Antsaklis and X. D. Koutsoukos, Hybrid Systems: Review and Recent Progress, in Software-Enabled Control: Information Technologies for Dynamical Systems, T. Samad andG.
Balas (Eds.), IEEE Press,pp 273298, 2003.
PANOS J. ANTSAKLIS
University of Notre Dame
Notre Dame, IN, 46556
11
KALMAN FILTERS AND OBSERVERS

FILTERING THEORY
GAUSSIAN PROCESSES
NONLINEAR FILTERS
Theoretically, a Kalman lter is an estimator for what is
called the linear quadratic Gaussian (LQG) problem, which
is the problem of estimating the instantaneous state of a
linear dynamic system perturbed by Gaussian white noise,
by using measurements linearly related to the state, but
corrupted by Gaussian white noise. The resulting estimator is statistically optimal with respect to any quadratic
function of estimation error. R. E. Kalman introduced the
lter in 1960 (1).
Practically, the Kalman lter is certainly one of the
greater discoveries in the history of statistical estimation
theory, and one of the greatest discoveries in the twentieth
century. It has enabled humankind to do many things that
could not have been done without it, and it has become as
indispensable as silicon in the makeup of many electronic
systems. Its most immediate applications have been for
the control of complex dynamic systems such as continuous manufacturing processes, aircraft, ships, or spacecraft.
In order to control a dynamic system, one must rst
know what the system is doing. For these applications, it is
not always possible or desirable to measure every variable
that one wants to control. The Kalman lter provides a
means for inferring the missing information from indirect
(and noisy) measurements. In such situations, the Kalman
lter is used to estimate the complete state vector from
partial state measurements and is called an observer. The
Kalman lter is also used for predicting the outcome of
dynamic systems that people are not likely to control, such
as the ow of rivers during ood conditions, the trajectories
of celestial bodies, or the prices of traded commodities.
From a practical standpoint, this article will present the
following perspectives:
1. Kalman ltering is an algorithm made from mathematical models. The Kalman lter makes it easier to
solve a problem, but it does not solve the problem all
by itself. As with any algorithm, it is important to understand its use and function before it can be applied
effectively. The purpose of this article is to ensure sufcient familiarity with the use of the Kalman lter
that it can be applied correctly and efciently.
2. The Kalman lter is a recursive algorithm. It has
been called ideally suited to digital computer implementation, in part because it uses a nite representation of the estimation problemby a nite number
of variables (2). It does, however, assume that these
variables are real numbers with innite precision.
Some of the problems encountered in its use arise
from the distinction between nite dimension and -
Figure 1. Foundational concepts in Kalman ltering.
nite information, and the distinction between nite

and manageable problem sizes. These are all issues
on the practical side of Kalman ltering that must be
considered along with the theory.
3. It is a complete statistical characterization of an estimation problem. It is much more than an estimator,
because it propagates the entire probability distribution of the variables it is tasked to estimate. This
is a complete characterization of the current state of
knowledge of the dynamic system, including the inuence of all past measurements. These probability
distributions are also useful for statistical analysis
and predictive design of sensor systems.
4. In a limited context, the Kalman lter is a learning process. It uses a model of the estimation problem that distinguishes between phenomena (what we
are able to observe), noumena (what is really going
on), and the state of knowledge about the noumena
that we can deduce from the phenomena. That state
of knowledge is represented by probability distributions. To the extent that those probability distributions represent knowledge of the real world, and the
cumulative processing of knowledge is learning, this
is a learning process. It is a fairly simple one, but
quite effective in many applications.
Figure 1 depicts the essential subjects forming the foundations for Kalman ltering theory. Although this shows
Kalman ltering as the apex of a pyramid, it is but part
of the foundations of another disciplinemodern control
theoryand a proper subset of statistical decision theory
(3).
Applications of Kalman ltering encompass many elds.
As a tool, the algorithm is used almost exclusively for estimation and performance analysis of estimators and as observers for control of a dynamical system. Except for a few
fundamental physical constants, there is hardly anything
in the universe that is truly constant. The orbital parameters of the asteroid Ceres are not constant, and even the
xed stars and continents are moving. Nearly all physical systems are dynamic to some degree. If we want very
precise estimates of their characteristics over time, then
we must take their dynamics into consideration. Table 1
gives examples of common estimation problems.
We do not always know the dynamics very precisely.
Given this state of partial ignorance, the best we can do
is express ignorance more preciselyusing probabilities.
Nonlinear Filters
The Kalman lter allows us to estimate the state of such

systems with certain types of random behavior by using
such statistical information. A few examples of such systems are listed in Table 1.
The third column of Table 1 lists some sensor types that
we might use to estimate the state of the corresponding
dynamic systems. The objective of design analysis is to determine how best to use these sensor types for a given set
of design criteria. These criteria are typically related to
estimation accuracy and system cost.
Because the Kalman lter uses a complete description
of the probability distribution of its estimation errors in determining the optimal ltering gains, this probability distribution may be used in assessing its performance as a
function of the design parameters of an estimation system,
such as:
The types of sensors to be used

The locations and orientations of the various sensor

types with respect to the system to be estimated

The allowable noise characteristics of the sensors
The preltering methods for smoothing sensor noise
The data sampling rates for the various sensor types
The level of model simplication to reduce implementation requirements
3. Consistency Linearity preserves Gaussianity. That

is, Gaussian probability distributions remain Gaussian under linear transformations of the variates.
4. Tractability These models allow us to derive estimators minimizing expected squared errors.
5. Good Performance The resulting estimators have
performed well for many important applications, despite apparent discrepancies between models and reality.
6. Adaptability These estimators can often be extended
to estimate parameters of the model or to track slow
random variations in parameters.
7. Extendability The variances used for calculating
feedback gains can also be used for comparing performance to modeled performance, detecting anomalous
behavior, and rejecting anomalous sensor data.
Vector-valued random processes x(t) and y(t) are called
uncorrelated if their cross-covariance matrix is identically
zero for all times t1 and t2 :
where E is the expected value operator and T is the transpose of the vector.
The random process x(t) is called uncorrelated if
This analytical capability of the Kalman lter enables

system designers to assign error budgets to subsystems
of an estimation system and to trade off the budget allocations to optimize cost or other measures of performance
while achieving a required level of estimation accuracy.
Furthermore, it acts like an observer by which all the states
not measured by the sensors can be constructed for use in
the control system applications.
where (t) is the Dirac delta function (actually, a generalized function), dened by
WHITE NOISE
Similarly, a random sequence xk in discrete time is called

uncorrelated if
It is common engineering practice to model uncertainty in

terms of Gaussian probability distributions and dynamic
uncertainty in terms of linear dynamic systems disturbed
by uncorrelated (white noise) processeseven though empirical analysis may indicate that the probability distributions are not truly Gaussian, the random processes are not
truly white, or the relationships are not truly linear. Although this approach may discard useful information, we
continue the practice for the following reasons:
1. Approximation Probability distributions may not be
precisely Gaussian, but it is close enough. Nonlinear
systems are often smooth enough that local linearization is adequate. Even though the icker noise observed in electronic systems cannot be modeled precisely using only white noise, it can often be done
closely enough for practical purposes.
2. Simplicity These models have few parameters to be
estimated. Gaussian distributions are characterized
by their means and variances, and white noise processes are characterized by their variances.
where () is the Kronecker delta function, dened by
Q1 (t) and Q2 (k) are the intensity matrices of the white noise
process and sequence. If Q1 (t) and Q2 (t) are constant, the
processes and sequences are stationary. If the probability
distribution of a white noise process at each instant of time
is Gaussian, then it is completely dened by its rst two
moments, mean and variance. If Ex(t) = 0, the process is
called zero mean.
A white noise process or sequence is an example of an
uncorrelated process or sequence. Generally, a white noise
process has no time structure. In other words, knowledge
of the white process value at one instant of time provides
no knowledge of what its value will be (or was) at any other
time point.
Nonlinear Filters
LINEAR ESTIMATION
Linear estimation addresses the problem of estimating the
state of a linear stochastic system by using measurements
or sensor outputs that are linear functions of the state. We
suppose that the stochastic systems can be represented by
the types of plant and measurement models (for continuous and discrete time) shown as equations in Table 2, with
dimensions of the vector and matrix quantities. The measurement and plant noise vk and wk , respectively, are assumed to be zero-mean Gaussian processes, and the initial
value xo is a Gaussian random variable with known mean
x0 and known covariance matrix Po . Although the noise sequences wk and vk are assumed to be uncorrelated, this
restriction can be removed, modifying the estimator equations accordingly.
The objective of statistical optimization is to nd an estimate of the n state vector xk represented by x k , a linear
function of the measurements zi , . . . , zk , which minimizes
the weighted mean-squared error
where M is any symmetric nonnegative denite weighting

matrix.
We will now derive the mathematical form of an optimal
linear estimator for the states of linear stochastic systems
given in Table 2. This is called the linear quadratic Gaussian estimation problem. The dynamic systems are linear,
the performance cost functions are quadratic, and the random processes are Gaussian.
Let us consider similar types of estimators for the LQG
problem:
Filters use observations up to the time that the state

of the dynamic system is to be estimated:
Predictors estimate the state of the dynamic system
or its discrete-time equivalent. This implementation automatically minimizes the expected risk associated with
any quadratic loss function of the estimation error.
The statistical performance of the estimator can be predicted a priori (i.e., before it is actually used) by solving
the nonlinear differential (or difference) equations used
in computing the optimal feedback gains of the estimator. These are called Riccati equations, named in 1763 by
Jean le Rond DAlembert (17171783) for Count Jacopo
Francesco Riccati (16761754), who had studied a secondorder scalar differential equation, although not the form
that we have here (13, 14). Kalman gives credit to Richard
S. Bucy for the discovery that the Riccati differential equation serves the same role as the Wiener-Hopf integral equation in dening optimal gains. The Riccati equation also
arises naturally in the problem of separation of variables
in ordinary differential equations and in the transformation of two-point boundary value problems to initial value
problems (15). The behavior of their solutions can be shown
analytically in trivial cases. These equations also provide
a means for verifying the proper performance of the actual
estimator when it is running.
THE LINEAR OPTIMAL ESTIMATOR IN DISCRETE TIME
(KALMAN FILTER)
Suppose that a measurement has been made at time tk and
that the information that it provides is to be applied in
updating the estimate of the state x of a stochastic system
at time tk . It is assumed that the measurement is linearly
related to the state by an equation of the form zk = Hxk + vk ,
where H is the measurement sensitivity matrix and vk is
the measurement noise.
The optimal linear estimate is equivalent to the general (nonlinear) optimal estimator if the random variables
x and z are jointly Gaussian. Therefore, it sufces to seek
an updated estimate x k (+) (observation zk is included in the
estimate) that is a linear function of the a priori estimate
and the measurement z:
beyond the time of the observations:
This is a relatively minor distinction, and the differences

between the respective estimators are correspondingly
slight.
A straightforward and simple approach using orthogonality principles is used in the derivation of estimators.
These estimators will have minimum variance and the unbiased and consistent. Interested readers may refer to Refs.
212.
The Kalman lter can be characterized as an algorithm
for computing the conditional mean and covariance of the
probability distribution of the state of a linear stochastic system with uncorrelated Gaussian process and measurement noise. The conditional mean is the unique unbiased estimate. It is propagated in feedback form by a
system of linear differential equations or by the corresponding discrete-time equations. The conditional covariance is propagated by a nonlinear differential equation
where x k () is the a priori estimate (observation zk is not

included in the estimate) of xk and x k (+) is the a posteriori
value of the estimate.
k are as yet unknown.
The weighting matrices K1 k and K
k such that the new esWe seek those values of K1 k and K
timate x k (+) will satisfy the orthogonality principle. This
orthogonality condition can be written in the form
Equations Table 2 and for x k (+) from Eq. (9) into Eq. (10),
then we will observe from Eqs. 1 and 2 that the data z1 , . . . ,
zk do not involve the noise term wk . Therefore, because the
random sequences wk and vk are uncorrelated, it follows
that Ewk zT i = 0 for 1 i k.
Nonlinear Filters
Using this result, we can obtain the following relation:
Substitute for xk , x k (+) and zk from Eqs. <xref

target="W1020-mdis-0001 W1020-mdis-0009"/>, and <xref
target="W1020-mdis-0020"/>, respectively. Then
It can be shown that E(wk1 zT i ) = 0 because wk is a white

noise sequence. But because zk = Hk xk + vk , Eq. (12) can be
rewritten as
However, by the system structure

We also know that Eq. (11) holds at the previous step, that
is
Substituting for K1 k , zk , and xk (),
and
Let
because vk is a white noise sequence uncorrelated with wk
white noise i = 1, . . . k 1. Then Eq. (13) can be reduced to
the form
be the error covariance matrix before update. Then
and therefore
Let
Equation (16) can be satised for any given xk if
Clearly, this choice of K1 k causes Eq. (9) to satisfy a portion

k is chosen such
of the condition given by Eq. (10) where K
that Eq. (11) is satised.
Let
Vectors xk (+) and xk () are the estimation errors after

and before updates, respectively.
From Eq. (11)
and also [subtract Eq. (11) from Eq. (21)]
be the error covariance matrix after update.

Substituting Eq. (17) into Eq. (9), we obtain
Subtract xk from both sides to obtain
Substituting Eq. (31) into Eq. (29),
Nonlinear Filters
k from Eq. (28),

Substituting for K
Therefore,
Let
Subtract xk from both sides
Postmultiply by xT k () both sides and take the expected

value:
Summary of Equations for the Discrete-Time Kalman

Estimator
The equations derived in the previous section are summarized in Table 3. In this formulation of the lter equations,
G has been combined with the plant covariance by multiplying Gk1 and GT k1 , for example,
The relation of the lter to the system is illustrated in the

block diagram of Fig. 2. The computational procedure for
the discrete-time Kalman estimator follows:
1. Compute Pk () with Pk1 (+), k1 , Qk1 given to initialize the procedure.
k with Pk () computed from step 1 and
2. Compute K
Hk , Rk given to initialize the procedure.
k computed from step 2 and
3. Compute Pk (+) with K
Pk () from step 1.
k from step
4. Compute x k (+) with computed values of K
3 and with given initial estimate x0 and data set zk .
Figure 3 shows a typical time sequence of values assumed by the ith component of the estimated state vector
(plotted with solid circles) and its corresponding variance
of estimation uncertainty (plotted with open circles). The
arrows show the successive values assumed by the variables, with the annotation (in parentheses) on the arrows
indicating which input variables dene the indicated transitions.
Note that each variable assumes two distinct values at
each discrete time: its a priori value corresponding to the
value before the information in the measurement is used
and the a posteriori value corresponding to the value after
the information is used.
At the beginning of the design phase of a measurement
and estimation system, when neither real nor simulated
data are available, just the covariance calculations can
be used to obtain preliminary indications of lter performance. Covariance calculations consist of solving the estimator equations with steps 13, repeatedly. It is important
to notice that the covariance calculations are independent
of data zk . Covariance calculations will involve the plant
noise matrix Q, measurement noise matrix R, state transition matrix , measurement matrix H, and initial covariance matrix Po .
Step 4 of the Kalman lter implementation [computation of x k (+)] can be implemented only for state vector propagation where simulator or real data sets are available.
In the design tradeoffs, the covariance matrix update
(steps 1 and 3) should be checked for symmetry and positive deniteness. Failure to attain either condition is a sign
that something is wrong. One possibility is that it is an illconditioned problem. In order to overcome ill-conditioning,
another equivalent expression for Pk (+) is called the Joseph
form, as shown Eq. (32)
Note that the right-hand side of this equation is the summation of two symmetric matrices. The rst of these is positive denite and the second is nonnegative denite, thereby
making Pk (+) a positive denite matrix. Other techniques
are described in the implementation methods to alleviate
the ill-conditioning.
k and Pk (+) are not that useful (817).
Other forms for K
It can be shown that state vector update, Kalman gain,
and error covariance equations represent an asymptotically stable system; therefore, the estimate of state x k becomes independent of the initial estimate x o , Po as k is increased. It is also obvious that the Kalman gain and error
covariance equations are independent of the observation.
These equations are used for covariance analysis purposes.
THE CONTINUOUS-TIME OPTIMAL ESTIMATOR
(KALMAN-BUCY FILTER)
Analogous to the discrete-time case, the continuous-time
random process x(t) and the observation z(t) are given by
Nonlinear Filters
Figure 2. Block diagram of discrete-time Kalman lter.
This section provides a formal derivation of the

continuous-time Kalman estimator. A rigorous derivation
can be achieved by using the orthogonality principle as in
the discrete-time case. In view of the main objective (to
obtain efcient and practical estimators), less emphasis is
placed on continuous-time estimators, except in academia.
Let t be the time interval [tk tk1 ]. The following relationships are used:
where 0(t2 ) consists of terms with powers of t greater

than and equal to two.
Discrete measurement noise covariance in terms of continuous covariance is given by
Figure 3. Sequence of values of lter variable in discrete time.
Discrete process noise covariance in terms of continuous

covariance is given by
Equations <xref target="W1020-mdis-0034"/> and <xref

target="W1020-mdis-0037"/> can be combined. By substituting these relations, we can get the result
where F(t), G(t), H(t), Q(t), and R(t) are n n, n n, l n,
n n, and l l matrices, respectively. The covariance matrices Q and R are positive denite.
It is desired to nd the estimate of n state vector x(t)

represented by x (t), which is a linear function of the measurements z(t)o t T which minimizes the scalar equation
where M is a symmetric positive denite matrix. The initial

estimate and covariance matrix are x 0 and P0 .
The Kalman gain of Eq. (28) becomes, in the limit,
Nonlinear Filters
Substituting Eq. (50) into Eq. (51) and taking the limit as
t 0, we obtain the desired result
with P(t0 ) as the initial condition. This is called the matrix

Riccati differential equation. Methods for solving it will be
discussed in the next section. The differential equation can
be rewritten by using the identity
to transform Eq. (52) to the form
In similar fashion, the state vector update equation can

be derived from Eqs. 30 and 35 by taking the limit as t 0
to obtain the differential equation for the estimate:
with
initial
condition
x (0).
Equations
<xref
target="W1020-mdis-0051 W1020-mdis-0054"/>, and <xref
target="W1020-mdis-0055"/> dene the continuous-time
Kalman estimator, which is also called the Kalman-Bucy
lter (119).
The Wiener lter is dened for stationary systems in
continuous time, and the Kalman lter is dened for either stationary or nonstationary systems in either discrete
time or continuous time, but nite state dimension. To
demonstrate the connections on problems satisfying both
sets of constraints, like the continuous-time Kalman Bucy
estimator Eqs. <xref target="W1020-mdis-0051"/>, <xref
target="W1020-mdis-0054"/>, and <xref target="W1020mdis-0055"/>, letting F, G, H be constants; the noises be
stationary (Q and R constants), and the lter reach steady
state (P constant). That is, as t , then P(xt) 0. The
Riccati differential Eq. (54) becomes the algebraic Riccati
equation for continuous time systems. Taking the Laplace
of Eq. (55) leads in a transfer function with constant gain K

and represents the steady state Kalman-Bucy lter, which
is identical to the Wiener lter (3).
NONLINEAR ESTIMATION
Linear estimators for discrete and continuous systems
have been derived. The combination of functional linearity,
quadratic performance criteria, and Gaussian statistics is
essential to this development. The resulting optimal estimators are simple in form and powerful in effect.
Many dynamic systems and sensors are not absolutely
linear, but they are not far from it. Following the considerable success enjoyed by linear estimation methods on liner
problems, extensions of these methods were applied to such
nonlinear problems. This section investigates the model extensions and approximation methods used for applying the
methodology of Kalman ltering to these slightly nonlinear problems. More formal derivations of these nonlinear
lters and predictors can be found in References 2, 4, and

79.
Suppose that a continuous or discrete stochastic system
can be represented by nonliner plant and measurement
models as shown in Table 4. Although afne (i.e., linear
and additive) transformations of Gaussian random variables have Gaussian distributions, the same is not always
true in the nonlinear case. Consequently, it is not necessary that w and v be gaussian. They may be included as
arguments of the nonlinear functions f and h, respectively.
However, the initial value x0 may be assumed to be a gaussian random variate with known mean and known n n
covariance matrix P0 . The objective is to estimate xk or x(t)
to satisfy a specied performance criterion (as given previously). Applying linearization techniques (comparison with
Taylor series expansion and discarding the 2nd and higher
order terms) to get simple approximate solutions to nonlinear estimation problems requires that f and h are twice
continuously differentiable (7, 20).
Linearization About a Nominal Trajectory
A trajectory is a particular solution of a stochastic system,
with a particular instantiation of the random variates involved. The trajectory is a vector-valued sequence {xk |k = 0,
1, 2, 3, . . . } for discrete-time systems, and a vector-valued
function x(t), 0 t, for continuous-time systems.
The term nominal in this case refers to that trajectory
obtained when the random variates assume their expected
values. For example, the sequence {xN k } obtained as a solution of the equation
with zero process noise and with the mean xN 0 as the initial
condition would be a nominal trajectory for a discrete-time
system.
The word perturbation has been used by astronomers
to describe a minor change in the trajectory of a planet
(or any free-falling body) due to secondary forcessuch as
those produced by other gravitational bodies. Astronomers
had learned long ago that the actual trajectory can be accurately modeled as the sum of the solution of the two-body
problem (which is available in closed form) and a linear
dynamic model for the perturbations due to the secondary
forces. This technique also works well for many other nonlinear problems, including the problem at hand. In this
case, the perturbations are due to the presence of random
process noise and errors in the assumed initial conditions.
If the function f in the previous example is continuous,
then the state vector xk at any instant on the trajectory will
vary smoothly with small perturbations of the state vector
xk1 at the previous instant. These perturbations are the
result of off-nominal (i.e., off-mean) values of the random
variates involved. These random variates include the initial value of the state vector x0 , the process noise wk , and
(in the case of the estimated trajectory) the measurement
noise vk .
If f is continuously differentiable, then the inuence of
the perturbations on the trajectory can be represented by a
Taylor series expansion about the nominal trajectory. The
likely magnitudes of the perturbations are determined by
Nonlinear Filters
the variances of the variates involved. If these perturbations are sufciently small relative to the higher-order coefcients of the expansion, then we can obtain a good approximation by ignoring terms beyond some order. (However,
we must usually evaluate the magnitudes of the higherorder coefcients before making such an assumption.)
Let the symbol denote perturbations from the nominal
Let h be sufciently differentiable, then the measurement can be represented by a Taylor series:
or
so that the Taylor series expansion of f(x, k 1) with respect

to x at x = xN k1 is
If the higher-order terms in this expansion can be ignored,
then we can represent the perturbation in zk as
where the rst-order variational term is
or
which is an l n constant matrix.

In the continuous case, the corresponding nonlinear differential equations for plant and observation are
If the higher-order terms in x can be neglected, then
where the rst-order approximation coefcients are given

by
a n n constant matrix.
with the dimensions of the vector quantities the same as

in the discrete case.
Similar to the case of the discrete system, the linearized
differential equations can be derived as
Equations <xref target="W1020-mdis-0075"/> and <xref

target="W1020-mdis-0077"/> represent linearized continuous model equations. The variables x (t) and z (t) are the
Nonlinear Filters
perturbations about the nominal values as in the discrete

case (3).
Linearization About the Estimated Trajectory
The problem with linearization about the nominal trajectory is that the deviation of the actual trajectory from the
nominal trajectory tends to increase with time. As the deviation increases, the signicance of the higher-order terms
in the Taylor series expansion of the trajectory also increases.
A simple but effective remedy for the deviation problem is to replace the nominal trajectory with the estimated
trajectorythat is, to evaluate the Taylor series expansion
about the estimated trajectory. If the problem is observable
(as evidenced by the covariance of estimation uncertainty
getting smaller until it reaches a steady state), then the
deviations between the estimated trajectory (along which
the expansion is made) and the actual trajectory will remain sufciently small that the linearization assumption
is valid (7, 20).
The principal drawback to this approach is that it tends
to increase the real-time computational burden. Even
for linearization about a nominal trathough , H, and K
jectory may have been precomputed ofine, they must be
computed in real time as functions of the estimate for linearization about the estimated trajectory.
The only modication required is to replace xN k1 by
x k1 and xN k by x k in the evaluations of partial derivatives.
Now the matrices of partial derivatives become
and
Figure 4. Estimator linearized about a nominal state.
The models and implementation equations of the extended Kalman lter are summarized in Table 6. The last
three equations in this table are the same as those for the
standard Kalman lter, but the other equations are noticeably different from those of the linearized Kalman lter
in Table 5.
It has been said that modeling is the hardest part of
Kalman ltering. This is especially true when there are
nonlinearities in the physical equation that must be linearized. Developing a good Kalman lter model is partly
art and partly science. As a general rule, we look for
models that are simple enough to be implemented but, at
the same time, still represent the physical situation with
a reasonable degree of accuracy (3).
THE MATRIX RICCATI DIFFERENTIAL EQUATION
In order to implement a Kalman lter, the Riccati equation
must be solved. This section presents a brief discussion of
solution methods for the Riccati differential equation for
the Kalman-Bucy lter. A more thorough treatment of the
Riccati equation can be found in Ref. 21.
Transformation to a Linear Equation
The matrices have the same general form as for linearization about a nominal trajectory, except for the evaluations
of the partial derivatives:
and
Linearized and Extended Kalman Filters. The block diagram of Fig. 4 shows the data ow of the estimator linearized about a nominal trajectory of the state dynamics.
Note that the operations within the dashed box have no
inputs. These are the computations for the nominal trajectory. Because they have no inputs from the rest of the estimator, they can be precomputed ofine. The models and implementation equations for the linearized discrete Kalman
lter are summarized in Table 5. Note that the last three
equations in this table are identical to those of the standard Kalman lter.
The Riccati differential equation was rst studied in the

eighteenth century as a nonlinear scalar differential equation, and a method was derived for transforming it to a linear matrix differential equation. That same method works
when the dependent variable of the original Riccati differential equation is a matrix. That solution method is derived here for the matrix Riccati differential equation of
the Kalman-Bucy lter. An analogous solution method for
the discrete-time matrix Riccati equation of the Kalman
lter is derived in the next section.
A matrix product of the sort AB1 is called a matrix fraction, and a representation of a matrix N in the form
will be called a fraction decomposition of N. The matrix A

is the numerator of the fraction, and the matrix B is its
denominator. It is necessary that the matrix denominator
be nonsingular.
The Riccati differential equation is nonlinear. However,
a fraction decomposition of the covariance matrix results
in a linear differential equation for the numerator and
denominator matrices. The numerator and denominator
matrices will be functions of time, such that the product
10
Nonlinear Filters
A(t)B1 (t) satises the matrix Riccati differential equation

and its boundary conditions.
By taking the derivative of the matrix fraction A(t)B1 (t)
with respect to t and using the fact that
we can arrive at the following decomposition of the matrix

Riccati differential equation:
Solution of the Algebraic Riccati Equation

We have seen in the previous subsections the difculty
of obtaining a solution of the general Riccati differential
equation in closed form (i.e., as a formula in the parameters of the mode), even for the simplest (scalar) problem.
There is no general formula for solving higher-order polynomial equations (i.e., beyond quartic).This is at the limit
of complexity for nding closed-form solutions to algebraic
Riccati equations by purely algebraic means. Beyond this
relatively low level of complexity, it is necessary to employ
numerical solution methods. Numbers do not always provide us as much insight into the characteristics of the solution as formulas do, but they are all we can get for most
problems of practical signicance.
The MacFarlane-Potter-Fath Eigenstructure Method
MacFarlane, Potter, and Fath discovered (independently)
that the solution P() of the continuous-time form of the
steady state matrix Riccati differential equation can be expressed in the form (22)
where the matrices A and B are n n and the 2n-vectors

eik are characteristic vectors of the continuous-time system
Hamiltonian matrix
This can be formalized in somewhat greater generality as

a lemma.
Combining Eqs. 88 and 89 gives
The last equation is a linear rst-order matrix differential

equation. The dependent variable is a 2n n matrix, where
n is the dimension of the underlying state variable.
The Hamiltonian matrix is given by
the initial values of A(t) and B(t) must also be constrained

by the initial value of P(t). This is easily satised by taking
A(t0 ) = P(t0 ) and B(t0 ) = I, the identity matrix.
In the time-invariant case, the Hamiltonian matrix is
also time-invariant. As a consequence, the solution for the
numerator A and denominator B of the matrix fraction can
be represented in matrix form as the product
where et is a 2n 2n matrix.
Lemma 1 If A and B are n n matrices such that B is

nonsingular and
for a n n matrix D, then P = AB1 satises the steady state

matrix Riccati differential equation
Proof Equation (95) can be written as two equations,
If we multiply both of these on the right by B1 and the

last of these on the left by AB1 , we obtain the equivalent
equations
Nonlinear Filters
or taking the differences of the left-hand sides and substituting P for AB1
11
is a solution at time tk+1 , where
which was to be proved.

In the case that A and B are formed in this way from
n characteristic vectors of c , the matrix D will be a diagonal matrix of the corresponding characteristic values.
Therefore, to obtain the steady state solution of the matrix
Riccati differential equation by this method, it sufces to
nd n characteristic vectors of c such that the corresponding B-matrix is nonsingular. As will be shown in the next
section, the same trick works for the discrete-time matrix
Riccati equation.
Proof The following annotated sequence of equalities

starts with the product Ak+1 B1 k+1 as dened and proves
that it equals Pk+1 :
The Matrix Riccati Equation in Discrete Time

The representation of the covariance matrix as a matrix fraction is also sufcient to transform the nonlinear
discrete-time Riccati equation for the estimation uncertainty into a linear form. The discrete-time problem differs from the continuous-time problem in two important
aspects:
1. The numerator and denominator matrices will be
propagated by a 2n 2n transition matrix, and not
by differential equations. The approach is otherwise similar to that for the continuous-time Riccati
equation, but the resulting 2n 2n state transition
matrix for the recursive updates of the numerator
and denominator matrices is a bit more complicated
than the coefcient matrix for the linear form of the
continuous-time matrix Riccati equation.
2. There are two distinct values of the discrete-time covariance matrix at any discrete time-stepthe a priori value and the a posteriori value. The a priori value
is of interest in computing Kalman gains, and the a
posteriori value is of interest in the analysis of estimation uncertainty.
The linear equations for matrix fraction propagation of the
a priori covariance matrix are derived later. The method is
then applied to obtain a closed-form solution for the scalar
time-invariant Riccati equation in discrete time and to obtain a method for exponential speedup of convergence to
the asymptotic solution.
Lemma 2 If the state transition matrices k are nonsingular and
is a nonsingular matrix solution of the discrete-time Riccati

equation at time tk , then
where the Hemes inversion formula is given in Ref. 3.

This completes the proof.
This lemma is used later to derive a closed-form solution
for the steady state Riccati equation in the scalar timeinvariant case.
The MacFarlane-Potter-Fath Eigenstructure Method
The method presented for the steady state solution of the
time-invariant matrix Riccati differential equation (i.e., in
continuous time) also applies to the Riccati equation in discrete time (22). As before, it is formalized as a lemma.
Lemma 3 If A and B are n n matrices such that B is
nonsingular and
12
Nonlinear Filters
for a n n nonsingular matrix D, then P = AB1 satises

the steady state discrete-time matrix Riccati equation
Proof If Pk = AB1 , then it was shown in Lemma 2 that

1 where
Pk+1 = AB
linear rst-order differential equations. We deal with only

continuous and linear differential equation models. Nonlinear models can be linearized as shown previously:
where x(t) represents the n-dimensional state vector and u

is r-dimensional deterministic control input. F(t) and G(t)
are known time-varying matrices. The objective of control
is to keep x(t) close to zero without excessive control effort.
This objective is to be achieved by minimizing the quadratic
cost function.
where
Consequently,
The solution is provided by the optimal state feedback control
0 (t) is an r n time-varying control gain matrix.

where K
0 (t) is given by
The value of K
That is, AB1 is a steady state solution, which was to be

proved.
In practice, A and B are formed from n characteristic
vectors of d . The matrix D will be a diagonal matrix of
the corresponding nonzero characteristic values.
The algebraic Riccati equation can be solved by using
MATLAB. The algebraic matrix Riccati equation for the
continuous case uses Schurs method which is slower, but
more robust when the system is not able to be diagonalized
(3, 23).
The matrix Riccati equation for the continuous case can
be converted to vector nonlinear coupled differential equations. Fourth-order Runge-Kutta (self-starting), fourthorder Adams method (non-self-starting), and AdamsMoultons predictor corrector can be used to solve these
equations (24).
CONTROLLERS, OBSERVERS, AND THE SEPARATION
PRINCIPLE
Optimal control theory was developed under the inuence
of such great researchers as Pontryagin, Bellman, Kalman,
and Bucy (2530). Kalman introduced a number of state
variable concepts. Among these were controllability, observability, optimal linear quadratic regulator (LQR), state
feedback, and optimal state estimation (Kalman ltering).
In LQR problems, the dynamics of the system to be controlled are represented by the state-space model, a set of
where the n n matrix P0 (t) is the solution of the Riccati

subject to boundary conditions at the terminal time T,

P0 (T) = M0 .
The same solving techniques can be applied as developed previously. This Riccati equation has boundary conditions given as compared to initial condition in the case
of estimation problems. For time-invariant cases, x(t) 0
as t . For this reason, a terminal weighing matrix M0
need not be included in Eq. (110) and F(t) = F, C(t) = C,
Q0 (t) = Q0 , R0 (t) = R0 . The optimal control correction is
and P0 is a constant symmetric position denite n n matrix, which is the solution of the algebraic matrix Riccati
equation
The same solving technique can be applied as developed

previously.
In implementation, the state variables, which are generally unavailable for direct measurement, would be substituted by their estimates (see Fig. 5) by an observer or
Nonlinear Filters
Figure 5. Block diagram of separation principle.
Kalman lter. The remarkable property of the LQG control

problem is that the optimal control u(t) is generated from
the estimated state x (t), generated by the Kalman lter by
means of the relationship
0 is precisely the one determined

where the gain matrix K
in the solution of the deterministic LQR [see Eq. (116)]
0 (t) x(t) under the assumption
problem. That is, u(t) = K
that the complete state vector is measured exactly. The
LQG problem solution separates into the solution of a linear quadratic deterministic problem and the solution of a
linear Gaussian estimation problem. The key theorem that
shows this property is often called the separation theorem.
The importance of the separation principle is that the
LQG regulator design procedures can be accomplished in
two separate stages: (1) the Kalman lter design and (2)
the control feedback design. This means that all results derived separately for the deterministic optimal control problem and the optimal estimation problems are still valid (20,
26).
IMPLEMENTATION METHODS
We have discussed what Kalman lters are and how they
are supposed to behave. Their theoretical performance has
been shown to be characterized by the covariance matrix
of estimation uncertainty, which is computed as the solution of a matrix Riccati differential and difference equation. A relationship between optimal deterministic control
and optimal estimation problems has been described via
the separation principle.
Soon after the Kalman lter was rst implemented
on computers, it was discovered that the observed meansquare estimation errors were often much larger than the
values predicted by the covariance matrix, even with simulated data. The variances of the lter estimation errors
were observed to diverge from their theoretical values, and
the solutions obtained for the Riccati equations were observed to have negative variances. Riccati equations should
have positive or zero variances.
Current work on the Kalman lter primarily focuses on
development of robust and numerically stable implementation methods. Numerical stability refers to robustness
13
against roundoff errors. Numerically stable implementation methods are called square root ltering because they
use factors of the covariance matrix of estimation uncertainty or its inverse, called the information matrix.
Numerical solution of the Riccati equation tends to be
more robust against roundoff errors if Cholesky factors of
a symmetrical nonnegative denite matrix P is a matrix
C such that CCT = P. Cholesky decomposition algorithms
solve for C that is either upper triangular or lower triangular. Another method is modied Cholesky decomposition.
Algorithms solve for diagonal factors and either a lower triangular factor L or an upper triangular factor U such that
P = UDu UT = LDL LT where DL and Du are diagonal factors
with nonnegative diagonal elements. Another implementation method uses square root information lters that use a
symmetric product factorization of the information matrix
P1 .
Alternative Kalman lter implementations use these
factors of the covariance matrix (or its inverse) in three
types of lter operations: (1) temporal updates, (2) observation updates, and (3) combined updates (temporal and
observation). The basic algorithm methods used in these
alternative Kalman lter implementations fall into four
general categories. The rst three of these categories are
concerned with decomposing matrices into triangular factors and maintaining the triangular form of the factors
through all the Kalman ltering operation. The fourth
category includes standard matrix operations (multiplication, inversion, etc.) that have been specialized for triangular matrices. These implementation methods have succeeded where the conventional Kalman lter implementations have failed (3, 31).
Even though uses are being explored in virtually every discipline, research is particularly intense on successful implementation of Kalman ltering to global positioning systems (GPS), inertial navigation systems (INS), and
guidance and navigation. GPS is a satellite-based system
that has demonstrated unprecedented levels of positioning accuracy, leading to its extensive use in both military
and civil arenas. The central problem for GPS receivers is
the precise estimation of position, velocity, and time based
on noisy observations of satellite signals. This provides an
ideal setting for the use of Kalman ltering. GPS technology is used in automobile, aircraft, missiles, ships, agriculture, and surveying.
In 1995 the United States began development of the
Wide Area Augmentation System (WAAS) under the auspices of the Federal Aviation Administration (FAA) and
the Department of Transportation (DOT), to provide precision approach capability for aircraft. Without WAAS, ionospheric disturbances, satellite clock drift, and satellite orbit errors cause too much error in the GPS signal for aircraft to perform a precision landing approach. Additionally,
signal integrity information as broadcast by the satellites
is insufcient for the demanding needs of public safety in
aviation. WAAS provides additional integrity messages to
aircraft to meet these needs.
WAAS includes a core of approximately twenty-ve wide
area ground reference stations (WRS) positioned throughout the United States which have precisely surveyed coordinates. These stations compare the GPS signal measure-
14
Nonlinear Filters
ments with the measurements that should be obtained at

the known coordinates. The WRS send their ndings to a
WAAS master station (WMS) using a land-based communications network, and theWMS calculates correction algorithms and assesses the integrity of the system. The WMS
then sends correction messages via a ground uplink system
(GUC) to geostationary (GEO) WAAS satellites covering
the United States. The satellites in turn broadcast the corrections on a per-GPS satellite basis at the same L, 1575.42
MHz frequency as GPS. WAAS-enabled GPS receivers receive the corrections and use them to derive corrected GPS
signals which enable highly accurate positioning.
On July 10, 2003, the WAAS system was activated
for general aviation, covering 95% of the United States
and portions of Alaska. In September 2003, improvements
enabled WAAS-enabled aircraft to approach runways to
within 250 feet altitude before requiring visual control.
Currently, there are two Inmarsat III GEO satellites serving the WAAS area, the Pacic Ocean Region (POR) satellite, and the East Atlantic Ocean Region (AOR-W) satellite.
In March 2005, two additional WAAS GEO satellites
were launched (PanAmSat Galaxy XV and Telesat (Anik
F1R), and are planned to be operational in 2006. These
satellites plus the two existing satellites will improve coverage of North America and all but the northwest part of
Alaska. The four GEO satellites will be positioned at 54 ,
107 , and 133 West longitude, and at 178 East longitude.
In 2006, WAAS is projected to be available over 99% of
the time, and its coverage will include the full continental United States and most of Alaska. Although primarily intended for aviation applications, WAAS will be useful for improving the accuracy of any WAAS-enabled GPS
receiver. Such receivers are already available in low-cost
handheld versions for consumer use.
Positioning accuracy using WAAS is currently quoted at
less than 2 meters of lateral error and less than 3 meters
of vertical error, which meets the aviation Category I precision approach requirement of 16 meters lateral error and
4 meters vertical error. Kalman lters are an integral part
of the WAAS system (32).
Kalman lters are used in bioengineering, trafc systems, photogrammetry, and myriad process controls. The
Kalman lter is observer, parameter identier in modeling, predictor, lter, and smoother in a wide variety of applications. It has become integral to twenty-rst century
technology.
BIBLIOGRAPHY
1. R. E. Kalman, A new approach to linear ltering and prediction problems, ASME J. Basic Eng., 82: 3445, 1960.
2. A. Gelb et al., Applied Optimal Estimation, Cambridge, MA:
MIT Press, 1974.
3. M. S. Grewal, A. P. Andrews, Kalman Filtering Theory & Practice Using MATLAB (Second Edition), New York, NY: Wiley &
Sons, 2001 (8th printing).
4. B. D. O. Anderson, J. B. Moore, Optimal Filtering, Englewood
5. A. E. Bryson, Jr., Y. C. Ho, Applied Optimal Control, Waltham,
MA: Blaisdell, 1969.
6. R. S. Bucy, Optimal ltering for correlated noise, J. Math Anal.

Appl., 20: 18, 1967.
7. M. S. Grewal et al., Application of Kalman Filtering to GPS,
INS, and Navigation, Anaheim, CA: KFCA,Jan. 2007.
8. A. H. Jazwinski, Stochastic Processes and Filtering Theory,
New York: Academic, 1970.
9. P. S. Maybeck, Stochastic Models, Estimation, and Control,
New York: Academic, 1979, Vol. 1.
10. P. S. Maybeck, Stochastic Models, Estimation, and Control,
New York: Academic, 1982, Vol. 2.
11. J. M. Mendel, Kalman ltering and other digital estimation
techniques, IEEE Individual Learning Package, 1987.
12. H. W. Sorenson (ed.), Kalman Filtering: Theory and Application, New York: IEEE Press, 1985.
13. M. A. Poubelle et al., A miscellany of results on an equation
of Count J. F. Riccati, IEEE Trans. Autom. Control, AC-31:
651654, 1986.
14. J. F. Riccati, Animadversationnes in aequationes differentiales secundi gradus, Acta Eruditorum Quae Lipside Publicantur Suppl., 8: 6673, 1724.
15. L. Dieci, Numerical integration of the differential Riccati
equation and some related issues, SIAM J. Numer. Anal., 29:
781815, 1992.
16. T. Kailath, R. A. Geesey, An innovations approach to least
squares estimation. Part IV: Recursive estimation given
lumped covariance functions, IEEE Trans. Autom. Control AC16: 720726, 1971.
17. T. Kailath, An innovations approach to least squares estimation. Part I: Linear ltering in additive white noise, IEEE
Trans. Autom. Control, AC-13: 646655, 1968.
18. R. E. Kalman, New methods in Wiener ltering, in Proc. First
Symposium on Engineering Applications of Random Function
Theory and Probability, New York: Wiley, 1963, Chapter 9.
19. R. E. Kalman, R. S. Bucy, New results in linear ltering and
prediction theory. ASME J. Basic Eng., Ser. D, 83: 95108,
1961.
20. M. Athans et al. (guest eds.), Special issue on linear-quadraticgaussian problem, IEEE Trans. Autom. Control, AC-16:
529552, 1971.
21. S. Bittanti, A. J. Laub, J. C. Willems (eds.), The Riccati Equation, New York: Springer-Verlag, 1991.
22. A. G. J. MacFarlane, An eigenvector solution of the optimal
linear regulator, J. Electron. Control, 14: 643654, 1963.
23. R. G. Brown, P. Y. C. Hwang, Introduction to Random Signals
and Applied Kalman Filtering, 3rd ed., New York: Wiley, 1997.
24. M. L. James, G. M. Smith, J. C. Wolford, Applied Numerical
Methods for Digital Computation, 4th ed. New York: HarperCollins College Publishers, 1993.
25. L. S. Pontrjagin, On the zeros of some elementary transcendental functions, Izv. Akad. Nauk SSSR, Ser. Mat., 6: 115134,
1942; Eng. Transl.: Amer. Math. Soc. Transl., 2: 95110, 1955.
26. R. E. Bellman, The theory of dynamic programming, Proc.
Natl. Acad. Sci. USA, 38: 716719, 1954.
27. R. E. Bellman, Introduction to the Mathematical Theory of
Control Processes, New York: Academic, 1967, Vol. 1.
28. R. E. Kalman, Contribution to the theory of optimal control,
Bol. Soc. Matem. Mex., pp. 102119, 1960.
29. R. E. Kalman, On the general theory of control systems, Proc.
1st IFAC Cong., Moscow, 1960, Vol. 1, pp. 481492, 1961.
30. R. S. Bucy, P. D. Joseph, Filtering for Stochastic Processes, with
Applications to Guidance, New York: Wiley, 1968.
Nonlinear Filters
31. G. J. Bierman, Factorization Methods for Discrete Sequential
Estimation, New York: Academic, 1977.
32. M. S. Grewal, L. R. Weill, and A. P. Andrews, Global Positioning Systems, Inertial Navigation, and Integration (Second Edition), New York, NY: Wiley & Sons, 2007.
MOHINDER S. GREWAL
California State University,
Fullerton Fullerton, CA,
92634
15
16
Nonlinear Filters
LARGE-SCALE AND DECENTRALIZED SYSTEMS
209

The size of a system is a subjective notion, and so is the notion of large-scale systems. In the following we will take a
pragmatic view, and consider a system large whenever it is
conceptually or computationally attractive to decompose it
into interconnected subsystems. Typically the subsystems are
of small size and can be solved efficiently. The individual solutions can then be combined in some way to obtain a solution
for the overall system.
If the system is decomposed along the boundaries of the
physical subsystems, the subsequent analysis may produce
important information about the interplay between subsystems behavior and the nature of interconnections. It may,
however, be computationally inefficient. To reduce the computational effort, one should develop universal decomposition
210
techniques, which need not be constrained by physical boundaries of subsystems, but may lead to efficient solutions of
large problems utilizing modern computer architectures. With
that in mind, our first objective will be to describe several
decomposition schemes that can either be used as preconditioners for decentralized control design, or can serve to speed
up computations involving the control of large-scale systems
(usually through the use of parallel processing).
To efficiently decompose a large-scale system, it is generally convenient to represent it in the form of a graph. Depending on whether or not the graph is weighted, we can
roughly distinguish between two broad classes of decomposition algorithms. In problems where weights are not assigned
to edges of the graph, decomposition schemes typically exploit
topological properties such as structure and sparsity to obtain
an appropriate partitioning (18). Whenever possible, it is
also useful to incorporate any existing information regarding
the physical attributes of the system (such as hierarchical
structures or repetitive blocks that are built into the design).
Decomposition algorithms of this type are commonly applied
for solving large sparse systems of linear equations, and a
number of them have been successfully utilized in parallelizing control-related computational problems.
A conceptually different class of algorithms arises when
the system is viewed as a weighted graph. In this case the
partitioning strategy changes significantly, since we can now
utilize edge weights to identify weakly coupled subsystems
and establish hierarchical relationships between them. Decomposition schemes based on this approach have found numerous applications not only in parallel computing, but also
in decentralized control design. In the following sections particular attention will be devoted to the epsilon decomposition
algorithm (911), due to its efficiency and simplicity; in this
context, we will also examine the concept of overlapping, and
its application to the control of large-scale systems.
Our second objective in this paper will be to provide the
motivation and describe the basic ideas and techniques for
decentralized control of dynamic systems. The accumulated
experience in controlling large complex systems suggests
three basic reasons for using decentralized control structures:
dimensionality, information structure constraints, and uncertainty (6). By decomposing a system of large dimension into
subsystems, a designer can devise decentralized strategies for
solving control problems that would be either impractical or
impossible to solve using a single centralized controller. Furthermore, in a large complex system where databases are developed around the plant with distributed sources of data, a
need for fast control actions in response to local inputs and
perturbations dictates use of distributed (that is, decentralized) measurement and control structures.
A restriction on what and where the information is delivered in a large system is a common structural constraint in
building controllers and estimators. A good example is the
standard automatic generation control in power systems,
where the decentralized schemes are used to reduce the cost
of communication that would be demanded by a centralized
control strategy spread over distant geographical areas.
In modeling and control of large systems, it has been long
recognized that models of subsystems can be obtained with
increasing levels of accuracy and versatility. The essential uncertainty resides in the interconnections between the subsystems, since these interconnections are often poorly known in
x1
y1
x2
y2
x3
y3
Figure 1. Bipartite graph B.
both deterministic and stochastic terms. One of the reasons

for using decentralized control strategies is their inherent robustness to a wide variety of structured and unstructured
perturbations in the subsystems and their interconnections.
Furthermore, the strategies can be made reliable with respect
to both interconnection and controller failures involving individual subsystems.
Epsilon Decompositions
A natural way to introduce epsilon decompositions is to consider the system of linear algebraic equations
Ax = b
(1)
and its solution by the Jacobi iterative method (12). In this

context, the epsilon decomposition algorithm will be used to
permute matrix A into a form that ensures rapid convergence
to the solution.
The algorithm itself is remarkably simplegiven a matrix
A (aij) and a value of parameter 0, all elements satisfying aij are set to zero. The resulting sparsified matrix
is then permuted into a block-diagonal form, and all the variables in the same block are considered to be strongly coupled.
After such a permutation, the matrix A can be represented as
A = AD + AC
(2)
where AD is block-diagonal and all elements of AC are less

than or equal to one in magnitude. The following example
illustrates how such a permutation can be identified, and subsequently utilized to iteratively solve Eq. (1).
Example 1. Consider the matrix
1
0.05
A = 0.01
1
0.1
0.1
0.1
2
(3)
and the corresponding bipartite graph B in Fig. 1, in which

vertices yi and xj are connected if and only if aij 0. If we
remove all edges that correspond to elements aij 0.1, we
obtain the subgraph B shown in Fig. 2. It is easily seen that
x1
y1
x2
y2
x3
y3
Figure 2. Subgraph B.
x1
y1
x3
y3
x2
y2
211
it is easily verified that no permutation can produce weakly

coupled diagonal blocks. On the other hand, if repeated verti in Fig. 5,
ces are allowed, we obtain the expanded bigraph B
which now has two disconnected components. This transformation corresponds to a rectangular permutation matrix V,
which is uniquely defined by the ordering of x and y vertices
in the graph. Specifically,
VA = AV
Figure 3. Components of B.
(8)
where
A =

---------
---------
the vertices and edges of B can now be regrouped into two

disconnected components, as indicated in Fig. 3. The permutation defined by the vector p (1 3 2) now produces a matrix which satisfies Eq. (2), with 0.1 and
1
2 0
0
0 0.5
AD = 0.1
AC = - 0
(4)
- - - - -0- - - - -1- -
- - - - - -2- - - -0- ,
0
0 1
0.1 1
0
(k = 0, 1, . . .)
1
0
V =
0
0
(5)
---------
Namely, if the original matrix A of Eq. (3) is partitioned so

that
1
0.05 0
AD = -0.01
(6)
- - - - - - - -1- - - - - -0-
0
0
2
we obtain I A1
D A2 1, and the process diverges; on the
other hand, the partitioning in Eq. (4) obtained by epsilon
decomposition results in rapid convergence.
Overlapping Epsilon Decompositions. Given a matrix A and
a particular choice of parameter , there is no guarantee that
A can be permuted into the form shown in Eq. (2). The obvious remedy in such cases is to repeat the decomposition with
a larger value of ; alternatively, we can use the concept of
overlapping. The following example illustrates the basic ideas
behind overlapping epsilon decomposition.
Example 2. Consider the following matrix
A =

0
1
0
1
0
0
1
0
(10)
Scaling. Another issue that arises in the practical application of epsilon decompositions is that elements of the matrix
A can widely vary in size. In such cases it may not be possible
to find a meaningful value for , and row scaling needs to be
utilized to obtain a more uniform distribution of element values. This process is demonstrated by the following example.
Example 3. Let matrix A be
10
A = 0.1
4.5
defined as
5
0.3
0.2 0.02
1
100
(11)
In this case 0.3 is obviously not a feasible choice, since

the entire second row would be eliminated. However, if each
row is scaled by the element with the maximal absolute value,
we obtain
A = AD + AC
x1
x1
y1
x2(1)
x2
y2
x3
x3
y3
x2(2)
Figure 4. Subgraph B.
(9)
now has an
It is easily verified that the expanded matrix A
epsilon decomposition in the sense of Eq. (2).
(7)
where all entries larger than in magnitude are denoted by

. The corresponding bipartite graph B is given in Fig. 4, and
0
0

and
The impact of weak coupling is easily seen when Eq. (1) is

solved using the Jacobi iterative method
xk+1 = xk A1
D (Axk b)
0
0
(12)
y1
y2(1)
y3
y2(2)
.
Figure 5. Expanded subgraph B
212
where 0.1 and
---------
1
0.5 0
AD = -0.5
- - - - - - -1- - - - 0
- - ,
0
0
1
0
0
0.3
AC = - -0- - - - - -0- - - - - -1- -

0.45 0.1
0
---------
(13)
Note that in this case A can be represented only as A

AD AC (without ), but scaling does identify block-diagonal
dominance, since
1
A1
D AC
=
AD AC
1
Figure 7. The undirected graph.
(14)
This feature is frequently used to enhance the convergence of

the iterative process in Eq. (5).
Structural Decompositions
By their very nature, epsilon decompositions are ideally
suited for iterative solutions of Eq. (1). In contrast, there is
an entire class of decompositions that are aimed at solving
Eq. (1) directly; in this approach, no matrix entries are discarded, and the decompositions are designed to achieve certain desirable structures. A structure that is of particular interest in the analysis and simulation of large-scale systems is
the bordered block-diagonal (BBD) form shown in Fig. 6. The
appeal of this structure lies in its inherent potential for parallel computation, a feature that has been widely exploited in
different areas of engineering.
Numerous algorithms have been developed for permuting
a matrix into the BBD form, based on diverse concepts ranging from node clustering (2) and diakoptics (1,13) to various
forms of graph dissection (3,4,7). Despite their obvious differences, all these methods have a common goal in identifying a
minimal border that induces a block-diagonal structure in the
rest of the matrix. The border size is an important issue in
the solution of the system in Eq. (1), since the computation
time depends heavily on the number of nonzero elements in
the border.
In this section, we will briefly describe three methods for
obtaining BBD structures. We begin our analysis with the
classic nested dissection algorithm of George and Liu (3).
Nested Dissection
Nested dissection is a typical representative of BBD decomposition methods. It is relatively simple algorithm, in which the
matrix is assumed to be structurally symmetric and can
therefore be represented by an undirected graph. The following example illustrates the decomposition procedure.
Example 4. Let us consider the structurally symmetric matrix
1 2
0
1
2B
BB
3 B0
B0
4B
B
5B
BB
6 @
7
3 4
6 7
0
0
0
0
0
0
0
0
0
1
CC
CC
CC
CC
CC
A
(15)
and the corresponding undirected graph in Fig. 7. The basic

idea of nested dissection is to arrange the vertices of this
graph into a rooted level structure, such as the one in Fig. 8.
To achieve optimal results, this structure should have as
many levels as possible, which is assured by choosing an appropriate vertex as the root (14,15).
Once a rooted level structure is formed, the nested dissection algorithm identifies the middle level (in this example,
vertex 2) and removes it from the graph, so that two disconnected components are left. The middle level is then placed in
the border, as illustrated in Eq. (16). In general, this proce-
5
1
Level 0
6
4
7
Level 1
Level 2
Figure 6. The BBD matrix structure.
0
0
0
0
Level 3
Level 4
Figure 8. A rooted level structure.
dure can be recursively repeated on each of the remaining

blocks:
6 3
----------------------
-------------------
(16)
-----------------------0 0
The nested dissection algorithm was found to be very successful for matrices with a regular structure (such as those
arising in the numerical solution of partial differential equations using finite elements). However, this decomposition is
much less effective for systems such as electric circuits, where
the matrix structure is typically highly irregular (4). Several
different solutions where proposed to alleviate this problem,
dating back to the work of Sangiovanni-Vincentelli et al. (2).
We now describe two recent decomposition algorithms, which
have been successful over a wide range of matrix structures.
Decompositions Using Eigenvectors of Graphs. The idea of
eigenvector decompositions was introduced by Pothen et al.
(5), and is based on the Laplacian matrix of a graph. This
matrix is defined as
QDA
The Balanced Bordered Block-Diagonal Decompositions. The

balanced BBD decomposition algorithm is recursive and consists of two basic steps.
7 2
0
1
5 0 0 0 0
1B
BB 0 0 0 CCC
6 B 0 0 0 C
B0- - - -0- - - -0- - - -- - - -- - -- - - CC
3B
B
C
4B
BB0 0 0 CCC
7 @ 0 0 0 0A
2
213
(17)
where A is the adjacency matrix of the graph, and D is a diagonal matrix whose entries represent vertex degrees. It can be
shown that matrix Q is always positive semidefinite, with at
least one zero eigenvalue; the smallest positive eigenvalue of
Q is denoted 2, and the corresponding eigenvector is denoted
by X2. The decomposition procedure can now be summarized
as follows:
1. Compute eigenvector X2, and determine its median component xl.
2. Partition the vertices of the graph in the following way:
for any vertex i, if xi xl, set i A; otherwise, i B.
In this way, the vertices will be partitioned into two
approximately equal sets, A and B.
3. All the edges connecting sets A and B constitute an edge
separator, H. The objective now is to find a minimal vertex cover for H (that is, the minimal number of vertices
that need to be removed so that all edges in set H are
removed). This vertex cover constitutes the separator,
which appears in the border of the BBD structure.
4. Repeat steps 13 on the remaining components after
the separator is removed.
Decompositions based on eigenvectors of graphs were
found to be effective and applicable to a wide range of matrix
structures. However, for large matrices computing the second
eigenvector can be difficult, if not impossible. This consideration has motivated the development of the balanced BBD
decomposition (11,16), which is described next.
Step 1. Select a maximal allowable block size Nmax. Given

this choice, move as many vertices as necessary to the
border so that each block has size Nmax. A typical situation after this step is shown in Fig. 9.
Step 2. The border is obviously too large after the first
step; consequently, in step 2 we reconnect border vertices one by one. In this process, the next vertex to be
reconnected is always the one that results in the smallest increase in block sizes (such an algorithm is called
greedy). The process continues as long as there are at
least two blocks left (in other words, it terminates when
we establish that the next reconnection will result in a
single block).
Once two diagonal blocks and an initial border have been obtained, steps 1 and 2 are repeated on each block (which makes
the algorithm nested); the local borders are then added to the
initial border. This procedure continues until we obtain a desired number of diagonal blocks of approximately equal size
(hence the term balanced).
Advantages of this algorithm are its execution speed and
numerical simplicity, which result from the fact that only the
sizes of blocks are considered, not their contents. In addition,
since all diagonal blocks are balanced in size, the workload
can be evenly distributed across different processors; this feature is critical for an efficient parallel solution of Eq. (1).
To illustrate the effectiveness of this decomposition, in
Figs. 10 and 11 we show how it is applied to a highly irregular
matrix that arises in the modeling of the US electric power
network.
DECENTRALIZED CONTROL: AN EXAMPLE
The underlying idea of decentralized control is decomposition.
A dynamic system is considered as an interconnection of subsystems, which have independent inputs and outputs. A satisfactory performance of the overall system is achieved by controlling each individual subsystem using local feedback,
whereby local inputs are connected to local outputs (or
states).
Figure 9. Situation after step 1.
214

0
u1
u2
1000
2
k
l
2000
3000
Figure 12. Inverted pendulums.
1000
2000
3000
4000
5000
Figure 10. A model for the US electric power network (5300 5300).
To introduce the decentralized control problem, let us consider the two inverted pendulums interconnected by a spring
as shown in Fig. 12. The control objective is to keep the pendulums upright by using the inputs u1 and u2. After linearization around the equilibrium state 1 2 0, the equations
of motion are
ml 2 1 = mgl1 ka2 (1 2 ) + u1
ml 2 2 = mgl2 ka2 (2 1 ) + u2
(18)
-----------------
5000
-----------------
4000
When the state and input vectors are chosen as x (1, 1,

2, 2)T and u (u1, u2)T, the interconnected system is represented by the state equations
0
1
0
0
0
0
2
g
1
ka2
ka
0
0
0
2
2
2
ml
ml
l
ml
S:
x = - - - - - - - - - - - - - - - - - - - - - - - - - - - - - x + - - - - - - - - - - - - u
0
0
0
1
0
ka2
ka2
g
0
0
0
ml 2
ml 2
l
ml 2
(19)
In choosing the feedback control laws to achieve the objective, the information structure constraint is essential: Each
input u1 and u2 can depend only on local states x1 (1, 1)T
and x2 (2, 2)T of each individual pendulum. In other words,
u1 = u1 (x1 ),
u2 = u2 (x2 )
(20)
Since the system S is linear, a reasonable choice is a linear

control law
u1 = kT1 x1 ,
u2 = kT2 x2
(21)
k2 = (k21 , k22 )T
(22)
where feedback gain vectors

k1 = (k11, k12 )T ,
1000
need to be selected to stabilize S, that is, to keep the pendulums in the upright position. When the two pendulums are
considered as two subsystems

0 1
0
S1 :
x1 =
x +
u
0 1
1

(23)
0 1
0
S2 :
x2 =
x2 +
u2
0
2000
3000
4000
5000
0
1000
2000
3000
4000
5000
Figure 11. The matrix after a balanced BBD decomposition.
the overall system breaks up into two interconnected subsystems as

0 1
0
0
0
0 0
S:
x1 =
x +
u +e
x +e
x
0 1
1
0 1
0 2

0 1
0
0 0
0
0
x2 =
x +
u +e
x +e
x
0 2
2
0 1
0 2
(24)
where g/l, 1/ml2, a2k/ml2, and e (a/a)2.
By choosing the decentralized control of Eq. (21), we effectively intend to stabilize the interconnected system S by stabilizing the two subsystems in Eq. (23). This turns out to be
a robust control strategy, since it can produce an overall
closed-loop system that can remain stable despite the essential uncertainty about the height a of the spring. This uncertainty is represented by the normalized interconnection parameter e, which can take any value between 0 and 1. When
an interconnected system is stabilized in this way, it is said
to be connectively stable (6).
By using the control law of Eq. (21) in Eq. (23) we obtain
the closed-loop subsystems as
0
x1 =
k11

0
x2 =
k21
S 1 :
S 2 :

1
x
k12 1

1
x
k22 2
(25)
trices have proper dimensions, and N 1, 2, . . ., N. At

present we are interested in disjoint decompositions, that is,
x = (xT1 , xT2 , . . ., xTN )T

u = (uT1 , uT2 , . . ., uTN )T
y=

1
0
0
0
x +e
x +e
k12 1
0 1

1
0 0
0
x1 + e
x2 + e
0
k22
:
S
0
x1 =
k11

0
x2 =
k21
Rn = Rn 1 Rn 2 Rn N
Rm = Rm 1 Rm 2 Rm N
Rl = Rl 1 Rl 2 Rl N
x = AD x + BD u + AC x + BC u
y = CD x + CC x

0
x
0 2

0
x
0 2
(26)
INTERCONNECTED PLANTS AND CONTROLLERS

To describe representations of plants, which are required in
the design of decentralized controllers, let us consider a linear
system
(27)
y = Cx
xi = Ai xi + Bi ui +
yi = Ci xi +
(Aij x j + Bij u j )
jN
Cij x j ,
AD = diag{A1 , A2 , . . ., AN }
BD = diag{B1 , B2 , . . ., BN }
iN
(33)
CD = diag{C1 , C2 , . . ., CN }
and the coupling block matrices are
AC = (Aij ),
BC = (Bij ),
CC = (Cij )
(34)
The collection of N decoupled subsystems is described by

SD :
x = AD x + BD u
y = CD x
(35)
which is obtained from Eq. (32) by setting the coupling matrices to zero.
Important special classes of interconnected systems are input (BC 0) and output (CC 0) decentralized systems, where
inputs and outputs are not shared among the subsystems. Inputoutput decentralized systems are described as
x = AD x + BD u + AC x
y = CD x
(36)
where both BC and CC are zero. This structural feature helps

to a great extent when decentralized controllers and estimators are designed for large plants.
A static decentralized state feedback
as an interconnection
S:
(32)
where
S:
x = Ax + Bu
(31)
A compact description of the interconnected system S is

S:
It is interesting that the system in this example belongs to

the class of interconnected systems that can always be stabilized by decentralized feedback. A decentralized control law
can be chosen to connectively stabilize the closed-loop system
S even if the spring is shifting up and down the length of the
pendulums in an unpredictable way. The class of decentrally
stabilizable systems is described next.
S:
(30)
(yT1 , yT2 , . . ., yTN )T
where x(t) n, u(t) m, and y(t) l are the state, input,

and output of the overall system S, such that
and the overall closed-loop system becomes
215
(28)
u = KD x
jN
(37)
is characterized by a block-diagonal gain matrix
of N subsystems
Si :
xi = Ai xi + Bi ui
yi = Ci xi ,
iN
KD = diag{K1 , K2 , . . ., KN }
(29)
where xi(t) ni, ui(t) mi, yi(t) li are the state, input,
and output of the subsystem Si at a fixed time t , all ma-
(38)
which implies that each subsystem Si has its individual control law
ui = Ki xi ,
iN
(39)
216
with a constant gain matrix Ki. The control law u of Eq. (37),
which is equivalent to the totality of subsystem control laws
of Eq. (39), obeys the decentralized information structure constraint requiring that each subsystem Si is controlled using
its locally available state xi. The resulting closed-loop system
is described as
S :
x = (AD BD KD )x + AC x
(40)
When dynamic output feedback is used under decentralized

constraints, then controllers of the following type are considered:
yi and inputs ui, but are collectively responsible for the stabilization of S. The closed-loop system is

x
A BKDC BHD x
(46)
S&CD :
=
z
z
GD C
FD
The basic result of Ref. 17 states that the closed-loop system
S&CD is stabilizable by decentralized controllers Ci if and
only if the set of decentralized fixed modes
D =
(A BKDC) =
z i = Fi zi + Gi yi
ui = Hi zi Ki yi ,
(41)
iN
which can be rewritten in a compact form as a single decentralized controller defined as

CD :
z = FD z + GD y
where
u=
y=
S&CD :

x
AD BD KDCD + AC
=
z
GDCD
BD HD
FD

x
(44)
z
DECENTRALIZED FEEDBACK STRUCTURES

Before a search for stabilizing decentralized feedback begins,
it is necessary to determine whether such a feedback exists
for a given plant. It is well known that if there are no restrictions on the information structure of the linear system S of
Eq. (27), it can be stabilized if and only if the uncontrollable
or unobservable modes of the system are stable. However,
this is not the case when information constraints are present
and the plant has unstable decentralized fixed modes.
Fixed Modes
Let us consider the system S of Eq. (27) in the form
S:
x = Ax +
B i ui
iN
yi = C i x,
(43)
are the state z r, input y l, and output u m of the

controller CD. By combining the system S and the decentralized dynamic controller CD, we get the composite closed-loop
system as
(45)
iN
where only the inputs and outputs are partitioned as in Eq.

(28), but the state (and thus the matrix A) is considered as a
whole. Either the subsystems are ignored for technical reasons, or there are no natural or useful decompositions of the
system into interconnected subsystems. In this case, the controllers Ci described in Eq. (41) still use local measurements
B i KiC i
(47)
iN
C j
1
Cj
= .2
.
.
C jNP
( yT1 , yT2 , . . ., yTN )T
(uT1 , uT2 , . . ., uTN )T
lies in the open left half plane. This result includes the interconnected system of Eq. (44) in an obvious way.
A simple characterization of decentralized fixed modes was
provided in Ref. 18. For any subset I i1, . . ., iP of the
index set N , let I C j1, . . ., jNP denote the complement of
I in N , and define
B I = [B i , B i , . . ., B i ],
z=
(42)
u = HD z KD y
(zT1 , zT2 , . . ., zTN )T ,
K1 ,...,KN
KD
Ci :
C I c
(48)
Then a complex number is a decentralized fixed mode

of S if and only if

A I B I
<n
(49)
rank
0
C I c
for some I N . Thus, the appearance of a fixed mode can
be attributed to a special polezero cancellation, which cannot be removed by constant decentralized feedback. However,
under relatively mild conditions, such fixed modes can be
eliminated by time-varying decentralized feedback.
Structurally Fixed Modes
Graph theory serves as a suitable environment for both conceptual and numerical analysis of large-scale systems (6), because it allows the designer to take advantage of the special
structural features of a large system before attempting a
costly quantitative analysis. In particular, the theory is suitable for handling the lack of exact knowledge of system parameters by considering the existence rather than the true
value of a connection between any two variables in the system (19).
A graph-theoretic characterization of controllability was
introduced by Lin (20). He established that in an uncontrollable pair (A, B), loss of controllability is either due to an insufficient number of nonzero parameters (indicating a lack of
sufficient linkage among system variables), or due to a perfect
matching of system parameters. In the former case, the pair
(A, B) is structurally uncontrollable in the sense that all pairs
having the same structure as (A, B) are uncontrollable. By
describing the structure of (A, B) using a directed graph,
structural controllability can be checked via efficient graphtheoretic algorithms (6). In this context, a pair (A, B) is structurally controllable if and only if:
1. The system graph is input-reachable, that is, each state

variable can be reached along a directed path by at
least one input, and
2. The system graph has no dilation, that is, there exists
no subset of state variables whose number exceeds the
total number of all state and input variables directly
affecting these variables.
The concept of structurally fixed modes represents a generalization of the idea of structural controllability (21). Let D
(V , E ) be a directed graph associated with the system S of
Eq. (27), where V U X Y is a set of vertices corresponding to inputs, states, and outputs of S, and E is a set of
directed edges corresponding to nonzero elements of the system matrices A, B, and C. To every nonzero aij there corresponds an edge from vertex xj to vertex xi, to every nonzero
bij an edge from uj to xi, and to every nonzero cij one from xj
to yi. Given a feedback pattern K, which adds a feedback edge
from yj to ui for every kij 1, one obtains a digraph DK (V ,
E E K) that completely describes the structure of both the
system S and the feedback constraint specified by K, a special
case of which is the decentralized constraint. In this case, permissible controllers have the structure

CK :
z i = Fi zi +
gij y j
jJ i
ui =
hTi zi
kij yi
(50)
jJ i
where J i j : kij 1.
If we choose a gain matrix K to conform with the feedback
structure K, then the set

K =
(A BKC)
(51)
1. All state vertices of DK are covered by vertex disjoint

cycles, and
2. No strong component of DK contains only state vertices,
where a strong component is a maximal subgraph
whose vertices are reachable from each other.
How this graph-theoretic criterion can be used to choose a
minimum number of feedback links that avoid structurally
fixed modes is explained in Refs. 6 and 22.
STABILIZATION
It has been common practice to use decentralized control in
the stabilization of large interconnected systems. Each subsystem is stabilized independently using local feedback, and
stability of the overall system is established using the MatrosovBellman concept of vector Liapunov functions. While a
vector Liapunov function may offer more flexibility and computational simplicity than a scalar one, it remains inherently
conservative. The use of vector functions, however, has been
justified by the presence of uncertainty in the interconnections, which can cause a breakup of the system during operation along the subsystem boundaries. The method of vector
Liapunov functions is a natural tool for making decentrally
controlled systems robustly stable to interconnection failures,
that is, connectively stable (6).
Vector Liapunov Functions
Let us assume that the plant is governed by linear time-invariant equations
S:
can be conveniently specified as the set of fixed modes with
respect to the decentralized feedback structure constraint defined by K. It is fairly easy to show (6) that the system S can
be stabilized by the constrained controller CK if and only if
K is contained in the open left half plane. To characterize
K as in Eq. (49), let us consider the index sets I M 1,
2, . . ., M and replace I C by J iI C J i, where I C now
refers to the complement of I in M .
In order to formulate graph-theoretic conditions for the existence of structurally fixed modes, let us recall that two systems are said to be structurally equivalent if they have the
same graph. Then, a system S is said to have structurally
fixed modes with respect to a given feedback structure K if
every system structurally equivalent to S has fixed modes
with respect to K. Structurally fixed is a generic concept:
Having structurally fixed modes is a property of a class of
systems sharing the same graph. Most importantly, if a system has no structurally fixed modes, then it either has no
fixed modes, or the fixed modes can be removed by arbitrarily
small perturbations of system parameters. This means that if
a system has no structurally fixed modes for a given K, then
(generically) it can be stabilized by a controller with a gain K
and structure K, decentralized structure being a special case.
It was shown in Ref. 21 that a system S has no structurally
fixed modes with respect to a feedback pattern K if and only
if:
217
xi = Ai xi + Bi ui +
N
eij Aij x j ,
iN
(52)
j=1
which are an obvious derivative of Eq. (28) save for the insertion of the interconnection parameters eij [0, 1]. The parameters are coefficients of the N N interconnection matrix
E (eij), which are used to model the uncertain strength of
interconnections.
To stabilize S we use the decentralized control laws of Eq.
(39) to stabilize each individual closed-loop system
:
S
i
xi = (Ai Bi Ki )xi ,
iN
(53)
This is presumably a relatively easy task, because the subsystems have low dimensions. Stability of each decoupled closedloop system follows from a subsystem Liapunov function
vi : ni . These individual functions are then stacked up
to form a vector Liapunov function v(x) [v1(x1), v2(x2), . . .,
vN(xN)]T. Finally, the vector Liapunov function is used to form
a scalar Liapunov function V : n of the form
V (x) = d T v
(54)
where d (d1, d2, . . ., dN)T is a positive vector (di 0, i

N ). The Liapunov function V(x) is utilized to establish stabil-
218
ity of the overall closed-loop system
:
S
xi = A i xi +
N
for the system S of Eq. (27). By applying the epsilon decomposition to M and regrouping the variables, we get S as
eij Aij x j ,
iN
(55)
j=1
S:
xi = A i xi + B i u i +
N
(56)
wij =
eij ij ,
i = j
(57)
The positive numbers i depend on the choice of vis, and the

nonnegative numbers ij are bounds on interconnection matrices Aij. The binary numbers eij are elements of the N N
fundamental interconnection matrix E (eij), which define
the nominal structure of S,

eij =
1 if S j acts on Si
0 if S j does not act on Si
(58)
It is a well-known result (6,19) that:
A matrix W (wij) with nonpositive off-diagonal elements

is an M matrix if and only if there exists a positive vector d
such that the vector
T
By applying the decentralized feedback of Eq. (39), we obtain

the closed-loop system as
S :
Epsilon Decompositions
Epsilon decompositions are ideal preconditioners for the stabilization of large-scale systems using decentralized control
and vector Liapunov functions (6,9). To see this, assume that
C 0 and form the augmented matrix

B
0
xi = A i xi +
N
A ij x j ,
iN
(62)
j=1
j = i
ij Aij BijKj. In this case, the aggregate matrix

where A
W (wij) is defined as
wij =

i ,
i= j
(63)
ij , i = j
We recall (6) that if the threshold is sufficiently small, then

W is an M matrix: The smaller the absolute values of wijs,
the easier it is for W to satisfy the M-matrix conditions. We
note, however, that the smaller is, the smaller the number
of subsystems will be, implying a smaller reduction in the
dimensionality of the stability problem via decomposition.
Vector Liapunov functions provide only sufficient conditions

for stability of interconnected systems, and one may search in
vain for stabilizing control. For this reason, there are a number of results aimed at identifying classes of interconnected
systems which can always be stabilized by decentralized
feedback.
The most popular (but also the most restrictive) conditions
for decentral stabilizability are the matching conditions (23),
Im Bi Im Aij
(59)
is a positive vector as well, and stability of S follows by the

(x)S 0.
standard Liapunov theorem involving V(x) 0 and V
The connective property of stability, which requires stability
to hold for all E E, is concluded from the fact that W(E)
W(E) element by element, and W is an M matrix whenever
W is.

A
M=
0
iN
Decentrally Stabilizable Systems
The interconnected system S is connectively stabilized by

decentralized control ui Kixi, i N , if the aggregate matrix W is an M matrix.
c=W d
(61)
where z (x1, x2, . . ., xN)T, and W (wij) is the aggregate

matrix defined as

i eii ii , i = j
Aij x j + Bij u j
j=1
j = i
i Ai BiKi.
where A
Taking the total time derivative of V(x) with respect to S,
after lengthy but straightforward computations (6), we get
V (x) d T W z
(64)
These conditions simply imply that the interconnections may

be made to enter the subsystem through the input matrices
Bi, that is, we have
xi = Ai xi + Bi
S:
ui +
N
eij Dij x j
iN
(65)
j=1
which means that Aij BiDij for some matrices Dij.

Nonmatching conditions for decentral stabilizability of single-input subsystems have been considered in Ref. 24. The
system S is described as
S:
xi = Ai xi + bi ui +
eij Aij x j ,
iN
(66)
jN
(60)
where, without loss of generality, the subsystem pairs (Ai, bi)

are assumed to be in the controllable canonical form. For each
interconnection matrix Aij, define an integer

mij =
max{q p : aijpq = 0}, Aij = 0

Aij = 0
n,
(67)
Thus, mij is the distance between the main diagonal and a

line parallel to the main diagonal that borders all nonzero
elements of Aij. Given an index set I N , let J denote any
permutation of I . Then, the following holds:
The system S of Eq. (66) is stabilizable by decentralized
state feedback ui kTi xi, i N , if
(mij 1) < 0
(68)
iI
jJ
for all I and all permutations J .

In the special case of matching conditions, mij nj ni.
The condition in Eq. (68) was obtained via vector Liapunov
functions, and therefore guarantees decentral stabilizability
even when the elements of the interconnection matrices Aij
are bounded nonlinear time-varying functions of the state x.
There are a large number of results that characterize
wider classes of decentrally stabilizable systems (e.g., Refs.
2427), most of them surveyed in Ref. 6. Some of the results
broadened the scope of the given framework to include multivariable systems, time delays in the interactions, time-varying models, and stochastic perturbations. Especially interesting are schemes for decentralized output control, which
utilize dynamic output controllers or decentralized estimators
for reconstruction of the state of the overall system from the
subsystem state estimates.
Robustness. There are many useful and interesting properties of systems stabilized by decentralized feedback. First,
stability of decentrally controlled systems can tolerate nonlinearities in the interconnections; the nonlinear interconnections need not be known, since only their size must be limited.
Once the closed-loop system S is shown to be stable, it automatically follows (6) that any nonlinear time-varying version
:
S
N
xi = A i xi + hi (t, x),
iN
(69)
of S is connectively stable, provided the conical constraints
hi (t, x)

N
eij ij
x j
,
iN
(70)
219
ized control have been obtained by Chen et al. (32,33), Zhang

et al. (34), and Hassan et al. (35).
Since the introduction of multiple control system concepts
for reliable stabilization (36), there have been numerous papers dealing with controller or sensor/actuator failures. Solutions to this problem in the algebraic setting were given by
zguler (38), Tan et al.
Vidyasagar and Viswanadham (37), O
(39), and Gundes and Kabuli (40); relying on adaptation, by
Cho et al. (41); using H , by Veillette et al. (42), Medanic
(43), and Park and Bien (44); and in the context of linear quadratic control by Veillette (45). An application of the multiple
control system concept to reliable control of steam generators
was offered by Wu and Lin (46).
There are a number of interesting recent results concerning fixed-gain designs of decentralized control for interconnected systems. In Ref. 47 a stabilization scheme was proposed in the parameter space of linear systems, which was
formulated in the standard framework of convex programming. This opened up the possibility of using a variety of concepts and algorithms available in the linear matrix inequalities approach (48) for robust decentralized designs. A
promising result is the decentralized quadratic stabilization
of interconnected systems (49), which can be reduced to an
H control problem (see also Refs. 50 and 51).
There is a large body of literature on decentralized control
design using frequency-domain methods, which is based on
the work of Rosenbrock (52). The initial results were obtained
in Ref. 53 and further developed in Refs. 5456 using the Mmatrix theory and block-diagonal dominance. Parametrization of decentralized stabilizing controllers by the fractional
representation approach was considered in Refs. 5760. Finally, we should mention a number of books (6165) where a
wide variety of useful methods and techniques can be found
for the design of stabilizing decentralized control.
ADAPTIVE CONTROL
Decentrally stabilizable systems with suitable interconnection structures can (always) be stabilized by using local states
or outputs, often employing high-gain feedback. How high the
gains should be depends on how strong the interconnections
are. More often than not, it is difficult, or impossible, to predict and limit the size of the coupling among the subsystems,
implying that fixed-gain controllers may be incapable in stabilizing the system. In such cases, one has to use adaptive controllers, which can adjust the gains to values needed for overall stability (66).
Let us consider a single-input, single-output version of S,
j=1
on the interconnection functions hi : n ni hold for all

(t, x) n. This robustness result is useful in practice
because, typically, either interconnections are poorly known,
or they change during the operation of the controlled system.
Obviously, the result includes the case when the interconnection parameters eij are considered as nonlinear time-varying
functions eij : n [0, 1].
Connective decentralized stabilization was considered by
Tan and Ikeda (28) and, when applied to robotic manipulators, by Stokic and Vukobratovic (29), Mills and Goldenberg
(30), and Mills (31). New results in robust design of decentral-
S:
xi = Ai xi + bi ui + Pi vi
yi = cTi xi
w i = Qi xi ,
(71)
iN
where xi ni, ui , and yi are the state, input, and

output of the subsystem Si, and vi mi and wi li are the
interconnection inputs and outputs of Si from and to other
subsystems Sj, j N , which are related as
vi = hi (t, w),
iN
(72)
220
The crucial restriction on the functions hi : l mi is

that they are bounded:
hi (t, w)

N
ij
w j
(73)
j=1
where ij are nonnegative, but unknown, numbers. The matrices Ai and vectors bi and ci defining the subsystem Si are not
specified, except for the fact that pairs (Ai, bi) are controllable
and that pairs (Ai, ci) are observable. The uncertainty about
the triples (Ai, bi, ci) compounds the essential uncertainty
about the overall system S caused by our ignorance of the
interconnections.
The control objective is to force each state xi(t) of Si to track
the state xmi(t) of the corresponding reference model
Mi :
xmi = Ami xmi + bmi ri

ymi = cTmi xmi ,
iN
(74)
where ri is the external (reference) input. To achieve this

objective, we assume that there exist subsets I N and
J N I such that
Pi = bi pTi ,
iI
qi cTi ,
i J
Qi =
(75)
for some constant vectors pi mi and qi li, which are

matching conditions requiring that either incoming disturbances enter through the control channel or outgoing disturbances pass through the measurement channel. These conditions, as shown in Ref. 67, make the system stabilizable by
high-gain decentralized feedback, and are crucial for the adaptation scheme to work.
The basic requirement of the adaptive regulator is to drive
the state of the overall system S to zero. For this purpose, the
local control laws are chosen as
ui = iT xi ,
iN
(76)
where i ni is the time-varying adaptation gain vector. To

arrive at suitable adaptation laws i(t), we choose Amis as (satisfactorily) stable constant matrices and set
i = Ri (kTi xi )xi ,
iN
(77)
where Ri and ki are appropriate constant matrices and i(t0)

is finite. Finally, the closed-loop system is
:
S
xi = (Ami + bi iT )xi + bi pTi hi

i = Ri (kTi xi )xi ,
iN
(78)
where i ni is the ith parameter adaptation error defined

as i i *i , and *i is a model-matching constant parameter vector. We denote the solutions of the closed-loop system
S by (x, )(t; t0, x0, 0), where x (xT1 , xT2 , . . ., xTN) and
(T1 , T2 , . . ., TN)T. Relying on the decentral stabilizability condition of Eq. (75), Gavel and Siljak (66) established the following basic result:
The solutions (x, )(t; t0, x0, 0) of S are globally bounded,
and x(t; t0, x0, 0) 0 as t .
The adaptive stabilizability conditions, which are based on

structural restriction in Eq. (75), allow the controller gains to
rise to whatever level is necessary to ensure that stability of
the subsystem overrides the perturbations caused by interconnection fluctuations, so long as they are finite. The boundedness part of the above result ensures the boundedness of
the adaptation gains and therefore the realizability of the decentralized adaptive control scheme.
The above basic result has many extensions. In the statetracking problem, the state x(t) of the plant S follows the
state xm(t) of the reference model M despite the change in the
size and shape of interconnections. Furthermore, the steadystate tracking error e(t) x(t) xm(t) can be made as small
as desired. When the subsystem models are known, which is
often the case in practice, the adaptive decentralized scheme
extends to multiinput, multioutput subsystems with added
simplicity in implementation; the scheme requires only one
adaptation parameter per subsystem. Under relatively mild
conditions the scheme can accommodate output feedback controllers as well.
Since the initial work on adaptive decentralized control by
Hmamed and Radouane (68), a wide variety of schemes have
been developed. Unmodeled dynamics with fast and slow
modes in the subsystems was considered by Ioannou and Kokotovic (69). The use of inputoutput models was initiated by
Wiemer and Unbehauen (70) for a discrete-time problem of
decentralized adaptive control. A state-space approach to the
same problem was proposed by Reed and Ioannou (71) as well
as Kamoun et al. (72) and Yang and Papavassilopoulos (73).
Motivated by models of mechanical systems, Shi and Singh
(74,75) considered higher-order interconnections with polynomial bounds. Poor transient behavior of the standard adaptive schemes, which is caused by insufficient knowledge of
subsystem parameters, can be improved by exchanging the
output signals between individual subsystems and assuming
weak coupling (76).
A major drawback of the early adaptive decentralized
schemes was the relative-degree restriction on the subsystems. This restriction was first removed by Ortega and Herrera (77) by using the concept of higher-order tuning. The
same result was later achieved by Wen (78) by applying the
integrator backstepping procedure. The original scheme of
Gavel and Siljak (66) was extended by Lyon (79) to the case
when the relative order of each subsystem does not exceed
two. Due to the high-gain nature of the adaptive scheme proposed in Refs. 66, 80, a variable-structure decentralized control (81,82) became a logical candidate for improving the design. Sasaki et al. (83) developed a variable-structure version
of their decentralized adaptive scheme for control of distributed systems.
Inputoutput models with relay-type controllers were used
by Brusin and Ugrinovskaya (84); see also Refs. 85 and 86. In
Refs. 87 and 88, coordinate transformations have been utilized to broaden the class of systems that can be stabilized by
decentralized adaptive control. Indirect adaptive schemes
have been proposed by Wen (89) and Spooner and Passino
(90). Finally, a partially decentralized adaptive control has
been developed in Ref. 91.
OVERLAPPING DECENTRALIZED CONTROL
In a wide variety of natural and industrial systems, subsystems share common parts (6). In these cases, for either con-
ceptual or computational reasons it is advantageous to use

overlapping decentralized control. We will use the overlapping epsilon decomposition to provide a justification for such
a practice.
Let us consider a linear constant system
x = Ax + Bu
S:
(79)
---------
------
-------
where x n, u m, and the block matrices
0
A11 A12 A13
B11
A = A21 A22 A23 ,

B = - - -0- - - 0
------------0
B32
A31 A32 A33
(80)
are decomposed along the dashed lines defining the two overlapping subsystems. By using a linear transformation
x = V x
where V is the n n matrix
I1
0
V =
0
0
0
I2
I2
0
(81)
0
0
0
I3
(82)
and the identity matrices are compatible with the blocks of

A, we get the expansion of S as
x = A x + Bu
:
S
(83)
where x n, n n1 2n2 n3, and n1, n2, n3 are dimensions

of the square matrices A11, A22 and A33. The system matrices
are
A = VAU + M,
B = V B + N
(84)
and
0
0
0
I1
U = 0 12 I2 21 I2 0
0
0
0
I3
1
1
A
2 A12
0
2 12
0
1
A
12 A22
2 22
M=
1
1
0 2 A22
2 A22
1
0 12 A32
2 A32
N=0
0
0
(85)
------------
------------
Then, the expansion S of Eq. (83) has the form
0
A13
0
A11 A12
B11
A
0
0
A
0
A
:
S
x = - - - -21- - - - - -22- - - - - - - - - - - - -23- x + - - - - - - - - - - - u (86)
A21
0
0
0
A22 A23
0
B32
A31
0
A32 A33
22
-------
21
-------
where the dashed lines delineate two disjoint subsystems, so

that we can write the expansion S as

B 11
0
A 11 A 12
x + - - - - - - - - - - u
(87)
S:
x =
0
B
A
A
22
221
It is crucial to note from Eq. (87) that the expansion S of

Eq. (86) has an epsilon decomposition, while the original system S of Eq. (80) does not. Decentralized stabilization
schemes can now take advantage of the weak coupling among
the two subsystems, and use local feedback to stabilize the
expansion S by independent stabilization of the two subsystems. It is easy to show that stability of the expanded system
S implies stability of the original system S.
The circle of ideas and methods surrounding the overlapping decompositions and control structures have been organized into a general mathematical framework known as the
inclusion principle (6). In a wide variety of problems arising
in the theory and practice of large-scale systems, the principle
produced interesting conceptual insights and useful solution
procedures (9298).
OPTIMIZATION
Despite considerable efforts and a large number of new results in the theory of large complex systems, the fundamental
problem of optimal decentralized control has remained unsolved for over two decades. The simple reason has been the
fact that the decentralized information structure constraints
have not been successfully incorporated into any of the standard optimization frameworks. Neither Pontryagins maximum principle nor Bellmans dynamic programming can handle the lack of complete state observation, a difficulty
recognized by Fleming (99) as far back as the late sixties. For
this reason, there have been a large number of results relying
on pragmatic suboptimality concepts, which have capitalized
on effective solutions of robustness issues in the suboptimality framework (6).
The standard practice has been to optimize each decoupled
subsystem using linear quadratic (LQ) control laws. Then,
suboptimality of the interconnected closed-loop system, which
is driven by the union of the locally optimal LQ control laws,
is determined with respect to the sum of the quadratic costs
chosen for the subsystems. Under relatively mild conditions,
suboptimality implies stability. Furthermore, the degree of
suboptimality, which is computed with respect to the globally
optimal union of decoupled subsystems, can serve as a measure of robustness with respect to a wide spectrum of uncertainties residing in both the subsystems and their interactions. It has been shown (100) how the classical measures of
gain and phase margins, as well as the gain reduction tolerance, can be incorporated in the decentralized LQ control of
large interconnected systems.
Recently, a solution to the optimal decentralized stochastic
control problem has been offered in the classical optimization
framework of Lagrange (101) relying on the constrained optimization approach proposed in Ref. 102. The principal idea is
to redefine the information structure constrains as differential equations, and attach Lagrange multipliers to each constraint to obtain sufficient as well as necessary conditions for
optimality of decentralized control laws. The multipliers are
functions of time and state, and, because of the role they play
in the optimization process, they are termed the Lagrange
Liapunov multipliers. The sufficient conditions are formulated in terms of the HamiltonJacobi-like equations and are
proved by using the method of global optimization (103). In
contrast, the necessary conditions of optimality, which are de-
222
rived very much in the spirit of the Flemings result, require

the solution of a two-point boundary value problem with a
minimum condition of the Pontryagin type.
Let us present the main idea of this new development with
a minimum of technical elements. A system S, which is composed of N interconnected subsystems, is described by a stochastic differential equation of Itos type,
S:
dxi =
Aii (t)xi + Bi (t)ui (t, x)

+
n
Aij (t)x j
(88)
dt + Ci (t, x) dwi ,
iN
j = i
With the system S we associate a quadratic cost
J=
N t1
1
E
[xT Q (t)xi + uTi Ri (t)ui ] dt
2 i=1 t 0 i i
(89)
The design objective is to determine a control u(t, x) that

drives the system S optimally with respect to cost J and, at
the same time, satisfies the essential information restriction
that the control law ui at each individual subsystem can utilize only the locally available state xi. More precisely, we have
the following problem:
Determine a control u(t, x) that minimizes the functional J
with respect to the system S under the decentralized information structure constraints
ui = ui (t, xi ),
iN
(90)
By reformulating the constraints in Eq. (90) as differential

equations
ui (t, x)
= 0,
x i
iN
(91)
where the vector xi is the state vector x without the state xi of

the ith subsystem, we can append Eq. (91) to the equations of
motion in Eq. (88) and use the standard Lagrange optimization framework in the manner suggested by Pontryagin in his
maximum principle and by Bellman in his version of the
HamiltonJacobi equation. When the additional assumption
of Gaussian state evolution is added, the optimality conditions obtained in the new approach provide a feedback structure for decentralized control, which involves Riccati-type
equations in the same way as in the classical regulator theory
of Kalman (104). This fact is expected to play a major role in
applications of optimal decentralized control to complex interconnected systems.
BIBLIOGRAPHY
1. H. H. Happ, Diakoptics and Networks, New York: Academic
Press, 1971.
2. A. Sangiovanni-Vincentelli, L. K. Chen, and L. O. Chua, An efficient heuristic cluster algorithm for tearing large scale networks, IEEE Trans. Circuits Syst., 24: 709717, 1977.
3. A. George and J. W. H. Liu, Computer Solution of Large Sparse

Positive Definite Systems, Englewood Cliffs, NJ: Prentice-Hall,
1981.
4. I. S. Duff, A. M. Erisman, and J. K. Reid, Direct Methods for
Sparse Matrices, Oxford: Clarendon Press, 1986.
5. A. Pothen, H. D. Simon, and K. P. Liou, Partitioning sparse matrices with eigenvectors of graphs, SIAM J. Matrix Anal. Appl.,
11: 430452, 1990.
6. D. D. Siljak, Decentralized Control of Complex Systems, Boston:
7. A. George, J. Gilbert, and J. W. H. Liu (eds.), Graph Theory and
Sparse Matrix Computation, New York: Springer-Verlag, 1993.
8. A. I. Zecevic and D. D. Siljak, Balanced decompositions of sparse
systems for multilevel parallel processing, IEEE Trans. Circuits
Syst., 41: 220233, 1994.
9. M. E. Sezer and D. D. Siljak, Nested decompositions and clustering of complex systems, Automatica, 22: 321331, 1986.
10. M. E. Sezer and D. D. Siljak, Nested epsilon decompositions of
linear systems: Weakly coupled and overlapping blocks, SIAM
J. Matrix Anal. Appl., 12: 521533, 1991.
11. A. I. Zecevic and D. D. Siljak, A block-parallel Newton method
via overlapping epsilon decompositions, SIAM J. Matrix Anal.
Appl., 15: 824844, 1994.
12. L. A. Hageman and D. M. Young, Applied Iterative Methods,
New York: Academic Press, 1981.
13. H. H. Happ, Piecewise Methods and Applications to Power Systems, New York: Wiley, 1980.
14. N. E. Gibbs, W. G. Poole, and P. K. Stockmeyer, An algorithm
for reducing the bandwidth and profile of a sparse matrix, SIAM
J. Numer. Anal., 13: 236250, 1976.
15. J. G. Lewis, Implementation of the GibbsPooleStockmeyer
and GibbsKing algorithms, ACM Trans. Math. Software, 8:
180189, 1982.
16. D. D. Siljak and A. I. Zecevic, A nested decomposition algorithm
for parallel computations of very large sparse systems, Math.
Prob. Eng., 1: 4157, 1995.
17. S. H. Wang and E. J. Davison, On the stabilization of decentralized control systems, IEEE Trans. Autom. Control, 18: 473
478, 1973.
18. B. D. O. Anderson and D. J. Clements, Algebraic characterization of fixed modes in decentralized control, Automatica, 17:
703712, 1981.
19. D. D. Siljak, Large-Scale Dynamic Systems: Stability and Structure, New York: North-Holland, 1978.
20. C. T. Lin, Structural controllability, IEEE Trans. Autom. Control, 19: 201208, 1974.
21. M. E. Sezer and D. D. Siljak, Structurally fixed modes, Sys. Control Lett., 1: 6064, 1981.
22. M. E. Sezer and D. D. Siljak, Decentralized control, in W. S.
Levine (ed.), The Control Handbook, Boca Raton, FL: CRC Press,
1996, pp. 779793.
23. G. Leitmann, One approach to the control of uncertain systems,
ASME J. Dynam. Syst. Meas. and Control, 115: 373380, 1993.
24. M. Ikeda and D. D. Siljak, On decentrally stabilizable largescale systems, Automatica, 16: 331334, 1980.
25. M. E. Sezer and D. D. Siljak, On decentralized stabilization and
structure of linear large-scale systems, Automatica, 17: 641
644, 1981.
26. Z. C. Shi and W. B. Gao, Stabilization by decentralized control
for large-scale interconnected systems, Large Scale Syst., 10:
147155, 1986.
27. G. H. Yang and S. Y. Zhang, Decentralized robust control for
interconnected systems with time-varying uncertainties, Automatica, 32: 16031608, 1996.

28. X. L. Tan and M. Ikeda, Decentralized stabilization for expanding construction of large-scale systems, IEEE Trans. Autom. Control, 35: 644651, 1990.
29. D. Stokic and M. Vukobratovic, Practical stabilization of robotic
systems by decentralized control, Automatica, 20: 353358,
1984.
30. J. K. Mills and A. A. Goldenberg, Global connective stability of
a class of robotic manipulators, Automatica, 24: 835839, 1988.
31. J. K. Mills, Stability of robotic manipulators during transition
to and from compliant motion, Automatica, 26: 861874, 1990.
32. Y. H. Chen and M. C. Han, Decentralized control design for interconnected uncertain systems, in C. T. Leondes (ed.), Control
and Dynamic Systems, Boston: Academic Press, 1993, pp.
219266.
33. W. J. Wang and Y. H. Chen, Decentralized robust control design
with insufficient number of controllers, Int. J. Control, 65: 1015
1030, 1996.
34. S. Y. Zhang, K. Mizukami, and H. S. Wu, Decentralized robust
control of uncertain large-scale interconnected nonlinear dynamical systems, J. Optim. Theory Appl., 91: 235256, 1996.
35. M. F. Hassan et al., Expert robust decentralized controller for
uncertain large-scale systems, IEE Proc. Control Theory Appl.,
143: 519529, 1996.
36. D. D. Siljak, Reliable control using multiple control systems, Int.
J. Control, 31: 303329, 1980.
37. M. Vidyasagar and N. Viswanadham, Reliable stabilization using a multicontroller configuration, Automatica, 21: 599602,
1985.
zguler, Decentralized control: A stable proper fractional
38. A. B. O
approach, IEEE Trans. Autom. Control, 35: 11091117, 1990.
39. X. L. Tan, D. D. Siljak, and M. Ikeda, Reliable stabilization via
factorization methods, IEEE Trans. Autom. Control, 37: 1786
1791, 1992.
40. A. N. Gundes and M. G. Kabuli, Reliable stabilization with integral action in decentralized control systems, Proc. 34th IEEE
CDC, New Orleans, 1995, pp. 26472652.
41. Y. J. Cho, Z. Bien, and B. K. Kim, Reliable control via an adaptive redundant controller, Int. J. Control, 50: 385398, 1989.
223
51. M. Ikeda, Decentralized H controller design for large-scale systems, Proc. 35th IEEE CDC, Kobe, Japan, 1996, pp. 16.
52. H. H. Rosenbrock, Computer-Aided Control System Design, New
York: Academic Press, 1974.
53. M. Araki and O. I. Nwokah, Bounds for closed-loop transfer
functions of multivariable systems, IEEE Trans. Autom. Control,
20: 666670, 1975.
54. W. H. Bennett and J. S. Baras, Decomposition and decentralized
control system design: A review of frequency domain methods,
Proc. 24th IEEE CDC, Ft. Lauderdale, 1985, pp. 18281835.
55. Y. Ohta, D. D. Siljak, and T. Matsumoto, Decentralized control
using quasi-block diagonal dominance of transfer function matrices, IEEE Trans. Autom. Control, 31: 420430, 1986.
56. J. H. Seo, A frequency domain method for decentralized stabilization based on quasi-block diagonal dominance, Int. J. Control,
64: 4159, 1996.
57. A. N. Gundes and C. A. Desoer, Algebraic Theory of Linear Feedback Systems with Full and Decentralized Compensators, Heidelberg: Springer-Verlag, 1990.
zguler, Linear Multichannel Control: A System Matrix
58. A. B. O
Approach, UK: Prentice-Hall Int., 1994.
59. V. Manousiouthakis, On the parametrization of all decentralized stabilizing controllers, Sys. Control Lett., 21: 397403, 1993.
60. D. D. Sourlas and V. Manousiouthakis, Best achievable decentralized performance, IEEE Trans. Autom. Control, 40: 1858
1871, 1995.
61. M. Jamshidi, Large-Scale Systems: Modeling, Control, and Fuzzy
Logic, Upper Saddle River, NJ: Prentice-Hall, 1997.
62. J. Lunze, Feedback Control of Large-Scale Systems, Englewood
Cliffs, NJ: Prentice-Hall, 1992.
63. H. Tamura and T. Yoshikawa, Large-Scale Systems Control and
Decision Making, New York: Marcel Dekker, 1990.
64. C. T. Leondes (ed.), Control and Dynamic Systems, Orlando, FL:
Academic Press, 1986, vol. 22, 1985; vols. 23, 24.
65. M. S. Mahmoud, M. F. Hassan, and M. G. Darwish, Large-Scale
Control Systems, New York: Marcel Dekker, 1985.
66. D. T. Gavel and D. D. Siljak, Decentralized adaptive control:
42. R. J. Veillette, J. V. Medanic, and W. R. Perkins, Design of reliable control systems, IEEE Trans. Autom. Control, 37: 290
304, 1992.
Structural conditions for stability, IEEE Trans. Autom. Control,

34: 413426, 1989.
67. O. Huseyin, M. E. Sezer, and D. D. Siljak, Robust decentralized
control using output feedback, IEE Proc., D-129: 310315, 1982.
43. J. V. Medanic, Design of reliable controllers using redundant

control elements, Proc. ACC, San Francisco, 1993, pp. 3130
3134.
68. A. Hmamed and L. Radouane, Decentralized nonlinear adaptive

feedback stabilization of large-scale interconnected systems,
IEE Proc., D-130: 5762, 1983.
44. S. H. Park and Z. Bien, Design of reliable control systems for

uncertain linear systems, Proc. Asian Control Conf., Tokyo,
1994, pp. 101104.
69. P. Ioannou and P. V. Kokotovic, Decentralized adaptive control

of interconnected systems with reduced-order models, Automatica, 21: 401412, 1985.
45. R. J. Veillette, Reliable linear-quadratic state-feedback, Automatica, 31: 137143, 1995.
70. P. Wiemer and H. Unbehauen, On the stability of decentralized

adaptive control systems, Int. J. Adaptive Control Signal Process., 4: 116, 1990.
46. W. Wu and C. Lin, Optimal reliable control system design for

steam generators in pressurized water reactors, Nucl. Technol.,
106: 216224, 1994.
71. J. S. Reed and P. Ioannou, Discrete-time decentralized adaptive

control, Automatica, 24: 419421, 1988.
47. J. C. Geromel, J. Bernussou, and P. L. D. Peres, Decentralized

control through parameter space optimization, Automatica, 30:
15651578, 1994.
72. M. Kamoun, A. Titli, and M. B. Kamoun, Robust decentralized

model reference adaptive control of discrete-time interconnected
systems, Int. J. Control, 37: 18411856, 1992.
48. S. Boyd et al., Linear Matrix Inequalities in System and Control

Theory, Philadelphia: SIAM, 1994.
73. W. Y. Yang and G. P. Papavassilopoulos, Decentralized adaptive

control in a game situation for discrete-time, linear, time-invariant systems, Proc. ACC, Baltimore, 1994, pp. 22943399.
49. G. K. Zhai and M. Ikeda, Decentralized H control of large-scale

systems, Proc. 32nd IEEE CDC, San Antonio, 1993, pp. 1652
1653.
50. R. A. Date and J. H. Chow, A parametrization approach to optimal H2 and H decentralized control problems, Automatica, 29:
457463, 1992.
74. L. Shi and S. K. Singh, Decentralized adaptive controller design

for large-scale systems with higher order interconnections,
75. S. K. Singh and L. Shi, Robust adaptive controllers for interconnected mechanical systems: Influence of types of interconnec-
224
LASER APPLICATIONS IN MEDICINE
tions on time-invariant and time-varying systems, ASME J. Dynam. Syst. Meas. Control, 116: 456473, 1994.
76. A. Datta, Performance improvement in decentralized adaptive
control: A modified model reference scheme, IEEE Trans. Autom.
Control, 38: 17171722, 1993.
LQG control, Proc. 13th IFAC World Congr., San Francisco,

1996, L: pp. 712.
98. S. S. Stankovic, M. Stanojevic, and D. D. Siljak, Decentralized
suboptimal LQ control of a platoon of vehicles, Proc. 8th IFAC
Symp. Transportation Sys., Chania, Greece, 1997, pp. 8186.
77. R. Ortega and A. Herrera, A solution to decentralized adaptive

stabilization problem, Syst. Control Lett., 13: 229306, 1993.
78. C. Wen, Decentralized adaptive regulation, IEEE Trans. Autom.
Control, 39: 21632166, 1994.
79. J. Lyon, Note on decentralized adaptive controller design, IEEE
80. R. Wei, M. E. Sezer, and O. Ocali, Robust adaptive stabilization
of a class of systems under structured nonlinear perturbations
with application to interconnected systems, Int. J. Control, 63:
183194, 1996.
81. H. Khurana, S. I. Ahson, and S. S. Lamba, On stabilization of
large-scale control systems using variable structure system theory, IEEE Trans. Autom. Control, 31: 176178, 1988.
82. G. P. Matthews and R. A. DeCarlo, Decentralized tracking for a
class of interconnected nonlinear systems using variable structure control, Automatica, 24: 187193, 1988.
83. M. Sasaki et al., Finite-dimensional decentralized VS-MRAC of
non-linear interconnected distributed parameter systems, Int. J.
Syst. Sci., 24: 11651188, 1993.
84. V. A. Brusin and E. Yu, Ugrinovskaya, Decentralized adaptive
control with a reference model, Avtom. i Telemekhan., 10: 29
36, 1992.
85. A. C. Wu and L. C. Fu, New decentralized MRAC algorithms
for large-scale uncertain systems, Proc. ACC, Baltimore, 1994,
pp. 33893393.
86. G. Feng and Y. A. Jiang, Variable structure based decentralized
adaptive control, Proc. ACC, Baltimore, 1994, pp. 34093410.
87. S. Jain and F. Khorrami, Decentralized adaptive control of a
class of large-scale interconnected nonlinear systems, IEEE
88. S. Jaoin and F. Khorrami, Decentralized adaptive output feedback design for large-scale nonlinear systems, IEEE Trans. Autom. Control, 42: 729735, 1997.
89. C. Wen, Indirect robust totally decentralized adaptive control
of continuous-time interconnected systems, IEEE Trans. Autom.
Control, 40: 11221126, 1995.
90. J. T. Spooner and K. M. Passino, Adaptive control of a class of
decentralized nonlinear systems, IEEE Trans. Autom. Control,
41: 280284, 1996.
91. M. T. Ho and A. Datta, Partially decentralized adaptive control
of interconnected systems with improved performance, Int. J.
Control, 64: 529561, 1996.
92. M. Ikeda and D. D. Siljak, Overlapping decompositions, expansions, and contractions of dynamic systems, Large Scale Syst.,
1: 2938, 1980.
93. M. Ikeda, D. D. Siljak, and D. E. White, Decentralized control
with overlapping information sets, J. Optim. Theory Appl., 34:
279310, 1981.
94. M. Ikeda and D. D. Siljak, Overlapping decentralized control
with input, state, and output inclusion, Control Theory Adv.
Technol., 2: 155172, 1986.
zguner, Contractible controller design and opti.O
95. A. Iftar and U
mal control with state and input inclusion, Automatica, 26: 593
597, 1990.
96. A. Iftar, Decentralized estimation and control with overlapping
input, state, and output decomposition, Automatica, 29: 511
516, 1993.
97. S. S. Stankovic, X. B. Chen, and D. D. Siljak, Stochastic inclusion principle applied to decentralized overlapping suboptimal
99. W. Fleming, Optimal control of partially observable diffusions,

SIAM J. Control, 6: 194214, 1968.
100. M. E. Sezer and D. D. Siljak, Sensitivity of large-scale control
systems, J. Franklin Inst., 312: 170197, 1981.
101. S. V. Savastyuk and D. D. Siljak, Optimal decentralized control
for stochastic dynamic systems, in R. P. Agarwal (ed.), Recent
Trends in Optimization Theory and Applications, Singapore:
World Scientific, 1995, pp. 337352.
102. M. M. Khrustalev and S. V. Savastyuk, Conditions for optimality of stochastic systems of diffusion type in problems with
constraints on the controlobservation process, Sov. Math.
Dokl., 41: 256260, 1990.
103. V. F. Krotov, A technique of global bounds in optimal control
theory, Control Cybernet., 17: 115144, 1988.
104. R. E. Kalman, Contributions to the theory of optimal control,
Bul. Soc. Mat. Mex., 5: 102119, 1960.
DRAGOSLAV D. S ILJAK
ALEKSANDAR I. ZECEVIC
Santa Clara University
LARGE SCALE INTEGRATION. See VLSI CIRCUIT

LAYOUT.
LARGE-SCALE SYSTEMS. See LARGE-SCALE AND DECENTRALIZED SYSTEMS.
LARGE SUPERCONDUCTING DETECTOR SOLENOIDS. See HIGH-ENERGY PHYSICS PARTICLE DETECTOR

MAGNETS.
LASER ABLATION. See LASER APPLICATIONS IN MEDICINE.
MODEL REFERENCE ADAPTIVE CONTROL

The aim of control is to keep the relevant outputs of a given
dynamic process within prescribed limits. Denoting the process to be controlled as a plant and denoting its input and
output as u and y, respectively, the aim of control is to keep
the error (e1 y yd) between the plant output and a desired
output yd within prescribed values. If yd is a constant, the
control problem is referred to as regulation and if yd is a function of time, the problem is referred to as tracking. In the
former case, the value of yd around which the system is to be
441
regulated is also referred to as a set point or operating point.

The goal of control, in both cases, is to ensure that the output
error e1 is as small as possible, in the presence of disturbances
and modeling errors, for all time, and that the controlled system is stable. Feedback control is one procedure by which regulation and tracking can be accomplished in a number of dynamic processes. When the differential equations describing
the behavior of the plant are linear and known a priori, powerful analytical techniques in both time domain and frequency domain have been developed. When the characteristics of the plant are unknown, both regulation and tracking
can be viewed as adaptive control problems.
The field of adaptive control in general, and model reference adaptive control in particular, has focused on problems
where the uncertainties in the system are parametric. Such
parametric uncertainties occur due to a variety of reasons on
practical applications. Typically system dynamics, which are
invariably nonlinear, are linearized to derive the requisite linear controller. The resulting linear model and its parameters
are therefore dependent on and vary with the operating condition. Parameters also may vary due to aging, disturbances,
or changes in the loading conditions. Parameters may be unknown due to approximations made in the modeling process.
In all these cases, a controller that provides a uniformly satisfactory performance in the presence of the parametric uncertainties and variations is called for. The adaptive approach to
this problem is to design a controller with varying parameters, which are adjusted in such a way that they adapt to and
accommodate the uncertainties and variations in the plant to
be controlled. By providing such a time-varying solution, the
exact nature of which is determined by the nature and magnitude of the parametric uncertainty, the closed-loop adaptive
system seeks to enable a better performance. The results that
have accrued in the field of adaptive control over the past
three decades have provided a framework within which such
time-varying, adaptive controllers can be designed so as to
yield stability and robustness in various control tasks.
Model reference adaptive control refers to a particular
class of adaptive systems. In this class, adaptive controllers
are designed by using a reference model to describe the desired characteristics of the plant to be controlled. The use of
such reference models facilitates the analysis of the adaptive
system and provides a stability framework. Two philosophically different approaches, indirect control and direct control,
exist for synthesizing model reference adaptive controllers. In
the indirect approach, the unknown plant parameters are estimated using a model of the plant before a control input is
chosen. In the direct approach, an appropriate controller
structure is selected and its parameters are directly adjusted
so that the output error is minimized. For the sake of mathematical tractability, the desired output yd needs to be characterized in a suitable form, which is generally accomplished by
the use of a reference model. Thus in a model reference problem formulation, the indirect approach employs both an identification model and a reference model while the direct approach uses a reference model only. We describe these models
in further detail below.
IDENTIFICATION MODEL
Mathematical modeling is an indispensable part of all sciences, whether physical, biological, or social. One often seeks
442
to characterize the cause-and-effect relations in an observed

phenomenon using a model and tune the model parameters
so that the behavior of the model approximates the observed
behavior for all cases of interest. One form of quantitative
models is mathematical and in the cases of dynamic systems,
these take the form of differential or difference equations.
Alternatively, a general mathematical model which represents the inputoutput behavior of a given process can also
be used for identification. The model obtained in the latter
case is often referred to as an identification model, since those
from the first approach are either not available or too complex
for control purposes. Often, especially for linear problems, frequency-domain methods are used to identify the system parameters. When measurement noise is present, the identification methods include statistical criteria so as to determine
the model that best fits the observed data. Systems identification, which is based on such approaches, is a well-developed
area of systems theory (1).
while in the latter the performance index implicitly includes

the reference model as

Ii =

(y p Am y p )T Qi (y p Am y p ) + uT Ru dt,
Qi > 0
In both cases, it can be shown that quadratic optimization

theory can be used to determine the control input. In the former case, the optimal input has the form
u(t) = Km ym (t) + K p x p (t)
and in the latter case we have
u(t) = K p x p (t)
The structure of the controller can be used in an adaptive
situation when the parameters of the plant are unknown,
though the control parameters have to be estimated to compensate for parametric uncertainties.
REFERENCE MODEL
Reference Model with Inputs
The use of a reference model for controls can be traced to
aircraft systems. Often, the situation therein is such that the
controls designer is sufficiently familiar with the plant to be
controlled and its desired properties; thus by choosing the
structure and parameters of a reference model suitably, its
outputs can be used as the desired plant response. While in
principle such a model can be linear or nonlinear, considerations of analytical tractability have made linear reference
models more common in practice.
Explicit and Implicit Model-Following
Two methods that have been studied extensively in this context include explicit and implicit model-following methods (2),
both of which include the use of a reference model described
by the homogeneous differential equation
ym = Am ym
(1)
where the constant matrix Am mm is chosen so that the

desired dynamics in terms of transient behavior, decoupling
of modes, bandwidth, and handling qualities is captured. Suppose that the plant to be controlled is described adequately
by an nth-order differential equation with m(m n) outputs
as
x p = A p x p + B p u
y p = Cp x p
(2)
The reference model in Eq. (1) is chosen so that the output

yp follows ym as closely as possible. The explicit and implicit
model-following methods are based on different performance
indices of the model-following error yp ym. In explicit modelfollowing, the performance index is of the form
Ie =
0
[(y p ym )T Qe (y p ym ) + uT Ru] dt
In Eq. (1), the output of the reference model was specified as

the output of a homogeneous differential equation. A more
general formulation of a reference model includes external inputs and is of the form
xm = Am xm + Bm r,
ym = Cm xm
(3)
where Am is a stable n n matrix with constant elements,

Bm and Cm are constant matrices with appropriate dimensions, and r is an arbitrary continuous uniformly bounded input. The goal of the control input u into the plant in Eq. (2)
so that the output yp(t) tracks the output ym(t) as closely as
possible. In this case, the reference input r along with the
model in Eq. (3) with the parameters Am, Bm, Cm determines
the output of the reference model. The introduction of the reference inputs significantly increases the class of desired trajectories that can be represented by a reference model. For
a perfect model following to occur, the differential equations
governing yp and ym as well as the initial conditions yp(t) and
ym(t) have to be identical. This imposes restrictive conditions
on the matrices Ap, Bp, Am, and Bm, in terms of their canonical
forms. It has been shown by Berger that the requisite control
input in this case is of the form
u(t) = K p x p (t) + Km xm (t) + Kr r(t)
In an adaptive situation, it is more reasonable to have the
objective of asymptotic model-following where yp(t) is desired
to follow ym(t) as t . The problem in this case is to determine the conditions under which this can be achieved amidst
parametric uncertainties.
The model reference adaptive control (MRAC) problem can be
qualitatively stated as the following: Given a plant P with an
inputoutput pair u( ), yp( ), along with a stable reference
model M whose inputoutput pair is given by r( ), ym( ),
where r is a bounded piecewise continuous function, determine the control input u(t) for all t t0 so that
lim |y p (t) ym (t)| = 0
443
Much of what is well known in MRAC concerns the case when

the plant and model are linear and time-invariant though
there have been a number of advances in the early 1990s in
adaptive control of nonlinear systems (3,4) by making use of
results in feedback linearization (5).
It becomes evident from the statement of the problem that
considerable prior information regarding the plant P is
needed to have a well-posed problem. Such information is critical while determining the structure of the reference model
and the controller. For instance, the controller must be such
that it makes use of all measurable signals in the system, is
differentiator-free, and results in a bounded control input. For
the plant output to follow the model output, the class of models M has to be constrained in some sense. Obviously, M depends on the prior information regarding the class P of
plants. For example, if the reference input r is a pulse train
and the model M has a unity transfer function, it is clear that
the output of the plant cannot follow ym asymptotically with
a bounded input u and a differentiator-free controller. To determine M for linear time-invariant plants, results related to
model-following in linear systems theory (6,7), LQG methods
(8), and pole-placement can be utilized. Once the classes of
plants P and M are determined, the structure of the controller
that generates u can be found. When the parameters of the
plant are known, the requisite controller has a linear structure. However, in order to compensate for the parametric uncertainty in P, the model reference adaptive controller has a
nonlinear structure where the nonlinearity arises due to the
fact that the controller parameters are adjusted on-line as a
function of the system variables that are measured.
To better illustrate the nature of the nonlinearity in
MRAC, we define the two parts of an MRAC, the algebraic
and the analytic. In what follows, we focus our attention only
on the case when the plant P and the model M are linear and
time-invariant.
Algebraic Part and Analytic Part
We parameterize the controller C by a vector : m,
where C is linear and time-invariant if is a constant. By
using model reference approaches, one can determine the controller structure and a parameter * such that if is equal to
* in C , the closed-loop system determined by the plant together with the model has an output which asymptotically
follows ym. Such a design process marks the first step of an
MRAC design and is referred to as the algebraic part.
The aim of adaptation is to generate the control input u
such that limtyp(t) 0 when the plant parameters are unknown. Since u(t) is determined by the manner in which the
parameter (t) is adjusted in the controller, the problem can
be equivalently stated in terms of (t). The second part of a
MRAC design, referred to as the analytic part, consists of determining the rule by which (t) is to be adjusted at each instant of time so that the closed-loop remains stable and the
output error e(t), defined as e(t) yp(t) ym(t), tends to zero
as t . The adjustment rule for (t) is referred to as the
adaptive law.
C( )
Figure 1. The MRAC problem.
The MRAC Problem

With the above definitions, the MRAC problem can be stated
below (Fig. 1). Suppose the inputoutput pair of a linear timeinvariant plant P with unknown parameters is u( ), yp( ).
1. Determine the class M of stable LTI reference models
such that if the inputoutput pair of the model is given
by r( ), ym( ), a uniformly bounded input u to the
plant P, generated by a differentiator-free controller, exists which assures that
lim |y p (t) ym (t)| = 0
(4)
2. Determine a differentiator-free controller C() parameterized by a vector (t) m, which generates u, such
that for a constant value *, the transfer function
of the plant together with the controller is equal to the
transfer function of M.
3. Determine a rule for adjusting (t) such that the closedloop system is stable and Eq. (4) is satisfied.
When a disturbance is present, such an asymptotic output
tracking may not be possible if very little information is available about the disturbance. In such a case, the goal of MRAC
is to minimize the error between yp and ym as much as possible.
Since stability is vital to the satisfactory operation of any
dynamic system and since in general adaptive systems are
nonlinear, one of the major difficulties in designing adaptive
systems is ensuring their stability properties. This often
serves as a guideline while solving the MRAC problem
stated above.
Error Model Approach
The solution of the MRAC problem is often significantly facilitated by an error model approach. This approach consists of
studying the relationship between two kinds of errors commonly present in any adaptive system: (1) the tracking error
e between the plant output and the model output and (2) the
parameter error between the estimated adaptive parameter
and its desired value. If the evolution of the error e is determined by the differential equation
e(t)
= f 1 (e(t), (t), t)
(5)
then the MRAC problem can be formulated as the determination of the adaptive law
(t) = f 2 (e(t), t)
(6)
444
in such a way as to ensure closed-loop stability and asymptotic tracking. Focusing attention directly on the error rather
than on the actual response of the plant or the reference
model enables the designer to concentrate on the essential
features of the problem and determine the adaptive law by
inspection. Such an approach has facilitated the design of
many adaptive systems both in the disturbance-free case as
well as when disturbances and modeling errors are present.
The structure of the model transfer function guarantees that

a constant vector * exists such that when (t) *, the plant
together with the controller has the same transfer function as
that of the model. The structure of the controller guarantees
that the underlying error model is of the form
Solution to the MRAC Problem
where *. The structure of the adaptive law in Eq.

(7) and the fact that Wm(s) is SPR enables one to select an
appropriate Lyapunov function of all of the states of the adaptive system. As a result, the closed-loop system remains
bounded and e1(t) tends to zero as t .
Case 2: n* 2. When the relative degree is greater than
unity, even though the same error model structure as in Eq.
(8) can be derived, it is not possible to choose an SPR model
transfer function. This requires additional processing of the
error signal and the sensitivity function in order to construct
the necessary adaptive law. In particular, an augmented error 1 is constructed as
By the year 1980, several solutions to the MRAC problem

when the plant to be controlled is linear and time-invariant
were proposed. One such solution is summarized below.
Suppose the plant P to be controlled is described by the
transfer function
Wp (s) = k p
Z p (s)
R p (s)
where kp as well as the coefficients of the monic polynomials

Zp(s) and Rp(s) are unknown. The degree of Rp(s), n, and the
degree of Zp(s), m n 1, are assumed to be known. Zp(s) is
assumed to be a Hurwitz polynomial. The sign of kp is assumed to be known. (This assumption was relaxed in (9) by
replacing sign(kp) in the adaptive law by a nonlinear gain. In
this article, we assume that sign(kp) is known for ease of exposition). The reference model M is chosen to have a transfer
function
kp
Wm (s)[ T ]
km
(8)
1 = e1 + e2
e2 = T Wm (s) Wm (s) T
Defining the filtered sensitivity function as where
= Wm (s)
one can show that the underlying error model, when kp is
known, is simplified from Eq. (8) to
Zm (s)
Wm (s) = km
Rm (s)
1 = T .
where Rm(s) and Zm(s) are monic Hurwitz polynomials with

degree n and n m. The structure of the controller and the
adaptive laws for adjusting the control parameters are given
separately for the cases when the relative degree n* n m
is unity and when it is greater than or equal to two. In both
cases, the objective is to solve problems 13 stated above and
accomplish the tracking stated in Eq. (4).
Case 1: n* 1. In this case, the model transfer function
is chosen to be strictly positive real (SPR) (10). This can be
accomplished since the relative degree is one and Wm(s) has
asymptotically stable poles and zeros, by interlacing the zeros
with the poles. The control input u is chosen as
u = T (t)(t)
1 = 1 + u
2 = 2 + y p
= [r, 1T , y p , 2T ]T
= [k, 1T , 0 , 2T ]T
where (n1)(n1) is asymptotically stable, with det(sI
) Zm(s), (, ) is controllable, n is the control parameter to be adjusted appropriately so that (4) is achieved, and
is a sensitivity function which essentially estimates the state
of the system on-line. The requisite adaptive law, assuming
that km 0, is given by
= sign(k p ) e1 ,
e1 =
> 0
(7)
(9)
As a result, an adaptive law of the form

.
=

1 + T
(10)
can be chosen. Such an adaptive law guarantees that the

closed-loop system remains bounded and that 1(t) 0 asymptotically. The normalization in Eq. (10) is needed to establish the global boundedness of signals. Recently, other
adaptive control structures and adaptive laws have been proposed (11) that do not employ such normalization, which has
the potential to lead to better transient performance. The
most important distinction between the approach in (10) and
that in (9) is that the former prescribes an explicit Lyapunov
function for the adaptive system and hence provides bounds
on the tracking errors and parameter errors that can be estimated a priori.
Results in model reference adaptive control have been extended in several different directions, including robustness
properties in the presence of disturbances and unmodeled dynamics, time-varying parameters and, most notably, adaptive
control of nonlinear dynamic systems (3,4,10,12,13). Extensions to multivariable adaptive control and stable adaptive
control in the presence of very few assumptions on the plant
have also been proposed (10). Improvement of the transient
response of the adaptive system by using multiple models and
switching and tuning has also been proposed (14). Adaptive
control techniques including self-tuning regulators, and auto-
MODULATION ANALYSIS FORMULA
445
matic tuning, as well as practical aspects of control implementation and applications, can be found in Ref. (15).
shown to be persistently exciting in 2n if the input has n

distinct frequencies and the system is controllable.
PARAMETER IDENTIFICATION
BIBLIOGRAPHY
The discussions earlier pertain to the global stability of the

adaptive system and conditions under which the output error
1 will converge to zero. However, if one desires to match the
closed-loop transfer function with that of the model, then the
parameter estimate (t) must approach * asymptotically. In
other words, parameter identification has to be carried out.
This also will ensure that the adaptive system will have an
improved transient response as initial conditions and reference input change and better robustness with respect to different kinds of perturbations.
In the context of Case 1 above, the problem of parameter
identification can be stated as follows: The output error 1 and
the parameter error satisfy the differential equations
e = Ae + bT ,
.
= e1
e1 = h T e
(11)
where hT(sI A)1b is SPR. Find the conditions on under

which e1 and will converge to zero. On the other hand, in
Case 2 the parameter error evolves as
.
= u uT
(12)
where
The problem once again is the determination of conditions on

u under which (t) 0 as t . These conditions are labeled
as persistent excitation of the corresponding signal and are
discussed further below.
PERSISTENT EXCITATION
A function u : m is said to be persistently exciting in
m if it satisfies the inequality
t+T0
u( )u ( ) d I
T
t t0
(13)
for some constants t0, T0, and . Several statements equivalent to (13) can be given, one of which is that, for every unit
vector w in m, we obtain
1
T0
t+T0
t
|uT ( )w| d 0
6. A. S. Morse, Structure and design of linear model following systems, IEEE Trans. Autom. Control, 18: 346354, 1973.
7. S. H. Wang and C. A. Desoer, The exact model matching of linear
multivariable systems, IEEE Trans. Autom. Control, 17: 347
349, 1972.
8. G. Stein and M. Athans, The LQG/LTR procedure for multivariable feedback control design, IEEE Trans. Autom. Control, 32:
105114, 1987.
9. D. R. Mudgett and A. S. Morse, Adaptive stabilization of linear
systems with unknown high frequency gains, IEEE Trans. Autom. Control, 30: 549554, June 1985.
10. K. S. Narendra and A. M. Annaswamy, Stable Adaptive Systems,
11. M. Krstic, I. Kanellakopoulos, and P. V. Kokotovic, Nonlinear design of adaptive controllers for linear systems, IEEE Trans. Autom. Control, 39: 738752, 1994.
u=
1 + T
1. L. Ljung and T. Soderstrom, Theory and Practice of Recursive

Identification, Cambridge, MA: M.I.T. Press, 1985.
2. H. Erzberger, Analysis and design of model following control systems by state space techniques, In Proc. JACC, 1968.
3. I. Kanellakopoulos, P. V. Kokotovic, and A. S. Morse, Systematic
design of adaptive controllers for feedback linearizable systems,
4. D. Seto, A. M. Annaswamy, and J. Baillieul, Adaptive control of
nonlinear systems with a triangular structure, IEEE Trans. Autom. Control, 39: 14111428, 1994.
5. A. Isidori, Nonlinear Control Systems, 2nd ed., New York:
t t0
(14)
It can be shown that for m 2n, if satisfies Eq. (13), then

limt (t) 0 in Eq. (12), and if satisfies Eq. (13), the errors
in Eq. (11) converge to zero asymptotically, which ensures
that parameter identification will take place.
Typically, a vector signal generated using n distinct frequencies can be shown to be persistently exciting in n. The
state of a 2nth-order asymptotically stable system can be
12. S. Sastry and M. Bodson, Adaptive Control, Englewood Cliffs, NJ:

13. P. Ioannou and J. Sun, Stable and Robust Adaptive Control, Englewood Cliffs, NJ: Prentice-Hall, 1995.
14. K. S. Narendra and J. Balakrishnan, Adaptive control using
switching and tuning, In Proc. Eighth Yale Workshop on Appl.
Adaptive Syst. Theory, New Haven, CT: Yale University, 1994.
15. K. J. Astrom and B. Wittenmark, Adaptive Control, 2nd Ed.,
Reading, MA: Addison-Wesley, 1995.
ANURADHA M. ANNASWAMY
Massachusetts Institute of
Technology
MODELS, FUZZY. See FUZZY MODEL FUNDAMENTALS.

MODELS OF ELECTRICAL MACHINES. See ELECTRIC
MACHINE ANALYSIS AND SIMULATION.
MODULAR INSTRUMENTATION. See CAMAC.

MODULARIZATION. See SUBROUTINES.
MODULATION. See DIGITAL AMPLITUDE MODULATION; INFORMATION THEORY OF MODULATION CODES AND WAVEFORMS.
MULTIVARIABLE SYSTEMS
51
MULTIVARIABLE LINEAR SYSTEMS
Introduction
With the development of miniaturized, cheap, sensor, and actuator technology, many of todays control problems must coordinate the actions of multiple actuators, based on multiple
output signals from diverse sensors. Such systems, illustrated
in Fig. 1, are referred to as multi-input, multi-output (MIMO)
systems. An important class of MIMO systems is linear
MIMO systems, whose relationship between input and output
signals is represented by linear transformations.
This article introduces the basic concepts for analyzing
linear time-varying and time-invariant MIMO systems for
continuous time input and output signals. Extensions of the
concepts in this article to discrete time signals are straightforward and are found in the references at the end. The chapter discusses input-output and state-space models of linear
MIMO systems and introduces the concepts of controllability
and observability for state-space models. It also discusses
modal analysis for state-space models of time invariant
MIMO systems, MIMO poles and zeros, and singular-value
analysis for characterizing the frequency response of linear,
time-invariant MIMO systems.
Input-Output Models
Input-output models capture the essential relationships between inputs to a system and the outputs of that system. Instead of focusing on the internal representation and operation
of a system, input-output models represent these internal effects implicitly within a transformation from inputs to outputs, as illustrated in Fig. 1. In this section, we review results
on input-output models of linear MIMO systems in continuous time.
Consider a system with input u and output y, where the
relationship between input and output is denoted by the map
y1
System
...
um
...
u1
yp
Figure 1. Multi-input, multi-output system.
52
y H(u). A system is linear if it satisfies the following properties:

1. Homogeneity. For any scalar multiple a, the response
to a scaled input is equal to the scaled output response.
That is
H(au) = aH(u)
2. Superposition. Given two inputs u1, u2, the response to
the combined input u1 u2 is the sum of the individual
output responses. That is
H(u1 + u2 ) = H(u1 ) + H(u2 )
For MIMO continuous time systems, the inputs consist of
vector-valued signals u(t) Rm and the outputs consist of vector-valued signals y(t) Rp, both of which are functions of the
continuous time parameter t. Assume that such a system is
well behaved, in that small changes in the input signals over
a finite time result in small changes in the output signal over
the same interval. The general form of the input-output description of such systems is given by

y(t) =

y(t) =
H(t, )u( ) d
(5)
Another important property of the example is that the impulse response depends only on the difference t . This
property is known as time invariance.
Definition. The input-output system Eq. (1) is time-invariant if
H(t, ) = H(t , 0) H(t )
(6)
for all t and . The last equality is a slight abuse of notation

introduced by convenience. If Eq. (1) is time-invariant, then
y(t) is the convolution of H(t) and u(t):

y(t) =

H(t )u( ) d =
H( )u(t ) d
(7)
Such a system is called a linear time-invariant (LTI) system.

An LTI system is causal if and only if H(t) 0 for all t 0.
State-Space Models
H(t, )u( ) d
(1)
In the previous integral equation, H(t, ) is the p m impulse

response or weighting pattern, that is, if the components of
u(t) are impulses of the form
u j (t) = (t t0 ),
uk (t) = 0, k = j
(2)
then
yi (t) = hi j (t t0 ), i = 1, . . ., p
(3)
where hij(t, ) is the ijth element of H(t, ).

Example. Consider a two-input, two-output system described by the following impulse response:
H(t, )

(t ) + 0.5 e(t ) u1 (t )
=
(t ) + e2(t ) u1 (t )
0
e3(t ) u1 (t )
where u1(t) is the unit step function, which is 0 for t 0 and

1 otherwise. The output of the system is given by
so that

u ( )
y1 (t)
H(t, ) 1
=
d
y2 (t)
u1 ( )
In the previous example, note that the impulse response

H(t,) is zero for t. Thus, the output y(t) depends only on
inputs up to time t. Systems with this special property are
known as causal.
In contrast with input-output models, state-space models provide an explicit representation of the internal operations of
a system, leading to the transformation between input time
functions and output time functions. The state of a system at
a given time provides a complete summary of the effects of
past inputs to the system, which is sufficient to uniquely characterize system output responses to future inputs. This article
focuses on linear MIMO systems where the state takes values
in a finite-dimensional space of dimension n. Such models are
analyzed with concepts from linear algebra and matrix differential equations.
The general form of a state-space model for a linear system
with m inputs u(t) Rm and p outputs y(t) is given by the
d
x(t) = A(t)x(t) + B(t)u(t)
dt
(8)
y(t) = C(t)x(t) + D(t)u(t)
(9)
with initial condition specified at time t0 as x(t0) x0. In Eqs.

(89), x(t) Rn is the system state at time t, A(t) is the n
n system matrix, B(t) is the n m input matrix, C(t) is the
p n output matrix, and D(t) is the p m feedthrough
matrix.
Example. Consider the state-space model specified by the
following matrices:
1
0
0
1 0
A = 0 2
0 ;
B = 1 0
0
0 3
0 1

0.5 0 0
1 0
C=
;
D=
0
1 1
1 0
Definition. The input-output system Eq. (1) is causal if

H(t, ) = 0 for t <
(4)
This state-space model is equivalent to the input-output

model in the previous subsection, as seen later.
An important property of state-space models is that they

generate causal input-output maps, because past inputs affect
only future outputs through the value of the current state. To
determine the implied relationship between inputs and outputs in Eqs. (89), we need expressions for the solution of the
vector differential equation in Eq. (8). First consider solutions
of the unforced (homogeneous) equation
d
x(t) = A(t)x(t)
dt
(10)
starting from arbitrary initial conditions x(t0) at time t0. Under the assumption that the matrix A(t) is piecewise continuous, there exists a unique continuous solution of Eq. (10)
given by
x(t) = A (t, t0 )x(t0 )
If the initial time at t0 and it is assumed that the

system starts at rest, the output y(t) becomes

y(t) =
1. A(t, t) I, where I is the n n identity matrix.

2. If A(t) A is time-invariant, then
C(t)A (t, )B( )u( ) d + D(t)u(t)
(14)
which is a causal input-output model for the system. Thus,

every state-space model leads to a corresponding input-output
model for the system. The question of whether input-output
models have a finite-dimensional state-space representation
is addressed in the next section.
Example (continued). Because of the special diagonal,
time-invariant form of the A matrix, it is straightforward to
compute the state transition matrix as
(t )
e
A (t, ) = 0
0
(11)
for some n n matrix continuous matrix A(t, t0). The matrix

A(t, ), known as the state transition matrix, has the following properties:
53
0
e
0
0
2(t )
e3(t )
Because C, B do not depend on time, the product CA(t, )B

is written as

CA (t, )B =
A (t, ) = eA(t )
0.5e(t )
e2(t )
0
e3(t )
Substituting Eq. (14) yields

where the matrix exponential is defined by the series
eAt =

i=0
(At)i
i!

=
3. A(t, t0) is the unique continuous solution of the n n

d
X (t) = A(t)X (t)
dt
with initial condition X(t0) I.
4. For every t, , t0, the following compositional property
holds:
A (t, )A ( , t0 ) = A (t, t0 )
5.
(t, )
t A
= A(t)A (t, ).
Given the state transition matrix A(t, ), the general solution of Eq. (8) is written as

x(t) = A (t, t0 )x(t0 ) +
t0
A (t, )B( )u( ) d ,
t t0 (12)
It is straightforward to use the properties of A(t, ) to verify

that Eq. (11) satisfies Eq. (8). Using Eq. (9), the input-output
relationship of a state-space model is written
y(t) = C(t)(t, t0 )x(t0 )

t
+
C(t)A (t, )B( )u( ) d + D(t)u(t)
t0

y(t) =
(13)

0.5e(t )
e2(t )
e3(t )
u( ) d +
(t ) + 0.5e(t )
(t ) + e2(t )
0
e3(t )

1
1

0
u(t)
0
u( ) d
which is equivalent to the input-output model in the previous subsection.

State-space models have important qualitative properties
which are useful for control design. Of particular interest are
the properties of stability, controllability, and observability,
discussed in greater length in the articles by Sontag, Freudenberg and Vidyasagar. Some of the relevant concepts are
summarized here.
Stability. Given an n-dimensional state-space model with
matrices [A(t), B(t), C(t), D(t)], there are two types of stability
properties of interest, internal stability and input-output stability. Internal stability is the stability of trajectories of the
homogeneous system Eq. (10) and thus involves only the matrix A(t). Different types of internal stability are possible. Asymptotic stability corresponds to all solutions to Eq. (10) converging to 0 as t , and exponential stability corresponds
to all solutions converging to zero exponentially fast, that is,
x(t) Keat
(15)
for some K, a 0, where the vector norm is the standard

Euclidean vector magnitude. Exponential stability is equivalent to
A (t, ) Mea(t ) , t
(16)
54
where M is the matrix norm corresponding to the square

root of the largest eigenvalue of the matrix MTM.
Input-output stability refers to the full input-output map
in Eq. (13). A MIMO system is said to be bounded-input/
bounded-output (BIBO) stable if bounded inputs lead to
bounded outputs, that is, if u(t) K1 for all t t0 implies y(t) K2 0 for all t t0. For state-space models, if
B(t), C(t), D(t) are bounded, then exponential stability guarantees BIBO stability. However, the converse is not true, as
shown by the example below:
d
x(t) = x(t) + u(t)
dt
y(t) = u(t)
Controllability and Observability. The concepts of controllability and observability of state-space models characterize the
degree to which inputs and outputs determine the internal
state trajectory of a state-space model. This section presents
an overview of these concepts for linear, state-space models.
For a more detailed exposition of these concepts see CONTROLLABILITY AND OBSERVABILITY.
Consider a linear system with a state-space model whose
matrices are A(t), B(t), C(t), D(t), which are assumed to be
continuous functions of time. The system is said to be controllable on the interval [t0, t1] if, given any initial state x0 at t
t0 and any desired final state x1 t t1, it is possible to
specify a continuous input u(t), t0 t t1 so that if x(t0)
x0, then x(t1) x1. The system is observable on the interval
[t0, t1] if, given knowledge of u(t), t0 t t1 and y(t), t0 t
t1, the initial state x(t0) (and thus the entire state trajectory
x(t), t [t0, t1]) is uniquely determined.
Conditions for verifying controllability and observability
are determined from the explicit representation of the trajectories of state-space models in Eqs. (11) and (13). Using Eq.
(11), controllability is equivalent to finding a control u(t), t
[t0, t1], which solves
t
1
x(t1 ) A (t1 , t0 )x(t0 ) =
A (t1 , )B( )u( ) d
(17)
t0
for any pair of states x(t1), x(t0).

Define the controllability Gramian as the n n matrix
t
1
WC (t0 , t1 ) =
A (t0 , )B( )BT ( )TA (t0 , ) d
(18)
t0
The system is controllable on [t0, t1] if and only if the matrix

WC(t0, t1) is invertible. To establish this, if the inverse exists,
then the control u(t) BT(t)TA(t0, t)W1
C (t0, t1)[x(t0) A(t0,
t1)x(t1)] is continuous and, when substituted in Eq. (17) yields
t
1
A (t1 , )B( )u( ) d

=
t1
t0

A (t1 , )B( )B
( )TA (t0 , )

WC1 (t0 , t1 ) x(t0 ) A (t0 , t1 )x(t1 )
= A (t1 , t0 )WC (t0 , t1 )WC1 (t0 , t1 )[x(t0 ) A (t0 , t1 )x(t1 )]
= x(t1 ) A (t1 , t0 )x(t0 )
t1
t0
zT A (t0 , )B( )BT ( )TA (t0 , )z d = 0
Since the integrand is nonnegative, it follows that zTA(t0, t)

B(t) 0 for all t [t0, t1]. Controllability implies that a control
exists which yields x(t1) 0 for x(t0) z. From Eq. (17), this
requires that
In this example, the state x does not affect the output y. Thus,
the system is internally unstable although it is BIBO stable.
t0
Conversely, assume that the system is controllable but

that the matrix WC(t0, t1) is not invertible. Then, there must
exist a nonzero vector z Rn such that zTWC(t0, t1)z 0, that
is,
t1
z=
t0
A (t0 , )B( )u( ) d
Because of the choice of z, it follows that
t1
zT z =
t0
[zT A (t0 , )B( )]u( ) d = 0
implying that z 0.
The controllability Gramian has many properties. It is
symmetric, positive-semidefinite for all t1 t0, and satisfies
the following matrix differential equation:
d
W (t, t1 ) = A(t)WC (t, t1 ) + WC (t, t1 )AT (t) B(t)BT (t)
(19)
dt C
WC (t1 , t1 ) = 0
Direct integration of this matrix equation is preferred to the
integral expression for numerically computing the controllability Gramian.
In the special case that the matrices A, B do not depend
on time, there is a simple algebraic test for controllability.
Define the matrix
Mc (A, B) = (B AB A2 B
An1 B)
where n is the dimension of the state. The state-space model

is controllable if and only if the rank of Mc(A, B) is n.
Observability is characterized similarly. Using Eq. (13), it
is sufficient to consider the case where u(t) 0 for t [t0,
t1], so that the output response is given by
y(t) = C(t)(t, t0 )x(t0 ),
t > t0
(20)
Define the observability Gramian as the n n matrix

W0 (t0 , t1 ) =
t1
t0
T ( , t0 )CT ( )C( )( , t0 )x d
The observability Gramian is again symmetric, positivesemidefinite, and satisfies the matrix differential equation
d
W (t, t1 ) = AT (t)W0 (t, t1 ) W0 (t, t1 )A(t) C(t)T C(t)
(21)
dt 0
W0 (t1 , t1 ) = 0
The system is observable on [t0, t1] if and only if W0(t0, t1)
is invertible. If the system is observable, then, in the absence
of external inputs, the initial condition is given by
x(t0 ) = W01 (t0 , t1 )
t1
t0
T ( , t0 )CT ( )y( ) d
In the special case where the matrices A, C are independent of time, observability is determined from the matrix
55
additional extraneous states can be added which do not affect

the input-output behavior of the system. It is important to
identify realizations with minimal numbers of states. If a realization with state dimension n exists and no other realization exists with dimension less than n, then realizations with
state dimension n are known as minimal realizations.
The following result provides an answer to the realization
problem:
Mo (A,C) = Mc (AT , CT )T
The state-space model is observable if and only if the matrix
Mo(A, C) has rank n.
Theorem (3). There exists a state space realization of dimension n for the weighting pattern G(t, ) if and only if there
exists a p n matrix matrix function H(t) and an n m
matrix function F(t), both continuous for all t, such that
State-Space Realization of Input-Output Models

An important question in MIMO systems is determining
when a causal input-output model of the form
G(t, )u( ) d + D(t)u(t)
(22)
t0
[where G(t, ) does not contain generalized functions such as

impulses] is represented by a finite-dimensional state-space
model of the form
d
x(t) = A(t)x(t) + B(t)u(t)
dt
y(t) = C(t)x(t) + D(t)u(t)
(23)
where x(t) Rn, u(t) Rm, y(t) Rp and A(t), B(t), C(t), D(t)
are continuous matrices.
The converse of the question is straightforward. Statespace models correspond to causal input-output models of the
form
t
t0
for all t, .
y(t) =
y(t) =
G(t, ) = H(t)F ( )
C(t)A (t, )B( )u( ) d + D(t)u(t),
t t0
In the problem of realizability, it is straightforward to

identify the correspondence between D(t) in Eqs. (22) and
(23). Thus, the focus is on identifying matrices A(t), B(t), C(t)
in Eq. (23) from G(t,) in Eq. (22). Two variations of this problem are of interest. In the first variation, the function G(t,)
is continuous and known for all values of t, . This information corresponds closely to that provided by a state-space
model, because the term C(t)A(t,)B() is defined for all t and
and is continuous by the assumptions for a state-space
model. In the second variation, the function G(t,) is known
only for values t , corresponding to causal observations of
impulse responses of the system. In this variation, the realization problem is more complex and requires additional
smoothness assumptions on G(t,). In this overview, we focus
on the first variation where G(t,) is known and continuous
for all t, . The interested reader should consult (1,2,3) for
further details on realization from impulse responses.
Definition. A state-space model Eq. (23) is a realization of a
weighting pattern G(t,) if, for all t, , G(t,) C(t)A(t,)B().
There are many possible realizations corresponding to a
specific weighting pattern G(t,). Any change of basis in the
state space results in an equivalent realization. In addition,
The proof of this theorem is straightforward. The invertibility properties of the state transitional matrix guarantee
that, for a state-space model, its input-output relationship is
factored as
C(t)(t, )B( ) = C(t)(t, 0)(0, )B( )
so that H(t) C(t)(t, 0), F(t) (0, t)B(t). Conversely, given
H(t), F(t), the state-space model
x(t)
= F(t)u(t)
y(t) = H(t)x(t)
(24)
is a realization for H(t)F(), because the state transitional

function (t, ) is the identity.
Next, consider the problem of determining whether a statespace realization is a minimal realization. The answer is tied
to the concepts of controllability and observability of statespace models discussed in greater depth in CONTROLLABILITY
AND OBSERVABILITY.
Theorem (3). Suppose the linear system Eq. (22) is a realization of the weighting pattern G(t,). Then, Eq. (22) is a minimal realization if and only if for some t0 and tf t0, Eq. (22)
is controllable and observable on the interval [t0, tf].
Another important question is determining the existence
of a time-invariant realization for a weighting pattern. The
answer is provided in the following result.
Theorem (3). A weighting pattern G(t,) is realizable by
a time-invariant linear system Eq. (23) if and only if it is
realizable, continuously differentiable in both t and , and
G(t,) G(t , 0) for all t and . Under these conditions,
there exists a time-invariant minimal realization of G(t,).
Linear Time-Invariant MIMO Systems
The analysis of linear MIMO systems in the time domain is
greatly simplified in the time-invariant case by using results
from linear algebra. Furthermore, the time invariance property allows applying transform techniques to represent the
behavior of the system in the frequency domain, as illustrated
in the next subsection. This subsection discusses the analysis
of causal LTI MIMO systems described by state-space models
56
in the time domain. Relevant concepts from linear algebra are

introduced as needed.
Consider a MIMO LTI system described in state space
form as
d
x(t) = Ax(t) + Bu(t)
dt
y(t) = Cx(t) + Du(t)
(25)
where the state x(t) R , the input u(t) R , the output

y(t) Rp, and the matrices A, B, C, D do not depend on time.
The algebraic structures of the matrices A, B, C, D completely
determine the qualitative behavior of the system, as is evident after some concepts from linear algebra are reviewed.
n
uH
i vj = 0
Eigenvalues and Eigenvectors. Let Cn denote the space of ndimensional, complex-valued vectors. Consider a vector v
Cn of dimension n, expressed in terms of its real and imaginary parts as
chapter, assume that right and left eigenvectors are scaled so

that the inner product of eigenvectors for the same eigenvalue
is 1, that is, uiHvi 1 for all i 1, . . ., n. A useful property
of eigenvectors is that the right and left eigenvectors corresponding to different eigenvalues are mutually orthogonal.
Let ui and vi denote, respectively, the left and right eigenvectors corresponding to eigenvalue i. Then,
whenever i j.
Now consider a matrix M Cnn with n distinct eigenvalues i, i 1, . . ., n, and, with corresponding left and right
eigenvectors ui, vi, i 1, . . ., n. It can be shown that the n
eigenvectors vi form a basis for the space Cn. In this case, the
matrix M is represented in terms of these eigenvalues and
eigenvectors in a dyadic expansion as
M=
a1
b1
v1

v = ... = ... + j ... = a + jb
vn
an
bn
where ai, bi, i 1, . . ., n are real-valued. Denote by v the
Hermitian of the vector v, defined as the complex conjugate of
the transpose of v:
vH = aT jbT
Given two vectors u, v C , the inner product of u and v
is defined as the complex-valued scalar product u, v uHv.
The Euclidean norm of a vector v Cn is given by
n

n

(26)
This expansion is useful for computing powers of M, because

Mk =

v2 = vH v =
i vi uH
i
i=1
ki vi uH
i
i=1
resulting from the orthogonality property of left and right eigenvectors corresponding to different eigenvalues.
System Modes. Consider the LTI state-space model in Eq.
(25). In the absence of inputs u(t), the response of the system
is determined by the initial condition x(0) and the system matrix A, as
x(t) = eAt x(0)
(a2i
b2i )
i=1
The norm 2 is used to determine the size of vectors v Cn

and corresponds to the standard notion of vector length.
Denote by Cnn the space of (n n)-dimensional, complexvalued matrices. Let M Cnn be a square matrix. An eigenvalue of M is a complex number which is a root of the characteristic polynomial of M:
det(I M) = 0
Associated with each distinct eigenvalue are nonzero left and
right eigenvectors u, v Cn, which satisfy the linear equations
Assume that the matrix A has distinct eigenvalues i, i

1, . . ., n, with corresponding left and right eigenvectors ui,
vi, i 1, . . ., n. A treatment of the general case with repeated eigenvalues is found in (12). Using Eq. (26) in the
expansion of eAt yields
t2
+
2
n
n
n

(it)2
vi uH
=
(i t)0 vi uH
(i t)vi uH
i +
i +
i +
2
i=1
i=1
i=1
eAt = (At)0 + At + A2
e i t v i u H
i
i=1
Thus, the unforced system response is given as
Mv = v
x(t) =
and
uH M = uH
In the special case where the matrix M is real-valued (M
Rnn), if is an eigenvalue of M with a nonzero imaginary
part, then its complex conjugate is also an eigenvalue of M.
Because the previous equations are linear any multiple of
an eigenvector is also an eigenvector. Thus, eigenvectors can
be scaled to have any nonzero magnitude. In the rest of this
e i t vi [uH
i x(0)]
i=1
This is interpreted as follows: The initial condition x(0) is decomposed into its contributions along n different system
modes using the left eigenvectors. The ith system mode is defined as eitvi and has its own characteristic exponent i. When
the initial condition x0 corresponds to a right eigenvector vi,
the state response x(t) eitvi is focused along the same direction vi.
The system modes are also used to understand the output

response of the system in the presence of input signals u(t).
Substituting the dyadic expansion of eAt into the output response Eq. (13) yields
y(t) = CeAt x(0) +

=
C(t)eA(t ) Bu( ) d + Du(t)
e i t [(Cvi )(uH
i x(0)]
i=1
n

(Cvi )(uH
i B)
i=1
e i (t ) u( ) d + Du(t)
Example. Consider the state-space model specified by the

following matrices:
1
1
2
A = 0 2
1
0
0 3
1 0
B = 1 0
0 1

0.5 0 0
C=
0
1 1

1 0
D=
1 0
The eigenvalues of A are 1 1, 2 2, 3 3. A set of
left and right eigenvectors is given by
MIMO Transfer Function Matrix. One of the powerful tools

of classical single-input, single-output (SISO) control theory is
frequency-domain analysis using transform methods for LTI
systems. SISO systems are often characterized by their transfer functions relating input signals to output signals in the
frequency domain with Laplace transforms. Transform techniques are also applied to LTI MIMO systems to obtain generalizations of system transfer functions to MIMO systems, as
follows.
Consider an LTI MIMO system, characterized by the impulse response matrix-valued function H(t), which describes
the input-output behavior of the system as
The term uiHB indicates how the control action affects the ith
mode. Similarly, the term Cvi shows how much the ith mode
affects the system output y(t). Thus, modal analysis of LTI
systems decomposes the performance of MIMO systems into
a superposition of n independent modes which are exited by
the input signals and initial condition.
Based on Eq. (27), one can derive intuitive conditions for
controllability and observability of LTI systems using the system modes. In particular, note that the ith mode is uncontrollable if uiHB 0, because the input has no effect on the ith
mode trajectory. Thus, controllability requires that uiHB 0
for all modes i 1, . . ., n. Similarly, the ith mode does not
affect the output if Cvi 0. In this case, an initial condition
of vi yields an identical output to an initial condition of 0 and
thus is unobservable. Observability requires that Cvi 0 for
all modes i 1, . . ., n.

1
1
0

u1 = 1 , v1 = 0 ; u2 = 1 ,
1.5
0
1

1
0
0.5

v1 = 1 ; u3 = 0 , v1 = 1
0
1
1
Using these modes, it is straightforward to verify that Cvi

0 and uiHB 0 for i 1, . . ., 3, which establishes that the
system is controllable and observable.
57
y(t) =
H(t )u( ) d
(27)
where H(t) includes generalized functions, such as the unit

impulse (t).
Let X(s) denote the bilateral Laplace transform of the function x(t):

X (s) =
x(t)est dt
(28)
For the MIMO LTI system Eq. (27), application of Laplace

transforms on both sides yields
Y (s) = H (s)U (s)
(29)
where Y(s) is the p-dimensional, two-sided Laplace transform

of the output y(t), U(s) is the m-dimensional, two-sided Laplace transform of the input u(t), and H (s) is the p m twosided Laplace transform of the impulse response H(t), called
the system transfer function matrix. In coordinates, this relationship is given by
Yi (s) =
Hik (s)Uk (s),
k = 1, . . ., p
k=1
where H ik(s) is the Laplace transform of Hik(t).

For causal LTI systems described in state-space form, as
in Eq. (25), the transfer function matrix is obtained directly
from the state-space representation. Assume that the system
is at rest with no initial conditions. Taking bilateral Laplace
transforms of both sides in Eq. (25) yields
sX (s) = AX (s) + BU (s)

and
Y (s) = CX (s) + DU (s)
Solving these equations simultaneously,
Y (s) = [C(sI A)1 B + D]U (s)
which yields the transfer function matrix H (s) C(sI
A)1B D. Note that, although there can be different statespace models for a given LTI system, the transfer function
matrix H (s) is unique.
There are some special properties of system transfer function matrices of MIMO LTI systems. First, the variable s en-
58
ters into the expression in the inverse (sI A)1. If A is an

(n n) matrix, this means that the entries of H (s) are rational functions, ratios of polynomials, with denominator degree
no greater than n. Furthermore, the numerator degree is no
greater than n either and is strictly less than n for all entries
unless D 0. Transfer function matrices with entries as rational functions with numerator degree less than or equal to
denominator degree are known as proper. If the numerator
degree is strictly less than the denominator degree for each
entry, the transfer function matrix is known as strictly proper.
Multivariable Poles and Zeros. For SISO LTI systems, the
poles and zeros of the system are determined from the transfer function, consisting of a ratio of a numerator polynomial
and a denominator polynomial. The roots of the numerator
polynomial determine the zero frequencies of the system, frequencies which, if present at the input, are blocked by the
system and are thus not present at the output. Similarly, the
roots of the denominator polynomial determine the poles that
are frequencies appearing at the output in response to initial
conditions with no external input.
Although a rich theory exists for generalizing the SISO
transfer function decomposition to transfer function matrices
of MIMO systems using polynomial matrices and matrix fraction descriptions (see e.g. [12]), the simplest definition of
MIMO poles and zeros is given in terms of state-space models.
Consider the LTI state-space model of Eq. (25). The poles of
the system are the complex frequencies that appear in the
output in response to initial conditions. Based on the discussion of the previous subsections, these frequencies are the eigenvalues of the matrix A. This is also seen directly from the
transfer function matrix H (s) C(sI A)1B D. Using the
expression for inverting a matrix, it is clear that the denominator of all of the entries in the transfer function matrix is
given by det(sI A). Thus, the poles correspond to roots of
the equation det(sI A) 0, which are the eigenvalues
of A.
In contrast with multivariable poles, there are several
ways in which zeros have been defined for LTI MIMO systems. First consider a system with equal number of inputs
and outputs (m p), and assume that the state-space model
in Eq. (25) is minimal (thus controllable and observable).
Multivariable transmission zeros are defined as complex frequencies where, given a particular nonzero combination of input directions at that frequency and initial conditions, there
is no output generated by the system. The formal definition
is given here:
Definition. The system Eq. (25) has a transmission zero at
the complex frequency sk if there exist complex vectors uk
Cm, xk Cn, one of which is nonzero, such that the system Eq.
(25) with initial condition x(0) xk, and input u(t) ukeskt,
t 0 has the property that y(t) 0 for all t 0.
The initial condition xk must be chosen carefully to ensure
that the state trajectory does not contain modes other than
those of the input eskt, because those modes are observable (the
minimality assumption) and lead to nonzero outputs. Thus,
x(t) xkeskt is a solution for the trajectory of the system. Substituting this solution in the system Eq. (25) with input
u(t) ukeskt gives the following equations after dividing by the

term eskt:
sk xk = Axk + Buk
0 = Cxk + Dxk
Rearranging these equations as a set of linear equations
in the unknowns xk, uk yields

sk I A B
0
xk
=
0
C
D
uk
This is in the form of a generalized eigenvalue problem. Indeed, under the assumption of a minimal realization, the
MIMO transmission zeros are obtained as the roots of the following equation:

I 0
A B
det s
=0
0 0
C D
For a given transmission zero sk, the generalized eigenvector
associated with that transmission zero provides the initial
condition and input directions xk, uk which yield zero output
at that input frequency.
Example. Consider the MIMO LTI system described by
state-space matrices
1
0
0
A = 0 2
0
0
0 3
1 0
B = 1 0
0 1

0.5 0 0
C=
0
1 1

1 0
D=
1 0
The transfer function matrix for this two-input, two-output
system is given by

s+1.5
0
s+1
H (s) = s+3
1
s+2
s+3
Because A is diagonal, the poles of the system are easily determined as 1, 2, 3. Solving the generalized eigenvalue
problem, one obtains two transmission zeros, at frequencies
1.5 and 2. In particular, the input

1
u(t) =
e1.5t
4.5
with initial condition

2

x(0) = 2
3
yields output y(t) 0. Similarly, the input

0 2t
u(t) =
e
1
uncontrollable. Then, any uncontrollable mode of the system

k with left eigenvector uk is a left transmission zero with direction k 0, k uk. Any unobservable mode of the system
with right eigenvector vk is a right transmission zero with directions xk vk, uk 0. Thus, the presence of unobservable
and uncontrollable modes gives rise to transmission zeros in
the same directions as the modes of the system, leading to
pole-zero cancellations.
with initial condition
0

x(0) = 1
1
Singular Values and MIMO Frequency Response
also yields y(t) 0 for t 0. Note the presence of both a zero

and a pole at 2, without a pole-zero cancellation. Note also
that the zero at 2 is not a zero of any individual transfer
function entry in the transfer function matrix.
Now consider the general case where the number of inputs
m is different from the number of outputs p and the state
space model Eq. (25) is still controllable and observable. If
the number of inputs is less than the number of outputs, the
appropriate generalization is the concept of a right transmission zero, as defined here:
Definition. The system Eq. (25) has a right transmission zero
at the complex frequency sK if there exist complex vectors uk
Cm, xk Cn, both of which are not identically zero, such
that
sk xk = Axk + Buk
0 = Cxk + Dxk
In essence, a right transmission zero is a complex frequency where, for an appropriate input direction and initial
condition, the output is identically zero. When the number of
inputs m is greater than the number of outputs p, there are
additional complications, because there can exist nonzero
proper (m 1) transfer functions U(s) such that H (s)U(s)
0 for all s! Instead of declaring every complex frequency a
transmission zero, one defines the concept of a left transmission zero, as follows:
Definition. The system Eq. (25) has a left transmission zero
at the complex frequency sk if there exist complex vectors k
Cm, k Cn, both of which are not identically zero, such
that
sk kT = kT A + kT C
0 = kT B + BTk D
Consider an LTI MIMO system, specified by its transfer function matrix H (s). Assume that the system is bounded-input,
bounded-output stable, with no initial conditions. The transfer function H (s) can be interpreted as the complex gain of
the linear system in response to bounded inputs of the form
est. That is, if the input is defined as u(t) ves0t for t 0 for
some complex number s0 with nonpositive real part and some
direction vector v Rm, the output y(t) is given by
y(t) = H (s0 )ves 0 t
The frequency response of the system is the set of transfer
functions H ( j) for all frequencies R. Thus, the frequency response of the system defines the outputs corresponding to sinusoidal inputs of the form e jt.
In single-input, single-output (SISO) systems, the transfer
function is a scalar. Thus, the frequency response is characterized by the complex-valued function H ( j), which is represented by a magnitude and phase. In contrast, the frequency
response of MIMO systems is a complex, matrix-valued function of the frequency, which has a range of gains, depending
on the direction a of the sinusoidal input. To understand how
to represent this effect, it is useful to review some concepts of
gains for complex-valued matrices.
Complex Matrices and Gains. At a specific frequency, the
transfer function matrix H ( j) is a complex-valued matrix of
dimension p m. Denote by Cpm the space of complex-valued
matrices of dimension p m. Any matrix M Cpm, is decomposed into its real and imaginary parts, as
M = A + jB
where A, B Rpm. In a manner similar to a vector, the Hermitian of a matrix is defined as the complex conjugate of its
transpose, that is,
M H = AT jBT
that is, a left transmission zero is a right transmission zero

of the state-space model
d
x(t) = AT x(t) + CT u(t)
dt
y(t) = BT x(t) + DT u(t)
59
Given a matrix M Cpm, the spectral norm of the matrix,

denoted as M2, is the maximum amplification of any input
unit vector, defined as
(30)
For square systems, any frequency that is a left transmission

zero is also a right transmission zero.
As a final note on this topic, consider a state-space realization that is not minimal, so that it is either unobservable or
M2 = maxMv2
v 2
A complex-valued square matrix is called Hermitian if

MH M. A nonsingular, complex-valued matrix is called unitary if M1 MH, which implies MMH MHM I. Hermitian
matrices have the property that all of their eigenvalues are
60
real-valued. This can be readily derived by noting that, for an

eigenvalue with right eigenvector u,
(uH Mu)H = (uH u)H = H uH u
= uH M H u = uH Mu = uH u
which establishes that is equal to its complex conjugate and
thus is a real number. Hermitian matrices also have the property that repeated eigenvalues have a full complement of eigenvectors, and thus Hermitian matrices are represented as
diagonal matrices with an appropriate change of basis.
The eigenvalues of unitary matrices have unit magnitude,
as is readily seen from
(uH M H )(Mu) = (H uH )(u) = ||2 uH u

= uH (M 1 M)u = uH u
Thus, unitary matrices acting on vectors preserve the Euclidean norm. Let M Cpp be unitary and u Cp be an arbitrary
vector. Then
Mu2 =

(Mu)H Mu = uH (M H M)u = uH u
Now consider an arbitrary matrix M Cpm. The square

matrices MHM and MMH are Hermitian and thus have realvalued eigenvalues. They also have the additional property
that the eigenvalues are nonnegative, and the nonzero eigenvalues of MMH are equal to the nonzero eigenvalues of MHM.
Let denote an eigenvalue of MMH with eigenvector v. Then,
vH (MM H )v = v22 = (M H v)H M H v = M H v22 0
which shows that 0. If is a nonzero eigenvalue of MMH
with eigenvector u, then
M H (MM H )u = M H u = (M H u) = (M H M)M H u
which establishes that is also an eigenvalue of MHM with
eigenvector MHu. Note that MHu must be nonzero if is
nonzero.
For a general matrix M Cpm with rank k, the singular
values of M are the k square roots of the nonzero eigenvalues
of MHM or MMH. Let i(M), i 1, . . ., k denote the k singular
values and i(MHM) denote the corresponding k nonzero eigenvalues of MHM. Then,

i (M) = i (M H M) = i (MM H ),
a nearly diagonal transformation. Define the matrix of singular values as
.
=
..
1 (M) 2 (M) . . . k (M) > 0

The singular-value decomposition of a matrix M Cpm
states that there are convenient changes of bases in Cp and
Cm, so that the linear transformation M can be visualized as
0
0
..
.
k
0k(mk)
0 ( pk)(mk)
Note that k min(m, p); if k m or k p, some of the zero

blocks in the above matrix are removed. The singular value
decomposition states the following:
Theorem. Given a matrix M Cpm, there exist a p p unitary matrix U and m m unitary matrix V such that
M = UV H
= U H MV
The column vectors of U are called the left singular vectors
of M, and the column vectors of V are called the right singular
vectors of M. Because U and V are unitary, they correspond
to an orthogonal change of basis in Cp and Cm, respectively. It
is easy to show that the left singular vectors of M are the
normalized right eigenvectors of the p p matrix MMH and
that the right singular vectors of M are the normalized right
eigenvectors of the m m matrix MHM. Reliable and efficient
numerical techniques for computing singular-value decompositions are available in commercial software packages.
The singular-value decomposition allows us to estimate the
gain of the matrix M when acting on an input u of unit Euclidean norm, as follows. Let y Mu denote the output of M
with input u. Using the singular-value decomposition yields
y = UV H u = Uv
where v is also a unit norm vector because V is unitary. Then,
y2 = U H y2 = U H Uv2 = v2
This establishes that
M2 1 (M)
If u is the first column of V, the unitary property of V gives

1
0
VHu =
..
.
0
i = 1, . . ., k
Because the k nonzero eigenvalues of MHM are real and

positive, the singular values are also real and positive. Assume that the singular values are ordered in descending order, that is,
0
2
..
..
.
.
0
0 ( pk)k
which shows that

M2 1 (M)
which establishes that the spectral norm of M is equal to the
maximum singular value.
It is also possible to establish a lower bound on the gain of
the matrix M under the condition that the number of outputs
p is greater than or equal to the number of inputs m. Let

m min(p, m). When the rank of M is less than m, define
the singular values k1 . . . m 0. Then, a similar
argument as previous establishes that, for unit norm vectors m,
in radians/second on a semilog scale. Figure 2 illustrates the

MIMO frequency response plot for the transfer function matrix

s+1.5
0
s+1
H(s) = s+3
1
s+2
Mu2 m 0
If the rank of M is m, the lower bound is strictly positive.
When the number of outputs is less than the number of inputs (p m), M must have a nontrivial null space, and thus
the lower bound is always 0.
Singular-Value Representation of MIMO Frequency Response. Now consider the MIMO frequency response of a
bounded-input, bounded-output stable system with transfer
function matrix H (s). Assume that the number of inputs m is
less than or equal to the number of outputs p. When the input
vector is a complex exponential of the form u(t) aejt, the
output vector is given by
y(t) = H ( j)u(t) = H ( j)ae jt = be jt
for some complex vector b H ( j)a. A useful characterization of the MIMO frequency response is provided in terms of
bounds on the gain of complex matrix H ( j) as a function
of frequency.
For each frequency , the singular-value decomposition of
H ( j) is obtained as
with nonnegative singular values 1(), . . ., m(). Assume

that a has unit Euclidean norm. Then, for each frequency, the
maximum and minimum singular values 1(), m() are
available, and
1 H ( j)a2 m
A simple way of visualizing the gain of the transfer function matrix is to plot the maximum and minimum singular
values, expressed in decibels, in a Bode plot against frequency
10
0
Gain (dB)
10
20
30
40
50
60
100
101
102
Frequency (rad/s)
s+3
The information contained in such a Bode plot provides

direction-independent information concerning the magnitude
of the frequency response at specific frequencies. In particular, at frequencies where the minimum singular value is
large, all of the singular values of the system are large, and
thus the system has a large gain in all directions. In regions
where the maximum singular value is small, all of the singular values are small, and the system has a small gain in all directions.
At other frequencies, it is necessary to use the direction
of the input a to determine the magnitude of the frequency
response. The maximum and minimum singular values provide bounds on the range of gains which are possible. The
unitary matrix V( j) is a change of basis transformation on
the input space. Thus, VH( j)a is an m-dimensional complex
vector which is a decomposition of a into components along
the right singular vectors which form a new basis in the input
space. The magnitude and phase of the frequency response
for each singular direction are readily evaluated from the matrices () and U( j).
BIBLIOGRAPHY
H ( j) = U ( j)()V H ( j)
70
101
61
103
Figure 2. Singular-value MIMO frequency response.
1. L. M. Silverman, Representation of linear dynamical systems,

2. E. W. Kamen, New results in realization theory for linear timevarying analytic systems, IEEE Trans. Autom. Control, 24: 866
877, 1979.
3. W. J. Rugh, Linear System Theory, Upper Saddle River, NJ: Prentice-Hall, 1996.
4. G. Strang, Linear Algebra and Its Applications, San Diego: Harcourt Brace Jovanovich, 1988.
5. J. M. Maciejowski, Multivariable Feedback Design, Wokingham,
UK: Addison-Wesley, 1989.
6. A. G. J. MacFarlane and N. Karcanias, Poles and zeros of linear
multivariable systems: A survey of the algebraic, geometric and
complex variable theory, Int. J. Control, 24: 3374, 1976.
7. H. H. Rosenbrock, State-Space and Multivariable Theory, New
York: Wiley, 1973.
8. M. K. Sain and C. B. Schrader, The role of zeros in the performance of multi-input, multi-output feedback systems, IEEE
Trans. Educ., 33: 244257, 1990.
9. C. B. Schrader and M. K. Sain, Research in systems zeros: A
survey, Int. J. Control, 50 (4): 14071433, 1989.
10. E. W. Kamen, Poles and zeros of linear time-varying systems,
Linear Algebra and its Applications, 98: 263289, 1988.
11. M. G. Safonov, A. J. Laub, and G. L. Hartmann, Feedback properties of multivariable systems: The role and use of the return difference matrix, IEEE Trans. Autom. Control, AC-26: 4765, 1981.
1980.
DAVID CASTANON
Boston University
NONLINEAR CONTROL SYSTEMS, ANALYTICAL

METHODS
A large number of methods exist for the analysis and design

of linear control systems. Unlike a linear system, a nonlinear system does not satisfy the superposition property,
which means not only that it may be difcult to deduce how
the system will respond to a specic input if its response is
known to a different input, but also nonlinear systems exhibit unique behavior due to the effects of the nonlinearity.
In many systems it is possible to separate the static nonlinear effects from the dynamic so that a nonlinear system
can often be accurately modeled as a combination of static
nonlinear elements and linear dynamic elements. Here the
concentration is primarily on analytical methods for nonlinear systems which are associated with those aspects of
linear theory usually referred to as classical control. This
means basically that for systems other than second order,
frequency domain, rather than state space, models and formulations are used. An exception to this is the material
on variable structure systems. The state space theme, and
some design methods presented within that framework are
given in the following article.
A block diagram of a simple nonlinear feedback system
which will receive signicant attention in this article is
shown in Fig. 1. Although it only contains one nonlinear
element, its presence can change the whole behavioral possibilities of the feedback loop compared with the linear situation, and its form is adequate for discussing many of the
analysis and design techniques presented in this article.
Since all practical systems contain some form of nonlinearity, it is important that basic concepts relating to the effects of nonlinearity are well understood. When this is the
case it will allow the designer to assess qualitatively, if not
quantitatively, the possible effects of nonlinear operation at
various points within a feedback system and to take them
into account in the design. This may allow the analysis and
design to be done using one or more linearized models. A
full nonlinear simulation may then be used to check that
the design works satisfactorily when the nonlinear effects
are included. This approach works satisfactorily in many
instances, particularly if gain scheduling is used to counteract the effects of changes produced by any nonlinearity;
however, this approach cannot be used for all situations.
Many nonlinear effects which take place in control systems may be modeled approximately using static nonlinearities. These include saturation in ampliers, dead zones
in valves, friction, and backlash in gears. Depending on
the approach to be used in the analysis or design when
these nonlinearities exist, it may be necessary to approx-
Figure 1. A simple nonlinear feedback system.
imate their characteristics by either simple continuous

mathematical functions, such as polynomials, or linear segmented approximations. To apply some methods which we
will discuss, it may even be necessary to use coarser approximations to a nonlinearity simply to say that it is conned within a sector. These types of nonlinearities are often referred to as inherent nonlinearities, since for a satisfactory design they will exist due to the devices used, although for analysis we may wish to neglect them. It is also
true that good designs will always be nonlinear, since trying to ensure linear operation of a system will involve the
selection of oversized components such as pumps, motors,
and heaters. Nonlinearity may be introduced intentionally
into control systems to compensate for existing nonlinear
effects, or to implement a design strategy which is either
preferable technically or more economical.
The simple feedback system of Fig. 1, provided that the
nonlinearity or transfer functions are suitably chosen, may
exhibit a variety of behaviors which are unique to nonlinear systems. First, the performance of the system, even for
a specic type of input, will depend upon the amplitude of
the input. The response, for example, to a small step input
may be quite different from that of a large step input. If the
autonomous systemthat is, the system with no input
is released from several initial states, then the resulting
behavior may be appreciably different for each state. For
example, instead of reaching a stationary equilibrium, the
system may move from some initial conditions into a limit
cycle, a continuous oscillation which can be reached from a
subset of initial conditions. This behavior is distinct from
an oscillation in an idealized linear system since the magnitude of this latter oscillation is dependent upon the initial
energy input to the system. A limit cycle is a periodic motion, but its waveform may be signicantly different from
the sinusoid of an oscillation. The autonomous nonlinear
system may also have a chaotic motion, a motion which is
repeatable from given initial conditions but which exhibits
no easily describable mathematical form, is not periodic,
and exhibits a spectrum of frequency components.
If a sinusoidal input is applied to the system, then the
output may be of the same frequency but will also contain harmonics or other components related to the input
frequency. This output too, for certain frequencies and amplitudes of the input, may not be unique but has an amplitude dependent upon the past history of the input or the
initial conditions of the system. The sinusoidal input may
also cause the system to oscillate at a related frequency
so that the largest frequency component in the output is
not the same as that of the input. Also if, for example, the
autonomous system has a limit cycle, then the addition
of a sinusoidal input will cause the limit cycle frequency
to change and possibly cause synchronization of the limit
cycle frequency with the input frequency or one of its harmonics. In many instances the phenomena just mentioned
are undesirable in a control system, so that one needs techniques to ensure that they do not occur. Control systems
must be designed to meet specic performance objectives,
and to do this one is required to design a control law which
is implemented based on measurements or estimation of
the system states or, by simple functions of the system
variables, typically the error signal. Many systems can be
Nonlinear Control Systems, Analytical Methods
made to operate satisfactorily with the addition of a simple controller in the error channel, which is shown by the
transfer function Gc (s) in Fig. 1. Typical performance criteria, which the system may be required to meet, are that it
is stable, has zero steady-state error and a good response to
a step input, suitably rejects disturbances, and is robust to
parameter variations. Although one reason for using feedback control is to reduce sensitivity to parameter changes,
specic design techniques can be used to ensure that the
system is more robust to any parameter changes. If the process to be controlled is strongly nonlinear, then a nonlinear
controller will have to be used if it is required to have essentially the same step response performance for different
input step amplitudes. Some control systemsfor example, simple temperature control systemsmay work in a
limit cycle mode, so that in these instances the designer is
required to ensure that the frequency and amplitude variations of the controlled temperature are within the required
specications.
In the next section, we examine in some detail various
approaches which can be used for investigating the analysis and design of nonlinear systems. The rst topic discussed is the phase plane method, which can normally only
be used to investigate second-order systems. It is a useful
technique, since it can be used when more than one nonlinearity exists in the system and since many control problems, such as position control systems, can be modeled approximately by second-order dynamics. As mentioned previously, one specication for the design may be that the
system must be stable. For linear systems, assessment of
stability is a simple problem, but this is not the case for a
nonlinear system, even when it is as simple as that shown
in Fig. 1. Several absolute stability criteria exist for checking whether the system of Fig. 1 will be stable, and these
are discussed in more detail later. The criteria presented
are easy to use; and the circle criterion in particular, being
basically an extension of the Nyquist criterion, is easy to
implement and follow. A disadvantage, however, is that all
these criteria only produce sufcient conditions so that if
the condition is violated the system may still be stable.
To try to obtain an estimate of the possibility of this
being the situation, the describing function method has
been used by engineers for many years. The difculty with
the describing function approach, which approximates the
nonlinearity by its gain to a sinusoidal input, is that the
results are not exact. It does, however, enable the designer
to obtain more insight into the situation, and, of course, the
ideas can often be further checked by simulation. The describing function approach can also be helpful for system
design in terms of shaping the frequency response of the
system to produce a more stable situation or for indicating possible nonlinear effects which can be added in the
controller to counteract the nonlinear effects in the plant.
Describing functions for other than a single sinusoid can
be obtained, and these allow some of the more complex
aspects of the behavior of nonlinear systems to be investigated. These include, for example, synchronization and
subharmonic generation as well as estimating more accurately the frequency content of any limit cycle. Relay-type
characteristics are often introduced in control system to
provide economic designs or to produce variable structure
systems. First, a method for the determination of limit cycles in relay systems is presented. This is an interesting
approach, since it allows the exact evaluation of a limit cycle and also an exact determination of whether it is stable
or not. The method in this sense is unique, since exact limit
cycle data for systems with any order dynamics containing
a relay can be obtained.
As with the design of linear control systems, the issue of
robustness to unmodeled dynamics and parameter uncertainty is also pertinent in the nonlinear control area. One
such robust technique is the so-called variable structure or
sliding mode approach. Variable structure control systems
(VSCS) are characterized by a set of feedback control laws
and an associated decision rule or switching function. This
decision rule has as its input some measure of the current
system behavior and produces as an output the particular
feedback control law which should be used at that instant
in time. The resulting variable structure system (VSS) may
be regarded as a combination of subsystems, where each
subsystem has a xed control law which is valid for specied regions of system behavior. One of the advantages of
introducing this additional complexity into the system is
the ability to combine useful properties of each of the subsystems into a very exible closed-loop control strategy. Indeed, it will be seen that a VSS may be designed to possess
new properties which are not present in any of the composite structures. Utilization of these natural ideas began
in the late 1950s in the Soviet Union and formed the foundations for signicant contributions to the area of robust
nonlinear control.
Of particular interest in the area of VSS is the so-called
sliding mode behavior, where the control is designed to
drive and then constrain the system state and lie within
a neighborhood of the switching function. There are two
signicant advantages with this approach to controller design. First, the dynamic behavior of the system may be
tailored by the particular choice of switching function. Second, the closed-loop response becomes totally insensitive to
changes in certain plant parameters and will completely
reject a particular class of external disturbances. This invariance property clearly renders sliding mode control a
strong candidate for robust control. In addition, the ability
to specify performance directly makes sliding mode control
attractive from the design perspective. This is seen from
the wide exposure of sliding mode control to many applications areas including robotics, aerospace, and automotive
industries.
The sliding mode design approach involves two stages.
The rst consists of the design of an appropriate switching
function to ensure that the system behavior during sliding
motion satises the required design specications. This is
termed the existence problem. In the simplest case, this
will be seen to amount to the design of a linear full-state
feedback controller for a particular subsystem. The second
design stage is concerned with the selection of a control law
which will make the switching function attractive to the
system state. This is termed the reachability problem. It is
important to note that this control law is not necessarily
discontinuous in nature.
THE PHASE PLANE METHOD

The phase plane method was the rst approach used by
control engineers for studying the effects of nonlinearity
in feedback control systems. The technique can generally
only be used for systems represented by second-order differential equations. It had previously been used in nonlinear mechanics and for the study of nonlinear oscillations.
Smooth mathematical functions were assumed for the nonlinearities so that the second-order equation could be represented by two nonlinear rst-order equations of the form
Equilibrium, or singular points, occur when
and the slope of any solution curve, or trajectory, in the

x1 x2 state plane is
Figure 2. Limit cycle solution of Van der Pol equation with

= 1.0.
where is a positive constant. The phase plane form of this

equation can be written as
The slope of a trajectory in the phase plane is

A second-order nonlinear differential equation representing a control system with smooth nonlinearity can typically
be written as
and if this is rearranged as two rst-order equations, choosing the phase variables as the state variablesthat is,
x1 = x, x2 = xit can be written as
which is a special case of Eq. (3). A variety of procedures

have been proposed for sketching state (phase) plane trajectories for Eqs. 3 and 5. A complete plot showing trajectory motions throughout the entire state (phase) plane
from different initial conditions is known as a state (phase)
portrait. Knowledge of these original methods, despite the
immense improvements in computation since they were
rst proposed, can be particularly helpful for obtaining
an appreciation of the system behavior. When simulation
studies are undertaken, phase plane graphs are easily obtained and they are often more helpful for understanding
the system behavior than displays of the variables x1 and
x2 against time.
Many investigations using the phase plane technique
were concerned with the possibility of limit cycles in the
nonlinear differential equations. When a limit cycle exists,
this results in a closed trajectory in the phase plane; typical of such investigations was the work of Van der Pol. He
considered the equation
which is only singular (that is, at an equilibrium point),

when the right-hand side of Eq. (4) is 0/0, that is, x1 = x2 = 0.
The form of the singular point, which is obtained from
linearization of the equation at the origin, depends upon ,
being an unstable focus for < 2 and an unstable node for
> 2. All phase plane trajectories have a slope of r when
they intersect the curve
One way of sketching phase plane behavior is to draw a

set of curves for various selected values of r in Eq. (5) and
marking the trajectory slope r on the curves, a procedure
known as the method of isoclines. Figure 2 shows a simulation result from a small initial condition leading to the
stable limit cycle solution for = 1.0.
Many nonlinear effects in control systems, such as saturation and friction, are best approximated by linear segmented characteristics rather than continuous mathematical functions. This is an advantage for study using the
phase plane approach, since it results in a phase plane divided up into different regions but with a linear differential
equation describing the motion in each region.
To illustrate the approach, consider a basic relay
position-control system with nonlinear velocity feedback
having the block diagram shown in Fig. 3. First, let us
assume that the hysteresis in the relay is negligible (i.e.,
= 0) and that h is large so that the velocity-feedback signal will not saturate. Denoting the system position output
by x1 and its derivative x1 by x2 , we note that the relay
output of 1 or 0 is equal to x1/K and that the relay input is

equal to x1 x2 = 1. Taking the dead zone of the relay
Figure 3. Block diagram of relay positioncontrol system.
to be equal to 1, the motion of the system is described

by
Thus in the phase plane, which has x1 as abscissa and x2 as

ordinate, the dashed lines x1 + x2 = 1 in Fig. 4 divide the
plane into the three regions, for each of which the motion
is described by one of the above three simple linear secondorder differential equations. The solution of
in terms of the phase-plane coordinates x1 and x2 is
where x10 and x20 are the initial values of x1 and x2 . Since
Eq. (6) describes a parabola, which for the special case
of K = 0 has the solution x2 = x20 , it is easy to calculate the systems response from any initial condition (x10 ,
x20 ) in the phase plane. Figure 4 shows the response from
(4.6, 0) with = 1 and K = 1.25. The initial parabola meets
the rst switching boundary at A; the ensuing motion is
horizontalthat is, at constant velocityuntil the second
switching boundary is reached at B. The resulting parabola
meets the same switching boundary again at C, at which
point motion from either side of the switching line through
C will be directed toward C, so that the resulting motion is
a sliding motion. Responses from any other initial conditions are obviously easy to nd, but, from the one response
shown, several aspects of the systems behavior are readily apparent. In particular, the system is seen to be stable
since all responses will move inward, possibly with several
overshoots and undershoots, and will nally slide down a
switching boundary to 1. Thus a steady-state error of unit
magnitude will result from any motion.
When the velocity-feedback signal saturatesthat is,
when |x2 | > hthe input signal to the relay is x1 h. The
switching boundaries change to those shown in Fig. 5, but
the equations describing the motion between the boundaries remain unaltered. Therefore for a large step input
the response will become more oscillatory when the velocity saturates. When the hysteresis is nite then the switch-
Figure 4. Initial condition response.
Figure 5. Changed switching boundaries due to saturation.
ing lines for positive (negative) x2 move to the right(left) at

their intersection with the x1 axis. If h is large it is then
easily shown that a limit cycle, as shown in Fig. 6 for = 1
and = 0.5, will occur. Trajectories both inside and outside
the limit cycle have their motion directed toward it. Similarly, it is straightforward to draw phase-plane trajectories
for a nite hysteresis and smaller values of h.
Figure 6. Response terminating in a limit cycle for = 1 and

= 0.5.
Figure 7. Graphical illustration of the Popov criterion.
ABSOLUTE STABILITY CRITERIA

A very important question in control is to be able to ascertain the stability of a feedback system. The problem for
linear systems was examined over a century ago in Cambridge, England, by Routh, who published his famous work
on the stability of motion in 1877. As a result of this work
and further contributions, most notably by Nyquist, several
approaches are now available for determining the stability
of a feedback loop such as Fig. 1 when the nonlinearity n(x)
is replaced by a linear gain K. The methods provide necessary and sufcient conditions for stability. The rst work
on the stability of nonlinear systems by Lyapunov was published in 1892, and since that time there have been many
attempts to determine necessary and sufcient conditions
for the stability of the autonomous feedback systemthat
is, r = 0of Fig. 1. Lyapunov formulated an approach for
determining sufcient conditions, but the difculty of his
method is that it requires determination of a function of
the system states which then must be shown to satisfy certain properties. There is no general approach for nding a
suitable function; when one is found, it does not guarantee that a better function does not exist which will prove
stability in a larger domain in the state space. The problem has therefore been researched by many people with
the objective of obtaining conditions for stability which are
relatively easy to use.
Several frequency-domain results (1) giving sufcient,
but not necessary, conditions for stability have been determined which use limited information about the nonlinearity, n(x), typically its sector bounds or the sector bounds of
its slope. The nonlinearity n(x) has sector bounds (k1 , k2 );
that is, it is conned between the straight lines k1 x and k2 x
if k1 x2 < xn(x) < k2 x2 for all x. Similarly, it has slope bounds
(k 1 , k 2 ) if k 1 x2 < xn (x) < k 2 x2 , where n (x) = dn(x)/dx. The
Popov criterion (2) states that a sufcient condition for the
autonomous system of Fig. 1 to be stable if G(s) is stable
and G() > k1 is that a real number q > 0 can be found
such that for all we obtain
where the nonlinearity n(x) lies in the sector (0, k). The
theorem has the simple graphical interpretation shown in
Figure 8. Illustration of the circle criterion.
Fig. 7, where for the system to be stable a line of slope q1

can be drawn through the point k1 so that the Popov
locus G*(j) lies to the right of the line. The Popov locus is
given by
The circle criterion (3) is more conservative than the Popov

criterion but can be used when both the nonlinearity is time
varying and there is a bounded input, r, to the system of
Fig. 1. Satisfaction of the circle criterion guarantees that
the autonomous system is absolutely stable and the system
with bounded input has a bounded output. The criterion
uses the Nyquist locus, G(j), and for stability of the system
of Fig. 1 with n(x) in the sector (k1 , k2 ) it is required that
G(j) for all real has the following properties. If the circle
C has its diameter from 1/k1 to 1/k2 on the negative
real axis of the Nyquist diagram, then (1) if k1 k2 < 0, G(j)
should be entirely within C, (2) if k1 k2 > 0, G(j) should
lie entirely outside and not encircle C, and (3) if k1 = 0 or
k2 = 0, G(j) lies entirely to the right of 1/k2 or to the left
of 1/k1 . The situation for stability in case (2) is shown in
Fig. 8.
Two simple transformations are also useful for investigating the absolute stability of the autonomous system
of Fig. 1. Feeding forward around the nonlinearity and
backward around the dynamics G(s), through a constant
gain , whose effects cancel out, changes the nonlinearity
sector to (k1 , k2 ) and the linear transfer function
to G(s)/[1 + G(s)]. Alternatively feeding backward around

n(x) and forward around G(s) changes the nonlinearity sector to (k1 /(1 k1 ), (k2 /(1 k2 )) and changes the linear
transfer function to + G(s). This is needed in order to apply the Popov criterion to the general nite sectorthat is,
n(x) in the sector (k1 , k2 ).
Prior to the derivation of these frequency-domain results, Aizermann had put forward a conjecture that the
autonomous system of Fig. 1 would be stable for a nonlinearity sector bounded by (k1 , k2 ) if for k1 k2 > 0 the Nyquist
locus G(j) of a stable transfer function did not touch or encircle the line between 1/k1 and 1/k2 , which is of course
the diameter of the circle of Fig. 8. Several counterexamples have been put forward to show that the conjecture is
incorrect; however, it can be shown that if the conjecture is
satised, the system may possess a limit cycle but its output cannot go unbounded (4). For a monotonic nonlinearity
with slope bounds (k 1 , k 2 ) and k 1 k 2 > 0, an off-axis circle
criterion exists (5). This states that the autonomous system
of Fig. 1 with a nonlinearity satisfying the aforementioned
conditions will be absolutely stable if the Nyquist locus of a
stable transfer function does not encircle a circle centered
off the real axis and which intercepts it at (1/k 2 , 1/k 2 ).
DESCRIBING FUNCTION METHOD

The describing function (DF) method was developed simultaneously in several countries during the 1940s. Engineers found that control systems which were being used
in many applicationsfor example, gun pointing and antenna controlcould exhibit limit cycles under certain conditions rather than move to a static equilibrium. They realized that this instability was due to nonlinearities, such
as backlash in the gears of the control system, and they
wished to obtain a design method which could ensure that
the resulting systems were free from limit cycle operation. They observed that when limit cycles occurred the
observed waveforms at the system output were often approximately sinusoidal, and this indicated to them a possible analytical approach. Initial investigations therefore
focused on the autonomous feedback system with a single nonlinear element shown in Fig. 1 containing a static
nonlinearity n(x) and linear dynamics given by the transfer function G(s) = Gc (s)G1 (s). It was recognized that if a
limit cycle existed in the autonomous system with the output c(t) approximately sinusoidal, then the input x(t) to the
nonlinearity could be assumed sinusoidal, the corresponding fundamental output of the nonlinearity could be calculated, and conditions for this sinusoidal self-oscillation
could be found, if the higher harmonics generated at the
nonlinearity output were neglected. This is the concept of
harmonic balance, in this case balancing the rst harmonic
only, which had previously been used by physicists to investigate such aspects as the generation of oscillations in
electronic circuits. The DF of a nonlinearity was therefore
dened as its gain to a sinusoidthat is, the ratio of the
fundamental of the output to the amplitude of the sinusoidal input. Since describing functions can be used for
other than a single sinusoidal input to a nonlinearity, as
discussed in the latter part of this article; this DF is often,
for clarity, called the sinusoidal DF (SDF).

The Sinusoidal Describing Function
We assume that if in Fig. 1 we have x(t) = a cos , where
= t and n(x) is a symmetrical odd nonlinearity, then the
output y(t) will be given by the Fourier series.
where
and
The fundamental output from the nonlinearity is a1 cos

+ b1 sin , so that the DF is given by
which may be written
where
Alternatively, in polar coordinates,
where
It is further easily shown that if n(x) is single valued, then

b1 = 0. Although Eqs. 7 and 8 are an obvious approach to
the evaluation of the fundamental output of a nonlinearity, they are somewhat indirect, in that one must rst determine the output waveform y() from the known nonlinear characteristic and sinusoidal input waveform. This is
avoided if the substitution = cos1 (x/a) is made, in which
case, after some simple manipulations, it can be shown that
The function p(x) is the amplitude probability density function of the input sinusoidal signal and is given by
An additional advantage of using Eqs. 9 and q0 is that

they easily yield proofs of some interesting properties of
the DF for symmetrical odd nonlinearities. These include
the following:
1. For a double-valued nonlinearity the quadrature

component Nq (a) is proportional to the area of the
nonlinearity loop, that is, Nq (a) = (1/a2 ) (area of
nonlinearity loop)
2. For two single-valued nonlinearities n (x) and
n (x), with n (x) < n (x) for all 0 < x < b, we obtain
N (a) < N (a) for input amplitudes less than b.
3. For the sector bounded single-valued nonlinearity
that is, k1 x < n(x) < k2 x for all 0 < x < bwe have
k1 < N(a) < k2 for input amplitudes less than b. This
is the sector property of the DF, and it also applies
for a double-valued nonlinearity if N(a) is replaced
by M(a).
When the nonlinearity is single-valued, it also follows directly from the properties of Fourier series that the DF,
N(a), may also be dened as follows:
1. The variable gain, K, having the same sinusoidal input as the nonlinearity, which minimizes the meansquared value of the error between the output from
the nonlinearity and that from the variable gain.
2. The covariance of the input sinusoid and the nonlinearity output divided by the variance of the input.
Figure 9. Saturation characteristic and output waveform.
ideal saturation characteristic shown in Fig. 9 the nonlinearity output waveform y() is as shown in the same gure.
Because of the symmetry of the nonlinearity the fundamental of the output can be evaluated from the integral
over a quarter period so that
which for a > gives
Evaluation of the Describing Function

Tables of DFs for a variety of nonlinear characteristics can
be found in many books (6, 7). However, to illustrate the
evaluation of the DF of a nonlinearity a few simple examples are considered below.
with = sin1 /a.

Evaluation of the integrals gives
Cubic Nonlinearity. For this nonlinearity n(x) = x3 and

using Eq. (16), one has
which on substituting for gives the result
Since for a < the characteristic is linear, giving N(a) = m,

the DF for ideal saturation is mNS (/a), where
giving N(a) = 3a2 /4.

Alternatively from Eq. (23) we have
Alternatively one can integrate Eq. (9) by parts to give
a
n
The integral n =
x p(x) dx is known as the nth moment of the probability density function; and for the sinusoidal distribution with p(x) = (1/)(a2 x2 )1/2 , n has the
value
n (x)(a2 x2 )1/2 dx
a1 = (4/a)
if
n(0) = 0
so that using the substitution x = a sin , this yields
as before.
Therefore a1 = (4/a) 34 a4 = 3a3 /4, as before.
Saturation Nonlinearity. The DF can also be found by taking the nonlinearity input as a sin , in which case for the
Relay with Dead Zone and Hysteresis. The characteristic

of a relay with dead zone and hysteresis is shown in Fig.
10 together with the corresponding input, assumed equal
to a cos , and the corresponding output waveforms. Using
Eqs. 7 and 8 over the interval /2 to /2 and assuming

that the input amplitude a is greater than + gives
valued or whether G(j) is available from a transfer function G(s) or as measured frequency response data. Typically the functions G(j) and N(a) are plotted separately
on Bode, Nyquist, or Nichols diagrams. Alternatively, stability criteria such as the HurwitzRouth or root locus plots
may be used for the characteristic equation
where = cos1 [( )/a] and = cos1 [( + )/a,] and
Thus
For the alternative approach, one must rst obtain the inphase and quadrature nonlinearities which are shown in
Fig. 11. Using Eqs. 23 and 24, one obtains
The DFs for other relay characteristics can easily be found

from this result. For no hysteresis, = 0; for no dead zone,
= 0; and for an ideal relay, = = 0.
It is easily shown that the DF of two nonlinearities in
parallel is equal to the sum of their individual DFs, a result which is very useful for determining DFs, particularly
of linear segmented characteristics with multiple break
points. Several procedures (6) are available for obtaining
approximations for the DF of a given nonlinearity either by
numerical integration or by evaluation of the DF of an approximating nonlinear characteristic dened, for example,
by a quantized characteristic, linear segmented characteristic, or Fourier series.
Stability and Limit Cycles
To study the possibility of limit cycles in the autonomous
closed loop system of Fig. 1, the nonlinearity n(x) is replaced
by its DF N(a). Thus, the open-loop gain to a sinusoid is
N(a)G(j) and a limit cycle will exist if
where G(j) = GC (j)G1 (j). This condition means that the

rst harmonic is balanced around the closed loop. Since
G(j) is a complex function of and N(a) may be a complex function of a, a solution to Eq. (13) will yield both the
frequency and amplitude a of a possible limit cycle.
Various approaches can be used to examine Eq. (13)
with the choice affected to some extent by the problem
for example, whether the nonlinearity is single- or double-
although here it should be remembered that the equation

is appropriate only for s j.
Figure 12 illustrates the procedure on a Nyquist diagram, where the G(j) and C(a) = 1/N(a) loci are plotted and shown intersecting for a = a0 and = 0 . The DF
method therefore indicates that the system has a limit cycle with the input sinusoid to the nonlinearity, x, equal
to a0 sin(0 t + ), where depends on the initial conditions. When the G(j) and C(a) loci do not intersect, the DF
method predicts that no limit cycle will exist if the Nyquist
stability criterion is satised for G(j) with respect to any
point on the C(a) locus. Obviously, if the nonlinearity has
unit gain for small inputs, the point (1, j0) will lie on C(a)
and it may then be used as the critical point, analogous to
the situation for a linear system.
When the analysis indicates that the system is stable,
its relative stability may be indicated by evaluating its gain
and phase margin. These can be found for every amplitude
a on the C(a) locus, so it is usually appropriate to use the
minimum values. In some cases a nonlinear block also includes dynamics so that its response is both amplitude and
frequency dependent and its DF will be N(a, ). A limit cycle will then exist if
To check for possible solutions of this equation, a family of

C(a, ) loci, usually as functions of a for xed values of ,
may be drawn on the Nyquist diagram.
A further point of interest when a solution to Eq. (40) exists is whether the predicted limit cycle is stable. This is obviously important if the control system is designed to have
a limit cycle operation, as in the case of an onoff temperature control system, but it may also be important in other
systems. If, for example, an unstable limit cycle condition
is reached, the signal amplitudes will not become bounded
but may continue to grow. The stability of a limit cycle, provided that only one solution is predicted, can be assessed
by applying the Nyquist stability criterion to points on the
C(a) locus at both sides of the solution point. If the stability
criterion indicates instability (stability) for a point on C(a)
with a < a0 and indicates stability (instability) for a point
on C(a) with a > a0 , then the limit cycle is stable (unstable).
The situation is more complicated when multiple limit cycle solutions exist and the above criterion is a necessary
but not sufcient result for the stability of the limit cycle
(8).
The stability of the limit cycle can then normally be ascertained by examining the roots of the characteristic equation
Figure 10. Relay and input/output waveforms.
Figure 11. In-phase and quadrature characteristics for the relay of Fig. 10.
evaluated from
where n (x) and p(x) are as previously dened. It can also

be shown that Ni (a) is related to N(a) by the equation
Figure 12. Illustration of limit cycle evaluation.
where Ni (a) is known as the incremental describing function (IDF). For a single-valued nonlinearity, Ni (a) can be
Thus, for example, for an ideal relay, putting = = 0 in Eq.

(12) gives N(a) = 4h/a, and substituting this value in Eq.
(14) yields Ni (a) = 2h/a.
As an example of using the DF to investigate the possibility of a limit cycle, consider Fig. 1 with n (x) = x (x3 /6),
Gc (s) = 1 and G2 (s) = K (1 s)/s(s + 1). For this nonlinearity,
N(a) = 1 (a2 /8), so that the C(a) locus starts at 1 on
the Nyquist diagram and, as a increases, moves along the
then, for a greater
negative real axis to for a = 2
than this value, the C(a) locus returns along the positive
real axis from to the origin as a becomes large. An os-
10
cillation will only start to build up, assuming zero initial

conditions, if the feedback loop with G(s) alone is unstable
since N(a) 1 for a 0. This requires the characteristic
equation
to have a root with a positive real part; that is, K > 1. G(j)
has 180 phase shift when = 1 when its gain is K. Thus
the DF solution for the amplitude of the limit cycle is given
by
which results in
giving
As K is increased, because of the shape of the nonlinearity, the limit cycle becomes more distorted. For example, if
K = 2.4, the DF solution gives = 1 and a = 2.10, whereas
if four harmonics are balanced, which requires a computer
program, the limit cycle frequency is 0.719 and the amplitudes of the fundamental, third, fth, and seventh harmonics at the input to the nonlinearity are 2.515, 0.467, 0.161,
and 0.065, respectively.
As the DF approach is a method for evaluating limit cycles, it is sometimes suggested that it cannot be used to
guarantee stability of a feedback system, since instability
may be exhibited by a signal in the system becoming unbounded, not oscillatory. It is, however, known for the autonomous feedback system of Fig. 1 that if the symmetric
odd, single-valued nonlinearity n(x) is sector-bounded such
that k1 x < n(x) < k2 x for x > 0 and n(x) tends to k3 x for large
x, where k1 < k3 < k2 , then the nonlinear system is either
stable or possesses a limit cycle, provided that the linear
system with gain K replacing N is stable for k1 < K < k2 .
Thus for this situation, which is often true in practice, the
nonexistence of a limit cycle guarantees stability.
Accuracy of the Describing Function
Since the DF method is an approximate analytical approach, it is desirable to have some idea of its accuracy.
Unfortunate consequences may result if a system is predicted to be stable, and in practice this turns out not to be
the case. Although many attempts have been made to nd
solutions for this problem, those that have been obtained
either are difcult to apply or produce results which are
often as conservative as the absolute stability criteria discussed earlier.
Since, as has already been shown, the C(a) locus of a
sector bounded, single-valued nonlinearity is the diameter of the circle in the circle criterion, errors in the DF
method are related to its inability to predict a phase shift
which the fundamental may experience in passing through
the nonlinearity, rather than an incorrect magnitude of the
gain. When the input to a single-valued nonlinearity is a
sinusoid together with some of its harmonics, it is easy
to show that the fundamental output is not necessarily in

phase with the fundamental input; that is, the fundamental gain has a phase shift. The actual phase shift which
occurs varies with the harmonic content of the input signal in a complex manner, since the phase shift depends
on the amplitudes and phases of the individual harmonic
input components.
From an engineering viewpoint, one can therefore obtain a good idea of the accuracy of a DF result by estimating the distortion, d, in the waveform at the input to
the nonlinearity. This is relatively straightforward when
a limit-cycle solution is obtained since the sinusoidal signal corresponding to the DF solution can be taken as the
nonlinearity input and the harmonic content of the signal
fed back to the nonlinearity input calculated. Experience
indicates that the percentage accuracy of the DF method
in predicting the fundamental amplitude and frequency of
the limit cycle is usually better than the percentage distortion in the fedback signal.
It is also important to note that the amplitude predicted
by the DF is an approximation to the fundamental of the
limit cycle, not its peak amplitude. It is possible to estimate
the limit cycle more accurately by balancing additional harmonics, as mentioned earlier. Although algebraically this
is difcult apart from loops with a nonlinearity having a
simple mathematical descriptionfor example, a cubicit
can be done computationally. The procedure involves solving sets of nonlinear algebraic equations, but good starting
guesses can usually be obtained for the magnitudes and
phases of the other harmonic components from the waveform fedback to the nonlinearity, assuming that its input
is the DF solution.
Further Aspects
Before concluding this section on the DF method, it is important to mention two other facets of its application. In
introducing the DF, it was indicated that the existence of
a limit cycle is usually undesirable; thus if the DF indicates such behavior, the system must be compensated to
remove the limit cycle. If the parameters of n(x) and G1 (s),
with Gc (s) = 1, in Fig. 1 are such that a limit cycle is indicated, because the loci G1 (j) and C(a) intersect, then a
compensator Gc (s) can be added with a transfer function
such that the loci Gc (j)G1 (j) and C(a) do not intersect.
Shaping frequency responses to achieve a specic form is a
familiar approach in linear control theory, so this approach
can be easily applied. Other approaches such as adding additional feedback paths to compensate for the effect of the
nonlinearity may also be possible. This procedure has the
advantage, as can the approach of designing a nonlinear
controller, of producing an approximately linear system.
A feature of nonlinear systems, as mentioned earlier, is
that they possess unique forms of behavior. One such interesting feature is the jump resonance which can occur
when a nonlinear feedback system, such as Fig. 1, has a
sinusoidal input. Equations can be set up using the DF
approach for the feedback loop to balance the harmonic
at the frequency of the input sinusoid. Two nonlinear algebraic equations are obtained; and for some situations
they can have three, rather than one, solutions for a small
range of input frequencies. The DF can also be used to show

that only two of the solutions will be stable, which means
that the approximately sinusoidal output from the feedback loop may have two possible values, within this frequency range, which, if it exists, is found near to the resonant frequency of the linearized system. When the input
frequency is changed so that the solution of the equations
moves from the two-stable-solution to the one-solution (or
vice versa) region, a discontinuous change, or jump, in the
magnitude of the output may occur.
11
Figure 13. Relay output waveform.
where UG (n) = gn cos n and VG (n) = gn sin n Eq. (15)

can be written as
LIMIT CYCLES IN RELAY SYSTEMS

In this section an exact method for the evalution of limit
cycles and their stability is discussed which makes use of
the fact that the output from a relay is not continuously affected by its input (6, 9). The input only controls the switching instants of the relay and has no further effect on the
output until it causes another switching. Therefore to investigate limit cycles in relay systems the analysis starts
by assuming a typical relay output waveform, y(t), which
for a symmetrical odd limit cycle in a loop having a relay
with dead zone and hysteresis takes the form shown in Fig.
13, where T and t are unknown and the initial switching
is at time t1 . Then to nd a possible limit cycle in the autonomous system of Fig. 1 the steady-state response of G(s)
to this waveform has to be determined. Several slightly different approaches are possible, but here we follow that used
by Tsypkin, primarily because for a relay with no dead zone
it allows a simple comparison with the DF method. y(t) is
expanded in a Fourier series which gives
y(t)
2h 1
{sin(nt)cos[n(t t1 )]
Similarly, c(t) can be shown to be given by
To satisfy the above-mentioned switching conditions at

times t1 and t1 + t, assuming t1 to be zero without loss of
generality, and bearing in mind that x(t1 ) should be positive and x(t1 + t)negative, we require that
where RP and IP denote the real and imaginary parts, respectively. The IP expressions give two nonlinear algebraic
equations which, if they have solutions, yield the unknown
parameters t and T of possible limit cycles. Using these
solution values, the corresponding relay input waveform
x(t) can be found, from which the RP conditions can be
checked, as can the continuity conditions
n=1(2)
+[1 cos(nt)]sin[n(t t1 )]}

The output c(t) is then given by
where gn = |G(jn)| and n = G(jn). Using the A loci, dened by
to conrm that the relay input signal does not produce

switchings other than those assumed.
Since closed-form expressions exist for the A loci of simple transfer functions, analytical solutions can be obtained
for the exact frequencies, 1/T, of limit cycles for some specic systems, especially those in which the relay has no
dead zone. Then t = and the above two nonlinear algebraic equations are identical since only one unknown,
T, remains. When the nonlinear algebraic equations are
solved computationally the closed-form expressions for the
A loci may be used, or their value may be determined by
taking a nite number of terms in the series of Eqs. 16 and
17 (6, 9). Another interesting feature of this method is that
it is also possible to determine whether a solution to the
nonlinear algebraic equations corresponds to a stable or an
unstable limit cycle (10). The analysis has assumed a symmetrical odd limit cycle but can be extended to situations
where this is not the case. More nonlinear algebraic equations have then to be solved to obtain any possible limit
cycle solutions. It is also possible to extend the approach to
12
Figure 14. Graphs giving exact and approximate solutions for

limit cycle.
Figure 15. Stable limit cycle solution.
nd multi-pulse type limit cycles which may exist in relay

control of resonant plant transfer functions.
Two Examples
As a simple application of the above method, consider a
relay with no dead zonethat is, = 0so that one has the
single relationship
which yields the frequency of the limit cycle. If G(s) = K/s(1+

s), then the above expression gives the equation
where = for the limit cycle frequency . This compares

with the DF solution for the same problem, which yields
the equation
It is also interesting that, since the line with RP < 0 and

IP = /4h corresponds to C(a), the negative reciprocal
of the DF, the exact and approximate DF solutions can be
compared graphically. This is done in Fig. 14, which shows
the G(j) and AG (0, ) loci for K = = 1 and the C(a) locus
for h/ = 3. The exact limit-cycle frequency is 1.365 rad/s,
and the approximate solution using the DF method is 1.352
rad/s. The accuracy of the DF result may be used to conrm the lter hypothesis, since it can be shown that as
is increased, thus making G(s) a better low-pass lter, the
error in the DF solution for the frequency of the limit cycle
decreases.
Consider as a second example a feedback system having a relay with output 1 and dead zone 1, along with
a transfer function G(s) = 5/s(s2 + 3s + 1). Use of the DF
method indicates that the system has two limit cycles, both
of frequency 1.000 rads/s, with the larger amplitude one
stable and the smaller amplitude one unstable. Two nonlinear algebraic equations need to be solved using the Tsypkin method to nd the frequency and pulse width of any
limit cycles. Software with graphical facilities is available
to do this and the two limit cycles shown in Figs. 15 and
Fig. 16 are found. The larger amplitude limit cycle of 15
are found. The larger amplitude limit cycle of 15 is shown
Figure 16. Unstable limit cycle solution.
by the method to be stable with frequency 0.988 rads/s and

pulse width 1.967 s, and the smaller amplitude one of Fig.
16 with frequency 0.736 rads/s and pulse width 0.716 s is
unstable. It should also be noted that the larger amplitude
limit cycle is much closer to a sinusoid so that its frequency
is nearer the DF solution of 1.000 rads/s.
SLIDING MODE CONTROL METHODS
For ease of exposition, consider the uncertain linear time
invariant system with m inputs given by
where A Rnn and B Rnm with 1 m. Without loss of

generality it is assumed that the inputs are independent.
The nonlinear function f: R Rn Rm Rn is assumed to
be unknown but bounded by some known functions and
represents the parameter uncertainties, nonlinearities, or
disturbance signals which are present in the system. Let
S be the hyperplane dened by
where S Rmn is of full rank. This will dene the switching function. It should be noted that the choice of S need
not be restricted to a hyperplane and more general, nonlinear, possibly time-varying switching functions may be
chosen.
If there exists a nite time ts such that the solution to
Eq. (61) satises
then a sliding motion is taking place for all t ts .

This section will rst consider how to design the switching function so that the sliding motion is stable. The problem of designing variable structure control laws so that in
nite time the system states are forced on to and subsequently remain on the hyperplane S is considered next.
The total insensitivity to a particular class of uncertainty
is then demonstrated. The section will conclude with a
straightforward example to illustrate the mathematical
concepts.
For sliding mode design it is necessary that the system
assumes an appropriate canonical form. This so-called regular form is obtained using the orthogonal matrix T1 Rnn
whereby
where B2 Rmm and is full rank. The transformation matrix exists as B is full rank and can be readily found via QR
decomposition. By using the coordinate transformation x
T1 x then the nominal linear system can be written as
where x1 Rnm , x2 Rm . Effectively, the system has been

decomposed into two connected subsystems, only one of
which is directly affected by the system input. If the switching function matrix from Eq. (62) is partitioned compatibly
in this coordinate system, then
where S1 Rm(nm) and S2 Rmm . During ideal sliding,

the motion is given by
Assuming S2 is chosen by design to be nonsingular, it follows that
where M = S1 2 S1 . It further follows that in the sliding

mode, m of the states can be expressed in terms of the remaining (n m) and thus a reduction in order occurs. The
reduced order motion is determined from substituting for
x2 (t) from Eq. (21) in Eq. (20) as
The hyperplane design problem can therefore be considered to be one of choosing a state feedback matrix M to
prescribe the required performance to the reduced order
subsystem dened by (A11 , A12 ). It can be shown that controllability of the nominal (A, B) pair is sufcient to guarantee controllability of the (A11 , A12 ) pair.
13
Having dened the switching function, it is necessary

to establish sufcient conditions which guarantee that an
ideal sliding motion will take place. These will amount to
ensuring that in a certain domain enclosing the sliding surface, the trajectories of s(t) must be directed toward it. The
associated so-called reachability condition is perhaps most
succinctly expressed as
This choice is readily justied by considering the function
which is positive denite. Its time derivative along any trajectory is
It follows that if Eq. (70) holds, then V tends to zero and

therefore s tends to zero. This guarantees that a sliding
mode is attained. The control signal must thus be dened
to satisfy Eq. (70). Subject to this constraint, there are obviously a great many possible control congurations. A common structure is given by
where u1 (t) is a linear state feedback law and un (t) is a

discontinuous or switched component of the form
The extensive interest in sliding mode control is primarily

due to its robustness properties. When sliding, a system is
completely insensitive to any uncertainty which is implicit
in the channels of the input distribution matrix; such uncertainty is termed matched uncertainty. The reason for
this invariance property is easily demonstrated by a consideration of the uncertain state space system
where f is an unknown but bounded forcing function. In

the sliding mode
and thus
Substituting from Eq. (75) into Eq. (77),
where ueq is not the applied control signalwhich will be

of the form of Eq. (73)but does denote the equivalent linear control that would be required to maintain the sliding
mode. This may be expressed as
Substituting this expression for the equivalent control in

Eq. (75) yields
14
The dynamics in the sliding mode are thus completely invariant to the signal f. The system behavior will be determined entirely by Eq. (69) when sliding despite the presence of such matched uncertainty. Any unmatched uncertainty will affect the dynamics of Eq. (69), but such unmatched effects can be minimized by designing M such that
the subsystem of Eq. (69) is maximally robust.
In order to illustrate the theoretical concepts of VSCS,
consider the double integrator system
The system can be expressed in the state-space form
where x1 = y and x2 = y. This system is already in an appropriate regular form for sliding mode design. Consider
application of a negative feedback law
The phase portraits when k = 0.5 and k = 1.5 are shown in

Fig. 17. Neither control law yields an acceptable transient
when employed as the sole controller; an oscillation is seen
to exist in both cases. Consider instead the VSCS dened
by
An asymptotically stable motion is seen to result as shown

in Fig. 18. By introducing a rule for switching between
two control structures, which independently do not provide stability, a stable closed-loop system is formed. Such
heuristic arguments can be used to motivate the advantages of a variable structure control approach. However, for
design purposes a more logical algorithmic approach is required.<gureAnchor gures="W1024-g-0017 W1024-g0018"/>
Consider now the switching function dened by
This is seen to provide a rst-order motion in the sliding

mode where the pole dening the transient response is determined by the selection of m. The control signal is dened
by solving for u from the relationship
This clearly ensures that the reachability condition [Eq.

(23)] is satised. Essentially, the switching function is differentiated and the resulting state derivatives are replaced
with the original system dynamics. Equation (27) thus
yields an expression for the control signal in terms of the
states and the value of the switching function. The resulting controller will nd the switching function at least
locally attractive. For simulation, the double integrator
model is subject to a disturbance signal a1 sin (x1 (t))
which acts in the range of the input distribution matrix.
In this way a controller designed for a nominal double integrator model is implemented upon a normalized pendu-
Figure 17. (a) u = 1.5x1 ; (b) u = 0.5x1 . Double integrator control.
lum system. The design parameters m and k are both set

equal to unity. Figure 19 shows the resulting phase plane
plot. The system enters a sliding mode and the normalized
pendulum is forced to behave as the free rst-order system
during this phase of motion. The dynamics in the sliding

mode have been wholly specied by the choice of switching function despite the presence of a matched uncertainty
contribution.
OTHER DESCRIBING FUNCTIONS
In a previous section the discussion on describing functions was primarily restricted to the sinusoidal describing function, SDF, since this is used extensively in looking
at the effects of nonlinearity in practical systems. Many
control systems, however, are subject to inputs or disturbances which cannot be dened deterministically but only
as random signals with a given frequency spectrum and
amplitude probability density function. The most common
amplitude probability density function, p(x), considered is
the Gaussian density function which, when it has a zero
Figure 18. Double integrator with variable structure control system.
15
ate p(x) is used. When the input x is a sinusoidal or Gaussian signal, however, it can also be shown that the error
signal between the nonlinearity and linear gain outputs
that is, n(x) Keq xis uncorrelated with the input x (6).
Typically, when dealing with Gaussian inputs to a simple
nonlinear feedback system, the mean-squared values of the
signals at various points in the loop can be calculated approximately using the RDF for the nonlinearity.
In many feedback systems it may be necessary to take
account of bias, as well as other signals, due to constant input or disturbance signals or because of asymmetry in nonlinear characteristics. In this case the nonlinearity, again
using the minimum mean-squared error denition, may
be modeled by two DFs, one for the bias, , and one for the
other input (6, 7). When the other input is considered as a
sinusoid of amplitude a, then the two DFs for the singlevalued nonlinearity n(x) are given by
and
the former being the DF for the sinusoid and the latter for
the bias. Here p(x) = 1/(a2 x2 )1/2 for the sinusoidal signal.
Use of this DF allows, amongst other possibilities, for the
determination of limit cycles with bias. For example, if in
Fig. 1 the input r(t) has a constant value R, then balancing
the bias and fundamental of the limit cycle gives the two
equations
Figure 19. Normalized pendulum with sliding mode control.
mean, is given by
A general way to dene a DF, as mentioned earlier, is that

it is that value of gain, Keq , which, when fed with the same
input as the nonlinearity n(x), will give a minimum value
for the mean-squared error between the nonlinearity output and the gain output. It is then relatively easy to show
that
when the input signal x has a Gaussian distribution. When

this result was rst found, the gain was called the equivalent gain, hence the notation Keq , not the random describing function (RDF) by which name it is now usually known.
The result for x with any zero-mean amplitude probability
density function p(x) is
a formula which can be used for a single-valued nonlinearity with sinusoidal or Gaussian inputs when the appropri-
The equations can be solved to nd the bias and sinusoidal amplitude a of the limit cycle at the input to the
nonlinearity.
The above approach can be used in principle to obtain a
DF representation for a nonlinearity whose input consists
of any number of uncorrelated signals, but for practical
reasons the approach is difcult to justify for more than
two or possibly three components. A difculty in applying
such multiple input describing functions is understanding
the errors which are caused by neglecting not only higher
harmonics of the input signals but also cross-modulation
products which may be produced at frequencies lower than
those in the nonlinearity input signal.
Reasonably successful use, however, of the DF approach
for two related frequency sinusoidal inputs to a nonlinearity has been achieved to give results of some value in
control system design. This requires consideration of two
inputs such as a cos t and b cos(3t + ) so that the describing functions for the two signals become functions of
the three parameters a, b and , not just a and b (6). Analytically, results can only be obtained for simple nonlinearities such as a cubic; but by using computational methods, other characteristics can be considered (11). This procedure has been used to investigate subharmonic oscillations and synchronization phenomena when the feedback
loop of Fig. 1 has an input r(t) which is sinusoidal, and it
16
Obviously, the relay onoff levels control the limit cycle

amplitude; and if these are varied, some information may
be found about any nonlinearity in the plant (14, 15). In
such cases it may be possible to make the system behavior
more linear by incorporating appropriate nonlinearity in
the PID elements of the controller.
Figure 20. Kc and wc from the Nyquist plot.
has also been used for the more accurate determination of

limit cycle waveforms by balancing both the fundamental
and another harmonic, usually the third.
RELAY AUTOTUNING
A relatively recent application of the DF approach in control system design is its use in the relay autotuning method
for setting PID controller parameters. The procedure is
very useful in those cases where it is difcult to obtain
a good mathematical model for the process or retuning has
to be done on site by an operator. The basic concept employed is that knowledge of the critical frequency, c , and
gain, Kc , of a process illustrated on a plant frequency response shown in Fig. 20 can often provide sufcient information for setting the parameters of a PID controller. The
approach was suggested many years ago by Ziegler and
Nichols (12), where Kc and c were found by placing the
PID controller in the P mode and adjusting the gain until oscillation took place. There were difculties doing this
in practice, however, one problem being that the oscillation
amplitude is only limited by saturation in the controller, actuator or plant. More recently therefore, Astrom and Hagglund (13) recommended estimating c and Kc from results
obtained by replacing the controller with an ideal relay, a
feature, as illustrated in Fig. 21, which can easily be built
into a modern microprocessor controller. When this is done
the amplitude of the limit cycle, a, at the relay input and
its frequency, c , are measured. Then according to DF theory, c = 0 and Kc = 4h/a. Strictly speaking, a should be
the amplitude of the fundamental frequency component,
and the results are only exact when the limit cycle is sinusoidal. In many cases, however, these formulae provide
reasonable estimates for c and Kc which may then be used
in an appropriate algorithm (many of which have been put
forward recently) for setting the PID parameters. If the
form of the plant transfer function is known but not its parameters, then it may be possible, certainly for low-order
transfer functions, to make use of the known Tsypkin solution for the limit cycle to estimate the plant parameters.
When the plant has several unknown parameters, more
than one autotuning test may have to be done using different values of hysteresis in the relay or possibly with bias
signals introduced into the loop.
So far in this presentation, only simple nonlinear feedback

systems such as Fig. 1 have been considered, apart from
when discussing the phase plane. In principle there is no
difculty in extending use of both the DF and Tsypkin approaches to feedback loops with more than one nonlinear
element, although for the latter approach the nonlinearities must be of the relay type. The problem with using
the sinusoidal describing function approach is that the assumption of a sinusoidal input to all the nonlinearities
must be reasonable for the situation under investigation.
For some congurations of nonlinearities and linear dynamic elements in a feedback loop, this will not be true. Using the Tsypkin approach, more nonlinear algebraic equations are formulated and their possible solutions must be
investigated (6, 16).
Several investigators have produced results for studies
on the multivariable, typically two-inputtwo-output version of Fig. 1. Here the nonlinearity consists of four individual, or in many cases only two on the diagonal, nonlinear elements. Using describing function analysis for this conguration can often be justied since good ltering may exist
for all theor in the case of two inputtwo output, two
feedback loops. Software written to investigate such systems, using both the DF and Tsypkin methods, has been described in the literature (1719). Absolute stability results,
similar to those given earlier but which result in more complicated graphical procedures, have been extended to the
two-inputtwo-output multivariable system (20). Like the
situation for the single-inputsingle-output system, however, the results they produce are often very conservative
and may not be of value for practical problems.
SLIDING MODE CONTROL
The detailed discussion here comments further on uncertain systems. It is obviously desirable to consider other
nominal nonlinear system descriptions in order to broaden
the applicability of the method. Some work in this area
has considered problems relating to a particular application areafor example, roboticsand has developed sliding mode controllers for such specic classes of nonlinear
system (21). Other work has tried to consider more general nonlinear system descriptions. Some of the conditions
placed upon the particular system representation can be
quite restrictive. One example of this is the class of feedback linearizable systems. It is possible to augment the
traditional linearizing feedback with a sliding mode component which will provide robustness to some uncertainty
in the sliding mode. However, the conditions which must be
satised to feedback linearize the system initially are quite
restrictive and so limit the applicability of the methodol-
17
Figure 21. Block diagram for relay autotuning.
ogy. Perhaps the most widely applicable method to date

has resulted from the development of sliding mode control
schemes for differential inputoutput system representations (22). These yield dynamic controllers which act as a
natural lter on any discontinuous elements of the control
signal and are applicable to a fairly broad class of nonlinear systems. This is thus a considerable breakthrough in
the development of robust controllers for nonlinear system
descriptions.
The previous exposition relates to state-feedback based
sliding mode control schemes (23). For practical application, controllers based upon measured output information
are required (24). There are two ways to approach this problem. A dynamical system, or observer, may be used to estimate unmeasurable system states. Because of the inherent robustness of sliding mode controllers, some signicant
work has considered the development of an associated sliding mode observer (25, 26). The robustness properties have
been shown to transfer. However, there are restrictions on
the (A, B, C) triple used for observer design. In particular,
the invariant zeros of (A, B, C) are shown to play a crucial
role in determining the zero dynamics in the sliding mode.
It thus follows that these invariant zeros must be stable.
Despite this restriction, the closed-loop robustness properties of such a sliding mode controllerobserver scheme are
excellent. The sliding observer is also nding a promising
area of application relating to fault detection and isolation. The second approach to output feedback based sliding mode control is to develop design methodologies which
produce output dependent control strategies (27). This restricts the class of systems which may be considered as
the switching surface must be output-dependent, and thus
S must lie in the range of the output distribution matrix. Again the sliding mode dynamics will be dependent
upon the location of the system transmission zeros. The
development of design methodologies and associated case
studies relating to output feedback-based sliding mode control strategies and sliding mode controllerobserver strategies require further development. The development of sliding mode controllers based upon output measurements for
nonlinear systems is very much an open research problem.
BIBLIOGRAPHY
1. K. S. Narendra, J. H. Taylor, Frequency Domain Criteria for
Absolute Stability, New York: Academic Press, 1973.
2. V. M. Popov, Absolute stability of nonlinear control systems of
automatic control, Autom. Remote Control, 22: 857858, 1962.
3. I. W. Sandberg, A frequency domain condition for the stability

of feedback systems containing a single time-varying nonlinear element, Bell Syst. Tech. J., 43 (4): 16011608, 1964.
4. D. P. Atherton, D. H. Owens, Boundedness properties of nonlinear multivariable feedback systems, Electron. Lett., 15 (18):
559ff, 1979.
5. Y. S. Cho, K. S. Narendra, An off-axis circle criterion for the
stability of feedback systems with a monotonic linearity, IEEE
Trans. Autom. Control, 13 (4): 413416, 1968.
6. D. P. Atherton, Nonlinear Control Engineering: Describing Function Analysis and Design, London: Van NostrandReinhold, 1975.
7. A. Gelb, W. E. Vander Velde, Multiple-Input Describing Functions and Nonlinear System Design, New York: McGraw-Hill,
1996.
8. S. K. Choudhury, D. P. Atherton, Limit cycles in high-order
nonlinear systems, Proc. Inst. Electr. Eng., 121: 717724, 1974.
9. P. A. Cook, Nonlinear Dynamical Systems, London: PrenticeHall International, 1986.
10. R. Balasubramanian, Stability of limit cycles in feedback systems containing a relay, IEE Proc., Part D, 1: 2429, 1981.
11. J. C. West, J. L. Douce, R. K. Livesley, The dual input describing function and its use in the analysis of nonlinear feedback
systems, Proc. Inst. Electr. Eng., Part B, 103: 463474, 1956.
12. J. G. Ziegler, N. B. Nichols, Optimum settings for automatic
controllers, Trans. ASME, 64: 759768, 1942.
13. K. J. Astrom, T. Hagglund, Automatic tuning of simple regulators with specications on phase and amplitude margins,
Automatica, 20 (5): 645651, 1984.
14. J. H. Taylor, K. J. Astrom, A Nonlinear PID Autotuning Algorithm, Seattle, WA: ACC, 1986, pp. 16.
15. D. P. Atherton, M. Benouarets, O. Nanka-Bruce, Design of nonlinear PID controllers for nonlinear plants, Proc. IFAC World
Cong. 93, Sydney, Australia, 1993, Vol. 3, pp. 355358.
16. D. P. Atherton, Conditions for periodicity in control systems
containing several relays, 3rd IFAC Cong., London, 1966, Paper 28E.
17. J. H. Taylor, Applications of a general limit cycle analysis
method for multivariable systems, inR. V. Ramnath, J. K.
Hedrick, andH. M. Paynter (eds.), Nonlinear System Analysis
and Synthesis, New York: ASME, 1980, Vol.2.
18. D. P. Atherton et al., Suns, the Sussex University Nonlinear
Control Systems Software, 3rd IFAC/IFAD Symp. Comput.
Aided Des. Control Eng. Syst., Copenhagen, 1985, pp. 173178.
19. O. P. McNamara, D. P. Atherton, Limit cycle prediction in free
structured nonlinear systems, IFAC Cong., Munich, 1987, Vol.
8, pp. 2328.
20. D. P. Atherton, Stability of Nonlinear Systems, New York: Wiley, Research Studies Press, 1981.
18
21. J. J. E. Slotine, W. Li, Applied Nonlinear Control, London:

Prentice-Hall International, 1991.
22. H. Sira Ramirez, On the dynamical sliding mode control of
nonlinear systems, Int. J. Control, 57: 10391061, 1993.
23. V. I. Utkin, Sliding Modes in Control Optimisation, Berlin:
24. C. Edwards, S. Spurgeon, Sliding Mode Control: Theory and
Applications, London: Taylor & Francis, 1997.
25. J. J. E. Slotine, J. K. Hedrick, E. A. Misawa, On sliding observers for nonlinear systems, J. Dyn. Syst. Meas. Control, 109:
245252, 1987.
26. C. Edwards, S. K. Spurgeon, On the development of discontinuous observers, Int. J. Control, 59: 12111229, 1994.
27. S. Hui, S. H. Zak, Robust output feedback stabilisation of uncertain systems with bounded controllers, Int. J. Robust Nonlinear Control, 3: 115132, 1993.
DEREK ATHERTON
SARAH SPURGEON
School of Engineering
University of Sussex Brighton,
England, BN1 9QT
Department of Engineering
University of Leicester
Brighton, England, BN1 9QT
NONLINEAR CONTROL SYSTEMS, DESIGN METHODS
NONLINEAR CONTROL SYSTEMS,

DESIGN METHODS
A basic problem in control theory may be described as follows:
Given a plant, design a control mechanism in such a way that
the plant together with the controller meets certain design
specifications. The above regulation problem arises in numerous situations; for instance, the temperature in a house is regulated by a thermostat to keep the temperature in the house
constant, notwithstanding changing external effects such as
outdoor temperature, wind, open doors, and so on. Other regulation devices in everyday life are easy to find: washing machines, modern automobiles, and so on.
Probably the first mathematical study on regulation ever
published was written by J. C. Maxwell (18311870). His paper On governors published in the Proceedings of the Royal
Society of London in 1868 treats the problem of tuning centrifugal governors to achieve fast regulation towards a constant
speed, thereby avoiding oscillatory motions (hunting) of a
steam engine.
Clearly, in the past century a lot of theoretical and practical work has been carried out on the regulation problem, and
it is certainly beyond the scope of this article to present a
historical review of the subject; readers further interested in
the subject are referred to Ref. 1. However, one particular
type of controllers, the so-called proportional integral differential (PID) controller, originally proposed by N. Minorsky in
1922, deserves separate mentioning.
In a PID controller the control signal is built up as a
weighted sum of three terms; a proportional term (proportional to the error between the actual and desired value of the
to-be-controlled plants output) drives the plants output to
the reference. An integral term (of the error) compensates for
the steady-state error caused by uncertainties in the plants
model, and a differential term (proportional to the time derivative of the plants output) speeds up the convergence towards
the desired reference. The PID controller has had and still
has many applications in technical systems.
More recent methods in nonlinear control theory that
should be mentioned are feedback linearization, passivitybased control, and Lyapunov control.
519
The feedback linearization approach applies to a small

class of systems for which it is possible to use a nonlinear
control law which, given an appropriate coordinate change,
cancels all nonlinearities in the system. The rationale behind
this approach is that the resulting closed-loop system is linear, and thus linear control theory is then applicable. A drawback is that this technique may fail if one does not know the
plants model accurately; this uncertainty can lead to instability or, in the best case, to a steady-state error.
An alternative approach is the so-called passivity-based
control. This technique applies to a certain class of systems
which are dissipative with respect to a storage function (2).
Passive systems constitute a particular case of dissipative
systems for which the storage function happens to be an energy function. Hence, the rationale behind the passivity-based
approach is physical: Roughly speaking, a passive system is a
system from which one cannot pull out more energy than is
fed in. A very simple example of a passive system is a conventional RLC network, which dissipates part of the supplied
electrical energy in the form of heat. A fundamental property
of passive systems is that the interconnection of two passive
systems is passive. With this motivation, in passivity-based
control one aims at designing passive controllers, so that the
closed-loop system have some desired energy properties. In
many cases, seeking for passive controllers results in compensating instead of canceling the nonlinearities of the system,
which can give considerably more robust results than using
feedback linearization.
Modern control theory leans on the so-called Lyapunov stability theory which was launched by the Russian mathematician A. M. Lyapunov in his celebrated article (3). The Lyapunov theory consists of a set of mathematical tools, including
comparison equations, which help us to analyze the asymptotic behavior of the solutions of a (possibly nonlinear and
time-varying) differential equation. The advantage of this
theory is that it allows us to know the asymptotic behavior of
the solutions without solving the differential equation. The
price paid for this is that one must find a suitable Lyapunov
function for the system in question, which satisfies some desired properties. More precisely, the Lyapunov function is
positive definite while its time derivative is negative definite.
This is in general not an easy task.
Thus, the Lyapunov control approach consists of proposing
a positive definite Lyapunov function candidate and designing
the control input in such a way that its time derivative becomes negative definite. Then, some conclusions about the
stability of the system can be drawn.
Although in general, it is very difficult to find a suitable
Lyapunov function, often a good start is to use the total energy function (if available) of the system in question. This
may motivate us to think that the passivity-based approach
and Lyapunov control are very related since, for a physical
system, the storage function is the total energy of the system.
Nevertheless, it must be pointed out that the passivity-based
approach is based upon the inputoutput properties of the
system; that is, the system is viewed as an operator which
transforms an input into some output, regardless of the internal state of the system. The Lyapunov control approach is
based upon the asymptotic behavior of the systems state.
Both methods are complementary to one another.
We consider in more detail passivity-based and feedback
linearization schemes, and the reader may consult Ref. 4 and
520
5 for introductory texts on Lyapunov theory. We consider

from a mathematical perspective the regulation problem of
an important class of nonlinear passive systems: the so-called
EulerLagrange (EL) systems. This class of systems includes
various mechanical systems such as the robot manipulators.
Our motivation to illustrate these control techniques by
addressing the regulation problem for EL systems is multiple:
Not only are EL systems passive (hence passive controllers do
a good job for this class), but also the stability theory of Lyapunov was inspired upon the previous work of J. H. Poincare
(18541912) on second-order nonlinear systems and, in particular, mechanical systems. Moreover, even though the Lagrangian formulation is most popular in mechanics, it must
be remarked that it applies to a wide class of physical systems
which are modeled using variational principles. Thus, on one
hand, the EL class is fairly wide; hence the available results
are applicable to many physical (industrial) systems. On the
other hand, the theory is simple enough to be treated in a few
pages and to give the reader a general appreciation of the
flavor of nonlinear systems control design in this context.
We consider the regulation problem which consists of designing a feedback that enforces asymptotic tracking of the
output to be controlled (e.g., the endpoint of a rigid manipulator) towards a given desired trajectory. Provided that an exact
model of the plant is given and assuming that full state (that
is, joint position and velocity) measurements are available, a
tracking controller can easily be designed, like for instance
the computed torque controller (6). See, for example, Refs. 7,
8, and 18 for a literature review.
Unfortunately, in many cases, neither an exact model is
available, nor are joint velocities measured. The latter problem can be resolved by introducing an observer, a dynamic
system which uses only position information (or more general,
the measurable output) to reconstruct the velocity signal (or
more general, the unmeasured part of the state). Then, the
controller is implemented by replacing the velocity by its estimate. It is interesting to note that even though it is well
known that the separation principle (see section entitled Linear Time-Invariant Systems) does not apply for nonlinear
systems (specifically, an observer that asymptotically reconstructs the state of a nonlinear system does not guarantee
that a given stabilizing state feedback law will remain stable
when using the estimated state instead of the true one), the
rationale behind this approach is precisely that the estimate
will converge to the true signal and, in some particular cases
of EL systems, this in turn entails stability of the closed loop.
To cope with a nonexact plant model, one may proceed in
different ways, depending on to what extent the model uncertainties appear. In essence, discussion in this article will be
limited to the occurrence of some unknown (linearly depending) parameters, such as an unknown mass of the endtool, or there will be limited discussion of the model structure.
In the first case a parameter adapting or a PID controller can
be used whereas in the last case we return to a (high-gain)
PD controller design. Some of the above-mentioned results for
EL plants are discussed in some detail in the following sections and can be found in the literature (8,9). See also Ref. 7
for a comprehensive tutorial on adaptive control of robot manipulators.
One may wonder whether results as announced above for
fully actuated systems also extend to other classes of nonlinear systems. Apart from the case where the system is as-
sumed to be linear (see Refs. 10), the observer and observer

controller problems turn out to be difficult in general. At the
moment, very few general results are at hand (11); as an illustration in the last section we discuss one specific mathematical example of a single-input single-output nonlinear system.
xx is the Euclidean norm

Notation. In this article, x
n
of x . The largest and smallest eigenvalues of a square
symmetric matrix K are kM and km, respectively. The extended
n
n
space of square integrable signals is L 2e

u , T
T
0 : 0 u(t)2 dt . We denote B x n : x a ball
of radius , centered at the origin in n.
THE REGULATION PROBLEM
We define the general regulation problem for NL systems as
follows:
Tracking Control of NL Systems. Consider the system
x = f (t, x, u)
(1)
y = h(x)
(2)
where x n is the state, u k is the control input, y

m is the output to be controlled, and functions f and h are
continuous in their arguments and t 0. Given a desired
reference output trajectory yd(t) assume that there exists a
(unique) desired state xd(t) and input ud(t) trajectories which
generate yd(t), that is,
xd = f (t, xd , ud )
(3)
yd = h(xd )
(4)
Then the tracking control problem consists in finding, if possible, a smooth control law u such that for any bounded initial
conditions (t0, x0), x0 x(t0),
lim y(t)
, t
lim (y(t) yd (t)) = 0
(5)
Remark 1. It is important to note that we have assumed that

there exists xd(t) and ud(t) which are solutions of Eq. (3). This
point is crucial since in practical applications one should pay
attention to the definition of the desired output trajectory,
which may not be realizable due to the structure of the system. In this article we do not address the so-called trajectory
generation problem (see, for instance, Ref. 12) or the inversion problem considered in Ref. 13, but we simply assume
that the triple (xd, ud, yd) is given.
Note that in the problem formulation above we have not
imposed any specific structure for the control law; however,
one can distinguish (at least) the following variants of the
tracking control problem depending on the measurements
which are available:
1. (State feedback). Assume that the full state is available
for measurement; then find, if possible, a control law u
of the form
(a) (Static state feedback) u (t, x, xd), or
(b) (Dynamic state feedback) u (t, x, xd, xc), xc (
t, x, xd, xc) with xc being the dynamic compensator
state, say xc l.
2. (Output feedback). Assume that only an output z

k(x) is measurable; then find, if possible, a control law
u of the form
(a) (Static output feedback) u (t, z, z, xd),or
(b) (Dynamic output feedback) u (t, z, xd, xc), xc
(t, z, xd, xc) with, again, xc l being the dynamic
compensator state.
LINEAR TIME-INVARIANT SYSTEMS

Before presenting some results on state and output feedback
control of nonlinear systems we briefly revisit some facts
about linear systems theory. It is well known that the controllerobserver design problem for linear systems is, completely
solved (10), but the static output feedback problem is only
partially understood. Consider the linear time-invariant (LTI)
forced system
x = Ax + Bu
y = Cx
(6)
where y m, x n, u k, and A, B, and C are matrices

of suitable dimensions. Assume that we wish to achieve the
tracking control for system (6), which in the linear case boils
down to solving the tracking problem for xd 0, with corresponding yd 0 and ud 0. In order to do so we use the
linear state-feedback controller
u = Kx
y = Cx
(8)
(9)
(10)
where L is an output-injection matrix. Note that the estimation error x x x dynamics has the form
x = (A LC)x
u = K x
(12)
where x follows from Eq. (9). To find an answer to this question we write the overall controlled system plus observer:
x
A BK
=
0
x

BK
x
A LC x
(13)
and calculate the roots of its characteristic polynomialthat

is, the roots of

BK
sI A + LC
sI A + BK
p(s) = det
0
or, equivalently, the roots of
p(s) = det(sI A + BK) det(sI A + LC)

That is, the characteristic polynomial of the overall system is
the product of the characteristic polynomials of the observer
[Eq. (11)] and the controlled system [Eq. (8)]. Thus one can
design the controller and observer separately without caring
if the true states are available or if the observer will be used
in open or closed loop. This nice property is called the separation principle and, unfortunately, in general it is exclusive to
linear systems.
For this reason, we are obliged to explore new techniques
to achieve output feedback control for nonlinear systems.
FEEDBACK LINEARIZATION
is exponentially stable; that is, the matrix (A BK) must be

Hurwitz. Indeed a necessary and sufficient condition for the
matrix K to exist is the stabilizability of the system [Eq. (6)].
Now, as we have mentioned we are also interested in the
output feedback control problem. To date, even for linear systems the tracking control problem using static output feedback is not yet completely solved, and we therefore look at
the observer-design problem. For system [Eq. (6)] a linear observer is given by
x = Ax + Bu + L(y y)
as if they were the true ones, that is,
(7)
where K is chosen in a way that the closed-loop system

x = (A BK)x
521
(11)
It is clear that one can find an L such that Eq. (11) be asymptotically stablethus x 0 as t if the pair (A, C) is
detectable (10).
At this point we have successfully designed an asymptotically stabilizing feedback and a asymptotically converging observer. The natural question which arises now is whether it
is possible to use the state estimates in the feedback Eq. (7)
Consider a single input nonlinear system

x = f (x) + g(x)u
(14)
where x n, u , and f and g are smooth vector fields on

n; that is, their derivatives exist and are continuous up to
infinite order. In this section we describe the feedback linearization approach for system [Eq. (14)] around an equilibrium
point x0 n; that is, f(x0 0. The local feedback linearization problem for the system [Eq. (14)] is to findif possiblea static state feedback law
u = (x) + (x)v,
(x0 )
= 0
(15)
and a smooth, local coordinate transformation

S(x0 ) = 0 Rn
z = S(x),
(16)
such that the closed-loop system [Eqs. (14)(15)] in the z coordinates is a controllable system and without loss of generality
may be assumed to be in the Brunovski form:

0
z1
.
.

d
.=
.
dt .

.
0
zn
1
..
.

z1
0
.
. .
. .
+ . v
.
0
1 ..
1
0
zn
..
(17)
522
The local linearization problem is solvable only under quite

restrictive conditions on f and g. In order to see this we will
derive a set of sufficient conditions for the solvability of the
linearization problem. Assuming there exists a feedback (15)
and a coordinate transformation (16) that linearize the closedloop system, we note the following. Let zi Si(x), i 1, . . .,
n; then we have, using the first equation of Eq. (17)
z 1 =
S1 (x)
S1 (x)
(x) x =
(x) f (x)
x
x
S1 (x)
+
(x) g(x)u = z2 = S2 (x)
x
(18)
Defining Lf S(x) S(x)/x f(x) as the directional or Lie derivative of function S(x) in the direction of f, we obtain from Eq.
(18) that
S2 (x) = L f S1 (x)
(19)
0 = Lg S1 (x)
(20)
In an analogous way we derive, using the ith equation of Eq.

(17)
Si+1 (x) = L f Si (x),
i = 1, . . . , n 1
(21)
0 = Lg Si (x),
i = 1, . . . , n 1
(22)
If we introduce the Lie bracket of two vector fields g1 and g2

on n denoted by [g1, g2] as
g
g
[g1 , g2 ](x) = 2 (x)g1 (x) 1 (x)g2 (x)
x
x
(23)
then it follows that the function S1(x) should satisfy S1(x0)

0 and the n 1 conditions
Lg S1 (x) = L[ f,g] S1 (x) = L[ f,[ f,g]] S1 (x) = L[ f,... ,[ f,g]... ] S1 (x) = 0
(24)
In other words, if the local feedback linearization problem is
solvable, then there exists a nontrivial function S1(x) that satisfies the n 1 partial differential equations [Eq. (24)]. The
functions and in the feedback [Eq. (15)] are given as follows. Since
z n = f (x)Sn (x) + Lg Sn (x)u = v
(25)
we obtain from Eq. (15)

(x) = (Lg Sn (x))1 L f Sn (x),
(x) = (Lg Sn (x))1
(26)
In general one may not expect that a nontrivial function

S1(x) exists that fulfills Eq. (24). Writing the iterated Lie
brackets of the vector fields f and g as adfkg [f, adk1
f g], k
1, 2, . . ., with adf0g g, one can derive the following necessary and sufficient conditions for the existence of S1(x) (see,
for example, Refs. 13 and 14).
Theorem 1. Consider the system [Eq. (14)] about an equilibrium point x0. The local feedback linearization problem is
solvable about x0 if and only if
1. The vector fields adfigi 0, . . ., n 1 are linearly independent.
2. For any two vector fields X1, X2 in the set

adfi g, i 0, . . ., n 2 we have
[X1 , X2 ](x) =
n2
i (x)ad if g(x)
1=0
for certain functions 0, . . ., n2.

Note that the theorem above gives necessary and sufficient
conditions for local feedback linearizability. For global results, further conditions on f and g are required. In the following example of a flexible joint pendulum the local solution
extends to the full state space and thus become global.
Example 1. Consider the model of a robot link with a flexible
joint (15,16)
Iq 1 + mgl sin q1 + k(q1 q2 ) = 0

J q 2 k(q1 q2 ) = u
(27)
where q1 is the angle of the link, q2 is the angle of the motor

shaft, and u is the torque applied to the motor shaft. The
flexibility is modeled via a torsional spring with constant k,
m is the mass of the link, and l is the distance from the motor
shaft to the center of mass of the link. J and I are the momenta of inertia of motor and link, respectively. With x
(q1, q1, q2, q2) we obtain a system of the form Eq. (14) with
x2
mgl sin x k (x x )
1
1
3
I
f (x) =
x4
k
(x1 x3 )
J
0
0
g(x) =
0
1/J
(28)
One may verify that conditions (1) and (2) for feedback linearization are fulfilled about the equilibrium x0 0. In order to
find the linearizing coordinate change [Eq. (16)] and feedback
[Eq. (15)], we first solve Eq. (24). Using Eq. (28), this set of
partial differential equations reads as
S1 (x)
S1 (x)
S1 (x)
=
=
=0
x2
x3
x4
having as a (nonunique) nontrivial solution
z1 = S1 (x) = x1
(29)
which via Eq. (21) implies

z2 = S2 (x) = L f S1 (x) = x2
(30)
z3 = S3 (x) = L f S2 (x) =
mgl
k
sin x1 (x1 x3 )
I
I
(31)
z4 = S4 (x) = L f S3 (x) =
mgl
k
x cos x1 (x2 x4 )
I 2
I
(32)
Finally using Eq. (26) we find the linearizing feedback Eq.

(15) as
IJ
k
(x) = (Lg S4 (x))1 L f S4 (x)

IJ mgl
mgl
k
=
sin x1 x22 +
cos x1 +
k
I
I
I

k
k
mgl
k
+ (x1 x3)
+ +
cos x1
I
I
J
J
(x) = (Lg S4 (x))1 =
In order to better understand the passivity concept we should

think of a system like a black box which transforms some
input into some output. More precisely, we say that a system
with input u and output y defines a passive operator u y if
the following energy balance equation is verified:
T
T
H(T ) H(0) +
i u(t)2 dt +
o y(t)2 dt
0
0

stored energy
dissipated
In a similar way, the feedback linearization problem may

be stated and solved for multivariable nonlinear systems but
it is beyond the scope of this article to go further into this
topic; interested readers are referred to Refs. 13 and 14. However, let us mention a simple example: the computed torque
controller for rigid joint robot manipulators (EL systems),
whose dynamical model is
(33)
where qp n is the vector of link positions (generalized position coordinates), D(qp) D(qp) 0 is the inertia matrix,
C(qp, qp) is the Coriolis and centrifugal forces matrix, g(qp) is
the gravitational vector, and u n is the vector of control
torques.
The tracking control problem for system [Eq. (33)] is to
make the link position qp follow a desired trajectory yd(t)
qpd(t). The computed torque controller is a feedback linearization approach which, since it was proposed in Ref. 6, has become very popular. The control law is given by
u = D(q p )v + C(q p , q p )q p + g(q p )
(34)
where v is a new input to be defined. It is easy to see that

by substituting Eq. (34) in Eq. (33) we obtain the linear
closed-loop dynamics qp v; then in order to solve the
tracking problem for Eq. (33) we choose v Kp(qp qpd)
Kd(qp qpd) qpd and we obtain that the closed loop is globally exponentially stable for any positive definite matrices of
Kp and Kd.
Note that in this simple example, the feedback linearizationwhich is globaldoes not require any change of coordinates like Eq. (17). On the other hand, one may verify that
system [Eq. (33)] does fulfill multivariable conditions similar
to those given in Theorem 1 that guarantee the feedback linearizability.
Except from several more recent modifications of the computed torque controller (see Ref. 17 and references therein)
the computed torque controller [Eq. (34)] requires full state
feedback in order to obtain a linear closed-loop system; and
in fact, output feedback will never lead to linear dynamics in
closed loop. It is therefore attractive to investigate alternative
tracking strategies.
PASSIVITY-BASED CONTROL
The Passivity Concept
As we mentioned in the introduction, the physical interpretation of the passivity concept is related to the systems energy.
u(t)y(t)dt

0
As a matter of fact, the above derivations are globally defined,

and in addition Eqs. (29)(32) have a physical interpretation.
D(q p )q p + C(q p , q p )q p + g(q p ) = u
523
(35)
supplied
where H(T) is the total energy of the system at time instant

T. Expressed in words, the energy balance equation (35) establishes that one cannot pull more energy out of a passive
system than the energy which was fed in. To illustrate this
simple idea, consider an ordinary RLC (with all elements con
i is the current
nected in series) network: In this case, y
running through the resistor, while u v is the input voltage.

T
Hence, if we look at Eq. (35), the term 0 iu(t)2 dt corresponds to the (electrical) potential energy stored in the capaciT
tor, while the term 0 oy(t)2 dt corresponds to the (electrical)
potential energy dissipated in the resistor (considering R
o). The energy stored in the inductance corresponds to the
(magnetic) kinetic energy which has been considered in the
terms H(T) H(0).
The stored energy in the capacitor plus the term H(T) is
called available storage and since H(0) 0, it satisfies

H(T ) +
i v(t)2 dt <
v(t)i(t) dt
0
o i(t)2 dt
0
that is, we can recover less energy than what was fed to the
circuit. Formally, the definition of passivity we will use is the
following (4):
Definition 1. Let T 0 be any. A system with input u
n
n
and output y L 2e
defines a passive operator : u y if
L 2e
there exists a such that

u(t)T y(t) dt
(36)
The operator is output strictly passive (OSP) if moreover,

there exists o 0 such that

u(t)T y(t) dt o
y(t)2 dt +
(37)
Finally, is said to be input strictly passive (ISP) if there

exists i 0 such that

T
0
u(t)T y(t) dt i
u(t)2 dt +
(38)
It should be noted that mainly every physical system has

some passivity property; this has motivated researchers to
use passivity-based control, that is, to exploit the passivity
524
properties of the plant in order to achieve the control task by

preserving the passivity in closed loop. The literature on passive systems is very diverse. We will illustrate this technique
on a class of passive systems, the EulerLagrange systems.
It is worth remarking that the robot manipulators belong to
this class.
EulerLagrange (EL) systems can be characterized by the EL

parameters
{T (q, q,
V (q), F (q)}
1
T (q, q)
= q T D(q)q
2
(40)
where the inertia matrix D(q) satisfies D(q) D(q) 0.

Next, V(q) represents the potential energy which is assumed
to be bounded from below; that is, there exists a c such
that V(q) c for all q n, and F (q) qRq with R
R 0 is the Rayleighs dissipation function.
EL systems are defined by the EL equations
L (q, q)
L (q, q)
=Q
q
(41)
where L (q, q)
T(q, q) V(q) is the Lagrangian function.
We assume that the external forces, Q n, are composed
only of potential forces (derived from a time-invariant potential V(q)) u n and dissipative forces F (q)/q, hence
Q=u
F (q)
F (q)
q
2
q
F (q)
q c 2
q
(46)
Dik (q) D jk (q) Di j (q)

+
q j
qi
qk

(47)
With these definitions of matrices D(q) and C(q, q) the following properties hold:
P1. The matrix D(q) is positive definite, and the matrix
(q) 2C(q, q) is skew symmetric, that is,
N(q, q) D
N N. Moreover, there exist some positive constants dm and dM such that
dm I < D(q) < dM I
(48)
P2. The matrix C(x, y) is bounded in x. Moreover, it is easy

to see from Eq. (46) that C(x, y) is linear in y, then for
all z n
C(x, y)z = C(x, z)y
C(x, y) kc y,
kc > 0
(49)
(50)
Furthermore we will focus our attention on those systems for

which the following additional property on the potential energy holds:
P3. There exists some positive constants kg and kv such
that
2

V (q)

, q Rn
kg sup
q2
q n

V (q)
n

kv sup
q , q R
q n
(51)
(52)
F (q)
=u
q
Proposition 1. (Passivity). An EL system defines a passive

operator from the inputs u to the actuated generalized velocities q, with storage function, the total energy function. Moreover, it is output strictly passive if there is a suitable dissipationthat is, if q(F (q)/q) oq2 for some o 0.
(43)
(44)
It is also well known (18) that the Lagrangian equations [Eq.

(41)] can be written in the equivalent form [note that Eq. (45)
is exactly the same as Eq. (33) with Raleigh dissipation zero]
D(q)q + C(q, q)
q + g(q) +
1
2
ci jk (q)q i
It is well known (7) that EL systems have some nice energy

dissipation properties:
An EL system is underdamped if
q T
n
(42)
At this point, we find it convenient to partition the vector q
as q
col[qp qc] where we call qp the undamped coordinates
and call qc the damped ones. With this notation we can distinguish two classes of systems: An EL system with parameters
[Eq. (39)] is said to be a fully damped EL system if ( 0)
q T
(39)
where q n are the generalized coordinates and n corresponds to the number of degrees of freedom of the system.
We focus our attention on fully actuated EL systemsthat is,
systems for which there is a control input available for each
generalized coordinate. Moreover, we assume that the kinetic
energy function is of the form
Ck j (q, q)
ci jk (q)
The Lagrangian Formulation
d
dt
where the entries of the matrix C(q, q) are called the Coriolis
and centrifugal forces; the kjth entry is
(45)
Below, we present other properties of EL systems which

are related to the stability in the sense of Lyapunov. For the
sake of clarity, we distinguish two classes of EL systems, fully
damped and underdamped systems.
The proposition below establishes conditions for internal
stability of fully damped EL systems. After Joseph L. La
Grange, the equilibria of a mechanical system correspond to
the minima of the potential energy function (see Ref. 19 for a
definition). Inspired by this well-known fact, we can further
establish the following:
Proposition 2. (GAS with full damping). The equilibria of a
fully damped free EL system (i.e., with u 0) are (q, q)
(q, 0), where q is the solution of

V (q)
=0
q
(53)
The equilibrium is unique and stable if it is a global and

unique minimum of the potential energy function V(q) and V
is proper (4). Furthermore, this equilibrium is globally asymptotically (GAS) stable if the map defined by the Rayleigh
dissipation function is input strictly passive.
As far as we know, the first article which establishes sufficient conditions for asymptotic stability of underdamped Lagrangian systems is more than 35 years old (20). In the proposition below we show that global asymptotic stability of a
unique equilibrium point can still be ensured even when energy is not dissipated in all directions, provided that the
inertia matrix D(q) has a certain block diagonal structure and
the dissipation is suitably propagated.
Proposition 3. (GAS with partial damping). The equilibrium (q, q) (0, q) of a free (u 0) underdamped EL system
is GAS if the potential energy function is proper and has a
global and unique minimum at q q, and if
1. D(q)

D p (q p )
0

0
, where Dc (qc ) Rn c n c .
Dc (qc )
F (q)
q c 2 for some > 0.

q
V (q)
3. For each qc , the function
= 0 has only isolated
qc
zeros in q p .
2. q T
Condition (2) establishes that enough damping is present

in the coordinates qc while the other two conditions help to
guarantee that the energy dissipation suitably propagates
from the damped coordinates to the undamped ones. Hence,
one can think of an underdamped EL system as the interconnection of two EL systems. As a matter of fact the feedback
interconnection of two EL systems yields an EL system.
Output-Feedback Set-Point Control
In this section we illustrate the passivity-based control approach by addressing the position-feedback set-point control
problem of EL systems. The results we will present are based
on the so-called energy shaping plus damping injection methodology. Launched in the seminal paper (21), this methodology aims at shaping the potential energy of the plant via a
passive controller in such a way that the new energy function has a global and unique minimum at the desired equilibrium. It is worth remarking that this methodology was originally proposed in the context of robot control; however, it has
been proved useful in the solution of other control problems
as it will be clear from this section. Also, it shall be noticed
that the passivity property of robot manipulators was first
pointed out in Ref. 7.
Motivated by the energy shaping plus damping injection
technique of Takegaki and Arimoto, as well as by the proper-
525
ties of Lagrangian systems described in previous sections, it

becomes natural to consider El controllers (22) with generalized coordinates qc nc and EL parameters Tc(qc, qc), Vc(qc,
qp), F c(qc). That is, the controller is a Lagrangian system with
dynamics
Tc (qc , q c ) Vc (qc , q p ) Fc (q c )
Dc (qc )q c + D c (qc )q c
+
+
=0
qc
qc
q c
(54)
Note that the potential energy of the controller depends on
the measurable output qp, and therefore qp enters into the
controller via the term Vc(qc, qp)/qc. On the other hand, the
feedback interconnection between plant and controller is established by
u=
Vc (qc , q p )
q p
(55)
then the closed-loop system is Lagrangian and its behavior is

characterized by EL parameters T(q, q), V(q), F (q), where
, Tp (q p , q p ) + Tc (qc , q c ),
V (q) , Vp (q p ) + Vc (qc , q p ),
F (q)
, F p (q p ) + Fc (q c )
T (q, q)
The resulting feedback system is the feedback interconnection

of the operator p : up qp, defined by the dynamic equation
[Eq. (33)] and the operator c : qp up, defined by Eqs. (54)
and (55).
Note that the EL closed-loop system is damped only
through the controller coordinates qc. From the results presented in section entitled The Lagrangian Formulation we
see that to attain the GAS objective, V(q) must have a global
and unique minimum at the desired equilibrium, q qd, and
F (q) must satisfy Eq. (44). These conditions are summarized
in the proposition below whose proof follows trivially from
Proposition 3.
Proposition 4. (Output feedback stabilization) (22).
Consider an EL plant (33) where u m, m n, with EL
parameters Tp(qp, qp), Vp(qp), F p(qp). An EL controller (54),
(55) with EL parameters Tc(qc, qc), Vc(qc, qp), F c(qc), where
q Tc
Fc (q c )
q c 2
q c
for some 0, solves the global output feedback stabilization

problem if
1. (Dissipation propagation) For each trajectory such that
qc const and Vc(qc, qp)/qc 0, we have that qp
const.
2. (Energy shaping) V(q)/q 0 admits a constant solution q such that qpd [Inp 0]q, and q q is a global
and unique minimum of V(q), and V is proper. For instance, this is the case if 2V(q)/q2 In 0, 0 q
n.
526
A simple example of EL controllers is the dirty derivatives

filter, widely used in practical applications:
q c = A(qc + Bq p )
(56)
= (qc + Bq p )
(57)
where A, B are diagonal positive definite matrices. With an

obvious abuse of notation this system lies in the EL class and
has the EL parameters:
Tc (qc , q c ) = 0, Fc (q c ) =
stability. As far as we know, the first nonlinear PID controller

is due to Ref. 24 [even though Kelly (24) presented his result
as an adaptive controller, in the sequel it will become clear
why we use the PID qualifier] which was inspired upon the
results of Tomei (25). In order to motivate the nonlinear PID
of Ref. 24, let us first treat in more detail the PD-like adaptive
control law of Tomei
u = K p q p Kd q p + (q p )
(60)
together with the update law
1 T 1 1
q B A q c
2 c

= (q )T q +
p
p
1
Vc (qc , q p ) = (qc + Bq p )T B1 (qc + Bq p )
2
The controller above injects the necessary damping to achieve
asymptotic stability and its action has the following nice passivity interpretation. First, we recall that the EL plant Eq.
(41) defines a passive operator u qp. On the other hand,
the controller Eq. (54) defines a passive operator qp Vc(qp,
qc)/qp. These properties follow, of course, from the passivity
of EL systems established in Proposition 1. It suffices then to
choose u as in Eq. (55) to render the closed-loop passive.
CONTROL UNDER MODEL AND

PARAMETER UNCERTAINTIES
In all the results presented above, we assumed that we had
accurate knowledge about the systems model and its parameters; however, this rarely happens to be the case in practical
applications. It is of interest then to use techniques such as
robust and/or PID control.
PID Control
PID control was originally formulated by Nicholas Minorsky
in 1922; since then it has become one of the most applied
control techniques in practical applications. In the western
literature, the first theoretical stability proof of a PID in
closed loop with a robot manipulator is due to Ref. 23. We
reformulate below the original contribution of the authors.
Proposition 5. Consider the dynamic model [Eq. (33)] in
closed loop with the PID control law
u = KP q p KD q p +
(58)
= KI q p (0) = 0 Rn
(59)
Then, if KP kgI and KI is sufficiently small, the closed loop

The proposition above establishes only local asymptotic
stability. By looking at the proof (see Ref. 23) of the above
result we see that what hampers the global asymptotic stability is the quadratic terms in qp contained in the Coriolis matrix. This motivates us to wonder about the potential of a linear controller designed for a nonlinear plant.
As a matter of fact, with some smart modifications one can
design nonlinear PIDs which guarantee global asymptotic
2q p
1 + 2q p 2
(61)
where is a suitably defined positive constant. Tomei (25)

proved that under this adaptive control law, the position error
is globally asymptotically convergent. The normalization used
in Eq. (61), probably first introduced by Koditschek (26), helps
in guaranteeing the globality, in the un-normalized case being
only semiglobally convergent.
Note that the result of Tomei is based on a PD plus gravity
cancellation; since the gravity vector is not well known, an
adaptive update law must be used. Let us consider that instead of cancelling the gravity term, we compensate it at the
desired position, then we will be aiming at estimating the constant vector (qpd) . More precisely, consider the control law
(24)
u = KP q p KD q p + (q pd )
(62)
together with the update law

1
q p
= = (q pd )T q p +
1 + q p
(63)
where 0 is a small constant. Kelly (24) proved that this

adaptive controller in closed loop with Eq. (33) results in a
globally convergent system. As a matter of fact, since the regressor vector (qpd) is constant, the update law [Eq. (63)],
together with the control input [Eq. (62)], can be implemented
as a nonlinear PID controller by integrating out the velocities
vector from Eq. (63):

t
1
q p
dt + (0)
= (q pd )T q p +
1 + q p
0
Note that the choice KP KP KI, with KI 1/ (qpd)
(qpd), yields the controller implementation
u = KP q p KD q p +
=
KI
q p
,
1 + q p
(64)
(0) = 0 Rn
(65)
Following Ref. 24, one can prove global asymptotic stability

of the closed-loop system [Eqs. (33), (64), (65)]. An alternative
trick to achieve GAS is the scheme of Arimoto (27), who proposed the following nonlinear PID:
Proposition 6. Consider the dynamic model [Eq. (33)] in

closed loop with the PID control law
u = KP q p KD q p +
= KI sat(q p ),
(0) = 0 R
(66)
n
(67)
Then, if KP kgI and if KI is sufficiently small, the closed loop

It is clear from the proof (see Ref. 27) that the use of a
saturation function in the integrator helps to render the system globally asymptotically stable, just as the normalization
did in Tomeis and Kellys schemes.
In the sequel we assume that velocities are not available
for measurement. In general it is a difficult problem to design
adaptive output feedback controllers for nonlinear systems
and achieve global convergence. The result we present below
is inspired by the work of Kelly and the key observation that
when compensating with the (unknown) gravity vector evaluated at the desired position, one can simply integrate the velocities out of the update law.
Proposition 7. Consider the dynamic model of the EL plant
Eq. (33) in closed loop with the PI2D control law
u = KP q p KD +
= KI (q p ),
(0) = 0 Rn
(68)
(69)
q c = A(qc + Bq p )
(70)
= qc + Bq p
(71)
Let KP, KI, KD, A

diagai, and B
diagbi be positive definite diagonal matrices with
4dM
B>
I
dm
KP > (4kg + 1)I
(72)
(73)
col[qp, qp, ,
where kg is defined by Eq. (49), and define x
g(qpd)]. Then, given any (possibly arbitrarily large) initial
condition x(0), there exist controller gains that ensure
limt x(t) 0.
In Ref. 28, precise bounds for the controller gains are

given, depending on bounds on the plants parameters.
Note that the PI2D controller is linear and, as in the case
of a conventional PID scheme, it only establishes semiglobal
asymptotic stability. The technical limitation is the high nonlinearities in the Coriolis matrix. See Ref. 28 for details.
In the sequel we give an interpretation of the nonlinear
PID controllers above, as well as the PI2D scheme, from the
passivity point of view, or more precisely passifiabilitythat
is, the possibility of rendering a system passive via feedback.
From Proposition 1 we know that the plants total energy
function T(qp, qp) V(qp) qualifies as a storage function for
the supply rate w(u, q) uqp. From this property, output
strict passifiability of the map u qp follows taking u
KDqp u1, with u1 an input which shapes the potential
energy.
Other applications, including the present study of PI controllers, require a passifiability property of a map including
527
also qp besides qp, at the output. This can be accomplished

with a storage function that includes cross terms. Very recently, Ref. 27 showed, by using a saturation function sat( ),
that the nonlinear PID (66) can be regarded as the feedback
interconnection of two passive operators 1 : z sat(qp)
qp and 2 : sat(qp) qp z; hence the closed loop system
is also passive. The same can be proven for the normalized
scheme of Kelly [Eqs. (64) and (65)]. As a matter of fact it can
be proven that 1 is OSP (17).
Instrumental in proving OSP for 1 is the use of either a
normalization or a saturation function; unfortunately in the
case of the PI2D controller, these tricks do not lead us to
OSP actually, and the output strict passifiability property we
can establish is only local. That is, the property holds only for
inputs that restrict the operators output to remain within a
compact subset (29). Nonetheless, this compact subset can be
arbitrarily enlarged with high gains, and this explains the
semiglobalinstead of globalnature of this result.
Robust Control
We have assumed so far that even though the plants model
is known, some uncertainties over the parameters exist. Nevertheless, in some applications it may happen that we have
only a very rough idea of the plants model. For instance, some
lightweight robot manipulators with direct-drive motors present highly nonlinear and coupled dynamics for which a model
is not known. It is good to know that, at least for a certain
class of EL plants, still a high-gain PD control can be used,
leading to some robustness satisfactory results. In particular,
for the EL plant model we have
D(q p )q p + C(q p , q p )q p + g(q p ) + F (q p ) + T = u
(74)
where F(qp) is the vector of frictional torques which satisfies

F(qp) kf1 kf2 qp for all qp and T is the vector of load
disturbances which is bounded as T kt; we have the following result (30):
Proposition 8. Consider the EL plant [Eq. (74)], let e
qp qpd be the position error, and let e an estimate of it. Consider the control law
u = Kd e K p e
(75)
e = w + Ld (e e)
(76)
w = L p (e e)
(77)
Then for any set of bounded initial conditions (t0, x(t0)) we can
always find sufficiently large gains Kp, Kd, Lp, and Ld such
that the trivial solution of the closed-loop system: x(t)

col[e, e, e, e 0 is uniformly ultimately bounded. That is, for
every bounded initial conditions (t0; x(t0)) there exist a finite
constant 0 and a time instant t1(, x(t0)) such that
x(t) ,
t t0 + t1
Moreover, in the limit case, when Lp, Ld the origin is

asymptotically stable.
A particular case of the result above is presented in Ref.
31 where velocity measurements are used. Like in Ref. 31 the
528
uniform ultimate boundedness result of Proposition 8 is of local nature because the controller gains depend on the initial
conditions x(t0); nevertheless, it is important to remark that
the Proposition states that for any set of finite initial conditions there always exist control gains . . .; that is, the result
is semiglobal.
However, even without the knowledge of bounds over the
plants parameters, the closed loop system can be made uniformly ultimately bounded by selecting the control gains sufficiently large. Hence there is no need to quantify these
bounds a priori.
It is also important to remark that the linear control
scheme [Eq. (75)(77)] allows quick response in an online implementation, due to its simplicity. Since this control scheme
completely ignores the system dynamics, however, the conditions of Proposition 8 (see [30] for details) may require that
Kd and Ld be large to obtain an acceptable tracking performance. Such high gain implementations are not always desirable in practical applications. For this reason it may be
profitable to add model-based compensation terms to the control input, when available. See, for instance, Ref. 17 and references therein.
in closed loop with Eq. (78) is GAS. Now, let us suppose that
only x2 and x3 are measurable. In this particular case, a nonlinear observer for x1 can be easily designed using the Lyapunov control approach (4). Motivated by the control law Eq.
(81), consider the control law u

u(x2, x3, x1):
THIRD-ORDER FEEDBACK LINEARIZABLE SYSTEMS
where 1, 2, and 3 are positive constants. Hence, by setting
So far we considered n coupled second-order (fully actuated

EL) systems to illustrate different control design methods for
nonlinear systems. Even though the class of EL plants includes a large number of physical systems it is interesting to
investigate output feedback control of higher-order systems.
As a matter of fact, the topic of output feedback control of
nonlinear systems is one of the most studied in the literature
and very few particular results exist guaranteeing global asymptotic stability; see, for example, a recent study of the semiglobal problem (11).
In this section we illustrate this problem by addressing the
partial state feedback control of a complex system, the socalled Rossler system (see, for instance, Ref. 32)
x1 = (x2 + x3 )
(78)
x2 = x1 + ax2
(79)
x3 = x3 (x1 b) + c + u
(80)
where a, b, and c are positive constants. It can be seen from

simulations that the trajectories of this system have a chaotic
behavior, for instance if a 0.2, b 5, and c 0.2. A behavior is chaotic if it has a sensitive dependence on initial conditions. By this, we mean that the difference between two solutions of a differential equation with a slight difference in the
initial conditions grows exponentially (32).
The motivation to consider the Rossler system is to investigate to what extent the techniques used for second-order systems can be successfully used for third-order feedback linearizable systems.
Note for the Rossler system that if the whole state is supposed to be measured, then it is easy to see that there exist
positive gains k1, k2, and k3 such that the feedback linearizing
control law
u = x3 (x1 b) c k1 x1 k2 x2 k3 x3
(81)
u = x3 (x1 b) c k1 x1 k2 x2 k3 x3
(82)
where x1 is the estimate of x1. Consider the function

V (x, x1 ) =

1 2
x + x22 + x23 + x21 +
x2 (x3 x1 )
2 1
(83)
where x1 x1 x1 is the estimation error and 0 is sufficiently small to ensure that V(x) is positive definite and radially unbounded. Then, let V(x) be a Lyapunov function candidate for system Eq. (78) in closed loop with the control law
Eq. (82). We proceed now with the design of a reduced observer x1 f(x2, x3, x1). First, evaluating the time derivative
of V(x, x1) along the trajectories of the closed loop Eqs. (78),
(82) we get after some bounding
V (x, x1 ) 1 x21 2 x22 2 x23 + (x3 +
x2 )(k1 + x3 )x1 + x1 x1
x1 = (x3 +
x2 )(k1 + x3 )
the Lyapunov function becomes negative semidefinite. A simple analysis using the KrasovskiiLaSalles invariance principle shows that the closed-loop system is GAS. Note, moreover,
that the observer can be implemented as
x1 = (x3 +
x2 )(k1 + x3 ) (x2 + x3 )
without measurement of x1.
In the case when more variables are unmeasurable, one
may think that a similar procedure leads to the design of an
observer for the unavailable variables. Unfortunately, this
seems not the case when only x2 or x3 are considered available
for measurement. Moreover, the lack of a physical interpretation for the Rossler system makes this task more difficult.
The lesson one can take from this illustrative example is that
observer-based schemes become complicated even if the system itself is feedback-linearizable. The lack of (physical) passivity properties hampers the use of passivity-based control.
CONCLUSION
We have briefly illustrated different control design methods
for nonlinear systems. We derived necessary and sufficient
conditions to solve the local feedback linearization problem
and illustrated this approach on the flexible joints robots case.
We focused our attention into a special class of second-order systems, the EL systems. However, the Lagrangian formulation applies to all fields in which variational principles
can be used to model the plant in question; hence this class
includes a wide number of physical systems such as robot manipulators.
We saw that the EL class has some nice energy properties
which can be exploited by using passivity-based control. The
goal of this methodology is to design a controller and a dy-
NONLINEAR DYNAMIC PHENOMENA IN CIRCUITS
namic extension to the plant which renders the closed-loop

system passive. This approach appeared very useful in solving the set-point control problem.
ACKNOWLEDGMENTS
This work was partially carried out while the first author was
with the Department of Applied Mathematics of the University of Twente. His work has been partially sponsored by the
project European Capital and Human Mobility, under grant
no. ERB CHRX CT 93-0380.
BIBLIOGRAPHY
1. IEEE Control Systems Magazine, Special issue on History of Control, June 1996.
2. J. C. Willems, Dissipative dynamical systems. Part I: General
theory, Arch. Rat. Mech. Anal., 45 (5): 321351, 1972.
3. A. M. Lyapunov, Proble`me de la stabilite de mouvement, Ann.
Fac. Sci. de Toulouse, 9: 203474, 1907. (Translation from the
original article published in Commun. Soc. Math., Kharkov, 1893,
reprinted in Ann. Math. Studies, 17: Princeton 1949). See also
Stability of Motion, New York: Academic Press: 1996.
4. M. Vidyasagar, Nonlinear Systems Analysis, Englewood Cliffs,
5. H. Khalil, Nonlinear Systems, New York: Macmillan, 1992.
6. A. K. Bejczy, Robot arm dynamics and control, Technical Report
TM 33-669, California Institute of Technology, Jet Propulsion
Laboratory, 1974.
7. R. Ortega and M. Spong, Adaptive motion control of rigid robots:
A tutorial, Automatica, 25-6: 877888, 1989.
8. J. T. Wen and S. D. Bayard, New class of control laws for robot
manipulators, Parts I and II, Int. J. Control, 47-5: 12881310,
1988.
9. H. Berghuis and H. Nijmeijer, Global regulation of robots using
only position measurements, Syst. Contr. Lett., 21: 289293,
1993.
1980.
11. A. Teel and L. Praly, Tools for semiglobal stabilization by partial
state and output feedback, SIAM J. Control Opt., 33 (5): 1443
1488, 1995.
12. P. Rouchon et al., Flatness, motion planning and trailer systems.
In Proceedings of the 32nd IEEE Conference on Decision Control
(San Antonio, TX), 1993, pp. 27002705.
13. H. Nijmeijer and A. J. van der Schaft, Nonlinear Dynamical Control Systems, New York: Springer-Verlag, 1990.
14. A. Isidori, Nonlinear Control Systems. Berlin: Springer-Verlag,
3rd ed., 1995.
15. R. Marino and S. Nicosia, On the feedback control of industrial
robots with elastic joints: a singular perturbation approach, Technical Report 84.01, Dip. Ing. Elettronica, Universita di Roma Tor
Vergata, June 1984.
16. R. Marino and S. Nicosia, On the feedback control of industrial
robots with elastic joints: a singular perturbation approach. In
First IFAC Symposium on Robot Control, (Barcelona, Espana),
1985, pp. 1116.
17. A. Loria, On output feedback control of Euler-Lagrange systems,
PhD thesis, University of Technology of Compiegne, Compie`gne,
France, October 1996. Available on http://www-ccec.ece.ucsb.edu/
people/aloria.
18. M. Spong and M. Vidyasagar, Robot Dynamics and Control, New
York: Wiley, 1989.
529
19. E. Marsden and A. J. Tromba, Vector Calculus. New York: W. H.

Freeman, 1988.
20. G. K. Pozharitskii, On asymptotic stability of equilibria and stationary motions of mechanical systems with partial dissipation,
Prikl. Mat. i Mekh., 25: 657667, 1961. English translation in J.
Appl. Math. Mech., 25: 1962.
21. M. Takegaki and S. Arimoto, A new feedback method for dynamic
control of manipulators, ASME J. Dyn. Syst. Meas. Control, 103:
119125, 1981.
22. R. Ortega et al., On passivity-based output feedback global stabilization of EulerLagrange systems, Int. J. Robust and Nonlinear
Control, special issue on Control of Nonlinear Mechanical Systems, 5 (4): 313325, 1995 (H. Nijmeijer and A. J. van der
Schaft, eds.).
23. S. Arimoto and F. Miyazaki, Stability and robustness of PD feedback control with gravity compensation for robot manipulator. In
F. W. Paul and D. Yacef-Toumi (eds.), Robotics: Theory and Applications DSC, Vol. 3, 1986, pp. 6772.
24. R. Kelly, Comments on: Adaptive PD control of robot manipulators, IEEE Trans. Robotics Automat., 9 (1): 117119, 1993.
25. P. Tomei, Adaptive PD control for robot manipulators, IEEE
Trans. Robotics Automat., 7 (4): 565570, 1991.
26. D. E. Koditschek, Application of a new Lyapunov function to
global adaptive attitude tracking. In Proceedings of the 27th IEEE
Conference on Decision Control (Austin, TX), 1988.
27. S. Arimoto, A class of quasi-natural potentials and hyper-stable
PID servo-loops for nonlinear robotic systems, Trans. Soc. Instrument Control Eng., 30 (9): 10051012, 1994.
28. R. Ortega, A. Loria, and R. Kelly, A semiglobally stable output
feedback PI2D regulator for robot manipulators, IEEE Trans. Automat. Control, 40 (8): 14321436, 1995.
29. H. Pota and P. Moylan, Stability of locally dissipative interconnected systems, IEEE Trans. Automat. Control, AC-38 (2): 308
312, 1993.
30. H. Berghuis and H. Nijmeijer, Robust control of robots via linear
estimated feedback, IEEE Trans. Automat. Control, 39 (10):
21592162, 1994.
31. Z. Qu and J. F. Dorsey, Robust tracking control of robots by a
linear feedback law, IEEE Trans. Automat. Control, 36: 1081
1084, 1991.
32. T. Kapitaniak, Controlling Chaos, New York: Academic Press,
1996.
ANTONIO LORIA
University of California
at Santa Barbara
HENK NIJMEIJER
University of Twente
NONLINEAR DISTORTION MEASUREMENT. See INTERMODULATION MEASUREMENT.
NYQUIST CRITERION, DIAGRAMS, AND STABILITY
19
Table 1. List of Manufacturers

AAI Corporation
P.O. Box 126
Hunt Valley, MD 21030-0126
Tel: 800-655-2616
Fax: 410-628-8616
Cape Software, Inc.
333-T N. Sam Houston Parkway, E.
Houston, TX 77060-2403
Tel: 281-448-5177
Fax: 281-448-2607
Design Technology Corporation
5 Suburban Park Drive
Billerica (Boston), MA 01821
Tel: 508-663-7000
Fax: 508-663-6841
MicroMath Research Ltd.
P.O. Box 71550
Salt Lake City, UT 84171-0550
Tel: 801-483-2949
Fax: 801-483-3025
Mitech Corporation
43-T Nagog Park
Acton, MA 01720
Tel: 978-263-7999
Fax: 978-263-8081
NYQUIST CRITERION, DIAGRAMS,

AND STABILITY
H. Nyquist (18891976), born in Sweden, is known for his
contributions to telephone transmission problems in the
1920s. He is also well known for his contributions in the stability of feedback systems.
In 1927 H. S. Black invented the negative feedback amplifier. Part of the output signal was returned to the amplifiers
input to reduce the total system gain. This resulted in a flatter frequency response, a wider bandwidth, and a decrease in
the nonlinear distortion, since improperly designed amplifiers
were unstable, producing undesired results. In the late 1920s
and early 1930s, H. Nyquist and H. W. Bode, a colleague at
the Bell Telephone Laboratories, developed a mathematical
analysis for feedback amplifiers. Later developments evolved
into sinusoidal frequency analysis and design techniques applied to feedback control systems. Although, at that time, the
stability criteria for vibrating mechanical systems already existed and had been applied to feedback systems, the idea of
algebraic problems on complex roots of polynomials were just
arising. The Nyquist criterion offered geometrical solutions
and was much easier to apply to amplifiers.
Later, the Nyquist criterion was used to provide vital information on stability essential in the analysis and design of
Murphy Software
Company
1000 Town Center, Suite
1950
Southfield, MI 48075
Tel: 248-351-0900
Fax: 248-351-0906
ProModel Corporation
1875-T S. State Street,
Suite 3400
Orem, UT 84097-8075
Tel: 801-223-4600
Fax: 801-226-6046
Scientific and Engineering
Software
4301-T Westbank Drive
Austin, TX 78746-6546
Tel: 512-328-5544
Fax: 512-327-6646
Wolfram Research, Inc.
100-T Trade Center Drive,
Suite 600
Champaign, IL 61820-4853
Tel: 217-398-0700
Fax: 217-398-0747
general control systems. It is a graphical method that relates

the stability of a closed-loop system to the open-loop frequency response and the locations of poles and zeros. The Nyquist diagram method was found to be very useful in the design of linear feedback systems of all types. An important
application in World War II was the feedback control of direction of guns that employed electromechanical feedbackcontrolled servomechanisms. Before computers became
widespread, Nyquist diagrams were largely obtained by calculations and hand-drawn graphics. But, today many companies
offer a diverse range of computer software for simulation,
analysis and design of control problems. Some of these manufacturers are listed in Table 1. Popular software such as
MATLAB, MATHCAD, and SIMULINK include control system tools.
In design and stability analysis, the Nyquist method exhibits distinct features. It provides the same information on the
stability of a control system, as does the RouthHurwitz criterion. In addition, the Nyquist method indicates the relative
stability or instability of the system. Properties of the frequency-domain plots of the loop transfer function G(s)H(s)
provide an alternative approach to root locus, and give information on the frequency-domain characteristics of the closedloop system. One other major feature of the Nyquist method
is that it relates a closed-loop system to the transient response and steady-state errors.
ENCIRCLEMENTS AND ENCLOSURES
Before embarking into the Nyquist stability criterion, the concepts of encirclement and enclosures need to be established.
These concepts are essential in the interpretation of the Nyquist plots and stability analysis.
20
By taking the absolute value of both sides of Eq. (1) yields

|c(t)| =
r(t ) g( ) d
(2)
j
s-plane
s1
C
s1
A
B
or
|c(t)|
|r(t )|| g( )| d
(3)
Figure 1. Encirclements and enclosures of points by contours.
Let the input r(t) be bounded by a finite positive number R

such that
In a complex plane, a point is said to be encircled by a

closed path if it lies inside the path. If the closed path has a
direction assigned for being in a clockwise or counterclockwise
direction, then the region inside the path is considered to be
encircled in that prescribed direction.
A point or a region is said to be enclosed by a path if it is
encircled in the counterclockwise direction. Alternatively, the
point or region is enclosed if it lies on the left of the path
when the path is traversed in any prescribed direction. The
second definition is more useful in situations where only some
portion of the closed path is drawn.
As an example, consider the two cases illustrated in Fig.
1. In accordance with the foregoing definitions, the contour
1 encircles the point A twice. The point B is encircled only
once, whereas points C and D are not encircled at all by 1.
As the contour 1 traverses in the counterclockwise direction,
the points A and B lie on the left side; therefore, they are
both enclosed, and the points C and D are not enclosed. The
contour 2 encircles only the point D in the clockwise direction. Interestingly enough, the contour 2 does not enclose the
point D, but it encloses other points A, B, and C. A path can
encircle a point N number of times, and the magnitude of N
can be determined by drawing a phasor from the point to an
arbitrary point s1 along the path , as illustrated for point A
in the contour 1. The point s1 is traversed along until it
returns to its starting position. The net number of revolutions
traversed by the phasor is N and the total angle traced is
2N radians.
|r(t)| R
STABILITY CONDITIONS
(4)
Thus, Eq. (2) becomes

|c(t)| R
| g( )| d
(5)
If c(t) is also to be bounded by a positive finite number C,

with
|c(t)| C <
(6)
then

| g( )| d C <
(7)
Dividing Eq. (7) through by R and letting C/R equal to Q, a

positive finite number, results in

| g( )| d Q <
(8)
For Eq. (8) to hold, the integral of the absolute value of g(t)
must be finite.
The Laplace transform may be used to show the relationship between the roots of the characteristic equation and Eq.
(8). For g(t)

L[ g(t)] =
g(t)est dt = G(s)
(9)
0
The stability of linear time-invariant systems depends upon

the roots of the characteristic equation on the s-plane. In contrast to root locus plots, the Nyquist criterion does not give
the exact locations of the roots of the characteristic equation
but indicates the locations of the roots with respect to the left
or the right half of the s-plane.
In many control systems, the relations of the bounded-inputs to the bounded-outputs (BIBO) define the stability. The
definition states that if bounded inputs yield to bounded outputs, then the system is considered stable. The BIBO relationship can also be related to the roots of the characteristic
equation as shown next.
Lets take a linear time-invariant system, as illustrated in
Fig. 2, with r(t) as the input, c(t) as the output, and g(t) as
the impulse response. The convolution integral relating the
function r(t), c(t), and g(t) is given by

c(t) =
r(t )g( ) d
(1)
0
Where s is a complex number having real and imaginary components on the s-plane as s j. Taking the absolute
value on both sides of the equation yields

st

|G(s)| =
g(t)e dt
|g(t)est | dt
(10)
0
But since
|est | = |e t |
r(t)
g(t)
(11)
c(t)
Figure 2. Block diagram representation of a linear time invariant

system. The relation between input and output is given by the impulse response transfer function g(t). The g(t) may be a simple gain
or a complex function involving derivatives and integrals.
substituting Eq. (11) into Eq. (10) gives

| g(t)e t | dt
Table 2. Calculation of Points for the Illustration of Mapping

for Contours in Fig. 3
(12)
s1j
4
7
j
5
5
s1j
7
4
j
5
5
s 1 j
7
4
j
13
13
s 1 j
7
4
j
13
13
2
F(s)
s5j
5 j4
s5j
5 j4
s3j
3 j4
s3j
3 j4
s 3 j
4
3
j
25
25
s 3 j
3
4
j
25
25
s 5 j
5
4
j
41
41
s 5 j
5
4
j
41
41
s 7 3j
4
7
j
3
3
s 7 3j
7
4
j
3
3
s 7 3j
21
12
j
65
65
s 7 3j
21
12
j
65
65
F(s)
Note that the imaginary part, j, of the complex variable s

does not bear any importance in the proof leading to the BIBO
stability. All that is needed is the mathematical relation between real parts of the poles in the s-plane, that is whether
they lie in the right half or the left half of the complex plane.
If one or more roots of the characteristic equation lies in
the right half of the s-plane, or on the j-axis if is greater
than or equal to zero, then
|e t | R = 1
Substitution of Eq. (13) into Eq. (12) yields

R| g(t)| dt =
| g(t)| dt
0
21
F(s)
4
F(s)
(13)
(14)
Note that Eq. (14) does not satisfy the BIBO relation because
the equation is not bounded as in Eq. (8). Hence, to satisfy
the BIBO stability, the roots of the characteristic equation or
the poles of G(s) must lie on the left side of the j-axis. A
system is classified to be marginally stable if the first-order
poles, poles of conjugate pairs, lie on the j-axis. However,
multiple-order poles or repeating conjugate pairs of poles represent an unstable system. In addition, a system is classified
as unstable if more than one pole exists at the origin.
Another definition that is worth mentioning and that helps
in the understanding of the Nyquist criterion is the steadystate error. Steady-state error is the difference between the
input and the output as t for a given test input.
The steady state errors are generally described for three
main types of test inputs: the step, the ramp, and the parabolic. Often, control systems are subjected to these inputs to
test their ability to give the required outputs. Usually, these
test inputs are in the form of electrical signals that have defined waveforms. For example, the parabolic input has a constant second derivative, which represents acceleration with
respect to accelerating targets. In general, the output of any
system can be represented by the sum of the natural response
and the forced response. In relation to the Nyquist stability
criterion, a steady state error can be calculated from the
closed-loop transfer function of the system M(s) or the loop
transfer function G(s)H(s).
trarily chosen continuous closed path s in the s-plane, then

the values of s in s maps a new closed continuous path F on
the complex F(s)-plane. Some examples of mapping are presented in Table 2 and illustrated in Fig. 3. In Fig. 3, it can be
seen that for every closed contour on the s-plane there is a
corresponding contour on the F(s)-plane. If the contour s traverses in a selected direction say counterclockwise, Fig. 3
shows that the direction of traverse of F can either be in the
same or opposite direction depending on the number of poles
and zeros of function F(s) located on contour s. Lets express
F(s) in the following familiar form as seen in the control theory
Z

F (s) =
(s + zi )
i=1
P

(15)
(s + pk )
k=1
By using the Lucas formula, F(s)/F(s) can be written as

Z
P

F (s)
1
1
=
F (s)
(s
+
z
)
(s
+
pk )
i
i=1
k=1
(16)
where F(s) is the first derivative of F(s) with respect to s.

To illustrate this important point, lets take an example
F (s) =
s+1
(s + 2)(s + 3)
(17)
s2 2s + 1
(s + 2)2 (s + 3)2
(18)
Calculate
THE PRINCIPAL ARGUMENT
The Nyquist criterion is based on a theorem using the theory
of complex variables, which leads to the principal argument.
The principal argument may be presented in a number of
ways. Here, two approaches will be presented.
CASE 1. In this case, a rigorous mathematical approach may
be employed by using theories of contours and mappings in
complex planes. Assume that F(s) is a function of s and singlevalued, that is, for each point in the s-plane there exist a corresponding point, including infinity, in the F(s)-plane, and the
function consists of a finite number of poles and a finite number of zeros in the s-plane. Now, suppose that there is an arbi-
F (s) =
The ratio of F(s)/F(s) can be found to be

s2 2s + 1
F (s)
=
F (s)
(s + 1))(s + 2)(s + 3)
(19)
Writing the partial fractions
s2 2s + 1
A
B
C
F (s)
=
=
+
+
F (s)
(s + 1)(s + 2)(s + 3)
(s + 1)
(s + 2)
(s + 3)
(20)
22

Im
3.00
the f(s)-plane. The theorem shows that if a complex function

f(s) is analytical (differentiable) bounded in a region by a simple closed curve , then

f (s) ds =
f (s) ds = 0
(22)
s-plane
s4
2.00
s3
1.00
s1
s2
0.
Re
1.00
F(s) = s + 4
s-4
2.00
3.00
7.00
5.00
3.00
1.00
1.00
3.00
5.00
However, consider a complex function f(s) 1/(s a) with a

pole at a on the s-plane. Lets draw a unit circle centered at
s a on the s-plane described by (t) a ej where the
angle is 0 2k, and k is the number of encirclements
of the point a by the unit circle.
Then the integral becomes

7.00
f (s) ds =
1
ds = ln(s + a)|
(s + a)
(23)
Im
The right-hand side of this equation can be evaluated by substituting values of s(t) a ej yielding
8.00
F(s)-plane
6.00
F2
ln(s + a)| = ln(e j a + a)| = j|
4.00
Substituting the values of as 0 2k gives

2.00
0.
(24)
F3
F1
F4
Re
2.00
4.00
6.00
1.00 3.00 5.00 7.00
it can be proven that A 1, B 1, and C 1, thus giving

1
1
1
F (s)
=
F (s)
(s + 1)
(s + 2)
(s + 3)
k=
9.00
Figure 3. Mapping of s-plane contours to F(s)-plane. If the addition

of number of poles and zeros of F(s) in the s-plane is other than zero,
the contour on the F(s)-plane encircles the origin. This is clearly illustrated by the corresponding contours of 2 and 3.
(21)
and verifying the general Eq. (16). Equations (16) and (21)
indicate that the zeros of F(s) appear as the denominators of
the new equation with positive signs. The poles of F(s) still
appear in the denominators, but they have negative signs.
After having stated this important point, we can turn our
attention to Cauchys theorem. Although Cauchys theorem
may be of great mathematical interest, the intention of this
article is not to discuss the intricacies involved in the theorem
but rather to use it to illustrate relevant points in order to
establish a solid understanding of Nyquist stability criteria.
Cauchys theorem is based on the complex integrals, mapping of points and closed contours between the s-plane and
1
ds = 2kj
(s + a)
(25)
On the contrast to Eq. (22) where f(s) is analytic in the splane, for f(s)) having a singular point in the contour in the
s-plane, the resulting closed contour is no longer zero but
equal to multiples of 2j. This indicates that the contour on
the f(s)-plane containing all the values of s on the s-plane goes
through k number of revolutions around the origin. The number of revolutions around the origin depends on the number
of times the point a is encircled in the s-plane. The encirclement k now can be expressed as
8.00
7.00 5.00 3.001.00
f (s) ds =
1
2 j
f (s) ds =
1
2 j
1
ds
(s + a)
(26)
If only one encirclement has taken place in the s-plane, k

1, the result of this integration equals 1, indicating one revolution around the origin. Previously, a unit circle was considered, but the theory can be generalized for any closed contour
around the pole a.
We know that a complex function expressed as a ratio, as
in Eq. (15), can be expressed as Eq. (16). Now, lets write Eq.
(16) as
f (s) =
Z
P

1
1
F (s)
=
F (s)
(s + zi ) k=1 (s + pk )
i=1
(27)
where f(s) can be viewed as the residues of F(s). Now, substitute this Eq. (27) into Eq. (25)
f (s) ds =

Z
i=1
1
ds
(s + zi )

P
k=1
1
ds
(s + pk )
(28)
Making use of the similarities between Eq. (23) and Eq. (28)
will yield the following conclusions. There will be Z number
of revolutions around the origin, one for each zi for k 1.
Also, there will be P number of revolutions around the origin

as a result of pole pk, but this time they will be in the opposite
direction, which is indicated by the negative sign of Eq. (28).
From Eq. (26), the following may be written
1
2 j
R(s) +
E(s)
G(s)
23
C(s)
H(s)
f (s) ds = Z P
(29)
This equation indicates that the number of times the origin

is encircled depends on the difference between the zeros and
poles, which are located on the contour of the s-plane.
There are many other approaches to arrive at similar conclusions, and further information may be found in the bibliography at the end of this article. This illustrative approach
clarifies for readers many points, which otherwise might have
been difficult to understand without intensive mathematical
knowledge.
As a result of these explanations, the principle argument
can now be stated as follows: Let F(s) be a singled-valued
function with a finite number of poles in the s-plane, and let
s be chosen such that it does not pass through the poles or
zeros of F(s). Thus the corresponding F locus mapped in the
F(s)-plane will encircle the origin given by the formula
N = ZP
(30)
Figure 4. Block diagram of a closed-loop system. A closed-loop system has a forward path transfer function G(s) and a feedback path
transfer function H(s). The relation between the input and output can
be expressed in terms of these two terms in the form of a system
transfer function to be used in analysis and design of systems.
Fig. 4. Writing the transfer functions in Laplace transforms

as
The values of N can be positive, zero, or negative depending

upon the number of zeros and the number of poles of F(s)
encircled by s.
1. N 0 (or Z P). The path s encircles more zeros than
poles of F(s) in either the clockwise or counterclockwise
direction, and F will encircle the origin of the F(s)plane N times in the same direction as that of s.
2. N 0 (or Z P). The path s encircles an equal number
of poles and zeros, or no poles and zeros of F(s) in s,
and F will not encircle the origin in the F(s)-plane.
3. N 0 (or Z P). This is similar to the case N 0 but
F encircles the origin of the F(s)-plane in the opposite
direction as that of s.
In this analysis, for convenience, the origin of the F(s)-plane
is selected to be the critical point from which the value of N
is determined. However, it is possible to designate other
points in the complex plane as the critical point depending on
the requirement of the application. In the case of the Nyquist
criterion, the critical point is the 1 on the real axis of the
F(s)-plane.
CASE 2. Another way of presenting the principal argument
may be to begin with the derivation of the relationship between the open-loop and closed-loop poles or zeros viewed
from the characteristic equation. Lets take a closed-loop control system with single input and single output as shown in
NG (s)
DG (s)
(31)
H(s) =
NH (s)
DH (s)
(32)
and
then
G(s)H(s) =
where
N number of encirclements of the origin made by the
path F,
Z number of zeros of F(s) encircled by the path s,
P number of poles of F(s) encircled by the path s.
G(s) =
NG (s)NH (s)
DG (s)DH (s)
(33)
Hence, the characteristic equation

1 + G(s)H(s) =
DG (s)DH (s) + NG (s)NH (s)

DG (s)DH (s)
(34)
and the closed-loop transfer function

M(s) =
NG (s)DH (s)
G(s)
=
1 + G(s)H(s)
DG (s)DH (s) + NG (s)NH (s)
(35)
From Eqs. (31) to (34), it can be observed that the poles of

1 G(s)H(s) are the same as the poles of G(s)H(s). But more
importantly, the zeros of the function 1 G(s)H(s) are the
same as the poles of M(s) of the closed-loop system. Although
simple, this observation bears particular importance in the
understanding of the Nyquist criterion.
Assume that (s) equals 1 G(s)H(s) and has poles and
zeros in the s-plane, as shown in Fig. 5. As any point s1 of
contour s is substituted in (s), it maps to a point on contour
.
For the purposes of illustration, assume that s encloses a
pole and two zeros. Also, two poles and a zero lie outside or
are unbounded by the contour. As the point s1 moves around
the contour in a chosen clockwise direction, each of the pole
and zero vectors connected to that point trace angles. Take
the general equation
(s) =
(s + z1 )(s + z2 ) (s + zZ )
, m, n {1, 2, . . .}
(s + p1 )(s + p2 ) (s + pP )
(36)
This is equivalent to
(s) = |(s)|\(s)
(37)
24
As a matter of interest, similar conclusions can be drawn

from the Eqs. (27)(29), taking (s) analogous to F(s)
j
s-plane
s1
p2
z3
p2
z 3
p3
p
3
1
2 j
z
1
p
1
p1
1
f (s) ds =
2 j
F (s) ds
1
=
F (s)
2 j
d[ln F (s)] = Z P
(40)
z1
z2
and writing this equation as
z2
1
2 j
d[ln F (s)] =
1
2 j
d[ln |F (s)|] + j arg[ln F (s)] = Z P

(41)
Figure 5. Angles traced by poles and zeros. As the point s1 traverses

around the closed contour, each pole and zero trace 360. The angles
traced by the poles and zeros outside the contour go to a minimum
and then to a maximum and then back to the original value tracing
the same path in the opposite direction. The net angle traced becomes zero.
where
|(s)| =
|s + z1 ||s + z2 | |s + zZ |
|s + p1 ||s + p2 | |s + pP |
(38)
and s z1 s zz, s p1 s pP are length of

vectors and the angles are
\(s) = \(s + z1 ) + + \(s + zZ ) \(s + p1 )

\(s + pP )
(39)
From Fig. 5, we can deduce that as the point s1 traverses

around the contour once, the poles and zeros encircled by the
contour s go through a complete rotation, each tracing an
angle of 2 radians. On the other hand, the poles or zeros that
lie outside of s undergo a net angular change of 0 radians.
Because of the positive and negative signs of the Eq. (39), the
net number of rotation is equal to the difference between the
number of zeros and poles lying inside contour s.
For a closed contour, the first part of the integration, d[ln

F(s)] will be zero, but the second term is the 2 times the net
encirclement of the origin.
As discussed earlier, the poles of characteristics equation
(s) 1 G(s)H(s) are also the poles of G(s)H(s). Because
the denominator DG(s)DH(s) of Eq. (34) is much simpler than
the numerator DG(s)DH(s) NG(s)NH(s), the poles of the equation can be determined relatively easily. Also, the zeros enclosed by s are the zeros of (s), and they are the unknown
poles of the closed-loop system. Therefore, P equals the number of enclosed open-loop poles, and Z equals the number of
enclosed closed-loop poles. Thus, Z N P indicates that
the number of closed-loop poles inside the contour equals the
number of open-loop poles of G(s)H(s) inside the contour,
hence the number of clockwise rotations of the mapping about
the origin.
If the contour in the s-plane includes the entire right half
plane (RHP), as illustrated in Fig. 6, the number of closedloop poles enclosed by the contour determines the stability of
the system. Because it is possible to count the number of
open-loop poles P (usually by inspection) inside the bounding
contour in the RHP, the number of enclosures of the origin N
indicates the existence of the closed-loop poles on the RHP.
The poles and zeros of G(s)H(s) are usually known and if
the mapping function is taken to be (s) equals G(s)H(s) instead of 1 G(s)H(s), the resulting contour is the same except
that it is translated one unit to the left. Thus the number of
rotations about the point 1 in the G(s)H(s)-plane may be
j
+j
s = j
s-plane
RHP
LHP
Figure 6. The Nyquist path. The Nyquist

path covers the entire right half of the splane, avoiding poles and zeros located on
the imaginary axis, as shown in (a). The
existence of closed-loop poles on the RHP
indicates unstable conditions. In (b), poles
and zeros on the imaginary axis are included in the Nyquist path.
RHP
LHP
s-plane
s = j
(a)
= Poles or zeros
R = Infinite radius
(b)
Table 3. The Possible Outcomes of Number of Poles and

Zeros on the s-Plane
(s)-Plane Locus
NZP
N0
N0
N0
Direction of
s-Plane Locus
Encircles
of the
Origin
Direction of
Encirclement
Clockwise
Counterclockwise
Clockwise
Counterclockwise
Clockwise
Counterclockwise
N
N
N
N
0
0
Clockwise
Counterclockwise
Counterclockwise
Clockwise
None
None
counted instead of the origin. Hence the Nyquist criterion

may be stated as follows:
If a contour s encircles the entire right half plane, the number of closed-loop poles Z in the RHP of the s-plane can be
determined by the number of open-loop poles P in the
RHP and the number of revolutions N of the resulting contour around the point 1 in the G(s)H(s)-plane.
This mapping is called the Nyquist diagram or the Nyquist
plot of G(s)H(s). A summary of all the possible outcomes of
the principle argument is given in Table 3.
The method discussed is also called the frequency response
technique. Around contour s in the RHP the mapping of the
points on the j-axis through G(s)H(s) is the same as using
the substitution s equals j, hence forming the frequency response G( j)H( j). Thus the frequency response over the positive j-axis from 0 to are used to determine the
Nyquist plot. That is, instead of tracing the entire RHP, it is
sufficient to use just a part of the contour s. The Nyquist
criteria could have easily been built upon the tracing of the
left half plane (LHP); however, the solution is a relative one.
In Fig. 6(a,b) it can be seen that the contour s encircles
the entire right half of the s-plane in the counterclockwise
sense. The reason for this is because in mathematics counterclockwise is traditionally defined to be positive.
Observe on Fig. 6(a) that small semicircles are drawn
along the j-axis because the Nyquist path must not go
through any of the poles or zeros of (s). If any poles or zeros
fall on the j-axis, then the path s should detour around
these points. Only the poles or zeros that lie in the RHP of
the s-plane need to be encircled by the Nyquist path.
From the principal argument the stability of a closed-loop
system can be determined, after the Nyquist path is specified,
by plotting the function (s) 1 F(s) where F(s) equals to
G(s)H(s) and the s variable is chosen along the Nyquist path.
The behavior of the (s) plot, or the new path , is referred
to as the Nyquist plot of (s), with respect to the critical point,
the origin. Because the function F(s) is usually known and is
much simpler to construct, the Nyquist plot of F(s) arrives at
the same conclusion about the stability of a closed-loop system. This is simply done by shifting the critical point from
the origin to the point (1, j0) on F(s)-plane. This is because
the origin of the [1 F(s)]-plane corresponds to (1, j0) of
the F(s)-plane.
25
With the new critical point at (1, j0), it will be necessary

to define two sets of N, Z, and P as follows:
N0 number of encirclement around the origin made by
F(s).
Z0 number of zeros F(s) encircled by the Nyquist path in
the right half of the s-plane.
P0 number of poles F(s) encircled by the Nyquist path in
the right half of the s-plane.
N1 number of encirclement around the point (1, j0)
made by F(s).
Z1 number of zeros 1 F(s) encircled by the Nyquist
path in the right half of the s-plane.
P1 number of poles 1 F(s) encircled by the Nyquist
path in the right half of the s-plane.
When the closed-loop system has only a single feedback system having the loop transfer function of G(s)H(s), then
F(s) G(s)H(s). Now it becomes clear that
P0 = P1
(42)
because F(s) and 1 F(s) always have the same poles. The
result is similar to the one derived earlier in the discussion.
The other stability requirements are that for the closed-loop
stability
Z1 = 0
(43)
and for the open-loop stability

P0 = 0
(44)
Z1 must be zero because of the zeros of 1 G(s)H(s) and the

poles of the closed-loop transfer function M(s) as discussed
earlier; any poles that lie in the left-hand plane causes system
instability. For the case of the open-loop stability, the P0 is
the number of poles of F(s) encircled by the Nyquist path in
the right half of the s-plane and must be zero for stability conditions.
The discussions presented so far may be summarized as
follows:
1. For a given feedback control system, the closed-loop
transfer function is given by Eq. (35), and the denominator function represent the closed-loop transfer function as given by Eq. (34), which is equal to (s). The
Nyquist path is defined in accordance with the pole and
zero properties of F(s) on the j-axis.
2. The Nyquist plot of F(s) is constructed in the [G(s)H(s)]plane.
3. The value of N0 is determined by observing the behavior
of the Nyquist plot of F(s) with respect to the origin.
Similarly the value N1 is determined with respect to the
point (1, j0).
4. After determining N0 and N1, the value of P0, if not already known, can be obtained from
N0 = Z0 P0
(45)
26
if Z0 is known. With P0 determined, P1 is also known

via Eq. (42), and Z1 can then be calculated with
N1 = Z1 P1
(46)
Also, two quantities 1 and 2 will be defined to represent

the total angle traversed by the Nyquist plots of F(s) with
respect to the point (1, j0), corresponding to the points s1
and s1, respectively. Thus the two new quantities may be
written as
and from Eq. (43) Eq. (46) simplifies to

N1 = P1
(47)
Now the Nyquist criterion may also be stated in the following manner.
For a closed-loop system to be stable, the Nyquist plot of
F(s) must encircle the critical point (1, j0) as many times
as the number of poles of F(s) that lie in the right half of
the s-plane. The encirclements, if any, must be in the
clockwise direction when the Nyquist path is defined in
the counterclockwise sense.
SIMPLIFIED NYQUIST CRITERIA

The Nyquist criterion discussed previously requires the construction of the Nyquist plot corresponding to the Nyquist
path in the s-plane. Complication arises when the F(s)-plane
poles or zeros that lie on the j-axis, as in Fig. 6(a), indicate
small indentations around these points. As pointed out in Kuo
(1), Yeung and Lai came up with a simplified version of the
Nyquist criterion for closed-loop systems that requires only
the Nyquist plot corresponding to the positive j-axis of the
s-plane.
In the development of this simplified criterion, two paths
as shown in Fig. 6(a,b) are considered. The first path s1 encircles the entire right half of the s-plane excluding all the poles
and zeros that lie on the j-axis. The second path s2 encircles
the excluded poles and zeros that may exist. Now new quantities may be defined as follows:
Z1 number of zeros of 1 F(s) in the right half of the
s-plane.
P1 number of poles of 1 F(s) in the right half of the
s-plane, and is equal to P0, which are poles of F(s) in the
right half of the s-plane.
P number of poles F(s) or 1 F(s) that lie on the jaxis including the origin.
N1,1 number of times the point (1, j0) of the F(s)-plane
is encircled by the Nyquist plot of F(s) corresponding to
s1.
N1,2 number of times the point (1, j0) of the F(s)-plane
is encircled by the Nyquist plot of F(s) corresponding to
s2.
1 = 2 N1,1 = 2 (Z1 P1 )
(50)
2 = 2 N1,2 = 2 (Z1 P P1 )
(51)
and
To analyze the Nyquist path, it is best to consider the path

having three major sections. The first section is the portion
from s equals to j to j along the semicircle having an
infinite radius, the second portion is the path along the jaxis excluding the small indentations, and the final sections
include the small indentations. Because the Nyquist plot is
symmetrical at j 0, the angles traversed are identical for
positive and negative values of . Hence
(48)
(49)
where Eq. (49) includes the number of poles of F(s) or 1

F(s) that lie on the j-axis.
2 = 211 12 + 13
(53)
where
11 angle traversed by the Nyquist plot of F(s) with respect

to (1, j0) corresponding to the positive or negative
side of the j-axis, excluding the small indentations
and the factor of two emmerging in Eq. (52) and Eq.
(53).
to (1, j0) corresponding to the small indentations
along the j-axis s1. Also, with the direction of the
small indentations of s2 different from that of its counterpart, the negative sign emerges in Eq. (53).
to (1, j0) corresponding to the semicircle with infinite
radius on the Nyquist path.
Generally, for a physical realizable transfer function, the
number of poles cannot exceed the number of zeros of F(s).
Therefore, the Nyquist plot of F(s) corresponding to the infinite semicircle must be a point on the real axis or a trajectory
around the origin of the F(s)-plane. The angle 13 traversed by
the phasor from the point at (1, j0) to the Nyquist plot along
the semicircle is always zero.
Combining Eq. (52) and Eq. (53) yields
1 + 2 = 411
(54)
1 + 2 = 2 (2Z1 P 2P1 )
(55)
11 = (Z1 0.5P P1 )
(56)
since 13 is zero,
and
N1,2 = Z1 P P1
(52)
and
According to the Nyquist criterion,

N1,1 = Z1 P1
1 = 211 + 12 + 13
hence
Equation (56) means that the net angle traversed by the phasor from the (1, j0) point to the F(s) Nyquist plot corresponding to the positive j-axis of the s-plane excluding any of the
small indentations, that is
The number of zeros of 1 F(s) in the right half of the splane minus the sum of half the poles on the j-axis and
the number of poles of F(s) in the right half of the s-plane
multiplied by radians.
This means that the Nyquist plot can be constructed corresponding to s 0 to s j portion of the Nyquist path. For
an unstable closed-loop system, the number of roots of the
characteristic equation that fall in the right half of the splane can be determined via Eq. (55).
As mentioned earlier, a closed-loop system is stable only if
Z1 is equal to zero.
Hence,
11 = (0.5P + P1 )
(57)
This indicates that for closed-loop system stability the phase

traversed by the Nyquist plot of F(s) where s varies from zero
to j with respect to (1, j0) point cannot be positive because
P and P1 cannot be negative.
SOLUTION. To obtain the Nyquist plot, rearrange this equation, substitute s j, and assign the nominal value, K 1,
G( j)H( j) =
Example 1. Plot the Nyquist diagram of a closed-loop control

system as in Fig. 4 with a loop transfer function
G(s)H(s) =
K(s + 1)(s + 20)

250s(s + 0.2)(s + 0.4)
(58)
(1 + j)(1 + 0.05 j)
j(1 + 5 j)(1 + 2.5 j)
(59)
Find magnitudes and angles in terms of variable as
1 + 2 1 + (0.05)2
|G( j)H( j)| =
1 + (5)2 1 + (2.5)2
(60)
and
G( j)H( j) = tan1 + tan1 0.05 90

tan1 5 tan1 2.5
(61)
Now Nyquist contour may be applied by substituting values of from zero to on the positive part of the imaginary
axis on the s-plane. By avoiding the pole located on the origin
and substituting small positive values on the positive and
negative side of zero, it is possible to observe the following
features:
0+
NYQUIST DIAGRAMS
The Nyquist analysis is based on the assumption that the
control systems are linear; hence, the dynamic performances
are described by a set of linear differential equations. Because
of the nature of feedback control systems, the degrees of numerator of the loop transfer function F(s) G(s)H(s) is always
less than or equal to the degree of the denominator. All the
Nyquist diagrams presented here are based on these two assumptions.
As explained previously, when plotting Nyquist diagrams,
it is sufficient to assign values for the complex variable s on
the j-axis avoiding possible poles and zeros on the imaginary
axis. The frequency response of G(s)H(s) can be determined
by substituting s j and by finding the imaginary and complex components of G( j)H( j). Alternatively, G( j)H( j)
can be written in polar form, and magnitudes and angles are
determined for plotting on a polar graph paper. These techniques will be illustrated in the following examples.
27
|G( j)H( j)|

|G( j)H( j)| 0
|G( j)H( j)| 0
|G( j)H( j)|
and
and
and
and
G( j)H( j) 90
G( j)H( j) 90
G( j)H( j) 90
G( j)H( j) 90
These features indicate that for a clockwise rotation in the

s-plane covering the entire right half plane, the graph starts
from the infinity on the imaginary axis (in either the fourth
or the third quadrant) and approaches zero again from the
90 for 0 . Similarly, the graph starts from 0 at
an angle 90 and approaches to infinity with the same angle
90 for 0.
By substituting intermediate values for , the results in
Table 4 may be obtained. We can see that the plot goes numerically above 180 between 0.4 rad/s and 0.5
rad/s. It also falls back to be numerically less than 180
after 2.0 rad/s. This means that it crosses the real axis
on the negative side of the s-plane twice.
At this point, a polar plot graph paper may be used to
sketch the curve outlined in Table 4. Or a second table, which
shows the real and imaginary components of G( j)H( j), may
be made from Table 4 by using the relation [Rej R cos
Rj sin ] as in Table 5. An alternative approach to calculate
the real and imaginary components of G( j)H( j) is introduced in the next example.
The Nyquist plot of Table 5 is shown in Fig. 7. It is worth
noting that Table 5 could also have been drawn by rearranging Eq. (59) as real and imaginary components. This is a long
procedure, but using it allows us to calculate the exact values
Table 4. Calculation of Magnitudes and Angles of G( j)H( j)

rad/s
G( j)H( j)
G( j)H( j)
0.1
0.4
0.5
1.0
2.0
4.0
90
124
9.0
175
0.79
181.6
0.55
188.9
0.1
182.8
0.021
174.2
0.005
90
0
28
Table 5. Calculation of Real and Imaginary Components of G( j)H( j)

rad/s
G( j)H( j)
G( j)H( j)
0.1
0.4
0.5
1.0
2.0
4.0
5.03
7.46
0.787
0.069
0.55
0.018
0.099
0.015
0.021
0.0001
0.005
0.0005
0
0
of the gain and phase margins if an approximate estimation

from the plot is not permissible.
By using Eqs. (60) and (61), it can be shown that
G( j)H( j) 0.51 and G( j)H( j) 180 for 0.481
rad/s. Remember that value is for the nominal K where K
1. For other values of K, Eq. (60) should have been written as
1 + 2 1 + (0.05)2
|G( j)H( j)| = K
1 + (5)2 1 + (2.5)2
(62)
Therefore, the curve will pass to the left-hand side of the 1

point on the real axis if G( j)H( j) 1. Hence K 0.51
1 gives the value K 1.9 in which the plot will encircle the
1 j0 point in the clockwise direction, thus leading to instability. This indicates that there is a zero of the characteristic
equation on the RHP of the s-plane or a RHP pole of the
closed loop transfer function, hence the instability.
From the preceding analysis, it is known that the plot
crosses the real axis again at a frequency slightly greater
than 2.0 rad/s. By substituting a value 2.57 rad/s,
from Eqs. (60) and (61), it can be shown that G( j)H( j)
0.011 and G( j)H( j) 180. As explained previously the
corresponding value of K 91 obtained from K 0.011 1.
For K 91 and above, the system becomes stable again.
Example 2. Plot the Nyquist diagram of unity feedback control system which has a forward gain transfer function
G(s) =
(1 + 2s)
s(s 1)
(63)
SOLUTION. In this example, because G(s) has a pole on the

RHP of the s-plane, it is open-loop unstable. As before, in order to obtain the Nyquist plot, substitute s j in the loop
Im
=0
G(s)H(s)-plane
R
1
Re
= 0+
Figure 7. The Nyquist plot of Example 1. As the Nyquist path on
the s-plane traverses in the clockwise direction, the corresponding
path of G(s)H(s) traverses in the same direction on the G(s)H(s)-plane.
Because the Nyquist plot does not encircle the critical 1 j0 point,
this system is stable.
transfer function G( j)H( j) and assign the nominal value

(K 1)
G( j) =
(1 + j2)
j( j 1)
Find magnitudes and angles in terms of variable as
1 + 42
G( j) =
1 + 2
(64)
(65)
and
G( j) = tan1 2 90 tan1 /(1)
(66)
Lets consider the nominal value of K. Observing the extreme values for in the clockwise direction and starting
from 0 gives
0+ |G( j)H( j)| and G( j)H( j) 270 or 90

|G( j)H( j)| 0 and G( j)H( j) +270 or 90
|G( j)H( j)| 0 and G( j)H( j) +90
0 |G( j)H( j)| and G( j)H( j) 90
It is important to highlight the angle equation tan1/(1)
because the negative sign in the denominator indicates what
quadrant the angle is for varying . These features indicate
that for a clockwise rotation of a contour in the s-plane covering the entire right half plane, the graph starts from the
infinity on the imaginary axis from the first or second quadrant and approaches zero from the 90 for 0 .
Similarly, the graph starts from 0 at an angle 90 and
approaches infinity with the same angle 90 for
0.
Table 6 may be obtained by substituting values for , but
this time for only 0 . The Nyquist plot is given in
Fig. 8. The open-loop transfer function has one pole on the
RHP, and therefore P 1. In order for this system to be stable, N must be equal to 1, that is, one counterclockwise encirclement of the 1 j0 point. As shown in Fig. 8, the rotation of the curve is in a counterclockwise direction, and it
encircles the origin once; hence, the system is stable.
From Table 6, we can see that the graph crosses the real
axis between 0.6 rad/s and 1.0 rad/s. By guessing
and by using the trial-and-error method, this crossover frequency may be determined as 0.7 rad/s. At this frequency, the magnitude is 2.0. As in the case of Example 1,
the critical value of the gain can be found from K 2.0 1
to be K 0.5.
An alternative mathematical approach can be employed to
find the real and imaginary components of the loop transfer
function as
G( j) =
(1 + j2)
32 + j( 23 )
(1 + j2)
= 2
=
(67)
j( j 1)
j
2 + 4
29
Table 6. Calculation of Magnitudes and Angles of G( j)H( j)

rad/s
0.1
0.4
0.6
1.0
4.0
10.0
G( j)H( j)
G( j)H( j)
90
107.0
10.15
150.5
2.97
171.16
2.23
198.4
1.58
248.8
0.49
261.4
0.02
270
0
The graph crosses the real axis when the imaginary part is
equal to zero, that is ( 23) 0 or 1/ 2 rad/s. The
intersection can be calculated by substituting the value of
in the real component of G( j)H( j) as
3 (1/2)
32
= 2
=
2
4
+
(1/2) + (1/4)
(68)
There are many examples of Nyquist plots in control engineering books (15). Figure 9 illustrates typical Nyquist plots
of some of the selected control systems.
Stability Margins
In the preceding examples, we have demonstrated that the
Nyquist plots of the loop transfer function, G( j)H( j), depends on the values of K. This is illustrated in Fig. 10. As the
K is increased or decreased, as the case may be, at a certain
value the locus passes through 1 j0 point. At this point,
the system exhibits sustained oscillations. As K increases further, the system becomes unstable.
Generally, oscillations increase as the locus of G( j)H( j)
gets closer to the 1 j0 point. The closeness of the locus to
the critical point is measured by the stability margins usually
expressed in the form of phase and gain margins, as illustrated in Fig. 10. These margins indicate relative stability
and hence help the design of control systems to achieve desired responses. The gain and phase margins may be defined
as follows.
Gain margin is the amount of gain that can be allowed to
increase before the closed loop system becomes unstable.
Im
=0
G(s)H(s)-plane
Re
= 0+
Figure 8. The Nyquist plot of Example 2. The open-loop transfer
function of this control system has a pole on the RHP; hence, the
system is open-loop unstable. However, the Nyquist plot encircles the
critical 1 j0 point once in the counterclockwise direction, indicating that there are no closed-loop poles on the RHP. Therefore, the
system is stable.
Phase margin is the angle, in degrees, by which the locus

must be rotated in order that gain crossover point
passes through 1 j0.
The gain margin is measured in decibels and expressed in
phase-crossover frequency as
GM = 20 log10 =
1
dB
|G( jc )H( jc )|
(69)
When the loop transfer function G( j)H( j) passes through

the 1 j0 point, the gain margin is 0 dB. The negative or
positive value of gain margin depends on the number of poles
and zeros of G( j)H( j) on the RHP. If the stability is evaluated when the locus crosses the real axis on the right of the
1 j0 point (Example 1), the gain margin is positive. If the
stability is evaluated on the left of the 1 j0 point (Example 2), then the gain margin is negative.
The phase margin can be determined by calculating the
gain-crossover frequency when the gain of the G( j)H( j)
1 and by evaluating the phase angle of the system at that
frequency. That is
PM = G( j)H( j) 180
(70)
As in the case of gain margin, the sign of phase margin is

relative to stability condition and the shape of the locus. In
Example 1, a negative value for the PM indicates unstable
condition whereas in Example 2, the negative value implies
stability.
Effects of Adding Poles and Zeros
Control systems are often designed by introducing additional
poles and zeros to the system. Extra poles and zeros in the
system change the shape of the Nyquist diagrams and alter
phase and gain margins. The influences of additional poles
and zeros on the Nyquist locus can be evaluated by comparing
the loci of different systems given in Fig. 9. Some observations may be made as follows.
The mathematical difference between parts (a) and (b) in
Fig. 9 is the additional pole. In this case, the Nyquist
locus is shifted by 90 as , occupying quadrants
3 and 4 instead of quadrant 4 only. Adding an extra pole
introduces further 90, and the locus occupies three
quadrants. In this case, the risk of instability exists because the possibility of encirclement of 1 j0 is introduced.
In Fig. 9, the effect of adding a pole at s 0 can be seen
by observing parts (a) and (d) or (b) and (e). In both
cases, Nyquist loci are rotated by 90 for all frequencies. Adding a finite pole increases the risk of instability.

Nyquist diagrams
Nyquist diagrams
0.6
K
G(s)H(s) =
(s1 + 1)
0.2
0.4
0.6
0.5
0
Real axis
0.5
0
0.2
0
2
4
0.6
6
0.5
0
Real axis
0.5
8
4
K
(s 1+1)(s2+1)(s3+1)
0.4
0.8
1
G(s)H(s) =
0.2
2
4
Real axis
(c)
(b)
(a)
Nyquist diagrams
Nyquist diagrams
1.5
1.5
1
Imaginary axis
0.8
1
Imaginary axis
Imaginary axis
0.4
G(s)H(s) =
K
(s 1+1)
0
0.5
0
0.5
1
1.5
1.5
1.5
2
2
K
(s 1+1)(s2+1)
0.5
G(s)H(s) =
0.5
2
0 0.5
2 1.5 1 0.5
Real axis
1.5 1 0.5 0 0.5

Real axis
(d)
1.5
(e)
Nyquist diagrams
Im
10
G(s)(H)s = KesT(1 esL)

where L > T
8
6
Imaginary axis
Imaginary axis
G(s)H(s) =
K
(s1 + 1)(s2 + 1)
0.6
0.4
0.2
Nyquist diagrams
8
0.8
Imaginary axis
30
G(s)H(s) =
K (s z +1)
(s 1+1)(s2+1)
4
2
0
2
Re
4
6
8
10
2 1.5 1 0.5
0 0.5
Real axis
(f)
1.5
2
(g)
Figure 9. The Nyquist plots of selected control systems.
10
31
Im
Im
G(s)H(s)-plane
G(s)H(s)-plane
Unit circle
Re
1
PM
GM
Re
=0
Increasing
Increasing K
Figure 10. Phase and gain margins. The closeness of the locus to
the critical 1 j0 point is measured by the margins. Good gain and
phase margins are obtained for K1. As K increases, both gain and
phase margins become zero (for K2) indicating critical stability. Further increase in K leads to unstable conditions. Note the changes in
the gain and phase margins for varying K.
The effect of adding a zero into the system can be seen in

parts (c) and (f) in Fig. 9. In this case, the loop transfer
function increases the phase of G(s)H(s) by 90 as
. This result confirms the general knowledge that addition of a derivative control or a zero makes the system
more stable.
EFFECTS OF TIME DELAYS

The Nyquist criterion can be utilized to evaluate the effects
of time delays on the relative stability of feedback control systems. With the pure time delays, the loop transfer function
may be written as
G(s)H(s) = esT G1 (s)H1 (s)
(71)
where T is the time delay. The term esT does not introduce
any additional poles or zeros within the contour. However, it
adds a phase shift to the frequency response without altering
the magnitude of the curve. This is because
|G( j)H( j)| = |e jT ||G1 ( j)H1 ( j)|

= | cos() j sin()||G1 ( j)H1 ( j)|
Figure 11. The effect of time delays. Pure time delays do not introduce any extra poles and zeros into the system. However, the magnitude is equal to unity for all frequencies, the phase ( T) affects
the stability. For large values of time delay T, the system may be unstable.
tion of a single loop, single input and single output of a

system may be written as
M(z) =
G(z)
C(z)
=
R(z)
1 + GH(z)
where z is the z-transform defined as z esT.

The stability of the system can be studied by investigating
the zeros of the characteristic equation 1 GH(z) 0. For
the system to be stable, all the roots of the characteristic
equation must be inside the unit circle in the z-plane. As in
the continuous-time systems, the investigation of the Nyquist
plot of GH(z) with respect to critical point, 1 j0, indicates
the system stability. The general theory presented for continuous time control systems is applicable to discrete time systems with minor modifications. Here, an example will be
given to illustrate the use of Nyquist in discrete time control
systems.
Im z
G(z)H(z)-plane
(72)
The term containing the time delay is cos() j sin() 1,

but the phase is tan1(sin T/cos T) T. This shows
that the phase grows increasingly negative in proportion to
the frequency. A plot of the effect of time delay is given in Fig.
11. Because of the addition of the phase shift, the stability of
the system is affected for large values of T.
NYQUIST STABILITY CRITERION FOR DIGITAL SYSTEMS

The Nyquist stability criterion can equally be applied to linear continuous-data discrete time systems to graphically determine the stability. Generally, the closed-loop transfer func-
(73)
0.0231
1
= 3.14
Re z
=0
Figure 12. Nyquist plot of Example 3. The Nyquist path on the zplane must have small indention at z 1 on the unit circle. The
Nyquist plot of path of GH(z) in the GH(z)-plane intersects the negative real axis at 0.231 when 3.14 rad/s. For stability, the value
of K must be less than 4.33.
32
THE INVERSE NYQUIST AND NYQUIST

PLOT FOR MULTIVARIABLE SYSTEMS
Im
G(s)H(s)-plane
Inverse Nyquist is simply the reciprocal of the complex quantity in the Nyquist plot. They find applications particularly
in multiple loop and multivariable systems where graphical
analysis may be preferred.
The Nyquist stability criterion applied to inverse plots can
be stated as a closed loop system stable, if the encirclement
of the critical 1 j0 point by the 1/G(s)H(s) is in the counterclockwise direction for a clockwise Nyquist path in the splane. As in the case of a normal Nyquist, the number of encirclements must equal the number of poles of 1/G(s)H(s) that
lie in the right half of the s-plane.
Inverse Nyquist plots is particularly useful in the analysis
of multi-inputmulti-output control systems. In the multivariable feedback control systems, the relations between inputs and outputs may be expressed in matrix form as
Re
1.5
G(s) =
s3 + 2s2 +2s+1
1
4s+1
C(s) = [I + KG(s)H(s)]1G(s)KR(s)
x
H(s) = I
Figure 13. Examples of Nyquist plots of multivariable systems. The
Nyquist plot for multivariable systems carries similar information as
in the single-inputsingle-output systems. The number and the direction of encirclements of the critical 1 j0 point conveys the message
about the stability. But rigorous mathematical analysis is necessary
because matrices are involved.
(75)
where G(s), H(s) and K are n n matrices.

Similar to single-inputsingle-output systems, the output
is exponentially stable iff det[I KG(s)H(s)]1 has no poles
in the right half of the s-plane. The Nyquist diagrams can be
obtained by appropriately considering the K values as K
diagk1, k2, . . ., kn and the gij(s) elements of G(s). A typical
example of a Nyquist diagram of a multivariable control systems is shown in Fig. 13. This example is also given by Westphal (5); interested readers could refer to that book for further details.
BIBLIOGRAPHY
Example 3. Show the Nyquist plot of a discrete time system

with transfer function of
GH(z) =
0.632z
(z 1)(z 0.368z)
(74)
for a sampling period of T 1 s.

SOLUTION. The loop transfer function GH(z) does not have
any poles outside the unit circle, but it has one pole on the
unit circle. As in the case of s-plane zeros on the imaginary
axis, the Nyquist path on the z-plane must have small indention at z 1 on the unit circle. The Nyquist path, shown in
Fig. 12, intersects the negative real axis of the GH(z)-plane
at 0.231 when the value of 3.14 rad/s. The critical 1
j0 point may be encircled if 0.231K 1, that is K 4.33.
1. B. J. Kuo, Automatic Control Systems, 6th ed., Englewood Cliffs,

2. E. Kreyszing, Advanced Engineering Mathematics, 7th ed., New
York: John Wiley, 1993.
3. W. S. Levine, The Control Handbook, Boca Raton, FL: CRC
Press, 1996.
4. K. Ogata, Modern Control Engineering, 3rd ed., Upper Saddle
River, NJ: Prentice-Hall, 1997.
5. L. C. Westphal, Sourcebook of Control Systems Engineering, Cambridge, UK: Chapman & Hall, 1995.
HALIT EREN
BERT WEI JUET WONG
Curtin University of Technology
NYQUIST STABILITY. See NYQUIST CRITERION, DIAGRAMS, AND STABILITY.
150
OPEN-LOOP OSCILLATORY CONTROL

Conventional control systems rely on feedback, feedforward,
or a combination of the two. In a feedback control system,
the controlled variable is usually compared with a reference
variable, and the difference between the two, the error, is
used to regulate the system. In a feedforward control system,
an appropriate additive control signal is introduced to compensate for disturbances. While feedback and feedforward
rely on different principles, both methods require measurements. In a feedback control system, the controlled variable
is measured. Likewise, in a feedforward control system the
measurement of disturbances is used in the implementation.
However, measurements of states or disturbances are often
costly, difficult, or even impossible to obtain. In these cases,
feedback and feedforward are not feasible means of control.
Consider the particle accelerator originally described in (1)
and later discussed in detail in (2). The control objective is
to focus a beam of particles along the accelerator. In cyclic
accelerators with azimuth symmetrical magnetic fields, the
plant, a beam of particles, is described by
d 2x
+ 2 (1 n) x = 0
d 2
d 2z
+ 2 nz = 0
d 2
(1)
where x and z are state coordinates representing small betatron oscillations of the beam, is the betatron wave number,
is an independent variable (azimuth), and n is the field index of refraction. For proper operation, the field index of refraction should satisfy 0 n 1. However, for this range of
n, the beam focusing is often unacceptable. Feedback is usually not possible due to the difficulty of measuring x and z.
Feedforward also has similar measurement difficulties.
In such cases, a natural question is whether control is possible for such unmeasureable systems. For many systems,
one alternative is open-loop oscillatory control, sometimes referred to as vibrational control (not to be confused with vibration control where the idea is to reduce vibrations). Open-loop
oscillatory control is a fairly recently developed control methodology that does not require measurements of states or disturbances. Instead, zero mean periodic excitation is used to
modify the plant behavior in such a way that control is
achieved as the result of the systems natural response to the
excitation. For example, oscillations in the cyclic accelerator
can be introduced by appropriately focusing and defocusing
sectors of the magnetic lens. This causes a suppression of
betatron oscillations and thereby makes the focus beam more
acceptable. An early heuristic description of this phenomena
was given by Livingston (1), but it was not until 1980 that the
heuristically controlled azimuth accelerator was explained in
the context of open-loop oscillatory control in (2).
Definition 1. Open-loop Oscillatory Control. The utilization of periodic (or almost periodic) control laws, without the
use of measurements in order to induce a desired dynamic
response in a system is referred to as open-loop oscillatory
control or vibrational control.
The simplicity of open-loop oscillatory control synthesis is
offset by the difficulty added in introducing explicit time dependence in the state system models. In order to simplify the
analysis, open-loop oscillatory control algorithms may restrict
the control action so the controlled system admits a small parameter. One way to obtain a small parameter is to introduce
periodic excitation whose frequency is an order of magnitude
larger than the highest system natural frequency. The small
parameter will then arise as the result of a rescaling of time.
For such systems, the time-varying open-loop controlled system can be approximated by the behavior of a time-invarient
averaged equation, to which the usual analytical techniques
for time-invariant systems may be applied. This result forms
the basis of classical averaging theory in applied mathematics
and dynamical systems. Within the context of forced mechanical systems and averaging, energy methods and a quantity
called the averaged potential provide the most direct method
of analysis. In the absence of distinct system time or length
scales, the local stability of an equilibrium or periodic orbit
can be studied by the analysis of the linearized systems first
return map, or monodromy matrix, obtained through Floquet theory.
One of the most compelling examples of open-loop oscillatory control is the stabilization of the simple pendulums inverted equilibrium by high frequency vertical oscillation of
the pendulums suspension point. This discovery is usually
attributed to Bogoliubov (3,4) and Kapitsa (5), although earlier references to similar phenomena exist (6). More recent
accounts of this stabilization may be found in (7,8,9,10),
151
where the averaged potential is used to study global structure

and stability of the periodically forced system. The main result is that for a given forcing amplitude, there is a unique
critical forcing frequency at which the inverted equilibrium
experiences a pitchfork bifurcation. For forcing frequencies
higher than the critical frequency, the inverted equilibrium is
stable. Reference 10 considers a generalization of the classic
problem, where the periodic forcing is directed along an incline with respect to the horizontal. In this case, the pendulum tends to stabilize in configurations aligned with the direction of the forcing, and in fact this phenomenon holds in the
general n-link case as well (11).
Open-loop oscillatory control has been applied to many systems, and new applications continue to emerge. In (12) the
technique was applied to exothermic chemical reactions in a
continuous stirred tank reactor (CSTR). In this work, it was
shown that by modulating the input and exit chemical feed
rates of the CSTR, it is possible to operate in stabilized averaged conversion rates that would otherwise be unstable unless expensive feedback is applied. Although, on average, the
same amount of input chemical has been used, the stable operating regimes of the CSTR change substantially with the
use of an oscillatory control input. In similar work, the results
of (12) are analytically extended to include chemical reactions
in a CSTR with delayed recycle stream (13,14). Historically,
all of this work on oscillatory open-loop control was prompted
by the work of periodic operation of chemical reactors using
the sometimes heuristic techniques of push-pull, periodic optimization, and asynchronous quenching (12,15).
Experimental applications of open-loop oscillatory control
have also included laser illuminated thermochemical systems
(16), stabilization of plasma (17), and car parking algorithms
(18). In (19), sufficient conditions are given for a periodic process to minimize periodic paths. This approach generalized
the result in (20) that showed that periodic paths improve
aircraft fuel economy. Other analytic applications of open-loop
oscillatory control include rotating chains (21,22), n-link pendula (11), axial compressors (23), and population models
(24,25).
In the work by Lehman et al. (13,25,26) the technique of
oscillatory open-loop control is developed for systems with
time-delays. Bentsman and Hong (27,28) have extended the
technique to parabolic partial differential equations (PDEs).
The application of open-loop control to delay systems and
PDEs shows interesting potential since these types of infinite
dimensional systems are often difficult to control when using
feedback. Likewise, there has been success in combining the
benefits of open-loop oscillations with conventional feedback
in order to robustly stabilize systems with zeros in the openright half plane and systems with decentralized fixed zeros
(2932).
As with all other control algorithms, the important issues
of designing open-loop oscillatory control include stability,
transient response, and accuracy of the controlled system.
Certainly, the most important issue is stability. Many classical results on stability of operating points for systems with
oscillatory inputs depend on eigenvalues of the averaged system lying in the left half plane, or equivalently the eigenvalues of the monodromy matrix lying within the unit disk. However, there has been growing interest in the stabilization of
systems to which such classical results do not apply. These
include the mechanical systems studied in (7,8,10,11,33),
152
where eigenvalues locations are typically symmetric with respect to the imaginary axis. Coron (34) has shown the existence of a time-varying feedback stabilizer for systems whose
averaged versions have eigenvalues on the imaginary axis.
Additional interest in this design derives from the observation that it provides a method of smooth feedback stabilization for systems which Brockett (35) had previously shown
were not stabilizable by smooth, time-invariant feedback.
Stability of a system is concerned with the asymptotic behavior of the system. Often it is important to study trajectories of systems as steady-state behavior is being approached.
Analysis of such trajectories when there is an oscillatory control input is a difficult task. The oscillatory control is usually
designed to be high frequency. As a result, the controlled system is composed of a fast zero average oscillatory trajectory
superimposed on a slow trajectory. Therefore, the designer
must attempt to control the slow part of the trajectory and
ignore (or filter out) the high frequency component.
One disadvantage of open-loop oscillatory control is its accuracy. It is well known that driving a nonlinear system with
a periodic signal generally excites an array of resonances, and
under appropriate conditions chaos in the homoclinic tangles
of unstable resonances [See (36) for a complete exposition on
this topic]. While subharmonic resonances and chaos tend to
be suppressed at high forcing frequencies, 1 : 1 resonances
(primary resonances, or periodic orbits), whose averages correspond to fixed points of an averaged representation of the
dynamics, persist. If a stable 1 : 1 resonance has no association with a fixed point of the time-varying system (i.e., it
arises through a bifurcation), it is called a hovering motion.
These high frequency responses limit the utility of open-loop
oscillatory control when control accuracy is important.
PROBLEMS IN OPEN-LOOP OSCILLATORY CONTROL
Classes of Systems
This section considers systems of ordinary differential equations, with inputs, of the form
x = f (x, u)
(2)
where x n and u m. The function f: n m n will

always be assumed sufficiently continuous so that solutions to
Eq. (2) exist. Models of this form describe most of the systems
appearing in the recent engineering literature on open-loop
oscillatory control, as discussed in detail in (37).
Stabilization
Here we introduce the problem of vibrational stabilization, as
found in (24,38). Suppose that (39) has an unstable equilibrium point, xs when u constant 0, and the goal is to
determine a control input u(t) that stabilizes this operating
point. In addition, suppose this stabilization is to be performed without any state or disturbance measurements.
For the stabilization problem being considered, the methods of oscillatory open-loop control are as follows. Introduce
into Eq. (2) oscillatory inputs according to the law u(t)
0 (t) where 0 is a constant vector and (t) is a periodic
average zero (PAZ) vector, that is, (t) (t T) with
T
0 (t)dt 0. Even though the average value of u(t) remains
equal to 0, it is hoped that the periodic forcing can impose a

stabilizing effect on the system.
In this case, Eq. (2) becomes
dx
= f (x, 0 + (t))
dt
(3)
Definition 2. Vibrationally Stabilizable. An equilibrium

point xs of Eq. (2) is said to be vibrationally stabilizable if for
any 0 there exists a PAZ vector (t) such that Eq. (3) has
an asymptotically stable periodic solution, x*(t), characterized
by
x xs ;
x =
1
T
x (t) dt
It is often preferable that Eq. (3) has a fixed equilibrium

point, xs. However, this is not usually the case since the right
hand side of Eq. (3) is time varying and periodic. Therefore,
the technique of vibrational stabilization is to determine vibrations (t) such that the (possibly unstable) equilibrium
point xs bifurcates into a stable periodic solution whose average is close to xs.
The engineering aspects of the problem consist of:
1. Finding conditions for the existence of stabilizing periodic inputs
2. Determining which oscillatory inputs, u( ), are physically realizable and
3. Determining the shape (waveform type, amplitude,
phase) of the oscillations to be inserted which will ensure the desired response
At this time, it may be useful to explain why it is necessary
to use time-varying control inputs as opposed to simply using
classical time-invariant open-loop control techniques. Suppose that there is a single-input single-output linear timeinvariant (LTI) system with proper transfer function Y(s)/
U(s) n(s)/d(s), where Y and U are the Laplace transform of
the output and the input, respectively, and n and d are polynomials in s. If all the roots of d(s) 0 have negative real
parts, then open-loop control can be used to arbitrarily place
system poles simply by letting U(s) d(s)/p(s), where p(s)
0 has the desired system pole location and the degree of p is
greater or equal to the degree of d. At times, this polezero
cancellation open-loop control strategy might give desired system performance, especially if there is no need for feedback
(perhaps because there are no disturbances).
Unfortunately, though, perfect polezero cancellation is
not possible. This may not be worrisome if all the roots are in
the left-half plane, but when there exists at least one root of
d(s) 0 with positive real part, LTI open-loop control cannot
stabilize a system. On the other hand, when u(t) is an oscillatory open-loop control input, stabilization is often possible,
even when there is a pole in the right-half plane. Indeed, oscillatory open-loop controls have also shown a remarkable robustness to disturbances in many experimental applications
(12,37). This is a quality that is absent in LTI open-loop
control.
Remark 1. This subsection has attempted to state the problem of stabilization in its broadest terms. There are classes of
systems, however, for which discussion of stabilization and

stability is problematic. Such systems include conservative
systems, or more specifically, Hamiltonian systems. Hamiltonian systems include dissipation-free mechanical systems,
and include many electrical and optical systems as well. The
primary defect of Hamiltonian systems as far as control theory is concerned is that the strongest stability these systems
can possess is neutral stability; that is, eigenvalues/poles on
the imaginary axis. For this reason, standard concepts from
control theory seldom yield strong stability results. Progress
has recently been made in developing techniques for the stability analysis of these systems. The new techniques make
use of the system energy, and in the case of periodically forced
systems the averaged potential, to assess the stability of equilibriums. A technique for the equilibrium and stability analysis of a large class of periodically forced Hamiltonian systems
is presented later in this article.
Transient Behavior and Performance
Once a system is determined to be stable, the next issue in
evaluating its performance is to determine how quickly the
solutions decay to their steady state. This finite time transient behavior is sometimes crucial to system performance.
For LTI systems, there are several methods that can be used
to obtain estimates for the transient behavior of the output.
For example, an estimated output trajectory is obtained from
information on the location of the dominant system eigenvalues. Even for nonlinear time-invariant systems, it is common
to examine the eigenvalues of a Jacobian linearization in order to examine the rates of decay of solutions.
For systems subject to oscillatory open-loop control, the
analysis techniques are not so straightforward. As previously
mentioned, the control inputs cause Eq. (3) to be time-varying, and analysis of time-varying systems remains an open
area of research. However, since it has been assumed that the
control inputs have a special structure, that is, periodic and
high-frequency, it will be possible to apply the method of averaging to find approximations of the system transient behavior.
Essentially the problem of controlling the transient behavior of time-varying system Eq. (2) is to
1. Determine control inputs, (t) in Eq. (3) so that the solutions to Eq. (3) can be approximated by the solutions
of a simpler equation
2. Control the transient behavior of the approximate
equation
Sometimes this simpler equation turns out to be purely timeinvariant and in the form of
dy
= P( y),
dt
w = q( y)
(4)
where w and y are both vectors in n, and w approximates x,

the solution to Eq. (3). Often, though, a time-varying output
equation is used and the approximate equation becomes
dy
= P( y),
dt
w = h(t, y)
(5)
153
where, once again, w approximates x. The oscillatory openloop control results in a superposition of fast oscillatory trajectories on slow trajectories. The slow dynamics are represented by y, and h can be a fast periodic function. In either of
the above two cases, it is hoped to find oscillatory control input u(t) such that the transient performance of w meets desired objectives. Since the state equation of the approximations are time-invariant, the analysis becomes simpler. In
fact, even though Eq. (5) is time-varying, it is only the output
equation which explicitly depends on t. Therefore, many of
the well established tools can be applied directly to the state
equation. In particular, when P(y) Ay B, then the eigenvalues of matrix A help determine the qualitative features of
transient behavior.
Steering and Path Planning for Kinematically
Nonholonomic Systems
An application of open-loop oscillatory control which lies
largely outside the boundaries of this chapter is the use of
periodic functions in path generation for so-called kinematically nonholonomic systems. Such systems include wheeled
vehicles such as the unicycle, autonomous wheeled robots,
and cars with or without trailers. More generally, kinematically nonholonomic systems are systems which possess nonintegrable constraints, and typically the state equations do not
include dynamic effects, such as torques, accelerations, and
forces. Since this class of problems does not involve the articles central themes of stabilization and improvement of transient performance, only a brief description is given.
Consider the special case of Eq. (2) in which
f (x, u) =
m
ui gi (x)
i=1
A large body of literature has been published on the use of

oscillatory inputs designed to force such systems along prescribed paths. The reader is referred to (37) and its references
for details of these types of problems. The types of problems
encountered in this application include the following:
1. The prescribed endpoint steering problem requires that
given any pair of points x0, x1 n, a vector of piecewise
analytic control inputs u( ) (u1( ), . . ., um( )) is to
be determined to steer from some state x0 at time t 0
to x1 at time t T 0.
2. The trajectory approximation steering problem requires
that given any sufficiently regular curve : [0, T] n,
a sequence [uj( )] of control input vectors is found such
that the corresponding sequence of trajectories converges (uniformly) to .
Several authors have suggested constructive methods for periodic controllers in this context, and further details may be
found in (37,3944).
STABILIZATION BY OSCILLATORY CONTROLS:
METHODS AND SOLUTIONS
Applications of Classical Averaging Theory
The goal of the open-loop oscillatory control is usually to stabilize an unstable equilibrium xs of Eq. (2). This is performed
154

Equilibrium and
stability analysis
Transform
and average
Open-loop
oscillatory input
Unstable plant
(autonomous)
Periodic, timevarying plant
Stable average
plant
Connection between systems

valid if small parameter
sufficiently small.
Stable periodic
time-varying plant
Figure 1. Flow graph of typical open loop

oscillatory control design procedure.
Controlled system
by selecting the (t) in Eq. (3) to be periodic zero average signals, such as sinusoidal inputs or zero average square waves.
The frequency of the input is selected to be large, or equivalently, as Fig. 1 shows, the period is small. The periodic system, in the form of Eq. (3) can then be transformed into the
form of Eq. (1.1) in Appendix 1, where turns out to be proportional to the period. At this point, the transformed system
can be averaged. If the averaged system has a uniformly
asymptotically stable equilibrium point, then this implies
that there will be a uniformly asymptotically stable periodic
orbit of the transformed time-varying system in the vicinity
of the equilibrium point. The final criteria for vibrational stabilization is that the periodic orbit satisfying Eq. (3) remain
in the vicinity of xs (even though a transformation is used
prior to averaging). This is the reason for introducing the
definition of x*, which is the average value of the periodic
solution of Eq. (3).
What follows is a step-by-step procedure for the analysis
of open-loop oscillatory control laws by the classical method
of averaging. A brief introduction to the topic of averaging
and references to more comprehensive accountings may be
found in Appendix A.1. Many summaries of this procedure
detailed in this section can also be found in the literature
(e.g. see Refs. 13,24,25,37). The following discussion is based
on (37).
Assume that f in Eq. (3) has a special structure so that Eq.
(3) can be rewritten as
dx
= f 1 (x(t)) + f 2 (x(t), (t))
dt
(6)
where f 1(x(t)) f 1(0, x(t)) and the function f 2(x(t), (t)) is linear with respect to its second argument. Additionally, assume
that (t) is periodic of period T (0 T 1) and of the form
(t) u
(t), where 2/T, and u ( ) is some fixed period2 function. Since the primary interest is high frequency forcing, the usual implication is that the amplitude of (t) is
large. It is possible, however, that u
( ) has small amplitude,
making the amplitude of (t) small also.
Then Eq. (6) can be rewritten as
dx
= f 1 (x(t)) + f 2 (x(t), u(t))
dt
(7)
In order to proceed with the stability analysis, Eq. (7) will be

transformed to an ordinary differential equation in standard
form Eq. (1.1) in Appendix 1. To make this desired transformation, consider the so called generating equation given as
dx
= f 2 (x(t), u(t))
dt
Suppose that this generating equation has a period T general
solution h(t, c), for some u
( ) and t t0, where h : n
n
n
and c is uniquely defined for every initial condition
x(t0) n.
Introduce into Eq. (7) the Lyapunov substitution x(t)
h(t, z(t)) to obtain

1
h(t, z(t))
dz
=
f 1 (h(t, z(t))
dt
z
(8)
If time is rescaled by letting t, 1/, then using the

standard abuse of notation of letting znew() zold( /), Eq. (8)
becomes

1
h( , z( ))
dz
=
f 1 (h( , z( ))
d
z
(9)
Equation (9) is a periodic differential equation in standard

form with normalized period T 2 and averaging can be
applied. The averaged equation (autonomous) corresponding
to Eq. (9) is given as
dy
= P(y( ));
d
P(c) =
1
2
2
0
h( , c)
c
1
f 1 (h( , c)) d
(10)
It is now possible to convert the averaged equation back to

fast time to obtain
dy
= P( y(t))
dt
(11)
By the theory of averaging, there exists an 0 0 such that

for 0 0, the hyperbolic stability properties of Eqs. (9)
and (10) are the same. This also implies that for sufficiently
large, the hyperbolic stability properties of Eqs. (8) and (11)
are also the same. Specifically, if ys is an asymptotically stable
equilibrium point of Eq. (11) (it will also be an asymptotically
stable equilibrium point of Eq. (10)), this implies that, for
sufficiently large, there exists a unique T-periodic solution,
*(t) satisfying Eq. (8), in the vicinity of ys that is asymptotically stable also. Furthermore, T is known to be equal to 2/
. Since the transform x(t) h(t, z(t)) is a homeomorphism,
there will exist an asymptotically stable T-periodic solution to
Eq. (7) given by x*(t) h(t, *(t)). Equation (2) is said to be
T
vibrationally stabilized provided that x* 1/T 0 x*(t)dt remains in the vicinity of xs.
Example 1. Oscillatory stabilization of scalar differential
equations. Consider the scalar linear differential equation
x(n) + (a1 + u1 (t)) x(n1) + . . . + (an + un (t)) x = 0
(12)
In (45), the problem of stabilizing Eq. (12) is studied using

zero average periodic control inputs in the form
ui (t) = ki sin(t + i )
i = 1, 2, . . ., n
(13)
where ki are constants. Furthermore, the results determined

in (45) show that the impact of the control u1 for stabilization
is nonexistent. Hence, assume that k1 0.
This system can easily be rewritten in state space form of
m
q Aq i1 ui(t)Biq. However, due to the results determined
in (45) there is no need for this. For sufficiently large the
hyperbolic stability properties of xs 0 in Eq. (12) are the
same as the hyperbolic stability properties of the equilibrium
point ys 0 of the corresponding differential equation with
constant coefficients given by
y(n) + (a1 + 1 )y(n1) + . . . + (an + n )y = 0
(14)
where
i =
k2 ki
cos(2 i )
2
i = 1, 2, . . ., n
The impact of the above result is that it presents a calculation

formula for the system. Without knowledge of any periodic
transformations or mathematical analysis, it is possible to select the gain and phase of each oscillatory control to stabilize
the zero equilibrium of Eq. (12) based on the stability properties of Eq. (14), for sufficiently large . Since all the coefficients in Eq. (14) are known, the analysis becomes simple.
Some important comments on Eq. (14) need to be made.
First, notice that since 1 0, this implies that the coefficient
of the n 1th derivative in Eq. (14) cannot be changed. This
coefficient is equal to the negative of the sum of all system
eigenvalues ( trace[A]). Hence, for vibrational stabilization
to take place, it must be that a1 0. Reference 45 shows this
to be a necessary and sufficient condition for scalar differenm
tial equations. (In fact, for all systems q Aq i1 ui(t)Biq
with ui(t) zero average, the trace[A] must always be less than
zero for vibrational stabilization to be possible.) This trace
condition is never satisfied for linearized versions of the mechanical systems treated in the following section, indicating
one direction in which the theory has been considerably extended in recent years. Next, note that 2 is always positive,
and therefore, the coefficient of the n 2th derivative in Eq.
(14) can only be increased. The quantitites i, i 3 can be
made either positive or negative; however, they depend on
k2. Therefore, oscillatory control must enter through the a2
coefficient or else all i will be zero and vibrational stabilization will not take place.
155
Example 2. Oscillatory stabilization of a second-order LTI

system in state space. This example is a slight modification
of the problem discussed by (46). Consider the second-order
system

x =

1.3
0
+
1.6
0
0.6
0.8

1
u(t) x
0
(15)
where u is the scalar control. It is easy to verify that when

u 0 the equilibrium point xs 0 is unstable (check the
eigenvalues of the system).
Suppose u(t) cos(t). Then

0.6
dx
=
dt
0.8

1.3
0
x+
1.6
0

cos(t)
x
0
(16)
which is in the form of Eq. (7). The generating equation is

therefore

x =

cos(t)
x
0
0
0
which has solution x2 c2 and x1 c1 sin(t)c2.

Now introduce the substitutions x2 z2 and x1 z1
sin(t)z2 into Eq. (16) and convert time to t with 1/
to obtain

1
dz
=
d
0
sin( )
1

0.6
0.8

1.3
1
1.6
0

sin( )
z( ) (17)
1
which is now in a form that averaging can take place. Taking

the average of Eq. (17) and converting back to regular time t
leads to the equation corresponding to Eq. (11) of

0.6
dy
=
dt
0.8

1.3 0.4 2
y(t)
1.6
(18)
The eigenvalues of Eq. (18) have negative real part when

2.5. The equilibrium point at zero remains unchanged. Therefore, for sufficiently large (equivalently sufficiently small
0) and for 2.5 the equilibrium xs 0 of Eq. (15) is
vibrationally stabilized.
Example 3. Oscillatory stabilization of a simple pendulum:
Classical Averaging. Consider a simple pendulum consisting
of a massless but rigid link of length to which a tip of mass
m and inertia I is attached, and let denote the counterclockwise rotation about the vertical hanging configuration. Suppose the hinge point of the pendulum is forced to oscillate
vertically, where the elevation of the hinge above some reference height at time t is given by R(t). An illustration of such
a system is given in Fig. 2. Accounting for Rayleigh damping
b and gravitational forces, the pendulum dynamics can be
written
I + b + mR sin + mg sin = 0
(19)
(t) sin t, where

Suppose R(t) sin t. Then R
() . Writing Eq. (19) as a system of first order equa-
156
the averaged equation, then for sufficiently large forcing frequencies there exists an asymptotically stable periodic orbit
near the inverted equilibrium. A simple linearization of the
averaged equation reveals that the stability condition for the
inverted equilibrium is given by 22 2Ig/m.
Remark 2. Note that in the absence of dissipative forces that

the linearized averaged system will possess eigenvalues either of the form 1,2 where , or of the form 1,2
i, where i . Hence the system is never asymptotically
stable in the absence of damping, and stability results in this
case are weak. The lack of asymptotic stability is a characteristic of Hamiltonian, and more generally, conservative systems. The averaging technique of the next subsection is more
suited to such systems and yields stronger stability results.
R(t)
Figure 2. A simple pendulum whose hinge point undergoes vertical

motion.
tions where x1 and x2 , it is clear that the first order

system can be written in the form of Eq. (7). Following the
steps detailed in the previous section, the generating equation
is found to be
x1 = 0,
x2 =
m
sin t sin x1
I
which has the solution
x1 = c1 = h1 (t, c),
x2 =
m
cos t sin c1 + c2 = h2 (t, c)
I
Introducing the transformation
x1 = z 1 ,
x2 =
m
cos t sin z1 + z2
I
letting t, and letting 1/, Eq. (9) specializes to

m
cos sin z1 + z2
z1 =
I

2
m
mg
sin z1
cos2 cos z1 sin z1
z2 =
I
I

m
mb
b
z cos cos z1 +
+
cos sin z1 z2
I 2
I2
I
Therefore the averaged equations, given by Eq. (11), are
y1 = y2 ,
y2 =
1
2
m
I
2
cos y1 sin y1
mg
b
sin y1 y2
I
I
Notice that the averaging preserves the upper equilibrium,

T
and xs 1/T 0 h(t, ys)dt. Therefore, by the previous discussion, if the inverted equilibrium is asymptotically stable for
Remark 3. Simple nonquantitative experiments demonstrating the stabilization described in this example are not difficult to build and work remarkably well. Such an experiment
is shown in Fig. 3. In this experiment, the rotary motion of a
dc motor is rectified to periodic linear motion by the mechanism shown in the left frame of Fig. 3. Note that by virtue of
the construction, the forcing amplitude is fixed and the forcing frequency can vary. It is observed that when current is
applied to the motor, the inverted equilibrium is unstable until the forcing frequency reaches a critical frequency at which
the inverted equilibrium experiences a bifurcation which renders it stable, as depicted in the right frame of Fig. 3. The
inverted equilibrium is then stable for all higher frequencies.
Remark 4. To this point, the main goal has been to use averaging as a means of studying the local stability properties of
periodically excited systems. Under certain conditions, however, the averaged system gives far more information about
the global structure of the periodically excited system. As essentially a perturbation technique, averaging theorems as
found in (36,47,48) give no clues as to how large the small
parameter can be perturbed off zero before the averaged
dynamics fail to describe the forced dynamics. For sufficiently large, a variety of undesirable nonlinear effects arise,
such as subharmonic resonance and stochasticity, which are
not captured in any way by the simple averaging of the nonautonomous dynamics. Because of the inherent difficulty of
the analysis, theory for the prediction of nonlinear effects in
this range has been slow to emerge. A later section briefly
illustrates some of the features of periodically excited systems
exhibit when is allowed to vary.
Averaging for Mechanical Systems
Recently, interest has emerged in using high frequency oscillatory forcing to control the dynamics of mechanical systems.
The typical applications setting is a controlled Lagrangian
system where only some of the degrees of freedom are directly
controlled:
d L
L
=u
dt q 1
q1
(20)
d L
L
=0
dt q 2
q2
(21)
157
where it is assumed dim q1 m and dim q2 n, and u is an

m-vector of controls. (Systems of this form have been called
super-articulated in the literature, and the reader is referred
to (49) for details and references.) Within this class of models,
it is further assumed that there is enough control authority
to always be able to completely specify any trajectory q1( )
over an interval of interest. When this is the case, q1( ),
q1( ), and q1( ) are viewed collectively as generalized inputs,
and are used to control the dynamics of the configuration variables q2( ). The starting point may thus be taken to be a (generalized) control system [see Fliess (50) for an introduction to
generalized control systems] prescribed by a Lagrangian
L (q, q;
x, v) =
1 T
q M (q, x)q + vT A (q, x)q Va (q; x, v)
2
(22)
If

L (q1 , q 1 ; q2 , q 2 ) =
1
(q T1 , q T2 )
2
M11
T
M12
M12
M22

q 1
V (q1 , q2 )
q 2
is the Lagrangian associated with Eqs. (20) and (21), then

with the identifications q1 x, q1 v, q2 q, M22 M ,
M12 A , and V a(q;x,v) V (x,qV ) VTM11V, the connection
between the Lagrangian dynamics prescribed by Eq. (22) and
Eq. (21) is clear.
To simplify averaging, perform the usual Legendre transform H pq L , where p L /q, and write the resulting
Hamiltonian in terms of the variables q, p; x, v
H (q, p; x, v) =
1
(p A T v)T M 1 (p M T v) + Va
2
(23)
This quantity is not a proper Hamiltonian since in general

H /t 0. It is remarkable that if the (generalized) input
functions x( ), and v( ) x( ) are restricted to be periodic
and the simple average of H over one period is computed, (i)
the resulting quantity H will itself be a proper Hamiltonian,
and (ii) in many cases the dynamics associated with H will
closely approximate the dynamics of the nonautonomous system prescribed by Eq. (23). Recall that the simple average is
the time average over one period of H (q, p; x(t), v(t)) where q
and p are viewed as variables which do not depend on the
time t. The averaged Hamiltonian Eq. (23) can be written
H (q, p)
1 T 1
1
p M p vT AM 1 p + vT AM 1 (M 1 )1 M 1 A T v
2
2
1 T
1
+ v AM 1 A v vT AM 1 (M 1 )1 M 1 A T v + V
2
2
1
1
1
T
= (M p M A v)T (M 1 )1 (M 1 p M 1 A T v)

2
=
averaged kinetic energy

1
1
+ vT AM 1 A T v vT AM 1 (M 1 )1 M 1 A T v + V
2

2
Figure 3. A simple experiment to demonstrate the stabilization of
the inverted equilibrium of the vertically forced pendulum. The picture on the left (a) shows the mechanism which rectifies the rotary
motion of the dc motor into periodic linear motion. The picture on the
right (b) shows the pendulum stabilized in the inverted equilibrium.
averaged potential
(24)
The averaged potential given in Eq. (24) is an energy-like
function of the generalized coordinates q which is abbrevi-
158
ated V A(q). A complete understanding of the relationship between the dynamics of nonautonomous Hamiltonian systems
of Eq. (23) and the appropriate counterparts for averaged
Hamiltonian S of Eq. (24) does not presently exist. There are
very broad classes of such systems, however, for which it is
possible to prove the validity of the following:
q0
Averaging Principle for Periodically Forced Hamiltonian Systems. The dynamics associated with Eq. (23) under periodic
forcing (x(t), v(t)) are locally determined in neighborhoods of
critical points of the averaged potential V A(q) as follows:
If q* is a strict local minimum of V A( ), then provided
the frequency of the periodic forcing (x( ), v( )) is sufficiently high, the system will execute motions confined to
a neighborhood of q*.
If (q, p) (q*, 0) is a hyperbolic fixed point of the corresponding averaged system (i.e., the Hamiltonian system
determined by Eq. (24)), then there is a corresponding
periodic orbit of the forced system such that the asymptotic stability properties of the fixed point (q*, 0) of the
averaged system coincide with the asymptotic stability
properties of the periodic orbit for the forced system.

Averaged potential. This example illustrates the use of the
averaged potential in analyzing the dynamics of a pendulum
whose hinge point is forced to undergo oscillatory linear motion which is not necessarily vertical as in the last example.
Suppose (x, y) gives the coordinates of the horizontal and vertical displacement of the hinge point of a pendulum attached
to a sliding block which is controlled to execute the oscillatory
motion

x(t)
cos
=
sin t
y(t)
sin
where prescribes the direction of the oscillatory motion, and
is the frequency. This system is illustrated in Fig. 4. If, as
in the last example, the pendulum has total mass m and inertia I about its hinge point, the motion under this oscillatory
forcing is described by a second order differential equation
I m2 cos( ) sin t + mg sin = 0
(25)
Figure 4. Periodically forced pendulum where the forcing is directed

along a line of angle with respect to the horizontal.
This equation may be derived from a Lagrangian of the form

1
L (, , v) = I 2 + m cos( )v(t) + mg cos
2
where v(t) cos t. The averaged potential for this system
is
VA ( ) =
This type of averaging for the analysis of periodically forced

mechanical systems has been treated in (7) in the case in
which M and A in Eq. (22) do not depend explicitly on the
variable x. A detailed stability analysis based on Floquet theory appears in (8), but this is restricted to the case in which
local minima of the averaged potential correspond to rest
point of the nonautonomous dynamics. In the more general
case, the motion of systems defined by Eq. (23) are organized
around local minima of the averaged potential which are not
rest points of Eq. (23). The theory is less well developed for
this case, but (33) and (10) analyzes the class of single input
systems and presents detailed results on the correspondence
between averaged system phase portraits and the corresponding Poincare maps of the nonautonomous system defined by
Eq. (23).
(m )2
cos2 ( ) mg cos
4I
When /2, the hinge of the pendulum undergoes the vertical oscillation described in the previous example. The averaged potential has two or four critical points in the interval
[0, 2) depending on whether or not 22 is less than or larger
than 2Ig/m. Clearly the equilibrium is a strict local
minimum of the averaged potential if and only if 22
2Ig/m. According to the theory of averaging presented in (7)
and (8), the pendulum will execute stable motions confined to
a neighborhood of this equilibrium for sufficiently large values of . This also recovers the result obtained in the previous example.
Remark 5. This example illustrates nonclassical behavior in
the case /2. For this case there will be, for sufficiently
large values of , strict local minima of the averaged potential
which are not equilibrium points of the nonautonomous Eq.
(25). Nevertheless, the pendulum will still execute motions
confined to neighborhoods of such local minima. For details
on this type of emergent behavior, see (33) and (10).
Remark 6. The strategy behind the very simple (open loop)
control designs associated with the averaged potential (and
more generally with systems having oscillatory control inputs) is to produce robustly stable emergent behavior which
is related to the critical point structure of the averaged potential. The design method for control laws in this category involves designing the averaged potential functions themselves
by means of appropriately chosen inputs. The guiding theory
for this approach remains very much under development.
Floquet Theory
Another body of theory used in the study of the stability of
equilibriums and periodic orbits of systems controlled by
open-loop oscillatory inputs is Floquet theory. As described in
Appendix 2, the central idea behind the theory is that the
local stability of an equilibrium or periodic orbit may be deter-
mined from the eigenvalues of the monodromy matrix M. The

monodromy matrix represents the growth or decay of solutions of the linearized system, where the linearization is
about that equilibrium or periodic orbit. In general, computing the monodromy matrix is not straightforward. The calculation is relatively easy, however, if the linearization of the
system state equations is piecewise constant in t. For example, suppose that the linearized time-varying system is
159
0.500
x = A(t)x
where A(t) A1 on 0 t t, and A(t) A2 on t t T,
such that A1A2 A2A1. Then the monodromy matrix M can be
obtained by computing the state transition matrix on the
interval [0, T]; that is
(T, 0) = (t , 0)(T, t )

=e
t
0 A 1 dt
T
t
A 2 dt
0.000
0.000
10.000
Figure 5. Regions of stability (darkened) and instability (light) for

the vertically forced simple pendulum. In this figure, m g
I 1.
=M
While in mechanical problems the assumption of piecewise
constant forcing is somewhat nonphysical, such approximations are often useful when the forcing frequency is sufficiently large. Piecewise constant forcing is often not a problem in the analysis electrical and electronic systems, where
such forcing is common.
Floquet theory. In the previous examples, it was shown that
the inverted pendulum may be stabilized with high frequency
vertical oscillation by averaging the time-varying equations
and studying the stability of the inverted equilibrium with
the averaged system. In this example, stabilization is inferred
by linearizing the pendulum dynamics about the inverted
equilibrium and studying the eigenvalues of the monodromy
matrix. To facilitate the calculation, assume that the pendulum is forced by square wave forcing, that is, in Eq. (19)
(t) 2u(t) where u(t) u(t 1/) is a square wave
R
which switches periodically between 1 and 1. Also, assume
that the damping term b 0. This assumption simplifies the
following calculations and more importantly allows us to
show that pendulum stabilization does not require dissipation. Linearizing Eq. (19) about the inverted equilibrium gives
rise to the linear system

0
1
1
1
= 1
2
2
[mg m2 u(t)] 0
I
Because the input u(t) is piecewise constant, the state transition matrix (t, 0) over one period of u(t) may be computed
as follows:
(0, 1/)
= (0, 1/2)(1/2, 1)

1
1

sin 1
sinh 2
cosh 2
cos 1
2
1
2
2
2
2
=
1
2
1
2
1 sin
2 sinh
cos
cosh
2
2
2
2
2
2
1
1
1
cosh
+
sinh
sin
cos
2
2 1
2
2
=
2

1
1
cosh
+ 2 cos
sinh 2
1 sin
2
2
2
2
1
2
1

1

sinh
+
cos
sin 1 cosh 2
2
2
2
1
2
2
2
1
1
1
1
cosh
sinh
sin
cos
2
2
2
2
2
where 12 1/I [mg m2] 0 and 22 1/I [mg

m2] 0. Stability of the fixed point may be determined
from the eigenvalues 1, 2 of (0, 1/), which are the roots of
the characteristic polynomial

2

2
1 sin 1 sinh 2 + 2 cos 1 cosh 2 + 1 = 0
1
2
2
2
2
2
and the usual stability condition is that both eigenvalues
must lie within the unit disk in the complex plane. Given the
form of the characteristic polynomial, an equivalent condition
for stability is trace (0, 1/) 2, which in this case may be
written

2
2

1
1
cosh
+
sin 1 sinh 2 < 2
2 cos
2
2
1
2
2
2
The boundary between regions of stability and instability may
be approximated by expanding the trigonometric functions in
Taylor series around zero and solving a truncated inequality
for in terms of . Note beforehand that in doing so, it has
been implicitly assumed that 1 /2 0 and 2 /2 which implies is large. For the present example, the stability regions
displayed in Fig. 5 have been obtained by numerically calculating 1 and 2 over the indicated ranges of and .
Remark 7. Note that in the absence of damping, (0, 1/) is
an area-preserving map. This fact implies that 1 and 2 are
constrained to either lie on the real axis such that 2 1/ 1,
or lie on the unit disk in the complex plane. In this case, stability results are fairly weak (lack of asymptotic stability), but
are typical of Hamiltonian, and more generally, conservative
systems. In the presence of dissipation, the eigenvalues may
occur as conjugate pairs inside the unit disk, implying asymptotic stability.
Remark 8. Note that results obtained through Floquet theory are strictly local results. In the case where the controlled
system is nonlinear, proofs of stability and instability are
strictly for the equilibrium or periodic orbit, and no information is given about the asymptotic behavior of solutions
nearby. This remark is not surprising for neutrally stable
equilibria, but it is in fact sometimes true for unstable equilibria as well. As an example, the reader is referred to the
parametrically excited pendulum example later in this article,
160
where in Fig. 8(b) it is seen that while the origin is unstable,

orbits which pass arbitrarily close to the origin are in fact
bounded by KAM (for Kolmogorov, Arnold, and Maser) tori
and subharmonic resonance bands. Understanding when
proofs of instability obtained by Floquet theory imply unbounded solutions for such systems is a current topic of research.
PERFORMANCE IMPROVEMENTS AND TRANSIENT BEHAVIOR

Sometimes stability in a system is not in question, but certain
performance specifications are the design constraints. This
section begins by revisiting the particle accelerator. Then a
method to analyze the transient behavior of a system is given
and followed by an example.
Example 6. Suppression of betatron oscillations in cyclic accelerators. Consider the cyclic accelerator subject to oscillations of focusing and defocusing sectors of the magnetic lens
given in Eq. (1). The work of (1) describes an experimental
method of betatron oscillations by alternating gradient focusing. To make the focusing improve, it is desirable that both
the 2(1 n) and the 2n terms increase simultaneously, with
0 n 1. Clearly, though, if n is viewed as the control variable, then it is not possible to both increase and decrease n
and 1 n simultaneously. Instead, the technique of focusing
and defocusing a lens is introduced, which, according to (2) is
modelled
d2x
+ 2 [1 (n + K(, ))] x = 0
d 2
d2z
+ 2 [n + K(, )] z = 0
d 2
(26)
where K is a PAZ function with frequency ; for example,

K(, ) () sin .
Suppose that the oscillatory control input is given by K(,
) sin(), that is, () , where is a constant.
Define x1 x, x2 x, x3 z and x4 z. Then the state space
representation of Eq. (26) becomes
dx1
d
dx2
d
dx3
d
dx4
d
= x2
Analysis Method For Transient Behavior

From the theory presented above, it is possible to approximate the transient behavior of Eqs. (7) and (9), using the
techniques found in Refs. (38) and (25). Assume that Eq. (11)
has an asymptotically stable equilibrium point, zs, in the vicinity of the equilibrium point of Eq. (2), xs, with u
constant 0.
By the method of averaging, for sufficiently large , the
solutions of Eqs. (8) and (11) remain arbitrarily close to
each other provided they have the same initial conditions. That is, y(t) z(t) for all time.
The solution to Eq. (7) is given by x(t) h(t, z(t)),
where z(t) is the solution to Eq. (8). This transformation
is a homeomorphism.
Therefore, the quantities in Eq. (5) can be selected so that
h(t, c) is the solution to the generating equation and P is as
defined in Eq. (11). Now define w h(t, y(t)) where y(t) is
the solution to y(t) P(y(t)) in Eq. (11). For sufficiently large
,
x(t) h(t, y(t)) = w(t)
It is possible to analyze the transient behavior of Eq. (7) in
two different manners:
1. Analyze the transient behavior of y(t) and then examine
the behavior of x(t) through the direct relation x(t)
w(t). This technique predicts many of the fast oscillatory parts of the trajectory x.
2. Analyze a moving average of x(t), given by x(t) where
x(t) H(z(t));
= 2 (1 n) x1 + 2 sin( ) x1
(27)
= x4
= 2 nx3 2 sin( ) x3
which is precisely the form of Eq. (7). Using the previously

discussed techniques, the equation corresponding to Eq. (11)
becomes
2
d y
+ 2 (1 n + 0.5 2 )y = 0
d 2
d 2
+ 2 (n + 0.5 2 ) = 0
d 2
It is seen that the net effect was to increase both the

2(1 n) and the 2n terms, which as a result, improved system performance by suppressing the betatron oscillations in
both the x and z directions (this simplified model did not take
into account the interaction of oscillations along x and z).
Hence, performance has been improved via the introduction
of oscillatory open-loop control, provided that is sufficiently
large. This example demonstrates that the benefits of introducing oscillations can, at times, help system performance
even when stabilization is not an issue.
(28)
H(c)
1
T
h(, c) d
(29)
and T is the period of h(t, ). This is done by approximating x(t) H(y(t)) and once again, analyzing the
transient behavior of y (making this technique more
typical of Eq. (4)). Since the fast dynamics in h are averaged out, this technique introduces some error. On the
other hand, since H does not explicitly depend on t, the
analysis becomes simpler.
In either of the two methods, controlling y(t) in Eq. (11) governs how to control x(t).
Example 7. Transient behavior of the vertically oscillating
pendulum. Referring back to Example 3 and using the same
notation, it is possible to now approximate the trajectories of
, the angular position, and , the pendulums angular velocity. The averaged equations of the system are
y1 = y2
1
y2 =
2
where
1
2
= y2
m
I
mg
b
sin y1 y2
cos y1 sin y1
I
I
The transformations utilized are
x1 = z 1
x2 =
m
cos t sin z1 + z2
I
Therefore, it is possible to obtain estimates on the transient

behavior of the vertically oscillating pendulum using one of
the two following methods:
Method 1. The estimate on is given by y1. The estimate on is given by

m
cos t sin y1 + y2
I
Method 2. The estimate on is given by y1. The estimate on is given by
Note that the approximation obtained by Method 2 is merely

the averaged system. The approximation obtained by Method
1, as well as the transient solution obtained from the original
equation of motion, are compared to the averaged solution in
Fig. 6. It is clear from the figure that the averaged system
accurately captures the averaged behavior of the system as it
stabilizes. The phase plot in the lower left of the figure shows
some discrepancies between the trajectory obtained by
Method 1 with the trajectory of the original system, but in
the lower right figure it is seen that the Method 1 approximation of is actually quite close to the velocity of the original
system. As Fig. 6 shows, Method 1 is more accurate than
Method 2. However, Method 1 utilizes a time-varying output
equation, making it a more complicated technique.
RESONANCE PHENOMENA IN
PERIODICALLY FORCED SYSTEMS
1.5
1.5
Averaged system
Method 1
Averaged system
Original system
0.5
Theta
Theta

m
cos s sin y1 + y2 ds
I
Resonances arise as the natural result of subjecting any system to periodic excitation. The resonances produced vary
H(y)
0
0.5
1
H(y)
0.5
0
0.5
2.6
2.8
3
Theta
3.2
3.4
2.6
2.8
3
Theta
3.2
3.4
1.5
Method 1
Original system
0.5
Theta
Theta
Method 1
Original system
1.5
1
0.5
0
0
0.5
0.5
1
2.6
2.8
161
3
Theta
3.2
3.4
10
15
20
Figure 6. A comparison of the behaviors of the averaged/Method 2 system, the Method 1 system,
and the original system. In these plots, m g I 1, b 0.5, 0.2, and 10.
25
162
which, if 1/2 and /2, is recognizable as a nonlinear

version of Mathieus equation
l(t) = + cos t
k
m
F0cos t
g
q
m
Figure 7. The periodically forced springmass system (left) and the

parametrically excited pendulum (right).
based on the type of system being excited; for example the

simple resonance produced by exciting the spring-mass system illustrated in the left frame of Fig. 7 is different from the
parametric resonance exhibited by the parametrically excited
pendulum illustrated on the right. The difference in the resonant behavior exhibited by the two systems is that the forced
mass-spring system has a resonance at a single frequency,
where the parametrically excited pendulum has resonances
at subharmonics of the natural frequency, similar to those of
Mathieus equation. As a nonlinear system, the parametrically excited pendulum also has resonances which form as the
result of the breaking and tangling separatrix solutions, as
visualized with the period-1 return map or Poincare map.
In designing open-loop oscillatory controls for the type of
stabilization described in this article, a primary objective
should be to choose the forcing frequencies such that undesirable resonances are avoided. The main obstacle to prevent the
control theorist from ensuring this using the methods presented in this article is that (i) averaging techniques do not
generally capture phenomena of periods different than the period over which the system is averaged, and (ii) Floquet theory gives only local stability information for a single equilibrium or periodic orbit. Experience and the application of more
powerful tools in the analysis of such systems has shown that
these effects are often avoided by choosing the forcing frequency to be sufficiently large (10). This observation is very
consistent with the sufficiently small conditions imposed
by averaging theory, although the arguments used by averaging theory are for purely analytical reasons. The literature
concerning the global dynamics of nonlinear systems excited
by periodic inputs is vast, fairly technical, and tends to lie
completely outside the boundaries of conventional control theory. In this section, only broad concepts and observations are
presented in the form of an example, and the reader is referred to such texts as (36,51,52) for extensive overviews of
the field.
Example 8. Qualitative features of parametrically excited
pendulum dynamics. To illustrate the kind of responses which
might arise in subjecting a nonlinear system to periodic excitation, consider the parametrically excited pendulum (PEP)
with no damping, which is often described by the differential
equation
q + (1 + sin t) sin q = 0
(30)
Remark 9. Note that after a rescaling of time t, Eq.

(30) takes the form
q +
+
sin
sin q = 0
2
2
q + ( + sin t)q = 0

Note also that by letting 2, Eq. (30) is merely the equation of the motion of the vertically forced pendulum discussed
in Examples 3, 4, and 5. In this example, however, it is assumed that 1.
The natural tool for visualizing the global dynamics of single-degree-of-freedom periodically forced system is the Poincare map, where Poincare sections are taken at the end of
every forcing period. Features of the phase portrait typically
have equivalent features in the Poincare map. For example,
fixed points and periodic orbits in the phase portrait are associated with periodic points of the Poincare map. The Poincare
map also preserves hyperbolic structures of the phase portrait; for example, normally hyperbolic periodic points of the
Poincare map reflect the normally hyperbolic structure of the
corresponding equilibriums and periodic orbits of the phase
portrait. Poincare maps clearly show the bands of stochasticity, indicating the presence of chaotic behavior and subharmonic resonances, which are periodic orbits periodic of some
rational multiple of the forcing period.
The results of simple simulations of the PEP are shown in
Figs. 8(af). Using averaging techniques described previously, it can be shown that the averaged system is merely
the unperturbed ( 0) system, the phase portrait of which
is shown in Fig. 8(a). In general, in implementing an openloop oscillatory control law, an objective is to choose the forcing parameters so that the Poincare map closely resembles
the averaged phase portrait. In the present examples, there
are three ways to systematically adjust and : (i) fix and
let vary, (ii) fix and let vary, or (iii) let () or
() and adjust the independent variable. Physically, it is often most reasonable to fix the forcing amplitude and control
the frequency, hence for the current example attention is restricted to this case.
The five other plots in Fig. 8 reflect the changes which take
place in the Poincare map when is fixed at 0.5 and increases from 1 to 9. Passing from the upper right frame [(b),
1)] to the frames in the middle row [(c, d), 3, 5] to
the lower frames [(e, f), 7, 9], the general trend is clearly
that subharmonic resonances and separatrix splitting is suppressed as increases. In Fig. 8(b), the origin is unstable,
and this instability would be predicted by a Floquet analysis
of the origin. In Figs. 8(c)(f), the origin is stable, as indicated
by the presence of KAM tori. What varies with the excitation
is the minimum radius at which subharmonic resonance
bands exist. Progressing from Figs. 8(b) to (f), observe that
the inner resonance bands are being pushed out towards the
separatrix as frequency increases. As a consequence, the region in which regular quasiperiodic flow dominates increases.
In addition to pushing subharmonic resonance bands out,
increasing frequency also has the effect of reducing the area
of lobes formed by the intersection of stable and unstable
manifolds of the two periodic points. The significance of this
observation is that as frequency increases, the set of initial
conditions which are transported out of the region between
the separatrices of the averaged system decreases. This is an
important observation, because the averaged phase portrait
1.5
1.5
0.5
0.5
q
0.5
0.5
1.5
1.5
0
q
1.5
1.5
0.5
0.5
0.5
1.5
1.5
2
0
q
1.5
1.5
0.5
0.5
q
0.5
1.5
1.5
2
0
q
0
q
0
q
0.5
(d) Gamma = 0.5, omega = 5
(c) Gamma = 0.5, omega = 3
0.5
(b) Gamma = 0.5, omega = 1
(a) Averaged phase portrait
163
(e) Gamma = 0.5, omega = 7
0
q
(f) Gamma = 0.5, omega = 9
Figure 8. Poincare maps showing separatrix splitting and resonance bands for the parametrically excited pendulum. The phase portrait for the unperturbed simple pendulum is shown in
the upper left frame, with Poincare maps of the forced system to the right and underneath.
Forcing parameters used in the Poincare maps are indicated in each plot. Note that the Poincare
maps of the forced system more closely resemble the averaged phase portrait as the forcing
frequency becomes large.
provides no information whatsoever on the existence of a set

of initial conditions which lies inside the separatrices but are
transported out. In (f), it is unlikely that any subharmonic
resonances exist, and the stable and unstable manifolds of the
periodic points have closed to form a barrier to transport as
indicated by the averaged phase portrait in (a).
Remark 10. The example gives somewhat anecdotal evidence that choosing a sufficiently high forcing frequency
tends to suppress the negative features of periodic excitation.
This has also been found to be the case in the cart and pendulum problem described in Examples 3, 4, and 5 [see (10)], the
vertically forced rotating chain (33), and in an entire class of
164
periodically forced single-degree-of-freedom systems (33).

Current work revolves around extending this understanding
to other classes of single-degree-of-freedom systems and
multi-degree-of-freedom systems.
Remark 11. It has also been observed that dissipation tends
to have beneficial effects beyond guaranteeing the asymptotic
stability of certain equilibriums. Dissipation generally has
the effect of breaking phase space separatrices and imposing
hyperbolicity on systems with elliptic structures. As a result,
elliptic structures, such as KAM tori and resonance bands,
are destroyed and initial conditions which, in the absence of
dissipation, belong to a KAM torus or resonance limit on a
fixed point or periodic orbit. In addition, with sufficient dissipation, intersecting stable and unstable manifolds completely
separate, giving rise to distinct basins of attraction. As with
frequency, the extent to which dissipation helps eliminate undesirable nonlinear effects is largely dependent on its magnitude. In (10), it was seen for the cart and pendulum problem
of Example 4 that there exists a minimum damping coefficient such that for all values less than this minimum value,
manifold intersections and resonances persist. Recent work
has suggested that the same issue arises in multi-degree-offreedom systems.
Remark 12. Unfortunately, there is no known generally applicable rule-of-thumb for deciding what constitutes a sufficiently large forcing frequency. Experiments, simulation, and
preliminary analysis suggest that for many systems, a rule of
thumb might be for the forcing frequency to be an order of
magnitude larger than the largest natural frequency of the
controlled system. This rule of thumb presents a problem for
many-degree-of-freedom systems like rotating chains or infinite dimensional systems like strings and beams, where natural frequencies tend to be very large. In addition, there exist
counterexamples where the averaged phase portrait and
Poincare map resemble each other for small forcing frequencies, but possess completely different features at high frequencies. These topics represent the current focus of much of
the research in this field.
ing theory addresses the relationship between the original,

time-varying Eq. (1.1) and the autonomous averaged system
given by
y = f 0 ( y)
(1.2)
where f 0 : n n is defined
f 0 (y)
1
T
f (s, y) ds
0
and where y is treated as a constant in the integration. For

sufficiently small , solutions of Eq. (1.2) provide good approximations to solutions of Eq. (1.1). Since there are many mathematical tools that can be used to analyze and control the
time-invariant system Eq. (1.2), the problem of determining
the behavior of time-varying periodic system Eq. (1.1) has
been greatly simplified.
Specifically, if it is assumed that x(t0) y(t0) then for sufficiently small , the following statements hold:
On any finite time interval, assuming the same initial
conditions at time t0, the solutions to Eqs. (1.1) and (1.2)
remain close to each other. As becomes smaller, then
the approximation becomes better and tends to zero in
the limit.
If the solution to Eq. (1.2) approaches a uniformly
asymptotically stable equilibrium point, then, under additional mild assumptions, the solutions to Eqs. (1.2) and
(1.1) remain close to each other on infinite time intervals.
As becomes smaller, then the approximation becomes
better and tends to zero in the limit.
If Eq. (1.2) has a uniformly asymptotically stable equilibrium point, then Eq. (1.1) has an asymptotically stable
periodic solution in the vicinity of this equilibrium point.
For a detailed discussion on the theoretical framework of averaging, the reader is referred to (36,47,53,54).
APPENDIX 2. FLOQUET THEORY
Floquet theory is concerned with local stability for systems of
the form
ACKNOWLEDGMENTS
Brad Lehman gratefully acknowledges the support of the National Science Foundation through an NSF President Faculty
Fellowship, grant CMS 9596268. John Baillieul would like to
express gratitude for support from the United States Air
Force Office of Scientific Research under grant F49620-96-10059.
APPENDIX 1. CLASSICAL AVERAGING THEORY
The classical method of averaging was originally developed
for periodic systems of the form
x = f (t, x)
x = A(t) x
(2.1)
where x n, A(t) : nn nn, and A(t T) A(t).

Such systems arise as linearizations of Eq. (2) around an
equilibrium or periodic orbit. As described in many standard
texts on differential equations, the fundamental result upon
which Floquet theory is built is that the fundamental matrix
solution of Eq. (2.1) can be written as the product of a periodic
matrix P(t) and a constant exponential growth or decay; that
is,
(t) = P(t)e Bt
(1.1)
where x n, f : n n, f(t T, ) f(t, ) and 0

1. For simplicity, assume that f has continuous second
partial derivatives in its second argument. Classical averag-
where (t) is a fundamental matrix associated with Eq. (2.1),

P(t) is a n n matrix periodic in t of period T, and eBt is the
matrix exponential of a constant n n matrix B. By the periodicity of A(t), if (t) is a fundamental matrix, so is (t T)
165
and (t T) is linearly dependent on (t). Then there exists

a constant n n matrix M such that
11. S. Weibel, J. Baillieul, and B. Lehman, Equilibria and stability

of an n-pendulum forced by rapid oscillations. In Proc. 36th IEEE
CDC, pp. 11471152, San Diego, 1997.
(t + T ) = (t)M
12. A. Cinar, J. Deng, S. Meerkov, and X. Shu, Vibrational control of

an exothermic reaction in a CSTR: theory and experiments.
AIChE J., 33: 353365, 1987.
where M eBT and B is a constant n n matrix. Without loss

of generality, let t 0 and (0) I where I is the n n
identity matrix. Then
(T ) = IM = M
and therefore M represents the rate of growth or decay of the
solution. Stable solutions decay or at least remain bounded.
Hence, the condition for an equilibrium to be stable is that all
the eigenvalues i of M satisfy i 1. M itself is called the
monodromy matrix, and eigenvalues i of M are called Floquet multipliers. Complex numbers i such that i eiT are
called Floquet exponents.
Floquet theory is a very classical method, and there is a
vast literature describing the basic theory and applications.
Some of the classic texts on the topic include Hale (47), Magnus and Winkler (55), Stoker (56), and Yakubovich and Starzhinskii (57). Floquet theory is most famous for its application
in the study of Hills equation and Mathieus equation, which,
because of their second order structure, bear special relevance
in the stability of periodically forced mechanical systems.
Hills equation is the topic of (55), and various classic applications to (electrical) engineering are given in (56). References
(47) and (57) give comprehensive summaries of the fundamental theory as well as examples.
BIBLIOGRAPHY
1. M. S. Livingston, High Energy Accelerators, New York: WileyInterscience, 1954.
2. S. M. Meerkov, Principle of vibrational control: theory and applications, IEEE Trans. Autom. Control, AC-25: 755762, 1980.
13. B. Lehman, Vibrational control of time delay systems. In J. Wiener and J. K. Hale, (eds.), Ordinary and Delay Equations, Pitman
Research Notes in Mathematical Series 272. Harlow, Essex, England: Longman Scientific & Technical, 1992.
14. B. Lehman, I. Widjaya, and K. Shujaee, Vibrational control of
chemical reactions in a CSTR with delayed recycle, J. Math.
Anal. & Appl., 193: 2859, 1995.
15. J. E. Bailey, Periodic operation of chemical reactors: A review,
Chem. Eng. Com., (1): 111124, 1973.
16. J. Fakhfakh and J. Bentsman, Experiment with vibrational control of a laser illuminated thermochemical system, ASME J. Dyn.
Syst., Meas. Control, 110: 353360, 1990.
17. S. M. Osovets, Dynamic methods of retention and stabilization of
plasma, Soviet Phys. - Usp., 17 (2): September 1974.
18. D. Tilbury, R. Murray, and S. Sastry, Trajectory generation for
the n-trailer problem using goursat normal form, IEEE Trans.
Autom. Control, AC40: 802819, 1995.
19. J. L. Speyer, A second variational theory for optimal periodic processes, IEEE Trans. Autom. Control, AC-29-2: 138148, 1984.
20. J. L. Speyer, Nonoptimality of steady-state cruise for aircraft.
AIAA J., 14 (11): 16041610, 1976.
21. S. Weibel and J. Baillieul, Averaging and energy methods for robust open-loop control of mechanical systems. To appear in Essays on Mathematical Robotics. Berlin: Springer-Verlag, 1998.
22. S. Weibel and J. Baillieul, Oscillatory control of bifurcations in
rotating chains. In Proc. 1997 ACC, Albuquerque, 27132717,
1997.
23. J. Baillieul, S. Dahlgren, and B. Lehman, Nonlinear control designs for systems with bifurcations and applications to stabilization and control of compressors. In Proc. 34th CDC, New Orleans,
30623067, 1995.
3. N. N. Bogoliubov, Perturbation theory in nonlinear mechanics,

Sb. Stroit. Mekh. Akad. Nauk Ukr. SSR, 14: 934, 1950.
24. R. Bellman, J. Bentsman, and S. M. Meerkov, Vibrational control

of nonlinear systems: Vibrational stabilizability, IEEE Trans. Autom. Control, AC-31: 710716, 1986.
4. N. N. Bogoliubov and Y. A. Mitropolsky, Asymptotic Methods in

the Theory of Nonlinear Oscillators, International Monographs on
Advanced Mathematics and Physics. New York: Gordon and
Breach Science Pub., 1961.
25. B. Lehman, J. Bentsman, S. V. Lunel, and E. I. Verriest, Vibrational control of nonlinear time lag systems with bounded delay:
averaging theory, stabilizability, and transient behavior, IEEE
5. P. L. Kapitsa, Dynamic stability of a pendulum with a vibrating

point of suspension, Zh. Ehksp. Teor. Fiz., 21 (5): 588598, 1951.
26. B. Lehman and J. Bentsman, Vibrational control of linear time

lag systems with bounded delay, IEEE Trans. Autom. Control,
AC-37: 15761582, 1992.
6. A. Stephenson, On induced stability, Phil. Mag., (17): 765766,

1909.
7. J. Baillieul, Stable average motions of mechanical systems subject to periodic forcing. In M. J. Enos (ed.), Dynamics and Control
of Mechanical Systems: The Falling Cat and Related Problems:
Fields Institute Communications. 123, Providence, R.I.: American Mathematical Society, 1993.
8. J. Baillieul, Energy methods for the stability of bilinear systems
with oscillatory inputs, Int. J. Robust Nonl. Cont., 285301, July
1995. Special Issue on the Control of Nonlinear Mechanical
Systems.
9. S. Weibel, J. Baillieul, and T. J. Kaper, Small-amplitude periodic
motions of rapidly forced mechanical systems. In Proc. 34th IEEE
CDC, New Orleans, 533539, 1995.
10. S. Weibel, T. J. Kaper, and J. Baillieul, Global dynamics of a
rapidly forced cart and pendulum, Nonl. Dyn., 13: 131170, 1997.
27. J. Bentsman and K.-S. Hong, Vibrational stabilization of nonlinear parabolic systems with neumann boundary conditions, IEEE
28. J. Bentsman and K.-S. Hong, Transient behavior analysis of vibrationally controlled nonlinear parabolic systems with neumann
boundry conditions, IEEE Trans. Autom. Control, AC-38: 1603
1607, 1993.
29. S. Lee, S. Meerkov, and T. Runolfsson, Vibrational feedback control: zero placement capabilities, IEEE Trans. Autom. Control,
AC-32: 604611, 1987.
30. K. Shujaee and B. Lehman, Vibrational feedback control of time
delay systems. In Proc. 34th IEEE CDC, 936941, 1995.
31. K. Shujaee and B. Lehman, Vibrational feedback control of time
delay systems, IEEE Trans. Autom. Control, AC-42: 15291545,
1997.
166
OPERATIONAL AMPLIFIERS
32. L. Trave, A. M. Tarras, and A. Titli, An application of vibrational

control to cancel unstable decentralized fixed modes, IEEE Trans.
Autom. Control, AC30: 9599, 1985.
53. P. Lochak and C. Meunier, Multiphase Averaging for Classical

Systems with Applications to Adiabatic Theorems. In Applied
Mathematical Sciences. vol. 72, New York: Springer-Verlag, 1988.
33. S. Weibel, Applications of Qualitative Methods in the Nonlinear

Control of Superarticulated Mechanical Systems, PhD thesis, Boston University, 1997.
54. J. A. Sanders and F. Verhulst, Averaging Methods in Nonlinear

Dynamical Systems, Applied Mathematical Sciences. vol. 59, Berlin: Springer-Verlag, 1985.
34. J. M. Coron, Global asymptotic stabilization for controllable systems without drift, Math. Control, Signals, Syst., 5: 295312,
1992.
55. W. Magnus and S. Winkler, Hills Equation. Tracts of Mathematics. vol. 20, New York: Interscience Publishers, 1966.
35. R. W. Brockett, Asymptotic stability and feedback stabilization.

In R. W. Brockett, R. S. Millman, and H. J. Sussmann, eds., Differential Geometric Control Theory. Basel: Birkhauser, 1983.
36. J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields. Vol. 42, Applied
Mathematical Sciences. Berlin: Springer-Verlag, 1983.
56. J. J. Stoker, Nonlinear Vibrations in Mechanical and Electrical

Systems. New York: Interscience Publishers, 1950.
57. Y. A. Yakubovich and V. M. Starzhinskii, Linear Differential
Equations with Periodic Coefficients, vols. 1, 2, Jerusalem: Keter
Publishing House Jerusalem, Ltd., 1975.
B. LEHMAN
S. WEIBEL
37. J. Baillieul and B. Lehman, Open-loop control using oscillatory

inputs. In W. S. Levine, ed., The Control Handbook, Boca Raton,
FL: CRC Press and IEEE Press, 1996, pp. 967980.
Northeastern University
J. BAILLIEUL
38. R. Bellman, J. Bentsman, and S. M. Meerkov, Vibrational control

of nonlinear systems: Vibrational controllability and transient behavior, IEEE Trans. Autom. Control, AC-31: 717724, 1986.
Boston University
39. J. Baillieul, Geometric methods for nonlinear optimal control

problems, JOTA, 25 (4): 519548, 1978.
40. J. Baillieul, Multilinear optimal control. In Proc. Conf. Geometry
Control Eng.: NASA-Ames, Brookline, MA: Math. Sci. Press, Summer 1976.
OPEN REGION PROBLEMS. See INTEGRAL EQUATIONS.

OPERATING SYSTEMS, NETWORK. See NETWORK OP-
41. G. W. Haynes and H. Hermes, Nonlinear controllability via lie

theory, SIAM J. Control, 8 (4): 450460, 1970.
OPERATING SYSTEMS, UNIX. See UNIX.
42. N. E. Leonard and P. S. Krishnaprasad, Motion control of driftfree, left invariant systems on lie groups, IEEE Trans. Autom.
Control, 40 (9): 1995.
43. H. J. Sussman and W. Liu, Lie bracket extension and averaging:
The single bracket case. In Z. Li and J. F. Canny (eds.), Nonholonomic Motion Planning. 109147. Boston: Kluwer Academic Publishers, 1993.
44. H. J. Sussman and W. Liu, Limits of highly oscillatory controls
and approximation of general paths by admissible trajectories. In
Proc. 30th IEEE CDC, December 1991.
45. S. M. Meerkov and M. Tsitkin, The effectiveness of the method
of vibrational control for dynamic systems described by a differential equation of order n, Automation and Remote Control (translation of Avtomatika i Telemekhanika), 525529, 1975.
46. R. Bellman, J. Bentsman, and S. M. Meerkov, Stability of fast
periodic systems, IEEE Trans. Autom. Control, AC-30: 289291,
1985.
47. J. K. Hale, Ordinary Differential Equations. Texts and Monographs in Pure and Applied Mathematics, Malabar, FL: Robert
E. Krieger Publishing, 1969.
48. J. A. Sanders, On the fundamental theory of averaging, SIAM J.
Math. Anal., 14: 110, 1983.
49. D. Seto and J. Baillieul, Control problems in superarticulated
mechanical systems, IEEE Trans. Autom. Control, AC-39-12:
24422453, 1994.
50. M. Fliess, Generalized controller canonical forms for linear and
nonlinear dynamics, IEEE Trans. Autom. Control, AC-35-9: 994
1001, 1990.
51. A. J. Lichtenberg and M. A. Lieberman, Regular and Chaotic Dynamics. Volume 38 of Applied Mathematical Sciences. Berlin:
Springer Verlag, 1992.
52. S. Wiggins, Introduction to Applied Nonlinear Dynamical Systems
and Chaos. In Texts in Applied Mathematics. vol. 2, Berlin:
ERATING SYSTEMS.
364
OPTIMAL CONTROL
with initial conditions

y(0) = y0
OPTIMAL CONTROL
Optimal control theory is concerned with the development of
techniques that allow one to control physical phenomena described by dynamical systems in such a manner that a predescribed performance criterion is minimized. The principal
components of an optimal control problem are the mathematical model in the form of a differential equation, a description
of how the control enters into this system, and a criterion
describing the cost.
The start of optimal control theory, as a mathematical discipline, dates back to the mid 1940s. The increasing interest
in and use of methods provided by optimal control theory is
linked to the rise of the importance of mathematical models
in many diverse areas of scienceincluding chemistry, medicine, biology, management, and financeand to ever increasing computing power, which allows the realization of optimal
control strategies for practical systems of increasing difficulty
and complexity. While optimal control theory has its roots in
the classical calculus of variations, its specific nature has necessitated the development of new techniques. In contrast
with general optimization problems, whose constraints are
typically described by algebraic equations, the constraints in
optimal control problems are given by dynamical systems.
The dynamic programming principle, the Pontryagin maximum principle, the HamiltonJacobiBellman equation, the
Riccati equation arising in the linear quadratic regulator
problem, and (more recently) the theory of viscosity solutions
are some of the milestones in the analysis of optimal control
theory.
Analyzing an optimal control problem for a concrete system requires knowledge of the systems-theoretic properties
of the control problem and its linearization (controllability,
stabilizability, etc.). Its solution, in turn, may give significant
additional insight. In some cases, a suboptimal solution that
stabilizes the physical system under consideration may be the
main purpose of formulating an optimal control problem,
while an exact solution is of secondary importance.
In the first section we explain some of the concepts in optimal control theory by means of a classical example. The following sections describe some of the most relevant techniques
in the mathematical theory of optimal control.
Many monographs, emphasizing either theoretical or control engineering aspects, are devoted to optimal control theory. Some of these texts are listed in the bibliography and
reading list.
DESCRIPTIVE EXAMPLE AND BASIC CONCEPTS

Control Problem
We consider the controlled motion of a pendulum described
by
m
d2
y(t) + mg sin y(t) = u(t),
dt 2
t>0
(1)
and
d
y(0) = v0
dt
Here y(t) is the angular displacement, m is the mass, and g is

the gravitational acceleration. Further, u(t) represents the applied force, which will be chosen from a specified class of functions in such a way that the system described by Eq. (1) behaves in a desired way. We refer to y and u as the state and
control variables. Due to the appearance of the sine function,
Eq. (1) constitutes a nonlinear control system. It will be convenient to express Eq. (1) as a first-order system. For this
purpose, we define x(t) col(x1(t), x2(t)), where x1(t) y(t) and
x2(t) (d/dt)y(t). Then we obtain the first-order form of Eq.
(1), which is of dimension n 2:
d
dt
x1 (t)
x2 (t)

=
x2 (t)
g sin x1 (t) + u(t)
(2)
with initial condition x(0) x0 (y0, v0) R2, where we assume m 1. In general, a control system is written in the
form
d
x(t) = f (t, x(t), u(t)),
dt
x(0) = x0
(3)
with state vector x(t) Rn, control input u(t) Rm, and f :
R1 Rn Rm Rn. If f is independent of time t [f f(x, u)],
then the system is said to be autonomous.
Next we formulate a sample control system associated to
Eq. (1). For that purpose, note that the stationary solutions
to the uncontrolled system, which are characterized by f(x,
u) 0 for u 0, are given by (0, 0) and (, 0). Our objective
is to regulate the state x(t) R2 to the stationary state (, 0).
Thus a control u must be determined that steers the system
described by Eq. (1) from the initial state x0 to the vertical
position (y ) or into its neighborhood (inverted-pendulum
problem). This objective can be formulated as an optimal control problem: minimize the cost functional
tf
J(x, u) =
0
[|x1 (t) |2 + |x2 (t)|2 + |u(t)|2 ] dt
(4)
+ [|x1 (t f ) |2 + |x2 (t f )|2 ]

subject to Eq. (2), over u L2(0, tf; R1), the space of squareintegrable functions on (0, tf). The nonnegative constants
and are the weights for the control cost and target constraint at the terminal time tf 0, respectively. The integrand x1(t) 2 x2(t)2 describes the desired performance
of the trajectory [the square of distance of the current state
(x1(t), x2(t)) to the target (, 0)]. The choice of the cost functional J contains a certain freedom. Practical considerations
frequently suggest the use of quadratic functionals.
A general form of optimal control problems is given by
tf
min
0
f 0 (t, x(t), u(t)) dt + g(t f , x(t f ))
(5)
subject to Eq. (3), over u L2(0, tf; Rm) with u(t) U a.e. in
(0, tf), where U is a closed convex set in Rm describing con-
OPTIMAL CONTROL
365
straints that must be observed by the class of admissible controls. In the terminology of the calculus of variations, Eq. (5)
is called a Bolza problem. The special cases with f 0 0 and
with g 0 are referred to as the Lagrange and the Mayer
problem, respectively. If tf 0 is finite, then Eq. (5) is called
a finite-time-horizon problem. In case g 0 and tf , we
refer to Eq. (5) as an infinite-time-horizon problem. The
significance of the latter is related to the stabilization of
Eq. (3).
If Eq. (5) admits a solution u*, we refer to it as the optimal
control, and the associated state x* x(u*) is the optimal
trajectory. Under certain conditions, the optimal control can
be expressed as a function of x*, that is, u*(t) K(t, x*(t)) for
an appropriate choice of K. In this case u* is said to be given
in feedback or closed-loop form.
If the final time tf itself is a free variable and f 0 1, then
Eq. (5) becomes the time optimal control problem.
For certain analytical and numerical considerations, the
treatment of the fully nonlinear problem (5) can be infeasible
or lengthy. In these cases, a linearization of the nonlinear
dynamics around nominal solutions will be utilized.
at x (0, 0) and
Linearization
is asymptotically stable, in the sense that x(t)Rn 0 as t

for all x0 Rn. Recall that a system of the form
We discuss the linearization of the control system (3) around

stationary solutions. Henceforth x stands for a stationary solution of f(x, u) 0, where u is a nominal constant control.
Let A Rnn and B Rnm denote the Jacobians of f at (x,
u); that is,
A = f x (x, u)
and B = f u (x, u)
(6)
Defining z(t) x(t) x and v(t) u(t) u and assuming

that f is twice continuously differentiable, Eq. (3) can be expressed as
|r(z(t), v(t))|R n const. [|z(t)|2 + |v(t)|2 ]

This implies that the residual dynamics r are dominated by
the linear part Az(t) Bv(t) if (z(t), v(t)) is sufficiently small.
We obtain the linearization of the control system (3) around
(x, u):
(7)
where now x(t) and u(t) represent the translated coordinates

x(t) x and u(t) u, respectively. We refer to Eq. (7) as a
linear control system.
For the example of the pendulum we find

A=
0
g
1
0

and B =
0
1
1
0

and B =
0
1
(6b)
at x (, 0). The linearized control system for the inverted

pendulum is given by
d
x(t) =
dt
0
g
1
x(t) +
0
0
u(t)
1
where x1(t) is the relative angle from .

Stability
In this and the following subsection we restrict our attention
to linear control systems of the form (7). One of the main objectives of optimal control is to find controls in state feedback
form u(t) Kx(t) with K Rmn such that the closed-loop
system
d
x(t) = Ax(t) BKx(t) = (A BK)x(t)
dt
(8)
d
x(t) = Ax(t)
dt
is asymptotically stable if and only if all eigenvalues of the
matrix A satisfy Re 0. For example, for the matrices in
Eqs. (6a) and (6b) we have
det(I A) = 2 + g = 0 = { gi}
(marginal stability)
and
(instability)
respectively. In particular, this implies that the uncontrolled

inverted-pendulum problem is unstable in the sense of Liapunov stability theory. For the closed-loop feedback system (8)
associated with the inverted pendulum with feedback matrix
chosen in the form
where the residual dynamics r satisfy
x(0) = x0 x
0
g
det(I A) = 2 g = 0 = { g}
d
z(t) = Az(t) + Bv(t) + r(z(t), v(t))
dt
d
x(t) = Ax(t) + Bu(t),
dt
A=
(6a)
K = (0
) R12
we find that the eigenvalues of A BK are given by (

2 4g) and hence the closed-loop system is asymptotically stable for appropriately chosen 0. Moreover, if we
apply the feedback u(t) Kz(t) with z(t) x(t) (, 0) to
the original system (2), then the closed system is locally asymptotically stable by Liapunov stability theory.
Linear Quadratic Regulator Problem
In order to construct the optimal stabilizing feedback law
u(t) Kx(t) for the linear system (7), we consider the infinite-time-horizon linear quadratic regulator (LQR) problem

min
[xt (t)Qx(t) + ut (t)Ru(t)] dt
(9)
0
366
OPTIMAL CONTROL
subject to Eq. (7), where Q, G Rnn are symmetric nonnegative definite matrices and R Rmm is symmetric and positive
definite. The optimal solution u*( ) to Eq. (9) is given in feedback form by
u (t) = R1 Bt Px (t)
(10)
Existence of Optimal Controls
where the optimal trajectory x*( ) satisfies

d
x (t) = (A BR1 Bt P)x (t),
dt
the initial and terminal times. Also we shall not systematically discuss the bangbang principle, which states that, for
certain control systems with controls constrained to lie in a
convex compact set, the optimal controls are achieved in the
extremal points of the admissible control set.
x (0) = x0
and the symmetric nonnegative definite matrix P Rnn satisfies the matrix Riccati equation
At P + PA PBR1 Bt P + Q = 0
(11)
In the section titled Linear Quadratic Regulator Problem

we shall return to a detailed discussion of this equation and
its significance in optimal control theory.
The problem of the existence of admissible controltrajectory

pairs (x, u) and of solutions to the optimal control problem
(12)(15) has stimulated a significant amount of research.
Here we can only give the flavor of some of the relevant aspects required to guarantee existence of optimal controls.
Let us assume in this subsection that tf is fixed. Then the
optimal control problem can be stated in the form of a nonlinear mathematical programming problem for (x, u) H1(0, tf,
Rn) L2(0, tf, Rm):
min
J(x, u)
(16)
subject to the equality constraints
x(t) f (t, x, u)
E(x, u) = dt
=0
(t f , x(t f ))
EXISTENCE AND NECESSARY OPTIMALITY

In this section we consider the optimal control problems of
Lagrange type

min
tf
J(x, u) =
f 0 (t, x(t), u(t)) dt
(12)
(17)
and
u K = {u(t) U a.e. in (0, t f )}
(18)
subject to the dynamical system

d
x(t) = f (t, x(t), u(t))
dt
(13)
control constraints
u(t) U
(a closed set in Rm )
(14)
and initial and target constraints

x(0) = x0
and (t f , x(t f )) = 0
(15)
over u L (0, tf; R ), where f : R R R R , f : R

Rn Rm R, and : R Rn Rp are C1 functions. In contrast with Eq. (5), we generalize the control problem in that
we restrict the trajectory to reach a target described by the
manifold (tf, x(tf)) 0. If, for example, tf 0 is free, f 0(t, x,
u) 1, and (t, x) x, then the objective is to bring the system to rest in minimum time. Typical forms of the control
constraint set U are given by U u Rm : u and U
u Rm : ui 0, 1 i m.
In order to obtain a first insight into the problem of (12)
(15), one needs to address the questions of (a) the existence of
admissible candidates (x, u) satisfying Eqs. (13)(15), (b) the
existence and uniqueness of solutions to the optimal control
problem (12)(15), and (c) necessary optimality conditions.
In the remainder of this section we shall present some of
the ideas that were developed to answer these questions. For
detailed information we refer to the bibliography and reading
list and to additional references given in the listed works.
In spite of their importance, in practice we shall not consider problems with constraints on the trajectories except at
2
with K a closed convex subset of L2(0, tf; Rm) and E considered

as a mapping from H1(0, tf; Rn) L2(0, tf; Rn) L2(0, tf; Rn)
Rp. Question (a) above is equivalent to the existence of feasible points satisfying the equality and control constraints (17)
and (18). The existence of optimal controls can be argued as
follows: Under an appropriate assumption the state function
x x( , u) L2(0, tf; Rn) can be defined as the unique solution
to Eq. (13) with initial condition x(0) x0, so that the control
problem (12)(15) can be written as
min J(x(u), u) over u K
with (t f , x(u)(t f )) = 0
(19)
Suppose that the admissible control set K is compact, that

is, every bounded sequence in K has a strongly convergent
subsequence in K, and the solution map u L2(0, tf; Rm)
(x(u), x(u)(tf)) L2(0, tf, Rn) Rn is strongly continuous. Moreover, assume that the functional J is lower semicontinuous,
that is,
J(lim xn , lim un ) lim inf J(xn , un )
for all strongly convergent sequences (xn, un) in L2(0, tf, Rn)
L2(0, tf, Rm). Then the control problem (12)(15) has a solution. In fact, let inf J(x(u), u) over u K with (tf,
x(u)(tf)) 0, and let un be a minimizing sequence, that is,
J(x(u1), u1) J(x(u2), u2) , with limn J(x(un), un)
and the constraints in Eq. (19) are satisfied. Due to the compactness assumption for K, there exists a subsequence unk
of un such that unk u* for some u* K. The continuity
assumption for the control to solution mapping implies that
OPTIMAL CONTROL
(tf, x(u*)(tf)) 0, and from the semicontinuity of J it follows

that
J(x(u ), u ) = J(lim x(un k ), lim un k )
lim inf J(x(un k ), un k ) =
This implies that J(x(u*), u*) and u* is a solution to
Eqs. (12)(15).
Alternatively to the compactness assumption for K, we
may assume that either limu J(x(u), u) , or that K is
bounded. We then also require that the solution map u
L2(0, tf; Rm) (x(u), x(u)(tf)) L2(0, tf; Rn) Rn be continuous
when L2(0, tf; Rm) is endowed with the weak topology and that
the functional J is weakly sequentially lower semicontinuous.
Then, using arguments similar to the ones above, the existence of a solution to Eqs. (12)(15) again follows.
367
the fact that it is in general not a sufficient optimality

condition, that is, (x*, u*, *) can be an extremal element without (x*, u*) being a solution to the control
problem in (12)(15).
2. We refer to the literature for the proof of the maximum
principle. A proof is sketched in the next subsection. We
also mention that the following fact plays an essential
role. Let s [0, tf], and consider the problem (12)(15)
with initial time 0 replaced by s and initial condition
x(s) x*(s). Then the optimal statecontrol pair restricted to [s, tf] is optimal for the control problem starting at s with x(s) x*(s).
3. Suppose tf is fixed, that is, 0(t) t tf and that the
target constraint is described by p additional conditions
i(x) 0, i 1, . . ., p. Then the transversality condition can be expressed as
f )), (t f ))
(H(t f , x (t f ), u (t f ), (t
Pontryagin Maximum Principle

An important step toward practical realization of optimal control problems is the derivation of systems of equations that
must be satisfied by the optimal controls and optimal trajectories. The maximum principle provides such a set of equations.
It gives a set of necessary optimality conditions for the optimal control problem (12)(15).
We shall require the Hamiltonian associated with Eqs.
(12)(15) given by
= 0 f 0 (t, x, u) + f (t, x, u)
H(t, x, u, )
(20)
where (0, ) R Rn.

Theorem 1. Assume that f 0, f, are sufficiently smooth, and
suppose that (x*, u*) minimizes the cost functional in Eq. (12)
subject to Eqs. (13)(15). Then there exists (t) (0, (t))
Rn1 with 0 0 such that (t) never vanishes on [0, tf], and
1. Maximum condition:
H(t, x (t), u (t), (t))

H(t, x (t), u, (t))
for all u U
2. Adjoint equation:
d
(t))
(t) = Hx (t, x (t), u (t),
dt
= 0 (1, 0, . . ., 0) + (0, x (x (t f ))
for some (0, ) R Rp. Here we set x(x*(tf))
p
i1 i grad i(x*(tf)).
4. If one can ascertain that 0 0 (normality), then without loss of generality we can set 0 1, and conditions
23 of Theorem 1 can be equivalently expressed as
d
(t) = Hx (t, x , u , )
dt
(t f ) = x (x (t f ))
(21)
If tf is fixed and no other target constraints are given,

then normality holds. In fact, from the adjoint equation
and the transversality condition we have
d
(t) = 0 f x0 (t, x (t), u ) (t) f x (t, x (t), u (t))
dt
with (tf) 0. If 0 were 0, then (t) 0 on [0, tf], which
gives a contradiction.
5. The maximum principle is based on first-order information of the Hamilton H. Additional assumptions involving, for example, convexity conditions or second-order
information are required to ascertain that a pair (x*,
u*) satisfying conditions 13 of Theorem 1 is, in fact, a
solution to the problems in Eqs. (16)(18). Some sufficient optimality conditions are discussed in the next
two subsections.
3. Transversality:
f )), (t f )) T f
(H(t f , x (t f ), u (t f ), (t
where Ttf is the tangent space to the manifold described by
(t, x) 0 at (tf, x*(tf)).
An admissible triple (x*, u*, *) that satisfies the conclusions of Theorem 1 is called an extremal element. The function x* is called the extremal trajectory, and u* is called the
extremal control.
Remarks
1. The maximum principle provides a necessary condition
for optimality. It is simple to find examples illustrating
Example. We conclude this subsection with an example. Let

us denote by x1, x2, x3 the rates of production, reinvestment,
and consumption of a production process. Dynamical constraints are given by
d
x (t) = x2 (t),
dt 1
x1 (0) = c > 0
and it is assumed that x1 x2 x3 and xi 0 for i 1, 2, 3.

The control function is related to the state variables and is
chosen as u(t) x2(t)/x1(t). The objective consists in maximizing the total amount of consumption given by
=
0
x3 (t) dt
368
OPTIMAL CONTROL
on the fixed operating period [0, T] with T 1. Setting x

x1, this problem can be formulated as a Lagrange problem:

min
J=
[u(t) 1]x(t) dt
(22)
the problem (22), it follows that the optimal control is given

by

1 on [0, T 1]
u (t) =
0 on (T 1, T]
Lagrange Multiplier Rule
subject to
d
x(t) = u(t)x(t),
dt
x(0) = c and u(t) U = [0, 1]
To apply the maximum principle, note first that x(t) 0

on [0, T] for all admissible control u. The Hamiltonian H is
given by
Here we present a necessary optimality condition based on

the Lagrange multiplier rule and establish the relationship to
the maximum principle. As in the section titled Existence of
Optimal Controls, it is assumed that tf is fixed. We recall the
definition of E in Eq. (19) and define the Lagrange functional
L : H1(0, tf, Rn) L2(0, tf, Rm) L2(0, tf, Rn) Rp R given by
L(x, u, , ) = J(x, u) + ((, ), E(x, u))L 2 (0,t
H = 0 (u 1)x + ux
d
= Hx = 0 (u 1) u
dt
and the transversality condition implies that (T) 0. Since
(0, (t)) 0 on [0, T], it follows that 0 0. Thus normality
of the extremals holds, and we set 0 1. The maximum
condition implies that
[1 u (t)]x (t) + (t)x (t)u (t) (1 u)x (t) + (t)x (t)u
for all u [0, 1]
Since necessarily x*(t) 0, the sign of (t) 1 determines
u*(t), that is,
u (t) =

1 if (t) 1 > 0
d
= (1 )u 1,
dt
Theorem 2. Assume that (x(u*), u*) minimizes the cost functional in Eq. (16) subject to Eqs. (17) and (18) and that the
regular point condition
0 int{E
(x(u ), u )(h, v u ) : h HL1 (0, t f , Rn ) and v K}
(23)
holds. Then there exists a Lagrange multiplier (, ) L2(0,
tf, Rn) Rp such that
Lx (h) = Jx (x , u )h + ((, ), Ex (x , u ))h = 0

for all h HL1 (0, t f , Rn )
(Lu , u u ) 0
We can now derive the explicit expression for the extremal

elements. Since is continuous, there exists a 0 such
that (t) 1 on [, T]. On [, T] we have u*(t) 0. It follows
that (t) T t on [, T], and hence reaches 1 at T
1. Since (d /dt)() 1 and (d /dt)() 0, there exists an
such that (d/dt) 0 on [, ]. This implies that (t)
1 and thus u*(t) 1 on [, ], and consequently
tf
0
x (t) = ce ,
[ f x0 (t, x , u ) f x (t, x , u )]h +
tf
t
ft
and (t) = e
(t )
We have thus derived the form of the only extremal on [0,

T]. Since one can easily argue the existence of a solution to
d
h dt
dt
+ x (x (t f ))h(t f ) = 0
[ f x0 (s, x , u ) f x (s, x , u )] ds
on [, ]
for some 0. Now we can argue that is necessarily 0 and

that on [0, ]
t
for all h HL1 (0, tf; Rn). An integration-by-parts argument implies that
for all u K
Let us establish the relationship between the Lagrange

multiplier (, ) and the adjoint variable of the maximum
principle. From the first line in Eq. (24) one deduces
(T ) = 0
(t) = e(t ) and x (t) = et
(24)
where the partial derivatives Lx and Lu are evaluated at (x*,

u*, , ).
0 if (t) 1 0
The adjoint equation is therefore given by
u (t) = 1,
,R n )R p
Further define HL1 (0, tf, Rn) as the set of functions in H1(0, tf,
Rn) that vanish at t 0.
We have the Lagrange multiplier rule:
the adjoint equation is
+ x (x (t f )) +
d
h(t) dt = 0
dt
and thus
tf
(t) =
t
[ f x0 (s, x , u ) + f x (s, x , u )] ds x (x (t f ))
a.e. in (0, tf). If f and f 0 are sufficiently regular, then

H1(0, tf, Rn) and (, ) satisfy Eq. (21).
OPTIMAL CONTROL
For certain applications, a Hilbert space framework may

be too restrictive. For example, f(u) sin u2 is well defined
but not differentiable on L2(0, tf; R). In such cases, it can be
more appropriate to define the Lagrangian L on W1,(0, tf;
Rn) L(0, tf; Rn) L(0, tf; Rn) Rp.
Let us briefly turn to sufficient optimality conditions of second order. To simplify the presentation, we consider the case
of minimizing J(x, u) subject to the dynamical system (13)
and initial and target constraints [Eq. (15)], but without constraints on the controls. If (x*, u*) satisfy the maximum principle and f 0, f are sufficiently regular, then
369
pose we introduce additional scalar components for the dynamical system and for the target constraint by
d
x
= 0 and p+1 (x) = g(x) txn+1
dt n+1
where x (x1, . . ., xn) as before, and the augmented cost
functional is
f0 = f 0 (t, x, u) + xn+1
We find
Huu (t) 0
for t [0, t f ]
where H(t) H(t, x*(t), u*(t), (t)). A basic assumption for

second-order sufficient optimality is given by the Legendre
Clebsch condition
Huu (t) < 0
for t [0, t f ]
(25)
This condition, however, is not sufficient for u* to be a local

minimizer for the control problem in Eqs. (12)(15). Sufficient
conditions involve positivity of the Hessian of the Lagrange
functional L at (x*, u*, , ) with (, ) as in Theorem 2. This
condition, in turn, is implied by the existence of a symmetric
solution Q to the following matrix Riccati equation:
Q = Q f x (t) f x (t)T Q + Hxx (t)

[Q f u (t) Hxu (t)]Huu (t)1 [ f uT (t)Q Hxu (t)]
Q(t ) = (x(t ))
on ker x (x(t f ))
xx
f
f
(26)
tf
f0 (t, x(t), u(t)) dt =
J(x, u) J(x , u ) + c|(x, u) (x , u
)|2L 2 (0,t ;R n+m )

f
for all (x, u) satisfying Eq. (13), Eq. (15), and (x, u) (x*,
u*)L(0,tf ;Rnm) .
The fact that perturbations (x, u) are only allowed in L so

as to obtain an L2 bound on variations of the cost functional
is referred to as the two-norm discrepancy.
tf
0
tf
=
0
f 0 (t, x(t), u(t)) dt + t f xn+1

f 0 (t, x(t), u(t)) dt + g(x(t f ))
For the augmented system, the initial conditions are x(0)

x0, while xn1(0) is free. The maximum principle can be generalized to allow for trajectories that are constrained to lie on
an initial manifold M0 Rn1. In this case a transversality
condition at t 0 must be added to part 3 of Theorem 1:
(H(0, x (0), u (0), (0)),

(0)) T0
where T0 is the tangent space to M0 at (0, x*(0)). For the Bolza
problem the initial manifold is characterized by t 0 and
x x0 0, and thus the transversality condition at t 0
implies n1(0) 0. The adjoint condition 2 of Theorem 1
turns out to be
d
(t))
(t) = Hx (t, x (t), u (t),
dt
where f(t) f(t, x*(t), u*(t)). We have the following result:

Theorem 3. Let (x*, u*) denote a pair satisfying Eqs. (13)
and (15), and assume the existence of a Lagrange multiplier
(, ) in the sense of Theorem 2 with U Rm. If, further, f
and f 0 are sufficiently regular, Eq. (25) holds, and Eq. (26)
admits a symmetric C1-solution, then u* is a local solution of
Eq. (12). Moreover, there exist c 0 and 0 such that
d
(t) = 0 ,
dt n+1
n+1 (0) = 0
where the Hamiltonian H is defined by Eq. (20), and the

transversality condition 3 of Theorem 1 is given by
(H(t f ), (t f )) + 0 (0, gx (x(t f ))) Tt
If we assume that tf fixed and that no target constraints at tf

are present, then normality holds and conditions 23 of Theorem 1 can be expressed as
d
)
(t) = Hx (x u ,
dt
(t f ) = gx (x (t f ))
For a restricted class of Bolza problems, the maximum

principle provides a sufficient optimality condition. We have
the following result:
Bolza Problem
Here we discuss the maximum principle for a Bolza type problem, where the cost functional of Eq. (12) is replaced by
min
Theorem 4. Consider the Bolza problem

tf
min
tf
J(x, u) =
0
f 0 (t, x(t), u(t)) dt + g(x(t f ))
with g : Rn R. Augmenting the system (13), the Bolza problem can be expressed as a Lagrange problem. For this pur-
(27)
[ f 0 (t, x(t)) + h0 (t, u(t))] dt + g(x(t f ))
subject to
d
x(t) = Ax(t) + h(t, u(t)),
dt
x(t) = x0
370
OPTIMAL CONTROL
and the control constraint u(t) U, where tf is fixed and g

and f 0 are C1 functions that are convex in x. If (0 1, (t),
x*(t), u*(t)) is extremal, then u* is optimal.
Proof. From the maximum condition
h (t, u (t)) + (t)h(t, u (t)) h (t, u) + (t)h(t, u)

0
for all u U
(28)
where (t) satisfies

d
(t) = f x0 (t, x (t)) (t)A,
dt
x(0) = x0
u (t) = R1 Bt t (t)
Combined with Eq. (28), this implies

d
d
(x ) f x0 (, x )x h0 (, u )
(x) f x0 (, x )x h0 (, u)
dt
dt
Integration of this inequality on [0, tf] yields
[ f x0 (t, x )x + h0 (t, u )] dt
(t f )x(t f )
tf
0
[ f x0 (t, x )x
(t f ) = xt (t f )G
and the maximum condition implies that
d
d
d
(x) =
x + x = ( f x0 A)x + (Ax + h)
dt
dt
dt
= f x0 (, x )x + h(, u)
tf
(30)
where f(t) L2(0, tf; Rm) represents a disturbance or an external force, Q, G Rnn are symmetric and nonnegative matrices, and R Rmm is symmetric and positive definite. This
problem is referred to as the finite-time-horizon linear quadratic regulator problem. The Hamiltonian H is given by
d
(t) = (t)A + xt (t)Q,
dt
we have
x(0) = x0
where we have used the fact that 0 1 established in the

last subsection. From Eq. (27) we obtain the form of the adjoint equation for (t):
For all admissible pairs (x, u) satisfying
(t f )x (t f )
d
x(t) = Ax(t) + Bu(t) + f (t),
dt
H = 12 (xt Qx + ut Ru) + (Ax + Bu + f (t))
(t f ) = gx (x (t f ))
d
x(t) = Ax + h(t, u),
dt
subject to the linear control system
Thus the maximum principle reduces to a two-point boundary

value problem. If we define p , then the optimal triple
(x, u*, p) is characterized by u*(t) R1Btp(t) and
d
x(t) = Ax(t) BR1 Bt p(t) + f (t),
x(0) = x0
dt
d
p(t) = At p(t) Qx(t),
p(t f ) = Gx(t f )
dt
(31)
In the section titled Linear Quadratic Regulator Theory and

Riccati Equations we discuss the solution to Eq. (31) in terms
of matrix Riccati equations. There we shall also consider the
infinite-time-horizon problem with tf .
+ h (t, u)] dt
0
By Eq. (27), the last inequality implies
gx (x (t f ))[x(t f ) x (t f )] +
tf
0
tf
Time Optimal Control
f x0 (t, x )(x
x ) dt
[h0 (t, u ) h0 (t, u)] dt
Note that (x) (x*) x(x*)(x x*) for all x, x* for any
convex and C1 function . Since g, f 0 are convex, we have

g(x(t f )) +
tf
[ f 0 (t, x) + h0 (t, u)] dt
g(x (t f )) +
tf
min
[ f 0 (t, x ) + h0 (t, u )] dt
tf =
tf
1 dt
(32)
d
x(t) = Ax(t) + Bu(t)
dt
which implies that u* is optimal.

LINEAR QUADRATIC REGULATOR PROBLEM
x(0) = x0
(33)
and control as well as target constraints
We consider the special optimal control problem
J(x0 , u)
t f

1
[xt (t)Qx(t) + ut (t)Ru(t)] dt + xt (t f )Gx(t f )
=
2
0
A time optimal control problem consists of choosing a control

in such a way that a dynamical system reaches a target manifold in minimal time. Without control constraints, such problems may not have a solution. In the presence of control constraints, the optimal control will typically be of bangbang
type. The following class of examples illustrates this behavior.
We consider the time optimal control problem
ui [1, 1] for 1 i m,
min
(29)
and x(t f ) = 0
Assume that (A, B) is controllable, that is, for every x0 Rn

and every target x1 at tf there exists a control u L2(0, tf,
Rm) that steers the system (33) from x0 to x1. We recall that,
OPTIMAL CONTROL
for the linear autonomous control system (33), controllability

is equivalent to the requirement that the Kalman rank (B,
AB, . . ., ABn1) n. A sufficient condition for the existence
of an optimal control to Eq. (32) for arbitrary initial conditions x0 in Rn is that (A, B) is controllable and that A is
strongly stable (the real parts of all eigenvalues of A are
strictly negative).
The Hamiltonian for Eq. (32) is
H = 0 1 + (Ax + Bu)
and the adjoint equation is given by
that u* 1. Then the equations (d/dt)x1(t) x2(t), (d/dt)x2(t)

1 have solutions of the form
x2 (t) = t + c1
(t)Bu (t) (t)Bu
and x1 (t) =
(t + c1 )2
+ c2
2
Thus, the orbit is on the manifold x1 x22 /2 c2 oriented upwards. Similarly, for u* 1 the orbit is on the manifold
x1 x22 /2 c2 oriented downwards. Since the optimal controls have at most one switch and the orbits must terminate
at (0, 0), it follows that the optimal control u* is given in
feedback form by
d
= A
dt
u (t) = U (x1 (t), x2 (t)) =
The transversality condition implies that H(tf) 0 and hence

(t) eA(tft) for some R1n. As a consequence of the maximum condition, we find
371
1 if (x1 , x2 ) is above S
1 if (x1 , x2 ) is below S
where S is the switching curve consisting of x1 x22 /2 (x1

0) and x1 x22 /2 (x1 0). The feedback law is in general robust, since it possesses the self-correcting property.
for all u [1, 1]n

DYNAMIC PROGRAMMING PRINCIPLE AND
HAMILTONJACOBIBELLMAN EQUATION
and hence
ui (t) = sign gi (t), for 1 i m
where g(t) : (t)B eA(tft)B. We claim that g(t) is nontrivial.
In fact, if g(t) 0 for some t [0, tf], then, since (A, B) is
controllable, 0 and (t) 0. We have
H(t f ) = 0 + (t f )[Ax(t f ) + Bu(t f )] = 0 = 0
and thus (0, (t)) is trivial if g(t) 0. This gives a contradiction to Theorem 1.
In the remainder of this subsection we consider a special
linear control system (rocket sled problem) and provide its
solution. Let y(t), the displacement of a sled with mass 1 on a
friction-free surface be controlled by an applied force u(t) with
constraint u(t) 1. By Newtons second law of motion,
(d2 /dt2)y(t) u(t). If we define x1(t) y(t) and x2(t)
(d/dt)y(t), then the state x(t) col(x1(t), x2(t)) satisfies Eq. (33)
with
A=
0
0
1
0
and B =
0
1
We observe that the system (A, B) is a single-input controllable system that is marginally stable. This implies existence of
an optimal control u*. From the above discussion it follows
that u* must satisfy
u (t) = sign 2 (t)
In this section we discuss Bellmans dynamic programming

principle (Refs. 1,2) for optimal control problems.
Derivation of the HamiltonJacobiBellman
Consider the Bolza problem
tf
J(s, y; u) =
min
f 0 (t, x(t), u(t)) dt + g(x(t f ))
(34)
subject to
d
x(t) = f (t, x(t), u(t)),
dt
x(s) = y,
u(t) U
(35)
where U is a closed convex set in Rm. Under appropriate conditions on f, Eq. (35) has a unique solution x x(t; (s, y)), and
moreover x C(s, tf; Rn) depends continuously on (s, y) [0,
tf] Rn and u L1(s, tf; Rm). As discussed in the preceding
section, sufficient conditions on f 0 and g, which guarantee the
existence of an optimal pair (x*, u*) for each (s, y) [0, tf]
Rn, are well known. We define the minimum-value function
V(s, y) by
V (s, y) = min J(s, y; u)
uK
Then V satisfies the optimality principle:
min

f (t, x(t), u(t)) dt + V ( , x( )) : u U on [s, ]
0
= V (s, y)
The adjoint equation implies that (1(t), 2(t)) is given by

1 (t) = 1
and 2 (t) = 1 (T t) + 2
for some nonzero (1, 2). Hence the optimal control assumes at most the values 1 and 1 (bangbang control) and
it has at most one switch between these two values. Assume
(36)
In fact, the cost functional J is additive in the first variable:

for [s, tf],
J(s, y; u) =
s
f 0 (t, x(t), u(t)) dt + J( , x( ); u)
(37)
372
OPTIMAL CONTROL
for all u(s) u U, and
and thus
Vt (s, y) + f (s, y, u (s))Vx (s, y) + f 0 (s, y, u (s)) = 0
f 0 (t, x(t), u(t)) dt + V ( , x( )) J(s, y; u)
which is Eq. (38). Moreover, we have the following dynamical

programming principle.
for all u K. Thus,
V (s, y)
min
s

f 0 (t, x(t), u(t)) dt + V ( , x( )) : u Uad on [s, ]
V (s, y)
which implies Eq. (36).
Suppose that V is continuously differentiable. Then V satisfies the so-called HamiltonJacobiBellman (HJB) equation:
Vt (s, y) + min[ f (s, y, u)Vx (s, y) + f 0 (s, y, u)] = 0
uU
(38)
Theorem 5. (Verification Theorem). Let V be a solution of

the HJB equation (38) such that V C1((0, tf) Rn) and V(tf,
x) g(x). Then we have
1. V(s, x) J(s, x; u) for any admissible control u.
2. If u* (t, x) U is the unique solution to
f (t, x, u )Vx (t, x) + f 0 (t, x, u )

= min[ f (t, x, u)Vx (t, x) + f 0 (t, x, u)]
uU
and the equation

d
x(t) = f (t, x(t), (t, x(t)))
dt
We derive Eq. (38) using the optimality principle (36). Let

u K be of the form
u =
u on ( , t f )
f 0 (t, x(t), u(t)) dt + V ( , x( ))
If we set u(t) u*(t) on [s, ], then, from Eq. (36) u

(t)
u*(t) on [, tf] minimizes J(, x*(); ) on [, tf] and
V (s, y) =
f 0 (t, x (t), u (t)) dt + V ( , x ( ))
Proof. Note that for u K

d
V (t, x(t)) = Vt (t, x(t)) + f (t, x(t), u(t))Vx (t, x(t))
dt
and for any pair (x, u) Rn U,
Vt + f (t, x, u)Vx (t, x) + f 0 (t, x, u) 0

and thus
d
V (t, x(t)) f 0 (t, x(t), u(t))
dt
(39)
Since V is assumed to be C1, we have
Hence we have
d
d
V (t, x(t)) = Vt + Vx x(t) = Vt + f (t, x(t), u(t)) Vx
dt
dt
(Vt + fVx ) dt
Now, since V(s, y) J(s, x; u), the equation above Eq. (39)
implies
tf
V (s, y)
s
f 0 (t, x(t), u(t)) dt + g(x(t f )) = J(s, y; u)
for all admissible controls u. Similarly, if u*(t) U attains

the minimum in Eq. (40) with x x*(t), then
or equivalently,
V ( , x( )) = V (s, y) +
x(s) = y
has a solution x*(t) C1(s, tf; Rn) for each (s, y) [0, tf]
Rn, then the feedback solution u*(t) (t, x*(t)) is
optimal, that is, V(s, y) J(s, y; u*).

u on (s, )
where u(t) U on (s, ) and u

minimizes J(, x(); u) over
the interval [, tf]. From Eq. (37) we have
J(s, x; u)
=
(40)
d
V (t, x (t)) = Vt (t, x (t)) + f (t, x (t), u (t))Vx (t, x (t))
dt
= f 0 (t, x (t), u (t))
and thus
[Vt (t, x(t), u(t)) + f (t, x(t), u(t))Vx (t, x(t))
+ f 0 (t, x(t), u(t))] dt 0

where, from Eq. (39), the equality holds if u u* on [s, ].
Thus,
lim
s +
1
s
[Vt + f (t, x(t), u(t)) Vx + f 0 (t, x(t), u(t))] dt
= Vt (s, y) + f (t, y, u(s))Vx (s, y) + f 0 (s, y, u(s)) 0
tf
V (s, y) =
s
f 0 (t, x (t), u (t)) dt + g(x (t f )) = J(s, x; u )
Relation to the Maximum Principle

In this section we discuss the relation between the dynamic
programming and the maximum principle. Define the Hamil by
tonian H
x, u, p) = f 0 (t, x, u) ( f (t, x, u), p) n
H(t,
R
(41)
OPTIMAL CONTROL
(t, x, u, p) attains the maximum over u

We assume that H
U at the unique point u (t, x, p) and that is locally
Lipschitz. Let us define
x, u, p)
H(t, x, p) = max H(t,
uU
V (x) = g(x) on x
uU
(47)
Vt H(t, x, Vx ) = 0
(42)
Assume that V C1,2((0, tf) Rn). Then, defining p(t) Vx(t,

x*(t)), we obtain
d
x (t, x (t), u (t), p(t)),
p(t) = H
dt
subject to Eq. (35), where inf x(t) is the exit time

from an open set in Rn. It can be shown that, if we define
the value function V(y) infuU J(y, u), then it satisfies the
HJB equation
min[ f 0 (x, u) + f (x, u) Vx ] = 0,
Then Eq. (38) can be written as
373
p(t f ) = gx (x (t f )) (43)
and u*(t) (t, x*(t), p(t)) is an optimal control. In fact, by

Eq. (42)
d
Vx
d
Vx (t, x (t)) =
(t, x (t)) + Vxx (t, x (t)) x (t)
dt
t
dt
= Hx (t, x, Vx (t, x (t)))
+ Vxx (t, x (t))H p (t, x, Vx (t, x (t)))
+ Vxx (t, x (t)) f (t, x (t), u (t))
x (t, x (t), u (t), Vx (t, x (t)))
=H
where Vxx 2V/xi xj Rnn. Here we have used the fact
that
For the specific case f(x, u) u, f 0 1, g 0, U [1, 1],

and (1, 1), with x, u R, the HJB equation (47) becomes
|Vx (x)| + 1 = 0
with V (1) = 0
It can be proved that Eq. (48) has no C1 solution, but there

are infinitely many Lipschitz continuous solutions that satisfy
it a.e. in (1, 1). The viscosity method is developed as a mathematical concept that admits non-C1 solutions and selects the
solution corresponding to the optimal control problem to the
HJB equation.
We return now to the general problem presented in the
first subsection of this section.
Definition 1. A function v C((0, tf] Rn) is called a viscosity solution to the HJB equation vt H(t, x, vx) 0 provided
that for all C1(), if v attains a (local) maximum at
(t0, x0), then
t H(t, x, x ) 0
H p (t, x, p) = f (t, x, u)
(48)
at (t0 , x0 )
and if v attains a (local) minimum at (t0, x0), then
and
t H(t, x, x ) 0
Hx (t, x, p) = f x (t, x, u)
t p f x0 (t, x, u)
t
(t, x, u, p) over U. We
where u (t, x, p) U maximizes H
observe that Eq. (43) represents the adjoint equation of the
maximum principle with adjoint variable given by Vx( ,
x*).
Next let us set U Vx. Then U satisfies
Ut (t, x) + f (t, x, (t, x, U (t, x)))
Ux (t, x) Hx (t, x, U (t, x)) = 0
(44)
Hence, setting u(t) (t, x(t), p(t)), the necessary optimality

conditions
d
x(t) = f (t, x, u(t))
dt
d
p(t) = Hx (t, x, p(t))
dt
d
V (t) = f 0 (t, x, u(t))
dt
It is clear that a C1 solution to Eq. (42) is a viscosity solution, and if v is a viscosity solution of Eq. (42) and Lipschitz
continuous, then vt H(t, x, vx) 0 a.e. in (0, tf) Rn. The
viscosity solution concept is derived from the vanishing viscosity method illustrated by the following theorem.
Theorem 6. Let V(t, x) C1,2((0, tf) Rn) be a solution to
the viscous equation
Vt H(t, x, V ) + V = 0,
V (t f , x) = g(x)
(49)
If V(t, x) V(t, x) uniformly on compact sets as 0, then

V(t, x) is a viscosity solution to Eq. (42).
(45)
are the characteristic equations of the first order partial differential equations (PDEs) (42) and (44).
Viscosity Solution Method
In this section we discuss the viscosity solution method for
the HJB equation. For motivation, we first consider the exit
time problem

min J(y, u) =
f 0 (x(t), u(t)) dt + g(x( ))
(46)
0
at (t0 , x0 )
Proof. We need to show that

t H(t, x, x ) 0
at (t0 , x0 )
for all C1((0, tf) Rn) such that V attains a local

maximum at (t0, x0).
Choose a function C1((0, tf) Rn) such that 0 1
for (t, x) (t0, x0) and (t0, x0) 1. Then (t0, x0) is a strict local
maximum of V . Define V , and note
that, since V V uniformly on compact sets, there exists a
sequence (t, x) such that (t, x) (t0, x0) as 0 and
attains a local maximum at (t, x). The necessary optimality
condition yields
t = 0,
x = 0,

xx
0
at (t, , x )
374
OPTIMAL CONTROL
It follows that
t H(t, x, x ) + t H(t, x, x ) +
at (t , x )
Since is an arbitrary function with the specified properties,

t H(t, x, x) 0 at (t0, x0).
For example, let V(x) be the solution to
|Vx (x)| + 1 + Vxx = 0,
V (1) = V (1) = 0
Here h* denotes the conjugate function of h, which is defined

by
h (v) = sup{vu h(u)}
We assume that h* is Gateaux-differentiable with locally

Lipschitz Gateaux derivative h*p . Then we have u h*p (v),
where u U attains the maximum of (v, u) h(u) over u
U. In this case, the HJB equation is written as
Vt + a(x) Vx h (b(x)tVx ) + l(x) = 0
Then the solution V is given by

V (x) = 1 |x| + (e1/ e|x|/ )
and we have lim V(x) 1 x as 0. Moreover, V(x)
1 x is a viscosity solution to Vx(x) 1 0. We can also
check that any other Lipschitz continuous solution is not a
viscosity solution. It can be proved, in a general context, that
the viscosity solution is unique (Refs. 3,4).
As we saw for the exit time problem, the value function V
is not necessarily differentiable. But it always superdifferentiable. Here we call a function superdifferentiable at y0 if
there exists p Rn such that
lim sup
yy 0
(50)
uU
(y) (y0 ) (p, y y0 )

0
|y y0 |
(51)
with V(tf, x) g(x). As a specific case, we may consider the

linear quadratic control problem where
f (t, x, u) = A(t)x + B(t)u + f (t),

f 0 (t, x, u) = 12 [xt Q(t)xtu R(t)u]
and U Rm. Then we have
Vt + [A(t)x + f (t)]Vx 12 | R1 (t)Bt (t)Vx |2R + 12 xt Q(t)x = 0
(52)
Suppose that g(x) xtGx. Then V(t, x) xtP(t)x xv(t) is a
solution to Eq. (52), where P(t) Rnn satisfies the differential
Riccati equation
and we denote the set of p such that the above inequality

holds by D(y0). Based on the notion of viscosity solution, one
can express the dynamic programming and the maximum
principle without assuming that V is C1 as follows (see, e.g.,
Ref. 2).
dP
(t) + At (t)P(t) + P(t)A(t)
dt
P(t)B(t)R1 (t)Bt (t)P(t) + Q(t) = 0
(53)
with P(tf) G, and the feedforward v(t) satisfies

Theorem 7. The value function V(s, y) is continuous on (0,
tf) Rn, and locally Lipschitz continuous in y for every s
[0, tf]. Moreover, V is a viscosity solution of the Hamilton
JacobiBellman equation, and every optimal control u* to the
problem Eqs. (34)(35) is given by the feedback law
u (t) = (t, x (t), (t))
for some
(t)
D+
x V (t, x (t))
d
v(t) = [A BR1 Bt P(t)]t v(t) = P(t) f (t),
dt
(54)
In the control-constrained case with U u Rm : u 1

and h(u) u2, we find
for every t [0, T], where x*( ) is the optimal trajectory of

Eq. (35) corresponding to u*.
Applications
v(t f ) = 0
1
h (p) =
|p|2
|p|
if |p| < 1
1
2
if |p| 1
and
Here we consider the case
f (t, x(t), u(t)) = a(x) + b(x)u and
f 0 (t, x, u) = l(x) + h(u)
where it is assumed that h : U R is convex, is lower semicontinuous, and satisfies

h(u) |u|2
for some > 0
We find
min{ f 0 (t, x, u) + p f (t, x, u)}

uU
= a(x) p + l(x) max{u b(x)t p h(u)}
hp (p)
if |p| < 1
p/|p| if |p| 1
so that h* C1(Rm) W2,(Rm).

LINEAR QUADRATIC REGULATOR THEORY
AND RICCATI EQUATIONS
In this section we first revisit the finite-horizon LQR problem
in Eqs. (29)(30) and show that the optimal control u*(t) can
be expressed in the feedback form
uU
= a(x) p h (b(x)t p)
u (t) = R1 Bt [P(t)x(t) + v(t)]
(55)
OPTIMAL CONTROL
where the symmetric matrix P(t) and the feedforward v(t) satisfy Eqs. (53) and (54). The matrix K(t) R1BtP(t) describing
the control action as a function of the state is referred to as
the feedback gain matrix. The solution in the form of Eq. (55)
can be derived from the dynamical programming principle in
the preceding subsection, but here we prefer to give an independent derivation based on the two-point boundary value
problem (31). Since this equation is affine, we can assume
that
p(t) = P(t)x(t) + v(t)
(56)
More precisely, let (x(t), p(t)) denote a solution to Eq. (31).

Then for each t [0, tf], the mapping from x(t) Rn to p(t)
Rn defined by forward integration of the first equation in Eq.
(31) with initial condition x(t) on [t, tf] and subsequent backward integration of the second equation of Eq. (31) with terminal condition p(tf) Gx(tf) on [t, tf] is affine. Substituting
Eq. (56) into the second equation of Eq. (31), we obtain
d
d
d
P(t)x(t) + P(t) x(t) + v(t) = Qx(t) At [P(t)x(t) + v(t)]
dt
dt
dt
and from the first equation in Eq. (31) we derive
d
dt
P(t) + AP(t) + P(t)A P(t)BR

t
d
x (t) = [A BR1 Bt Pt (t)]x (t) BR1 Bt v(t) + f (t) (57)
f
dt
where v(t) is a solution to Eq. (54), and if we set p(t)
P(t)x*(t) v(t), then the pair (x*(t), p(t)) is a solution to Eq.
(31). Thus, the triple (x*(t), u*(t), pt(t)) satisfies the maximum principle. From Theorem 3 or from Eq. (58) below, it
follows that the feedback solution (55) is optimal.
The formula (56) is called the Riccati transformation. It
transforms the TPBV problem (31) into a system of initial
value problems backwards in time. Moreover, the feedback
solution given by Eq. (55) is unique. This follows from the fact
that for arbitrary u L2(0, tf; Rm), multiplying Eq. (57) from
the left by [Ptf (t)x*(t)]t using Eqs. (53) and (54), and integrating the resulting equation on [0, tf], we have
tf
0
|u(t) + R1 Bt [Pt x(t) + v(t)]|2R dt
1
2
xt0 Pt (0)x0 + 2v(0)t x0

f
tf
+
0
1
2
[xt (t)Qx(t) + u(t)Ru(t)] dt
(60)

d
x(t) = Ax(t) + Bu(t),
dt
x(0) = x0
This problem need not admit a solution. For example, consider the system
d
x(t) =
dt
1
0
1
x(t) +
1
1
u(t)
0
and let Q I and R be arbitrary. Then there exists no admissible control u L2(0, ; Rm) such that J(x0, u) is finite, unless x2(0) 0.
Under the assumption that for each x0 Rn there exists at
least one admissible control such that J(x0, u) is finite (finitecost condition), it can be shown that the optimal control is
given in feedback form by
where the nonnegative symmetric matrix P is defined in the

following theorem. A sufficient condition for the finite-cost
condition is that the pair (A, B) is stabilizable, that is, there
exists a matrix K Rmn such that A BK is asymptotically
stable. In this case, the closed-loop system with feedback control u(t) Kx(t) is exponentially stable, and we have J(x0,
Kx(t)) Mx02 for some M 0 independent of x0 Rn.
The following result is referred to as LQR theory. It relies
on the notions of detectability and observability. The pair (A,
Q1/2) is called detectable if there exists a matrix G such that
A GQ1/2 is asymptotically stable. Further (A, Q1/2) is called
observable if, for some 0, the kernel of the mapping x
Q1/2eAx is trivial. Observability of (A, Q1/2) is equivalent to controllability of (At, Q1/2).
Theorem 8 (LQR)
1. Assume that for each x0 Rn there exists at least one
admissible control such that J(x0, u) is finite. For any
tf 0, let Ptf ( ) denote the solution to the Riccati equation Eq. (53) with G 0. Then Ptf (0) converges monotonically to a nonnegative symmetric matrix P as tf ,
and P satisfies the algebraic Riccati equation
At P + P A P BR1 Bt P + Q = 0
(61)
The control
(58)
where yR2 ytRy and
J(x0 , u ) =
J(x0 , u) =
B P(t) + Q x(t)
This equation holds if Eqs. (53) and (54) are satisfied. By

standard results from the theory of ordinary differential
equations, there exists a unique symmetric and nonnegative
solution Ptf (t) C1(0, tf; Rnn) with Ptf (t) G to the Riccati
equation (53). If x*(t) is a solution to
1
2
min
u (t) = R1 Bt P x (t)
d
v(t) + [A BR1 Bt P(t)]t v(t) + P(t) f (t) = 0
dt
J(x0 , u) = J(x0 , u ) +
We turn to the infinite-time-horizon problem:
375
[vt (t)BR1 Bt v(t) + 2vt (t) f (t)] dt
u (t) = R1 Bt P x (t)
is the unique solution to the LQR problem (60), and
J(x0 , u ) = 12 xt0 P x0 =
min
J(x0 , u)
uL 2 (0,;R m )
(59)
Conversely, if there exists a nonnegative symmetric solution P to Eq. (61), then for all x0 Rn there exists an
376
OPTIMAL CONTROL
admissible control u such that J(x0, u) is finite and

P P.
2. Suppose that (A, Q1/2) is detectable and that Eq. (61)
admits a solution. Then the closed-loop matrix A
BR1BtP is asymptotically stable, and P is the unique
nonnegative symmetric solution to Eq. (61). If, moreover, (A, Q1/2) is observable, then P is positive definite.
Proof. For part 1 note that, due to Eq. (59), we have for
tf tf
xt0 Pt (0)x0 xt0 Pt (0)x0
f
Thus, Ptf (0) Ptf (0) for tf tf. The assumption on the existence of admissible controls shows that eitPtf (0)ei and (ei
ej)tPtf (0)(ei ej) are monotonically nondecreasing and bounded
with respect to tf. Here ei denotes the ith unit vector in Rn.
Defining P limtf Ptf (0), it follows that P is symmetric, is
nonnegative, and moreover satisfies the steady-state equation
Eq. (61). It can then be argued that the feedback control
u*(t) R1BtPx*(t) is the unique optimal solution to the
LQR problem (60). To prove the last assertion of part 1, suppose that P is a nonnegative symmetric solution to Eq. (61).
Let x(t) be the solution to (d/dt)x(t) (A BR1BtP)x(t) with
x(0) x0, and let u(t) R1BtPx(t). Then
d t
[x (t)Px(t)] = 2xt (t)P(A BR1 Bt P)x(t)
dt
= xt (t)(Q + PBR1 Bt P)x(t)
Integration of this equation on [0, tf] implies
tf
0
(xt Qx + ut Ru) dt + x(t f )t Px(t f ) = xt0 Px0
and thus J(x0, u) x0t Px0 and x0t Px0 x0t Px0 for every
x0 Rn.
To verify part 2, note that
(A BR1 Bt P )t P + P (A BR1 Bt P )
+ Q + P BR1 Bt P = 0
It can be shown that (A BR1BtP, (Q PBR1BtP)1/2) is
detectable (observable) if (A, Q1/2) is detectable (observable).
Hence, it follows from the Liapunov criterion (5) that A
BR1BtP is asymptotically stable and moreover that P is
positive definite if (A, Q1/2) is observable. For the proof of
uniqueness we refer to the literature (see, e.g., Ref. 5).
In the following theorem we consider the LQR theory with
external forcing.
Theorem 9. Consider the infinite-time-horizon problem
min
1
2
[x (t)Qx(t) + u (t)Ru(t)] dt
t
subject to
d
x(t) = Ax(t) + Bu(t) + f (t),
dt
x(0) = x0
where f L2(0, ; Rn). Assume that (A, B) is stabilizable and

that (A, Q1/2) is detectable. Then there exists a unique optimal
solution u* given by
u (t) = R1 Bt [P x (t) + v(t)]
where P is the unique solution to Eq. (61) and v L2(0, ;
Rn) satisfies
d
v(t) + (A BR1 Bt P )v(t) + P f (t) = 0,
dt
v() = 0
Proof. From Theorem 8 it follows that A BR1BtP is asymptotically stable and thus v L2(0, ; Rn) and u*(t)
L2(0, ; Rm). Since (A, Q1/2) is detectable, there exists a matrix
G Rnn such that A GQ is asymptotically stable. For arbitrary admissible controls u L2(0, ; Rn)
x(t) = e(AGQ)t x0 +
e(AGQ)(ts) [GQx(t) + Bu(t) + f (t)] dt
From the Fubini inequality

|x(t)|2 dt M(|x0 |2 + |u|2L 2 (0,;R m ) + | f |2L 2 (0,;R n ) )
0
for some M 0. Thus limt x(t) exists and is zero. Taking

the limit tf in Eq. (58), we obtain

1
J(x0 , u) = J(x0 , u ) +
|u(t) + R1 Bt [P x(t) + v(t)]|2R dt
2 0
which proves the theorem.
Assume that (A, B) is stabilizable. Then there exists a solution P to Eq. (61), for which, however, A BR1BtP is not
necessarily asymptotically stable. The following theorem
shows that there exists a maximal solution P to the Riccati
equation (61) and gives a sufficient condition such that A
BR1BtP is asymptotically stable.
Theorem 10. Assume that (A, B) is stabilizable. For 0
let P be the nonnegative symmetric solution P to the Riccati
equation
At P + PA PBR1 Bt P + Q + I = 0
Then P converges monotonically to a nonnegative symmetric
matrix P as 0. The matrix P is a solution to Eq. (61),
and P P for all nonnegative symmetric solutions P to Eq.
(61). Moreover, if we assume that the Hamiltonian matrix
H=
A
Q
BR1 Bt
At
(62)
has no eigenvalues on the imaginary axis, then A

BR1BP is asymptotically stable. For the stability-constrained LQR problem of minimizing Eq. (60) subject to
d
x(t) = Ax(t) + Bu(t) + f (t),
dt

|x(t)|2 dt <
x(0) = x0 and
0
OPTIMAL CONTROL
Assume that U11 is nonsingular, and define P U21U1

11 . Since
PU11 U21, we have
the unique optimal control u* is given by

u (t) = R1 [Bt P + x(t) + v(t)]
(A WP)U11 = U11 S11
where v(t) L2(0, , Rn) satisfies

d
v(t) + (A BR1 Bt P+ )v(t) + P+ f (t) = 0,
dt
377
v() = 0
Due to the importance of finding the stabilizing feedback

gain, solving Eq. (61) is of considerable practical importance.
We therefore close this section by describing the PotterLaub
method. We also refer to Refs. 6, 7 for iterative methods based
on the NewtonKleimann and Chandrasekhar algorithms.
The PotterLaub method uses the Schur decomposition of the
Hamiltonian matrix (62) and is stated in the following
theorem.
(Q At P)U11 = PU11 S11
and
Thus P(A WP)U11 (Q AtP)U11, and moreover A WP

U11S11U1
11 .
Conversely, if P satisfies AtP PA PWP Q 0. Then
and
I
P
0
I

I
P
0
I
A WP
0
A WP
Q At P
W
At + PW
W
At

=
A WP
P(A WP)
W
At
Theorem 11
1. Let Q, W be symmetric n n matrices. Solutions P to
the algebraic Riccati equation AtP PA PWP Q
0 coincide with the set of matrices of the form P
VU1, where the n n matrices U [u1, . . ., un], V
[v1, . . ., vn] are composed of upper and lower halves of
n real Schur vectors of the matrix
H=
A
Q
W
At
and U is nonsingular.
2. There exist at most n eigenvalues of H that have negative real part.
3. Suppose [u1, . . ., un] are real Schur vectors of H corresponding to eigenvalues 1, . . ., n, and i j for 1
i, j n. Then the corresponding matrix P UV1 is
symmetric.
4. Assume that Q, W are nonnegative definite and (A,
Q1/2) is detectable. Then the solution P is symmetric and
nonnegative definite if and only if Re k 0, 1 k
n.
Proof. We prove part 1. Let S be a real Schur form of H, that
is, HU US with UtU I and
S=
Thus
U11
H
U21
where
S11
0
S12
S22
U11
U21
S11
and
I
P

0
I
I
P
0
I
A WP
0
W
At + PW
The proof of assertions 24 can be found in Ref. 6.

In summary, the stabilizing solution P corresponds to the
stable eigen subspace of H, and the eigenvalues of the resulting closed-loop system coincide with those of S11.
NUMERICAL METHODS
In this section we discuss numerical methods for the nonlinear regulator problem

min
J(x0 , u) =
[l(x(t)) + h(u(t))] dt + g(x(T ))
(63)
subject to
d
x(t) = f (x(t), u(t)),
dt
x(0) = x0 ,
u(t) U
(64)
for all u U
(65)
We assume that
and moreover either that U Rm is bounded or that h(u)

c1u2 and
( f (x, u), x) |x|2 + c2 |u|2
(66)
for constants , c1, c2 0, independent of x, y Rn and u

U. Also we assume that for each (x, p) Rn Rn the mapping
u h(u) + (p, f (x, u))
is made up of n Schur vectors of H corresponding to the S11

block. We observe that
AU11 WU21 = U11 S11
( f (x, u) f (y, u), x y) |x y|2

U11
U21
Thus
QU11 At U21 = U21 S11
admits a unique minimizer over U, denoted by (x, p). Finally, we assume that l, h, g, and f are sufficiently smooth
with l and g bounded from below.
378
OPTIMAL CONTROL
Discrete-Time Approximation
We consider the discretized problem
min
J N (uN ) =
N

[l(xk ) + h(uk )] t + g(xN )
(67)
k=1
Proof. First, we show that xN and pN are uniformly Lipschitzian in N. It will be convenient to drop the tilde in the notation for these sequences. In the case that U is bounded, we
proceed by taking the inner product of Eq. (68) with xk and
employing Eq. (65):
1
(|xk |2
2
|xk1 |2 ) t [|xk |2 + | f (0, uk )| |xk |]
subject to
t [( + 12 )|xk |2 + 12 | f (0, uk )|2 ]
xk xk1
= f (xk , uk ) and uk U
t
1kN
(68)
where N t T, which realizes the implicit Euler scheme for

time integration of Eq. (64) and first-order integration of the
cost functional (63). Note that if t 1, then the mapping
(x) x t f(x, u) is dissipative, that is, (F(x1, u) F(x2,
N
in U
u), x1 x2) (1 t )x1 x22. Thus for u ukk1
k N
there exists a unique x x k1, satisfying the constraint Eq.
(68) and depending continuously on u. Moreover, if t 1,
then there exists an optimal pair (xk, uk) to the problem Eqs.
(67), (68). The necessary optimality condition for that problem
is given by
xk xk1
= f (xk , uk )
t
pk+1 pk
= f x (xk , uk )t pk + lx (xk )
t
uk = (xk , pk ) U
(69)
xk xk1
(t tk1 )
t
and, by assumption on h, k1 uk2 t is bounded uniformly

in N. In either case, by the discrete-time Gronwalls inequality we obtain that xk M1 for some M1 0 uniformly in k
and N. The condition (65) implies that
N
( f x (x, u)p, p) |p|2

and taking the inner product of Eq. (69) with pk, we obtain
|pk+1|2 ) t [|pk |2 + |lx (xk )| |pk |]
Thus pk M2 for some M2 uniformly in k and N. Due to the

Lipschitz continuity of , we find that uk bounded uniformly
in k and N, and from Eq. (69),

k
x xk1 pk pk+1
,
are bounded uniformly

t
t
Using Lipschitz continuity of a second time, we find that
(uk uk1)/t is uniformly bounded as well. By the compactness of Lipschitz continuous sequences in L2(0, T), there ex such that (xN, uN, pN) converges to (x*,
ists a subsequence N
2
n
u*, p*) in L (0, T; R Rm Rn) and pointwise a.e. in (0, T).
From Eq. (69),

xN (t) = x0 +
f (xN (t), uN (t)) dt

0
where xN is the piecewise constant sequence defined by xk,

1 k N. By Lebesgues dominated convergence theorem,
we find that x* coincides with the solution x(t; u*) to Eq. (64)
associated with u*. For v L2(0, T; Rm) let vN be the piecewise
t
constant approximation of v, defined by vk (1/N) tk1 v(t)
k
dt, 1 k N. Then
and
pk+1 pk
(t tk1 )
t
Then the sequence (xN, u

N, pN) in H1(0, T; Rn) L2(0, T; Rm)
1
n
H (0, T; R ) has a convergent subsequence as t 0, and
for every cluster point (x, u, p), u K is an optimal control of
Eqs. (63), (64), and (x, u, p) satisfies the necessary optimality
condition
d
x(t) = f (x(t), u(t)),
x(0) = x0
dt
d
p(t) = f x (x(t), u(t))t p(t) + lx (x(t)),
dt
u(t) = (x(t), p(t)) U
|xk1 |2 ) t (|xk |2 + c2 |uk |2 )
t [( + 12 )|pk |2 + 12 |lx (xk )|2 ]
Theorem 12. Assume that is Lipschitz continuous, that
t 1, and that uNN1

is a sequence of solutions to Eqs.
(67), (68) with associated primal and adjoint states (xN,
pN)N1
such that Eq. (69) holds. Let u
N denote the step function defined by u
N(t) uk on (tk1, tk), 1 k N, and let xN
and pN be the piecewise linear functions defined by
p N (t) = pk +
1
(|xk |2
2
1
(|pk |2
2
for 1 k N, with x0 x0 and pN1 gx(xN). It is noted that

Eq. (69) is a sparse system of nonlinear equations for
col(col(x1, . . ., xN), col(p1, . . ., pN)) RnN RnN. We have the
following result:
xN (t) = xk1 +
In case U is unbounded, Eq. (66) implies that
p(T ) = gx (x(T ))
(70)
J N (uN ) J N (vN )
for all v
and x(t, vN) x(t, v) as N , and thus, by the Lebesgue

dominated convergence theorem, J(x0, u*) J(v; x0) for all
admissible control, that is, (x*, u*) is an optimal pair.
It is not difficult to argue that the triple (x*, u*, p*) satisfies the necessary optimality [Eq. (70)].
Construction of Feedback Synthesis
The numerical realization of feedback synthesis for problems
that are not of the LQR type has not received much research
attention up to now. In this subsection we propose a method
OPTIMAL CONTROL
for the construction of the feedback synthesis K to the problem (63)(64), which is still under investigation.
As we discussed in the Section titled Dynamic Programming Principle and HamiltonJacobyBellman Equation,
the optimal feedback law is given by K(t, x(t)) KT(t, x(t))
(x(t), Vx(t, x(t)), where V(t, x) is the solution to HJB Eq. (38)
and we stress the dependence of K on T. Let us assume that
f(x, u) f(x) Bu, and h /2u2. Then
1
KT (t, x(t)) = Bt Vx (t, x(t))
x0 Rn
(72)
1. Choose T 0 sufficiently large, as well as a grid

Rn, and calculate the solutions (x, p)x0 to the TPBV problem for all initial conditions determined by x0 . Thus
we obtain the values of K at x0 .
2. Use an interpolation method based on K at the grid
points of to construct a suboptimal feedback synthesis K.
The interpolation in step 2 above can be based on appropriate Greens functions, for example,
M

G(x, x j ) j
with
G(xi , x j ) j = K(xi ),
1iM
j=1
(73)
The Greens function interpolation (73) has the following variational principle: Consider

min
Rn
|K(x)|2 dx
subject to K(xi ) = i ,

x0
|Wx (x0 ) px 0 (0)|2
(74)
over Wad, where x max(0, x). Then we set K(x) (x, Wx).
ACKNOWLEDGMENTS
The dependence of the feedback gain KT on T is impractical,

and we replace it by a stationary feedback law v(t) K(x(t)),
which is reasonable if T is sufficiently large.
Based on these observations, a suboptimal feedback law
can be constructed by carrying out the following steps:
K(x)
=
best interpolation W Wad based on the stationary equation

f(x) Vx h*(BtVx) l(x) 0 by

min
|[ f (xk ) Wx (xk ) h (Bt Wx (xk )) + l(xk )]+ |2
(71)
Since the problem under consideration is autonomous, we can

write KT(t, x(t)) KTt(0, x(t)). Using the relationship between
the HJB equation and the Pontryagin maximum principle, as
described in the section titled Dynamic Programming Principle and HamiltonJacobyBellman Equation, we construct
a suboptimal feedback law. It is based on the fact that if we
set p(t) Vx(t, x(t)), then the pair (x(t), p(t)) satisfies the TPBV
problem (70). Thus, if we define the function x0 Rn px0(0)
Rn, where (x, p)x0 is the solution to Eq. (70) with x(0) x0,
then
KT (0, x0 ) = (x0 , px 0 (0)),
379
1iM
Then the optimal solution K is given by Eq. (73), where

Greens function G satisfies the biharmonic equation 2G(x)
(x). For instance, G(x, y) x y (biharmonic Greens
function) in R3. In our numerical testings we found that G(x,
y) x y, 1.5 4, works very well. Alternatively, we
may employ the following optimization method. We select a
class Wad C1(Rn) of parametrized solutions to the HJB equation and collocation points xk in Rn. Then we determine the
K. Itos research was supported in part by AFSOR under contracts F-49620-95-1-0447 and F-49620-95-1-0447. K. Kunischs research was supported in part by the Fonds zur Forderung der wissenschaftlichen Forschung, SFB Optimization
and Control.
BIBLIOGRAPHY
1. R. Bellman, Dynamic Programming, Princeton, NJ: Princeton
Univ. Press, 1957.
2. W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Control, New York: Springer-Verlag, 1975.
3. M. G. Crandall and P. L. Lions, Viscosity solutions of Hamilton
Jacobi equations, Trans. Amer. Math. Soc., 277: 142, 1983.
4. W. Fleming and M. Soner, Controlled Markov Processes and Viscosity Solutions, Berlin: Springer-Verlag, 1993.
5. W. M. Wonham, Linear Multivariable Control: A Geometric Approach, Berlin: Springer-Verlag, 1974.
6. D. L. Russell, Mathematics of Finite-Dimensional Control Systems,
Theory and Design, New York: Marcel Dekker, 1979.
7. H. T. Banks and K. Ito, A numerical algorithm for optimal feedback gains in high dimensional linear quadratic regulator problems, SIAM J. Control Optim., 29: 499515, 1991.
Reading List
V. M. Alekseev, V. M. Tikhomirov, and S. V. Fomin, Optimal Control,
New York: Plenum, 1987.
M. Athans and P. Falb, Optimal Control, New York: McGraw-Hill,
1963.
L. D. Berkovitz, Optimal Control Theory, New York: Springer-Verlag, 1974.
A. E. Bryson and Y. Ho, Applied Optimal Control, New York: Wiley,
1975.
M. R. Hestenes, Calculus of Variations and Optimal Control, New
York: Wiley, 1963.
H. Hermes and J. L. LaSalle, Functional Analysis and Time Optimal
Control, New York: Wiley, 1969.
H. Knobloch, Higher Order Necessary Conditions in Optimal Control
Theory, Berlin: Springer-Verlag, 1981.
H. Kwakernaak and R. Sivan, Linear Optimal Control Systems, New
York: Wiley-Interscience, 1972.
B. Lee and L. Marcus, Foundations of Optimal Control Theory, New
York: Wiley, 1967.
L. W. Neustadt, Optimization: A Theory of Necessary Conditions,
Princeton, NJ: Princeton Univ. Press, 1976.
KAZUFUMI ITO
North Carolina State University
KARL KUNISCH
Karl-Franzens-Universitat Graz
380
OPTOELECTRONICS IN VLSI TECHNOLOGY
OPTIMAL FILTERING. See WIENER FILTERS.

OPTIMIZATION, CIRCUIT. See MINIMIZATION.
OPTIMIZATION, CONVEX. See CONVEX OPTIMIZATION.
OPTIMIZATRION, FUZZY. See MINIMIZATION.
OPTIMIZATION IN SCHEDULING. See SCHEDULING.
OPTIMIZATION, LIFE CYCLE. See LIFE CYCLE OPTIMIZATION.
OPTIMIZATION OF LOGIC CIRCUITS. See LOGIC

SYNTHESIS.
OPTIMIZATION PROBLEMS/TECHNIQUES. See

TRAVELING SALESPERSON PROBLEMS.
OPTION CONTRACTS. See STOCK MARKETS.

OPTODES, OPTRODES. See CHEMICAL SENSORS.
OPTOELECTRONIC PHENOMENA. See PHOTOCONDUCTIVITY.
OPTOELECTRONICS. See LIGHT EMITTING DIODES, DEVICES.
OPTOELECTRONICS, INTEGRATED. See INTEGRATED

OPTOELECTRONICS.
PERIODIC CONTROL
59
PERIODIC CONTROL
The fact that a periodic operation may be advantageous has
been well known to humankind since time immemorial. All
farmers know that it is not advisable to grow the same product repeatedly in the same field because the yield can be improved by rotating crops. So, cycling is good.
More recently, similar concepts have been applied to industrial problems. Traditionally, almost every continuous industrial process was set and kept, in the presence of disturbances, at a suitable steady state. However, there are
circumstances under which a periodic time-varying action
proves to be better. This observation germinated in the field
of chemical engineering where it was seen that the performance of a number of catalytic reactors improved by cycling;
see the pioneering contributions in Refs. 13. Unfortunately,
as pointed out in Ref. 4, periodic control was still considered
too advanced in the industrial control scenario, in that the
steady-state operation is the norm and unsteady process behaviour is taboo. Its use was therefore confined to advanced
(aerospace or classified) applications, such as those treated in
Refs. 5 and 6. Today, however, the new possibilities offered
by current control technology, together with the theoretical
developments of the field, have opened the way for using periodic controllers in place of the traditional stationary ones. In
fact, the term periodic control takes a wider significance in
the contemporary literature. In addition to the control problems that arise when operating a plant periodically, periodic
control also includes all situations where either the controller
or the plant is a proper periodic system. One of the reasons
behind such an extension is the possible improvement of the
performances, in terms of stability and robustness, of plants
described by time-invariant models, when using a periodic
controller (see Ref. 7).
The diffusion of digital apparatuses in control has also
contributed to the increasing importance of periodic control
because computer-controlled systems are often based on sample-and-hold devices for output measurements and input updating. In multivariable control, it may also be necessary, for
technological or economical reasons, to adopt different sampling and/or hold intervals for the various actuators or transducers. For example, certain variables may exhibit a much
slower dynamic than others so that different sampling interJ. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering. Copyright # 1999 John Wiley & Sons, Inc.
60
PERIODIC CONTROL
vals must be adopted. In such situations, we usually resort to

an internal clock for setting a synchronization mode; the digital equipment complementing the plant performs the selection of the output sampling instants and the control updating
instants out of the clock time points. It turns out that these
sampling selection mechanisms are described by periodic
models in discrete time, with period equal to the least common factor of the ratios between the sampling and updating
intervals over the basic clock period. The overall control system obtained in this way is known as a multirate sampleddata system (8,9).
Finally, resorting to control laws that are subject to periodic time variations is natural to govern phenomena that
are intrinsically periodic. An important field where we encounter such dynamics is helicopter modeling and control,
as witnessed by the fact that a full chapter of the classical
reference book in the field (10) is devoted to periodic systems. The main interest in this framework is rotor dynamics modeling. Indeed, consider the case of level forward
flight, when the velocity vector of the flying machine is
constant and parallel to its body. Those flight conditions
are achieved by imposing a periodic pattern on the main
control variables of the helicopter (i.e., the pitch angles of
each blade). Consequently, the aerodynamic loads present a
cyclic pattern, with period determined by the rotor revolution period, and any model of the rotor dynamics is periodic
(see Refs. 1113). The interest for periodic systems goes far
beyond these situations. Periodicity arises in the study of
nonlinear time-invariant systems dealing with closed orbit
operations. Classical examples of such situations are relayoperated control plants, hysteretic oscillators, and processes
subject to seasonal-load effects. For the study of system
behavior against small perturbations, a linearized approximated model is often used. And, although the original system is time-invariant, the linearization procedure generates
periodicity in the approximated linear model (see Refs.
1416).
In this article, the most important techniques of periodic
control will be outlined, avoiding, however, overly technical
details. The article is organized as follows. The first part deals
with the analysis of periodic systems. Initially, it is shown
how state-space periodic models arise from multirate sampling or linearization around closed orbits. The periodic
input/output representation is also introduced as an alternative to state-space modelization. Then, the possibility of analyzing a periodic system via time-invariant models is investigated and a number of techniques are introduced. Further,
the frequency-response concept for periodic systems is outlined. The fundamental concept of stability comes next. It
calls for the definition of the monodromy matrix and involves
the theory of Floquet and Lyapunov. In passing, the notion of
cyclostationary stochastic process is touched on and briefly
discussed.
The second part is devoted to periodic control, and discusses three main problems: (1) choice of the control signal in
order to force a periodic regime with better performance than
any possible steady state operation, (2) periodic control of
time-invariant plants, and (3) periodic control of periodic
systems.
The literature on the subject is so vast that it is impossible
to cover all aspects of theoretical and application interest. The
interested reader will find a rather detailed list of references
in the bibliography, including survey papers (1722) and

early reference books (23,24).
BASICS IN PERIODIC SYSTEMS ANALYSIS

A basic classification of linear periodic systems depends on
the nature of the time variable t. We focus our attention
herein on continuous-time or discrete-time periodic systems.
In the former case t is a real variable, whereas in the latter t
is an integer.
State-Sampled Representation
Nowadays, the most widely used mathematical modelization
of dynamical systems hinges on the concept of state-variable.
The state variables are latent variables that establish a
bridge between the input variables u(t) and the output variables y(t). They are collected in a vector denoted by x(t), and
the basic state-space model is the set of difference equations
x(t + 1) = A(t)x(t) + B(t)u(t)
y(t) = C(t)x(t) + D(t)u(t)
in discrete-time, or the set of differential equations
x(t)
= A(t)x(t) + B(t)u(t)
y(t) = C(t)x(t) + D(t)u(t)
in continuous time.
Matrices A( ), B( ), C( ), and D( ) are real matrices, of
appropriate dimensions, that depend periodically on t:
A(t + T ) = A(t),
B(t + T ) = B(t)
C(t + T ) = C(t),
D(t + T ) = D(t)
The smallest T for which these periodicity conditions are met

is called the system period.
These state-space models may be generalized and extended
in various ways, among which are the class of descriptor models (25).
Periodicity Induced by Linearization. As mentioned earlier,
a linear periodic system can be used to describe the small
perturbation behavior along a periodic regime. For example,
consider the continuous-time nonlinear system
(t) = f ( (t), v(t))
(t) = h( (t), v(t))
and let v( ), ( ), ( ) be an associated periodic regime of
period T. This means that ( ) is a periodic solution of period
T associated with the periodic input v ( ) and that ( ) is the
corresponding periodic output. The linearized equations result in a linear continuous-time system with
u(t) = v(t) v(t),
x(t) = (t) (t),
y(t) = (t) (t)
PERIODIC CONTROL
and matrices

f ( , v)
A(t) =
,
= ,v= v

h( , v)
C(t) =
,
= ,v= v

f ( , v)
B(t) =
v = ,v= v

h( , v)
D(t) =
v = ,v= v
These matrices are obviously periodic of period T. Mutatis

mutandis, the same reasoning applies in discrete-time as
well. The linearization rationale is illustrated in Fig. 1.
Periodicity Induced by Multirate Sampling. Multirate
schemes arise in digital control and digital signal processing
whenever it is necessary to sample the outputs and/or update
the inputs with different rates. To explain in simple terms
how multirate sampled-data mechanisms generate periodicity, consider a system with two inputs and two outputs described by the time-invariant differential equations

u1 (t)
x(t)
= Ax(t) + B
u2 (t)

u (t)
y1 (t)
= Cx(t) + D 1
y2 (t)
u2 (t)
The overall behavior of the obtained system is ruled by the

discrete-time output variables yi(k), i 1, 2 and the discretetime input variables u
j(k), j 1, 2 defined as
yi (k) = y(ky i ),
u j (t) = u j (k),
y(k)
= y(k),
u=v~
v
Periodic regime
~~~
u,x,y
j = 1, 2
u(k)
= u(k)
In contrast, y(k) and u

(k) can be considered as the slow-rate
samples. The sampling selector, namely the device operating
the passage from the fast sampled-output to the slow sampled-output, is described by the linear equation
y(k)
= N(k)y(k)

N(k) =
tu j (k) = ku j
= f( ,v)
= h ( ,v)
t [ku j , ku j + u j ),
For the modelization, however, it is advisable to introduce the

fast-rate signals
n1 (k)
0
ni (k) =

1
0
if k is a multiple of y i /
otherwise
Note that matrix N( ) is periodic with the period given by the

integer Ty defined as the least common multiple of y1 / and
y2 /.
As for the hold device, introduce the holding selector matrix

s (k)
S(k) = 1
0

0
s2 (k)
with

s j (k) =
y = ~

0
n2 (k)
with
Time-invariant
nonlinear system
v
i = 1, 2
where
The two outputs yi( ), i 1, 2, are sampled with sampling

intervals yi, i 1, 2, whereas the two inputs uj( ), j 1, 2,
are updated at the end of intervals of length uj, j 1, 2, and
kept constant in between. The sampling and updating instants are denoted by tyi(k), i 1, 2 and tuj(k), j 1, 2, k
integer, respectively. Typically, these instants are taken as
multiples of the basic clock period . Moreover, for simplicity,
assume that
ty i (k) = ky i ,
61
0 if k is a multiple of u j /
1 otherwise
Then, the analog input signal u( ) is given by

u(t) = u(k),
t [k, k + )
where the fast updated signal u(k) is obtained from the slow
one u
(k) according to the holding selector mechanism
u
A(. ), B(. ), C(. ), D( .)
v(k + 1) = S(k)v(k) + (I S(k))u(k)
u(k)
= S(k)v(k) + (I S(k))u(k)
Periodic linear
system
Figure 1. Linearization around a periodic orbitthe dynamics of the
nonlinear system in the orbit vicinity is governed by a linear periodic system.
Matrix S(k) is periodic of period Tu given by the least common

multiple of u1 / and u2 /. This situation is schematically illustrated in Fig. 2, where u1 3, u2 4, y1 2, y2
5, so that Tu 12 and Ty 10.
62
PERIODIC CONTROL
~
u
^
u
S (.)
Holding selector
(period 12)
u
ZOH
A,B,C,D
Zero-order
holder
System
u1
y^
N (.)
Sampler
( )
~
y
Sampling selector
(period 10)
y1
y
u
u2
y2
~
u
i (slow updating)
~
y i (slow sampling)
^
u
i (fast updating)
y^i (fast sampling)
Figure 2. A multirate sampled-data system with two inputs and two outputs. The symbol
denotes the clock period. The first output signal y1 is sampled at rate y1 2 and the second y2
at rate y2 5. Hence, the sampling selector is a periodic discrete-time system with period
Ty 10. Moreover, the first input signal u1 is updated at rate u1 3 and the second u2 at rate
u2 4. The holding selector is a periodic discrete-time system with period Tu 12. The period
of the global system is therefore T 60.
The overall multirate sampled-data system is a discretetime periodic system with state

x(k)
x(k)
=
v(k)
where
A(k)
=

B(k)
=
and equations
u(k)
x(k
+ 1) = A(k)
x(t)
+ B(k)
x(k)
y(k)
= C(k)
+ D(k)
u(k)
A
0

0
e
0
d BS(k)
S(k)
e A d B(I S(k))
I S(k)
C(k)
= N(k)[C DS(k)]
D(k)
= N(k)D(I S(k))
PERIODIC CONTROL
The so-obtained system is periodic with period T given by the

least common multiple of Tu and Ty (T 60 in the example of
Fig. 2).
Lagrange Formula. The free motion of the periodic system,
i.e., the solution of the homogeneous equation

x(t)
= A(t)x(t)
in continuous time
x(t + 1) = A(t)x(t) in discrete time

starting from state x() at time is obtained as
where the transition matrix A(t, ) is given by

I
t=
A (t, ) =
A(t 1)A(t 2) . . . A( ) t >
in discrete time and by the solution of the differential matrix
equation
( , ) = I
in continuous time. Therefore, the state solution with a generic initial state x() and input function u( ) is
x(t) = A (t, )x( ) +
t
A (t, j)B( j 1)u( j 1)
j= +1
in discrete time and

x(t) = A (t, )x( ) +
Input-Output Representation
Another mathematical representation of periodic systems is
based on a direct time-domain relationship between the input
and the output variables without using any intermediate latent variables. In discrete time, this leads to a model of the
form
+ G1 (t)u(t 1) + G2 (t)u(t 2) + + Gs (t)u(t s)

where Fi( ) and Gj( ) are periodic real matrices. Such a representation is frequently used whenever a cyclic model must be
estimated from data, as happens in model identification, data
analysis, and signal processing. For the passage from a statespace periodic system to an input-output periodic model and
vice versa, see, for example, Refs. 26 and 27. Input-output
periodic models can also be introduced in continuous time by
means of differential equations with time-varying coefficients.
Note that state-space or input-output periodic models are
used in the stochastic modeling of cyclostationary processes, a
type of stochastic nonstationary processes with periodic characteristics. In such a context, the input u( ) is typically a remote signal described as a white noise. Then, the input-output models are known as PARMA models, where PARMA
means periodic auto-regressive moving average.
In the following sections, the main attention will be focused on state-space models.
A (t, )B( )u( )
TIME-INVARIANT REPRESENTATIONS
in continuous time. These expressions are known as Lagrange

formulas (also called variations-of-constants formulas).
We can easily see that the periodicity of the system entails
the biperiodicity of matrix A(t, ), namely that
A (t + T, + T ) = A (t, )
The transition matrix over one period
A ( ) = A ( + T, )
Reversibility. In continuous time, there is no analytic expression for the transition matrix. However, its determinant
can be worked out from the so-called Jacobi formula. In other
words,

The simplest way to achieve stationarity is to resort to a sample-and-hold procedure. Indeed, with reference to a continuous or a discrete-time periodic system, suppose that the input
is kept constant over a period, starting from an initial time
point . That is,
u(t) = u(k),
t [kT + , kT + T + )
As seen before, periodicity is often the result of ad hoc operations over time-invariant systems. On the other hand, in periodic systems analysis and control, a major point is to address
the backward problem of finding a way to transform a periodic system into a time-invariant one. In such a way, we can
resort to the results already available in the time-invariant
realm.
Sample and Hold
plays a major role in the analysis of periodic systems and is

known as monodromy matrix at time .
det[A (t, )] = exp
suming that input u( ) over the interval [, t) is known]. This

is not true, in general, in discrete-time since the transition
matrix is singular when A(i) is singular for some i.
y(t) = F1 (t)y(t 1) + F2 (t)y(t 2) + + Fr (t)y(t r)
x(t) = A (t, )x( )
(t, ) = A(t)A (t, ),

t A
63
trace[A( )] d
Therefore, for any choice of t and , the transition matrix is

invertible. This means that the system is reversible, in that
the state x() can be uniquely recovered from x(t), t [as-
Then the evolution of the system state sampled at kT

[i.e., x(k) x(kT )] is governed by a time-invariant equation in discrete time. Precisely,
x (k + 1) = A (T + , )x (k) +
( )u(k)
64
PERIODIC CONTROL
where
The time-lifted reformulation can then be introduced:
( ) =

T +
A (T + , )B( ) d

T
+ 1
A (T + , i + 1)B(i)
in continuous time
in discrete time
i=
Generalized Sample and Hold. It is possible to generalize

the sample-and-hold representation by equipping the holding
mechanism with a time-varying periodic modulating function
H( ) acting over the period as follows:
u(t) = H(t)u(k),
t [kT + , kT + T + )
In this way, the evolution of the sampled state x(k) is still

governed by the previous equations provided that B(t) is replaced by B(t)H(t). Such a generalized sample-and-hold representation allows for a further degree of freedom in the design
of periodic controllers. Indeed, function H( ) is a free parameter to be chosen by the designer; see Ref. 28.
Lifted and Cyclic Reformulations
Despite its interest, the sample-and-hold representation is by
no means an equivalent reformulation of the original periodic
system because the input function is constrained into the
class of piecewise constant signals. Truly equivalent reformulations can be pursued in a number of ways, depending on the
transformations allowed, in frequency or in time, on the input, state, and output signals.
For ease of explanation, it is advisable to focus on discrete
time, where the most important reformulations are timelifted, cyclic, and frequency-lifted representations.
The time-lifted reformulation goes back to early papers (29,
30). The underlying rationale is to sample the system state
with a sampling interval coincident with the system period T
and to organize the input and output signals in packed segments of subsequent intervals of length T, so as to form input
and output vectors of enlarged dimensions. That is, let be a
sampling tag and introduce the packed input and packed
output segments as follows:
u (k) = [u(kT + ) u(kT + + 1) . . . u(kT + + T 1) ]
y (k) = [y(kT + ) y(kT + + 1) . . . y(kT + + T 1) ]
The vectors u
( ) and y ( ) are known as lifted input and lifted
output signals. The introduction of the lifting concept enables
us to determine x(k 1) x(kT T ) from x(k)
x(kT ) and then to work out y (k) from x(k). More precisely,
define F Rnn, G RnmT, H RpTn, and E RpTmT as
B( + T 1)]
H = [C( )
A ( + 1, ) C( + 1) . . .
A ( + T 1, )C( + T 1) ]
E = {(E )ij },
(E )ij =
y (k) = H x (k) + E u (k)

Note that if u( ) is kept constant over the period, then the
state equation of this lifted reformulation boils down to the
sample-and-hold state equation.
In this reformulation, only the output and input vectors
were enlarged, whereas the dimension of the state-space was
preserved. In the cyclic reformulation, (31,32), every systems
signal, say v(t) of dimension q, is transformed into an enlarged signal v(t) of dimension qT. This transformation takes
place according to the following rule (given over one period
starting from a given initial instant ):
v( )

..
v ( ) = . ,
v( + 1)

v ( + 1) =
, . . .,
..
.
..
v ( + T 2) =
,
v( + T 2)

..
v ( + T 1) =
v( + T 1)
where is any element, typically set to zero. Obviously, the
previous pattern repeats periodically for the other periods.
This signal transformation is used for the input, output, and
state of the system. Then, we can relate the cyclic input to
the cyclic state by means of a time-invariant state-equation
and the cyclic output to the cyclic state via a time-invariant
transformation. In this way, we obtain an nT-dimensional
time-invariant system with mT inputs and pT outputs.
Finally, the frequency-lifted reformulation is based on the
following considerations. For a discrete-time (vector) signal
v(t), let V(z) be its z-transform. Now, one can associate with
V(z) the frequency augmented vector Vf (z) as follows:
F = A ( )
G = [A ( + T, + 1)B( ) A ( + T, + 2)B( + 1) . . .
x (k + 1) = F x (k) + G u (k)
i, j = 1, 2, . . ., T
i< j
D( + i 1)
i= j
C( + i 1) ( + i 1, + j)B( + j 1) i > j

A
V (z)
V (z)
V (z 2 )
V f (z) =
..
T 1
V (z
)
where e2j/T. By applying this procedure to the z-transforms of the input and output signals of the periodic system,
it is possible to establish an input-output correspondence described by a matrix transfer function; see Ref. 33. Such a
PERIODIC CONTROL
transfer function is referred to as the frequency-lifted representation.

The three reformulations are input-output equivalents of
each other. Indeed, for any pair of them it is possible to work
out a one-to-one correspondence between the input-output
signals. For the correspondence between the cyclic and the
time-lifted reformulations, see Ref. 22.
Lifting and Cycling in Continuous Time
In continuous time, the frequency-lifted reformulation can be
appropriately worked out as well leading to infinite-dimensional time-invariant systems. For example, the time-lifted
reformulation appears as in discrete time, but now G, H,
and E are linear operators on/from Hilbert spaces. On this
topic, the interested reader is referred to Refs. 34 and 35.
PERIODIC SYSTEMS IN FREQUENCY DOMAIN

The frequency domain representation is a fundamental tool
in the analysis and control of time-invariant linear systems.
It is related to the well-known property that, for this class of
systems, sinusoidal inputs result into sinusoidal outputs at
the same frequency and different amplitude and phase.
A similar tool can be worked out for periodic systems by
making reference to their response to the so-called exponentially modulated periodic (EMP) signals. Herein, we limit our
attention to continuous-time systems. Then, given any complex number s, a (complex) signal u(t) is said to be EMP of
period T and modulation s if
u(t) =
uk e s k t
kZ
65
Consider now the Fourier series for the periodic matrix coefficients. That is,
A(t) =
Ak e jkt
kZ
and similarly for B(t), C(t), and D(t), and plug the expansions
of the signals x(t), u(t), x(t) and the matrices A(t), B(t), C(t),
D(t) into the system equations. By equating all terms at the
same frequency, we obtain an infinite-dimensional matrix
equation of the following kind:
X = (A
A N )X
X + BU
sX
Y = CX + DU
where X , U , and Y , are doubly infinite vectors found with the
harmonics of x, u and y respectively, organized in the following fashion:
X
= [. . ., xT2 , xT1 , xT0 , xT1 , xT2 , . . .]
and similarly for U and Y . A , B , C , and D are doubly infinite

Toeplitz matrices formed with the harmonics of A( ), B( ),
C( ), and D( ), respectively, as
..
A =
..
.
A0
A1
A2
A3
A4
..
.
..
.
..
.
..
.
..
.
A1
A0
A1
A2
A3
..
.
A2
A1
A0
A1
A2
..
.
A3
A2
A1
A0
A1
..
.
A4
A3
A2
A1
A0
..
.
..
.
and similarly for B , C , and D . As for matrix N , it is the block

diagonal matrix
where
sk = s + jk
The quantity T 2/ is the named period of the EMP signal. The class of EMP signals is a generalization of the class
of T-periodic signals. As a matter of fact, an EMP signal with
s 0 is just an ordinary time-periodic signal. Indeed, as it is
easy to verify, an EMP signal is such that
u(t + T ) = u(t),
= esT
In much the same way as a time-invariant system subject

to a (complex) exponential input admits an exponential regime, a periodic system of period T subject to an EMP input
of the same period admits an EMP regime. In such a regime,
all signals of interest can be expanded as EMP signals as follows:
x(t) =
xk e s k t
kZ
x(t)
=
s k xk e s k t
kZ
y(t) =

kZ
yk e s k t
N = blkdiag{ jkI},
kZ
Then, we can define the harmonic transfer function as the

operator
G (s) = C [sI
I (A
A N )]1B + D
Such an operator provides a most useful connection between
the input harmonics and the output harmonics (organized in
the infinite vectors U and Y , respectively). In particular, if we
take s 0 (so considering the truly periodic regimes), the
appropriate input/output operator is
N A ]1B + D
G (0) = C [N
If u( ) is a sinusoid, this expression enables us to compute
the amplitudes and phases of the harmonics constituting the
output signal y( ) in a periodic regime.
In general, the input/output operator representation of a
periodic system may be somewhat impractical, given that it
is infinite-dimensional. From an engineering viewpoint, anyway, this model can be satisfactorily replaced by a finitedimensional approximation obtained by truncation of the
Fourier series of the system matrices, which in turn implies
66
PERIODIC CONTROL
that matrices A , B , C , D , and N also are truncated and have

therefore finite dimensions.
Analyzing the frequency domain behavior of a continuoustime periodic system in terms of the Fourier expansion of its
coefficients is a long-standing idea in the field; see Ref. 36 for
a classical reference and a more recent paper (37).
Interestingly enough, it can be shown that the discretetime version of this rationale leads to a finite-dimensional
time-invariant system whose transfer function coincides with
that of the frequency-lifted reformulation.
MONODROMY MATRIX AND STABILITY
The monodromy matrix A() relates the value of the state in
free motion at a given time-point to the value after one period T. Precisely, if u( ) 0 over the considered interval
of time,
x( + T ) = A ( )x( )
Therefore, the sampled state x(k) x( kT) is governed in
the free motion by the time-invariant discrete-time equation
x (k + 1) = A ( )x (k)
This is why the eigenvalues of A() play a major role in the
modal analysis of periodic systems. In the literature, such eigenvalues are referred to as the characteristic multipliers of
A( ). Note that, although the monodromy matrix may depend
upon , the characteristic multipliers are constant (21). Moreover, in continuous time, all characteristic multipliers are different from zero as can be easily seen from the Jacobi formula. Conversely, a discrete-time system may exhibit null
characteristic multipliers. This happens when at least one
among matrices A(i), i 0, 1, . . ., T 1 is singular, so that
the system is nonreversible.
Obviously the family of periodic systems includes that of
time-invariant ones, in which case the monodromy matrix
takes the expression

e AT
A ( ) =
AT
in continuous time
in discrete time
Therefore, denoting by an eigenvalue of A, the characteristic

multipliers of a time-invariant system seen as a periodic system of period T are given by eT and T in continuous time and
discrete time, respectively.
In general, the monodromy matrix is the basic tool in the
stability analysis of periodic systems. Indeed, any free motion
goes to zero asymptotically if and only if all characteristic
multipliers have modulus lower than one. Hence, a periodic
system (in continuous or discrete time) is stable if and only if
its characteristic multipliers belong to the open unit disk.
To be more precise, this stability concept is usually referred to as asymptotic stability. However, there is no need
for this article to introduce all possible notions of stability, so
the attribute asymptotic is omitted for the sake of conciseness.
Notice that there is no direct relation between the eigenvalues of A(t) and the system stability. In particular, it may
well happen that all eigenvalues of A(t) belong to the stable
region (i.e., the left half plane in continuous time and the unit
disk in discrete time) and nevertheless the system is unstable. Notable exceptions in continuous time are slowing-varying matrices or high-frequency perturbed matrices, see Refs.
38 and 39, respectively.
A celebrated stability condition can be formulated in terms
of the so-called Lyapunov equation.
There are two possible formulations of such an equation,
known as filtering Lyapunov equations and control Lyapunov
equation, reflecting the fact that Lyapunov equations may
arise in the analysis of both filtering and control problems. In
continuous time, the filtering Lyapunov equation takes the
form
P(t)
= P(t)A(t) + A(t)P(t) + Q(t)
and the control Lyapunov equation is
P(t)
= A(t) P(t) + P(t)A(t) + Q(t)
where Q( ) is a periodic [Q(t T) Q(t), t] and positive
definite [xQ(t)x 0, t, x 0] matrix.
It turns out that the continuous-time periodic system is
stable if and only if the Lyapunov equation (in any of the
two forms above) admits a (unique) periodic positive-definite
solution P( ).
An analogous result holds in discrete time, by making reference to
P(t + 1) = A(t)P(t)A(t) + Q(t)
P(t) = A(t) P(t + 1)A(t) + Q(t)
as filtering and control Lyapunov equations, respectively. As
before, Q( ) is periodic and positive definite.
The Lyapunov stability theorem can be expressed in a
more general form by referring to positive semidefinite matrices Q( ) provided that further technical assumptions on the
pair (A( ), Q( )) are met with; see Ref. 40 for more details on
the theoretical aspects and Ref. 41 for the numerical issues.
It is useful to point out that the above Lyapunov stability
condition can also be stated in a variety of different forms. In
particular, it is worth mentioning that one can resort to the
Lyapunov inequality, i.e., in continuous time
P(t)
> A(t)P(t) + P(t)A(t)
(filtering)
P(t)
> A(t) P(t) + P(t)A(t) (control)
and in discrete time
P(t + 1) > A(t)P(t)A(t)
(filtering)
P(t) > A(t) P(t + 1)A(t) (control)

Here it is meant that, given two square matrices M and N,
M N is equivalent to saying that M N is positive definite.
Then, an equivalent stability condition is that the system is
stable if and only if the Lyapunov inequality admits a periodic
positive definite solution. The advantage of expressing the
condition in this form is that no auxiliary matrix Q( ) is required.
PERIODIC CONTROL
Cyclostationary Processes
In the stochastic realm, the so-called cyclostationary processes
are well suited to deal with pulsatile random phenomena and
are the subject of intense investigation in signal processing;
see Ref. 42. Specifically, a stochastic process with periodic
mean and covariance function (t, ) satisfying the biperiodicity condition (t T, T) (t, ) is said to be a cyclostationary process. In particular, its variance (t, t) is T-periodic.
The periodic Lyapunov equation serves as a fundamental
tool in the analysis of these processes. Assume that the initial
state x(0) of the system is a random variable with zero mean
and covariance matrix P0, and assume also that the input of
the system is a white-noise process, independent of x(0), with
zero mean and unitary intensity. Then (21), under the stability assumption, the state of the periodic system asymptotically converges to a zero mean cyclostationary process with
variance (t, t), which can be computed via the periodic filtering Lyapunov equation by letting Q(t) B(t)B(t) and P(0)
P0. It turns out that
lim { (t, t) P(t)} = 0
Floquet Theory
One of the long-standing issues in periodic systems is
whether it is possible to find a state-coordinate transformation leading to a periodic system with constant dynamic matrix. In this way, the eigenvalues of such a dynamic matrix
would determine the modes of the system. With reference to
linear differential equations, this issue was considered by various mathematicians of the nineteenth century. Among them,
a prominent role was played by the French scientist Gaston
Floquet (18471920) who worked out a theory to solve linear
homogeneous periodic systems, which is now named after
him (43).
This theory can be outlined in a simple form as follows. If
S( ) is a T-periodic invertible state-space transformation,
(t)
x(t) S(t)x(t), then, in the new coordinates, the dynamic A
is given by

1
in continuous time
S(t)A(t)S(t)1 + S(t)S(t)
A(t) =
1
S(t + 1)A(t)S(t)
in discrete time
The Floquet problem is then to find S(t) (if any) in order to
(t) A
.
obtain a constant dynamic matrix A
In continuous time, it can be shown that such a transformation S( ) does exist, and the Floquet problem can be
can be obtained by solving eAT A(),
solved. Indeed, A
where is any given time point. The appropriate transformation S( ) is simply given by
S(t) = e A(t ) A ( , t)
67
tion, as can easily be seen in the simple case T 2, A(0)

0, A(1) 1, for which the equation S(t 1)A(t)S(t)1 constant does not admit any solution.
In the reversible case, such a representation always exists,
can be obtained by solving A
T A() and the
and matrix A
transformation S( ) is given by
S(t) = A t (t, )1
Again, it can be seen that such S( ) is periodic of period T
and satisfies the linear difference equation
1
S(t + 1) = AS(t)A(t)
with initial condition S() I.

Whenever a Floquet representation exists, the eigenvalues
are named characteristic exponents. In continuous time,
of A
the correspondence between a characteristic multiplier z and
a characteristic exponent s is z esT, whereas in discrete time,
such a correspondence is z sT.
Main references for the Floquet theory and stability issues
are Refs. 24, 36, 44, and 45. It should be emphasized that
Floquet theory does not consider systems driven by external
inputs. This nontrivial extension is touched upon in the
sequel.
PERIODIC CONTROL
The early developments of periodic control were concentrated
on the problem of forcing a periodic regime in order to improve the performance of an industrial plant (periodic optimization). At present, the term periodic control has taken a
wider sense, so as to include the design of control systems
where the controller and/or the plant are described by periodic models.
Periodic Optimization
In the 1970s, it was observed that there is evidence that periodic operation [of catalytic reactors] can produce more reaction products or more valuable distribution of products, [and
that] the production of wastes can perhaps be suppressed by
cycling (4). Ever since, the same idea has been further elaborated in other application fields, such as aeronautics
(5,46,47), solar energy control (48), and social and economic
sciences (49). This list of papers is largely incomplete, but in
the bibliography the interested reader can find many more
useful references.
In continuous time, the basic periodic optimization problem can be stated as follows. Consider the system
x(t)
= f (x(t), u(t))
y(t) = h(x(t))
subject to the periodicity constant
Such a matrix is indeed periodic of period T and satisfies the

linear differential equation
x(T ) = x(0)
S(t)
= AS(t)
S(t)A(t)
and to further constraints of integral or pathwise type. The

performance index to be maximized is
with initial condition S() I.

The discrete-time case is rather involved. Indeed, certain
nonreversible systems do not admit any Floquet representa-
J=
1
T
g(x(t), u(t)) dt
0
68
PERIODIC CONTROL
If we limit the desired operation to steady-state conditions,

then an algebraic optimization problem arises, which can be
tackled with mathematical programming techniques. Indeed,
letting u(t) const u and x(t) const x, the problem
becomes that of maximizing J g(x, u) under the constraint
f(x, u) 0. When passing from steady-state to periodic operations, an important preliminary question is whether the optimal steady-state regime can be improved by cycling or not.
Denoting by u0 the optimal input at the steady state, consider
the perturbed signal
u(t) = u 0 + u
where u(t) is a periodic perturbation. A problem for which
there exists a (nonzero) periodic perturbation with a better
performance is said to be proper. The issue of proper periodicity was originally dealt with in Refs. 50 and 51, by means of
calculus of variation concepts. The underlying rationale is to
express u( ) in its Fourier expansion:
u =
Uk e
k=
jkt
2
=
T
By means of variational techniques, it is possible to work out

a quadratic expression for the second variation of the performance index
2J =
Uk (k)Uk
k=
where U*k is the conjugate transpose of Uk. Matrix () is a

complex square matrix defined on the basis of the system
equations linearized around the optimal steady-state regime
and on the basis of the second derivatives of the so-called
Hamiltonian function associated with the optimal control
problem, again evaluated at the optimal steady-state regime.
Notice that () turns out to be a Hermitian matrix, namely
it coincides with its conjugate transpose [*() ()].
Thanks to the preceding expression of 2J, it is possible to
work out a proper periodicity condition in the frequency domain, known as -test. Basically, this test says that the optimal control problem is proper if, for some 0, () is partially positive [i.e., there exists a vector x 0 such that
x*()x 0]. The partial positivity of () is also a necessary
condition for proper periodicity if we consider the weak variations u( ). In the single-input single-output case, the test
can be given a graphical interpretation in the form of a circle criterion.
Variational tools have been used in periodic control by
many authors (see Refs. 5257). Moreover, along a similar
line, it is worth mentioning the area of vibrational control (see
Ref. 58), dealing with the problem of forcing a time-invariant
system to undertake a periodic movement in order to achieve
a better stabilization property or a better performance specification.
In general, if we leave the area of weak variations, periodic
optimization problems do not admit closed-form solutions.
There is, however, a notable exception, as pointed out in Refs.
59 and 60. With reference to a linear system, if the problem
is that of maximizing the output power

J=
1
T
y(t) y(t) dt
under the periodicity state constraint

x(0) = x(T )
and the input power constraint
1
T
u(t) u(t) dt 1
then the optimal input function is given by a sinusoidal signal

of suitable frequency. In the single-input single-output case,
denoting by G(s) the system transfer function, the optimal frequency is that associated with the peak value of the Bode
diagram
|G( j | |G( j)|,
(In modern jargon, is the value of the frequency associated
with the H norm of the system.) In particular, the problem
is proper if 0. Otherwise, the optimal steady-state operation cannot be improved by cycling.
Periodic Control of Time-Invariant Systems
The application of periodic controllers to time-invariant linear
plants has been treated in an extensive literature. Again, the
basic concern is to solve problems otherwise unsolvable with
time-invariant controllers or to improve the achievable control performances.
A typical line of reasoning adopted in this context can be
explained by referring to the classical output stabilization
problem, namely the problem of finding an algebraic feedback
control law based on the measurements of the output signal in
order to stabilize the overall control system. If the system is
x(t)
= Ax(t) + Bu(t)
y(t) = Cx(t)
with the control law
u(t) = Fy(t)
the problem is to find a matrix F (if any) such that the closedloop system
x(t)
= (A + BFC)x(t)
is stable. Although a number of necessary and sufficient conditions concerning the existence of a stabilizing matrix F have
been provided in the literature, no effective algorithms are
available for its determination, as discussed in Ref. 61. Moreover, it may be difficult or impossible to stabilize three linear
time-invariant plants (62). Periodic sampled control seems to
offer a practical way to tackle this problem. Indeed, consider
the time-varying control law based on the sampled measurements of y( )
u(t) = F (t)y(kT ),
t [kT, kT + T )
PERIODIC CONTROL
The modulating function F( ) and the sampling period T have

to be selected in order to stabilize the closed-loop system, now
governed by the equation
Periodic Control of Periodic Systems

A typical way to control a plant described by a linear periodic
model is to impose
x(kT + T ) = Ac x(kT )
u(t) = K(t)x(t) + S(t)v(t)

where K( ) is a periodic feedback gain [K(t T) K(t), t],
S( ) is a periodic feedforward gain [S(t T) S(t), t], and
v(t) is a new exogenous signal. The associated closed-loop system is then
where

Ac = e

AT
A(T )
BF ( )C d
The crucial point is the selection of matrix F( ) for a given

period T. A possibility, originally proposed in Ref. 28, is to
consider an F( ) given by the following expression
F (t) = B eA
(T t )
e A(T ) BB e A
(T )
69
x(t)
= (A(t) + B(t)K(t))x(t) + B(t)S(t)v(t)
in continuous time and
x(t + 1) = (A(t) + B(t)K(t))x(t) + B(t)S(t)v(t)
1
d
with matrix Z still to be specified. Note that this formula is

valid provided that the matrix inversion can be performed
(this is indeed the case under the so-called reachability condition). In this way, the closed-loop matrix Ac takes the form
Ac = e AT + ZC
Then, provided that some weak condition on the pair (A, C) is
met, period T and matrix Z can be selected so as to stabilize
Ac, (or, even, to assign its eigenvalues). The generalized sample-and-hold philosophy outlined previously in the simple
problem of stabilization has been pursued in many other contexts, ranging from the problem of simultaneous stabilization
of a finite number of plants (28) to that of fixed poles removal
in decentralized control (63), from the issue of pole and/or
zero-assignment (6469), to that of gain margin or robustness
improvement (7,70), from adaptive control (71) to model
matching (28), and so on.
When using generalized sample-data control, however, the
intersample behavior can present some critical aspects, as
pointed out in several papers, such as Refs. 72 and 73. Indeed,
the action of the generalized sample-and-hold function is a
sort of amplitude modulation, which, in the frequency domain, may lead to additional high-frequency components centered on multiples of the sampling frequency. Consequently,
there are nonnegligible high-frequency components both in
the output and control signals. To smooth out these ripples,
remedies have been studied, see Refs. 9 and 74. An obvious
possibility is to continuously monitor the output signal and to
adopt the feedback control strategy u(t) F(t)y(t), with a periodic gain F(t), in place of the sampled strategy before seen.
This point of view is adopted in Ref. in 75, where a poleassignment problem in discrete time is considered.
In the control of time-invariant systems, linear-quadratic
optimal control theory represents a cornerstone achievement
of the second half of the twentieth century. We can wonder
whether, by enlarging the family of controllers from the timeinvariant class to the class of periodic controllers, the achievable performance can be improved. To this question, the reply
may be negative, even in the presence of bounded disturbances, as argued in Ref. 76.
in discrete time. In particular, the closed-loop dynamic matrix

is the periodic matrix A(t) B(t)K(t).
The main problems considered in the literature follow:
1. Stabilization. Find a periodic feedback gain in such a
way that the closed-loop system is stable [any K( )
meeting such a requirement is named stabilizing gain].
2. Pole Assignment. Find a periodic feedback gain so as
to position the closed-loop characteristic multipliers in
given locations in the complex plane.
3. Optimal Control. Set v( ) 0 and find a periodic feedback gain so as to minimize the quadratic performance
index
J=
[x(t) Q(t)x(t) + u(t) R(t)u(t)] dt
in continuous time
x(k) Q(k)x(k) + u(k) R(k)u(k)
in discrete time
k=0
4. Invariantization. Find a feedback control law such that

the closed-loop system is time-invariant up to a periodic
state-space coordinate change.
5. Exact Model Matching. Let y(t) C(t)x(t) be a system
output variable. Find a feedback control law such that
the closed loop input-output relation [from v( ) to y( )]
matches the input-output behavior of a given periodic
system.
6. Tracking and Regulation. Find a periodic controller in
order to guarantee closed-loop stability and robust zeroing of the tracking errors for a given class of reference
signals.
We now briefly elaborate on these problems by reviewing the
main results available in the literature.
As a paradigm problem in control, the stabilization issue
is the starting point of further performance requirement problems. A general parametrization of all periodic stabilizing
gains can be worked out by means of a suitable matrix inequality. Specifically, by making reference to discrete time,
the filtering Lyapunov inequality seen in the section devoted
to the monodromy matrix and stability enables us to conclude
that the closed-loop system associated with a periodic gain
K( ) is stable if and only if there exists a positive definite
70
PERIODIC CONTROL
periodic matrix Q( ) satisfying the inequality:

Q(t + 1) > (A(t) + B(t)K(t))Q(t)(A(t) + B(t)K(t)) , t
(t) and D
(t) are to be tuned by the designer. In this
where C
way, the performance index can also be written in the (perhaps more popular) form
Then, it is possible to show that a periodic gain is stabilizing

if and only if it can be written in the form:
J(t, tf , xt ) = x(tf ) Pt x(tf ) +

f
tf
{x( ) Q( )x( )
+ 2u( ) S( )x( ) + u( ) R( )u( )} d
K(t) = W (t) Q(t)1

where W( ) and Q( ) 0 are periodic matrices (of dimensions
m n and n n, respectively), solving the matrix inequality
Q(t + 1) > A(t)Q(t)A(t) + B(t)W (t) A(t) + A(t)W (t)B(t)
+ B(t)W (t) Q(t)1W (t)B(t) , t
()C
(), S() D
(), and R()
()C
where Q() C
()D
(). We will assume for simplicity that the problem is
D
nonsingular [i.e., R() 0, ].
This problem is known as the linear quadratic (LQ) optimal control problem. To solve it, the auxiliary matrix equation
P(t) + P(t)A(t)
P(t)
= A(t)
P(t)B(t)R(t)1B(t) P(t) + Q(t)
that can be equivalently given in a linear matrix inequality

(LMI) form. The pole assignment problem (by state feedback)
is somehow strictly related to the invariantization problem.
Both problems have been considered in an early paper (77),
where continuous-time systems are treated, and subsequently
in Refs. 7880. The basic idea is to render the system algebraically equivalent to a time-invariant one by means of a first
periodic state feedback (invariantization) and then to resort
to the pole assignment theory for time-invariant systems in
order to locate the characteristic multipliers. Thus, the control scheme comprises two feedback loops, the inner for invariantization and the outer for pole placement.
Analogous considerations can be applied in the discretetime case (81), with some care for the possible nonreversibility of the system.
The model matching and the tracking problems are dealt
with in Refs. 82 and 83, respectively.
Finally, the optimal control approach to periodic control
deserves an extensive presentation and therefore is treated in
the next section.
is introduced. This is the well-known differential Riccati equation, in one of its many equivalent forms. More precisely, because the coefficients are periodic, the equation is referred to
as the periodic differential Riccati equation.
Let ( , tf) be the backward solution of the periodic Riccati
equation with terminal condition (tf, tf) Ptf . Assuming that
the state x( ) can be measured, the solution to the minimization problem can be easily written in terms of ( , tf) as follows
PERIODIC OPTIMAL CONTROL
Moreover, the value of the performance index associated with

the optimal solution is
As for the vast area of optimal control, attention focuses

herein on two main design methodologies, namely (i) linear
quadratic control and (ii) receding horizon control. Both will
be presented by making reference to continuous time.
Linear Quadratic Periodic Control
For a continuous-time periodic system, the classical finite horizon optimal control problem is that of minimizing the quadratic performance index over the time interval (t, tf):

J(t, tf , xt ) = x(tf ) Pt x(tf ) +

f
tf
z( ) z( ) d
where xt is the system initial state at time t, Ptf 0 is the

matrix weighting the final state x(tf), and z( ) is a performance evaluation variable. Considering that the second term
of J(t, tf, xt) is the energy of z( ), the definition of such a
variable reflects a main design specification. A common choice
is to select z(t) as a linear combination of x(t) and u(t), such
that
z(t) = C(t)x(t)
+ D(t)u(t)
where
A(t)
= A(t) B(t)R(t)1 S(t),
Q(t)
= Q(t) S(t) R(t)1 S(t)
u( ) = o ( , tf )x( )
where
o ( , tf ) = R( )1 [B( ) ( , tf ) + S( )]
J o (t, tf , xt ) = xt (t, tf )xt

The passage from the finite horizon to the infinite horizon
problem (tf ) can be performed provided that (t, tf) remains bounded for each tf t and converges as tf : In
other words, if there exists P(t) such that
lim (t, tf ) = P(t), t
tf
Under suitable assumptions concerning the matrices [A( ),

B( ), Q( ), S( )], the limit matrix P( ) exists and is the
unique positive semidefinite and T-periodic solution of the periodic differential Riccati equation. The optimal control action
is given by
u( ) = Ko ( )x( )
where Ko() is the periodic matrix obtained from o(, tf) by
letting tf . Finally, the optimal infinite horizon performance index takes on the value
lim J o (t, tf , xt ) = xt P(t)xt
tf
PERIODIC CONTROL
If the state is not accessible, we must rely instead on the measurable output
y(t) = C(t)x(t) + D(t)u(t)
A first task of the controller is then to infer the actual value
of the state x(t) from the past observation of y( ) and u( ) up
to time t. This leads to the problem of finding an estimate
x(t) of x(t) as the output of a linear system (filter) fed by the
available measurements. The design of such a filter can be
carried out in a variety of ways, among which it is worth mentioning the celebrated Kalman filter, the implementation of
which requires the solution of another matrix Riccati equation with periodic coefficients. When x(t) is available, the control action is typically obtained as
u( ) = Ko ( )x(
)
systems in Ref. 98. The problem can be stated as follows.

Consider the optimal control problem with S( ) 0 and
R( ) I, and write the performance index over the interval
(tf T, tf) as
J = x(tf ) Pt x(tf ) +
f
tf
tf T
Receding Horizon Periodic Control

The infinite horizon optimal control law can be implemented
provided that the periodic solution of the matrix Riccati equation is available. Finding such a solution may be computationally demanding so that the development of simpler control
design tools has been considered. Among them, an interesting
approach is provided by the so-called receding horizon control
strategy, which has its roots in optimal control theory and
remarkable connections with the field of adaptive and predictive control (see Refs. 95 and 96). Among the many research streams considered in such a context, the periodic stabilization of time-invariant systems is dealt with in Ref. 97
under the heading of intervalwise receding horizon control;
see also Ref. 96. The approach was then extended to periodic
{x( ) Q( )x( ) + u( ) u( )} d
Assume that the solution ( , tf) of the Riccati equation with

terminal condition (tf, tf) Ptf is such that
t = Pt (t f T, tf ) 0
f
This condition is usually referred to as cyclomonotonicity condition. Now, consider the periodic extension Pe( ) of ( , tf)
Pe (t + kT ) = (t, tf ),
Thus, the control scheme of the controller takes the form of a

cascade of two blocks, as can be seen in Fig. 3. Periodic optimal filtering and control problems for periodic systems have
been intensively investigated (see Refs. 8493). For numerical issues see, for example, Ref. 94.
71
t (tf T, tf ],
integer k
Then, under mild assumptions on [A( ), B( ), Q( )], it turns

out that the receding horizon control law
u( ) = B( ) Pe ( )x( )
is stabilizing. Although such a control law is suboptimal, it
has the advantage of requiring the integration of the Riccati
equation over a finite interval (precisely over an interval of
length T, which must be selected by the designer in the case
of time-invariant plants and coincides with the system periodor a multiplein the periodic case). However, for feedback stability, it is fundamental to check if the cyclomonotonicity condition is met. If not, we are led to consider a different
selection of matrix Ptf . Some general guidelines for the choice
of Ptf can be found in the literature. The simplest way is to
choose Ptf indefinitely large, ideally such that P1
tf 0. Indeed, as is well known in optimal control theory, such condition guarantees that the solution of the differential Riccati
equation enjoys the required monotonicity property.
CONCLUSION
Optimal periodic
gain
A(.), B( .), C( .), D(.)
(Control Riccati eq.)

Plant
x^
Periodic Kalman
filter
(Filtering Riccati eq.)
Overall periodic controller

Figure 3. Periodic optimal control based on the measurement of the
output signal. The controller is constituted of two blocks. The first
one (Kalman filter) elaborates the external signals of the plant (u
and y) to provide an estimate x of the unmeasureable state. The second block consists in an algebraic gain providing the command u from
the estimated state x .
Optimal control ideas have been used in a variety of contexts,

and have been adequately shaped for the needs of the specific
problem dealt with. In particular, ad hoc control design techniques have been developed for the rejection or attenuation of
periodic disturbances, a problem of major importance in the
emerging field of active control of vibrations and noise; see
Refs. 99 and 100.
BIBLIOGRAPHY
1. F. J. Horn and R .C. Lin, Periodic processes: A variational approach, I and EC Proc. Des. Development, 6: 2130, 1967.
2. J. E. Bailey and F. J. Horn, Improvement of the performances
of a fixed-bed catalytic reactor by relaxed steady-state operation,
AIChE J., 17: 550553, 1971.
3. J. E. Bailey, Periodic operation of chemical reactor: A review,
Chem. Eng. Commun., 1: 111124, 1973.
4. J. E. Bailey, The importance of relaxed control in the theory of
periodic optimization, in A. Marzollo, (ed.), Periodic Optimization, pp. 3561, New York: Springer-Verlag, 1972.
72
PERIODIC CONTROL
5. J. L. Speyer, On the fuel optimality of cruise, AIAA J. Aircraft,
10: 763764, 1973.
6. G. Sachs and T. Christopoulou, Reducing fuel consumption of
subsonic aircraft by optimal cycling cruise, AIAA J. Aircraft, 24:
616622, 1987.
7. P. P. Khargonekar, K. Poolla, and A. Tannembaum, Robust control of linear time-invariant plants using periodic compensation,
8. M. A. Dahleh, P. G. Vulgaris, and L. S. Valavani, Optimal and
robust controllers for periodic and multirate systems, IEEE
9. M. Araki, Recent developments in digital control theory, 12th
IFAC World Cong., Sidney, vol. 9, 251260, 1993.
10. W. Johnson, Helicopter Theory, Princeton, NJ: Princeton University Press, 1981.
11. D. P. Schrage and D. A. Peters, Effect of structural coupling of
parameters on the flap-lag forced response of rotor blades in
forward flight using Floquet theory, Vertica, 3 (2): 177185,
1979.
12. R. M. McKillip, Periodic control of individual blade control helicopter rotor, Vertica, 9 (2): 199225, 1985.
13. D. Teves, G. Niesl, A. Blaas, and S. Jacklin, The role of active
control in future rotorcraft, 21st Eur. Rotorcraft Forum, Saint
Petersburg, Russia, III.10.117, 1995.
14. W. Wiesel, Perturbation theory in the vicinity of a stable periodic orbit, Celestial Mechanics, 23: 231242, 1981.
15. J. Hauser and C. C. Chunt, Conserse Lyaupov functions for exponentially stable periodic orbits, Systems and Control Letters,
23: 2734, 1994.
16. C. C. Chung and J. Hauser, Nonlinear H control around periodic orbits, Syst. Control Lett., 30: 127137, 1997.
17. G. O. Guardabassi, A. Locatelli, and S. Rinaldi, Status of periodic optimization of dynamical systems, J. Opt. Theory Appl., 14
120, 1974.
18. E. Noldus, A survey of optimal periodic control of continuous
systems, Journal A, 16: 1116, 1975.
19. G. Guardabassi, The optimal periodic control problem, Journal
A, 17: 7583, 1976.
20. S. Bittanti and G. O. Guardabassi, Optimal periodic control and
periodic system analysis: An overview, 25th IEEE Conf. Decision
Control, pp. 14171423, 1986.
21. S. Bittanti, Deterministic and stochastic periodic systems, in S.
Bittanti (ed.), Time Series and Linear Systems, Lecture Notes in
Control and Information Sciences, vol. 86, pp. 141182, Berlin:
22. S. Bittanti and P. Colaneri, Analysis of discrete-time linear periodic systems, in C. T. Leondes (ed.), Control and Dynamic Systems, vol. 78, San Diego: Academic Press, 1996.
23. A. Marzollo, Periodic Optimization, vols. 1, 2, New York:
24. V. A. Yakubovich and V. M. Starzhinskii, Linear Differential
Equations with Periodic Coefficients, New York: Wiley, 1975.
25. T. Kaczorek, Two-Dimensional Linear Systems, New York:
26. S. Bittanti, P. Bolzern, and G. O. Guardabassi, Some critical
issues on the state representation of time-varying ARMA models, IFAC Symp. Identification System Parameter Estimation,
York, pp.15791583, 1985.
27. P. Colaneri and S. Longhi, The minimal realization problem for
discrete-time periodic systems, Automatica, 31: 779783, 1995.
28. P. T. Kabamba, Control of linear systems using generalized
sampled-data hold functions, IEEE Trans. Autom. Control, 32:
772783, 1987.
29. E. I. Jury and F. J. Mullin, The analysis of sampled-data control

systems with a periodic time-varying sampling rate, IRE Trans.
Autom. Control, 24: 1521,1959.
30. R. A. Mayer and C. S. Burrus, Design and implementation of
multirate digital filters, IEEE Trans. Acoust. Speech Signal Process., 1: 5358, 1976.
31. B. Park and E. I. Verriest, Canonical forms for discrete-linear
periodically time-varying systems and a control application,
28th IEEE Conf. Decision Control, Tampa, pp. 12201225,1989.
32. D. S. Flamm, A new shift-invariant representation for periodic
systems, Syst. Control Lett., 17: 914, 1991.
33. C. Zhang, J. Zhang, and K. Furuta, Performance analysis of periodically time-varying controllers, 13th IFAC World Cong., San
Francisco, pp. 207212, 1996.
34. P. Colaneri, Hamiltonian matrices for lifted systems and periodic Riccati equations in H2 /H analysis and control, 30th IEEE
Conf. Decision and Control, Brighton, UK, pp. 19141919, 1991.
35. B. A. Bamieh and J. B. Pearson, A general framework for linear
periodic systems with application to H sampled data control,
36. H. DAngelo, Linear Time-varying Systems: Analysis and Synthesis, Boston: Allyn and Bacon, 1970.
37. N. M. Wereley and S. R. Hall, Frequency response of linear time
periodic systems, 29th IEEE Conf. Decision Control, Honolulu,
1990.
38. C. A. Desoer, Slowing varying systems x A(t)x, IEEE Trans.
Autom. Control, 14: 780781, 1969.
39. R. Bellman, J. Bentsman, and S. M. Meerkov, Stability of fast
periodic systems, IEEE Trans. Autom. Control, 30: 289291,
1985.
40. P. Bolzern and P. Colaneri, The periodic Lyapunov equation,
SIAM J. Matrix Anal. Application, 9: 499512, 1988.
41. A. Varga, Periodic Lyapunov equations: Some application and
new algorithms, Int. J. Control, 67: 6987, 1997.
42. W. A. Gardner (ed.), Cyclostationarity in Communications and
Signal Processing, New York: IEEE Press, 1994.
43. G. Floquet, Sur les equations differentielles lineaires a coefficients periodiques, Annales de lEcole Normale Superieur, 12:
4789, 1883.
44. A. Halanay, Differential EquationsStability, Oscillations,
Time-lags, New York: Academic Press, 1986.
45. P. Van Dooren and J. Sreedhar, When is a periodic discretetime system equivalent to a time-invariant one?, Linear Algebra
and its Applications, 212/213: 131151, 1994.
46. E. G. Gilbert, Periodic control of vehicle cruise: Improved fuel
economy by relaxed steady-state, quasi steady-state and quasi
relaxed steady-state control, Automatica, 12: 159166, 1976.
47. J. Speyer, Non-optimality of steady-state cruise for aircraft,
AIAA J., 14: 16041610, 1976.
48. P. Dorato and H. K. Knudsen, Periodic optimization with applications to solar energy control, Automatica, 15: 673679, 1979.
49. G. Feichtinger and A. Novak, Optimal pulsing in advertising
diffusion models, Optim. Control Appl. Meth., 15: 267276, 1994.
50. G. Guardabassi, Optimal steady-state versus periodic operation:
A circle criterion, Ricerche di Automatica, 2: 240252, 1971.
51. S. Bittanti, G. Fronza, and G. Guardabassi, Periodic control: A
frequency domain approach, IEEE Trans. Autom. Control, 18:
3338, 1973.
52. S. Bittanti, A. Locatelli, and C. Maffezzoni, Second variation
methods in periodic optimization, J. Opt. Theory Appl., 14: 31
49, 1974.
53. E. G. Gilbert, Optimal periodic control: A general theory of necessary conditions, SIAM J. Control Optimization, 15: 717746,
1977.
PERIODIC CONTROL
54. D. S. Bernstein and E. G. Gilbert, Optimal periodic control: The
-test revisted, IEEE Trans. Autom. Control, 25: 673684, 1980.
55. J. Speyer and R. T. Evans, A second variational theory of optimal periodic processes, IEEE Trans. Autom. Control, 29: 138
148, 1984.
56. Q. Wang and J. Speyer, Optimal periodic control: A general theory of necessary conditions, SIAM J. Control and Optimization,
15: 717746, 1990.
57. Q. Wang, J. Speyer, and L. D. Dewell, Regulators for optimal
periodic processes, IEEE Trans. Autom. Control, 40: 1777
1778, 1995.
58. S. Lee, S. Meerkov, and T. Runolfsson, Vibrational feedback control: Zero placement capabilities, IEEE Trans. Autom. Control,
32: 604611, 1987.
59. S. Bittanti, G. Fronza, and G. Guardabassi, Periodic optimization of linear systems under control power constraints, Automatica, 9 (1): 269271, 1973.
60. S. Bittanti and G. Guardabassi, Optimal cyclostationary control:
A parameter-optimization frequency domain approach, VIII
IFAC World Congr., 6: 191196, 1981.
61. V. L. Syrmos, C. T. Abdolloh, D. Doreto, and K. Grigoriedis,
Static output feedbackA survey, Automatica, 33: 125137,
1997.
62. V. Blondel and M. Gevers, Simultaneous stabilization of three
linear systems is rationally undecidible, Math. Control, Signals
Syst., 6: 135145, 1994.
63. B. D. O. Anderson and J. Moore, Time-varying feedback laws
for decentralized control, IEEE Trans. Autom. Control, 26: 1133
1139, 1981.
64. A. B. Chammas and C. T. Leondes, On the design of linear timeinvariant systems by periodic output feedback, Int. J. Control,
27: 885903, 1978.
65. B. A. Francis and T. T. Georgiou, Stability theory for linear
time-invariant plants with periodic digital controllers, IEEE
66. H. M. Al-Rahmani and G. F. Franklin, Linear periodic systems:
Eigenvalues assignment using discrete periodic feedback, IEEE
67. A. B. Chammas and C. T. Leondes, Pole assignment by
piecewise constant output feedback, Int. J. Control, 44: 1661
1673, 1986.
68. J. L. Willems, V. Kucera, and P. Brunovski, On the assignment
of invariant factors by time-varying feedback strategies, Syst.
Control Lett., 5: 7580, 1984.
69. T. Kaczorek, Pole placement for linear discrete-time systems by
periodic output feedbacks, Syst. Control Lett., 6: 267269, 1985.
70. A. W. Olbrot, Robust stabilization of uncertain systems by periodic feedback, Int. J. Control, 45 (3): 747758, 1987.
71. R. Ortega and G. Kreisselmeier, Discrete-time model reference
adaptive control for continuous time systems using generalized
sample data hold functions, IEEE Trans. Autom. Control, 35:
334338, 1990.
72. K. L. Moore, S. P. Bhattacharyya, and M. Dahleh, Capabilities
and limitations of multirate control schemes, Automatica, 29:
941951, 1993.
73. A. Feuer and C. A. Goodwin, Generalized sampled-data functions: Frequency domain analysis of robustness, sensitivity, and
intersample difficulties, IEEE Trans. Autom. Control, 39: 1042
1047, 1994.
74. A. Feuer, C. A. Goodwin, and M. Salgado, Potential benefits of
hybrid control for linear time-invariant plants, Amer. Control
Conf., Albuquerque, 1997.
73
75. D. Aeyels and J. Willems, Pole assignment for linear time-invariant systems by periodic memoryless output feedback, Automatica, 28: 11591168, 1992.
76. H. Chapellat and M. Dahleh, Analysis of time-varying control
strategies for optimal disturbance rejection and robustness,
77. P. Brunovski, A classificaton of linear controllable systems, Kybernetika, 3: 173187, 1970.
78. M. Kono, Pole-placement problem for discrete-time linear periodic systems, Int. J. Control, 50 (1): 361371, 1989.
79. V. Hernandez and A. Urbano, Pole-placement problem for discrete-time linear periodic systems, Int. J. Control, 50: 361
371, 1989.
80. P. Colaneri, Output stabilization via pole-placement of discretetime linear periodic systems, IEEE Trans. Autom. Control, 36:
739742, 1991.
81. O. M. Grasselli and S. Longhi, Pole placement for nonreachable
periodic discrete-time systems, Math. Control, Signals Syst., 4:
437453, 1991.
82. P. Colaneri and V. Kucera, The model matching problem for discrete-time periodic systems, IEEE Trans. Autom. Control, 42:
14721476, 1997.
83. O. M. Grasselli and S. Longhi, Robust tracking and regulation
of linear periodic discrete-time systems, Int. J. Control, 54: 613
633, 1991.
84. J. R. Canabal, Periodic geometry of the Riccati equation, Stochastics, 1: 432435, 1974.
85. E. C. Bekir and R.S. Bucy, Periodic equilibria for matrix Riccati
equations, Stochastics, 2: 1104, 1976.
86. H. Kano and T. Nishimura, Periodic solutions of matrix Riccati
equations with detectability and stabilizability, Int. J. Control,
29: 471481, 1979.
87. S. Bittanti, P. Colaneri, and G. Guardabassi, Periodic solutions
of periodic Riccati equations, IEEE Trans. Autom. Control, 29:
665667, 1984.
88. V. Hernandez and L. Jodar, Boundary problems for periodic Riccati equations, IEEE Trans. Autom. Control, 30: 11311133,
1985.
89. M. A. Shayman, On the phase portrait of the matrix Riccati
equation arising from the periodic control problem, SIAM J.
Control Optim., 23: 717751, 1985.
90. S. Bittanti, P. Colaneri, and G. Guardabassi, Analysis of the
periodic Riccati equation via canonical decomposition, SIAM J.
Control Optim., 24: 11381149, 1986.
91. C. E. de Souza, Riccati differential equation in optimal filtering
of periodic non-stabilizing systems, Int. J. Control, 46: 1235
1250, 1987.
92. S. Bittanti, P. Colaneri, and G. De Nicolao, The difference Riccati equation for the periodic prediction problem, IEEE Trans.
Autom. Control, 33: 706712, 1988.
93. S. Bittanti, A. J. Laub, and J. C. Willems (eds.), The Riccati
Equation, Berlin: Springer Verlag, 1991.
94. J. J. Hench and A. J. Laub, Numerical solution of the discretetime periodic Riccati equation, IEEE Trans. Autom. Control, 39:
11971210, 1994.
95. D. W. Clarke, C. Mohtadi, and P. C. Tuffs, Generalized predictive control: part 1The basic algorithm; part 2Extensions
and interpolators, Automatica, 23: 137160, 1987.
96. R. R. Bitmead, M. R. Gevers, I. R. Petersen, and R. J. Kaye,
Monotonicity and stabilizability properties of solutions of the
Riccati difference equation: Propositions, lemmas, theorems, fallacious conjectures and counterexamples, Syst. Control Lett., 5:
309315, 1986.
74
PERIODIC NONLINEAR CIRCUITS
97. W. H. Kwon and A. E. Pearson, Linear systems with two-point

boundary Lyapunov and Riccati equations, IEEE Trans. Autom.
Control, 27: 436441, 1982.
98. G. De Nicolao, Cyclomonotonicity, Riccati equations and periodic receding horizon control, Automatica, 30: 13751388, 1994.
99. S. Bittanti and M. Lovera, A discrete-time periodic model for
helicopter rotor dynamics, 10th IFAC Symp. Syst. Identification,
2: 577582, 1994.
100. P. Arcara, S. Bittanti, and M. Lovera, Periodic control of helicopter rotors for attenuation of vibrations in forward flight, IEEE
Trans. Control Syst. Technol., 1998 (in press).
S. BITTANTI
P. COLANERI
Politecnico di Milano
446
PID CONTROL
PID CONTROL
PID control strategies are by far the most widely employed of
all strategies used in the automatic feedback control of industrial systems. The acronym PID stands for proportional, integral, derivative. Although the central concept of PID control
can be gleaned from an examination of how these three terms
are blended to form a control signal, the intelligent application of PID control in any given case requires an understanding of linear and nonlinear properties of the system that is
being controlled, and practical PID controllers incorporate
features such as bumpless transfer and anti reset windup.
The proportional term of the PID controller forms a part
of the control signal based on a proportional gain times a system error. The derivative term of the PID controller forms a
part of the control signal based on a gain times the rate of
change of a system error. Similarly, the integral term of the
PID controller forms a part of the control signal based on a
gain times the integral of a system error. This integral term
basically forces the error from a nonzero value (a value that
would exist without integral action present) to a zero value
the integral term in effect resets the error to zero. Because of
this phenomenon, the integral term of the PID controller is
sometimes called the reset term. This terminology is especially prevalent in older literature; Eckman, for example, uses
reset consistently in place of integral (1).
Much of this article is devoted to practical issues of PID
controllers. Several alternative forms of PID algorithms are
examined, and their relative merits are discussed. Rules for
tuning the parameters of the controller are considered, the
literature base devoted to this topic is exceedingly large. Tuning rules are concerned with ways of assigning controller parameters (proportional gain, integral gain, and derivative
gain) to achieve good performance based on various process
model assumptions. That there are a variety of tuning rules
should come as no surprise; generally there are tradeoffs to be
made in alternative performance criteria and different tuning
rules weight some performance criteria more than others.
Tuning rules tend to emphasize system performance in the
neighborhood of an operating point, and therefore generally
are based on linearized models of the process being controlled.
Tuning rules are but one aspect of PID controller design.
Of equal importance is how the system performs when large
input changes occur. For most systems, the control signal input to the system is limited by maximum and minimum
bounds; when the control signal is saturated at one of these
bounds for any length of time, the error between the desired
system output and the actual system output can be quite
large. Within the PID controller, this error signal is being integrated, and unless some mechanism is employed to control
the integration process (either by selectively turning it off and
on or by selectively forcing the integrator output to track an-
other signal) the output by the PID integrator can literally

wind up to exceedingly large values, requiring large periods
of time to unwind and leading to unacceptably long transient
response. There are a variety of ways to counter integrator
wind up, all of which are classified under the heading of antireset-windup strategies; some of the more common ones are
described in this article.
Prior to the 1960s, most PID controllers were implemented
with analog techniques. With the steady advance in digital
controller technology since that time, the majority of industrial PID controllers are now implemented digitally to take
advantage of flexibility of programming and decreased sensitivity to environmental changes. A major section of this article therefore is devoted to digital implementation issues.
PID controllers are the workhorses of industrial process
control, and they provide excellent results under a variety of
conditions. There are circumstances, however, where other
controllers (or at least enhanced PID controllers) offer superior results. Systems that are highly oscillatory often are difficult to control with PID alone, as are systems that exhibit
long pure time delays. Systems having three dominant modes
of comparable time constants generally can be controlled better by using a third-order controller, perhaps one that augments the desirable properties of a basic PID controller.
This article is arranged in sections: (1) characteristics of
controlled processes; (2) the essence of the PID terms; (3) alternative PID forms; (4) practical derivative action; (5) velocity or incremental PID form; (6) proportional band; (7) antireset-windup; (8) bumpless transfer; (9) digital PID control
(with subsections on analogous implementation of digital
PID, incremental digital PID, recursive digital PID, signal
property considerations, other digital PID issues, and an emulation method of digital PID design); and (10) PID tuning.
CHARACTERISTICS OF CONTROLLED PROCESSES

Before a controller for a process is specified, the process to be
controlled should be characterized, at least in some broad
sense. There are many different types of processes that are
controlled automatically. Examples range from fluid levels in
tanks to readwrite heads of computer disk storage devices.
The control inputs to a process are supplied through one or
more actuators. For example a motor driven valve can be an
actuator for fluid flowing into a tank. Process outputs that are
being controlled (e.g., the fluid level in a tank) are measured
using appropriate sensors. The basic control of one output
variable by the use of one control input variable is called single-input single-output (SISO) control. For more complex systems, multi-input multi-output (MIMO) control may be required. PID control was developed initially as an SISO control
strategy, but it has been extended in various ways to MIMO
control.
Consider a process that responds to an increase in the control signal by having an increase in the controlled output. The
output response generally is delayed from that of the controlled input because of time lags caused by system dynamics.
Both dynamic and static characteristics of the process are of
interest. Static (steady-state) characteristics of the process often can be measured in the following way: first, the control
input is set at a specific value; next, after the output of the
process has settled to a steady-state value, this steady-state
PID CONTROL
value is tabulated next to the associated input value; and the

preceding two steps are then repeated over the entire range
of inputs that are realizable to create a steady-state or static
inputoutput plot. Although static characteristics generally
are nonlinear, they often exhibit regions of operation over
which linear approximations apply, and it is possible to include nonlinear gain in the control to broaden the effective
linear range of operation.
The preceding approach must be modified in those cases
where a constant control input results, after transients have
settled, in a constant rate of change in the controlled output.
In that case, the process contains a pure integration term,
and the static characteristic of interest is a plot of the rate
of change of the controlled output as a function of the control
input. Disturbance signals can also influence the behavior of
a process. In the static measurements described in the preceding paragraph, it is assumed that disturbance signals have
had negligible influence.
The controller for an SISO system is supplied with realtime information concerning the relationship of the controlled
output of the system to some desired controlled output. The
desired controlled output generally is called the reference input to the system. It is preferable to have the controller supplied with real-time values of both the reference input r(t) and
the controlled output c(t); in some cases, however, only the
error signal e(t), e(t) r(t) c(t), is available to the controller.
The control problem is classified as a regulator problem if the
reference input remains constant for long periods of time, and
the controller strives to maintain the controlled output at a
constant value in spite of disturbance inputs. The control
problem is classified as a tracking problem if the controller is
required to make the controlled output track a time-varying
reference input such that the error between the two is maintained near zero, even in the presence of disturbance inputs.
PID controllers are used in both regulator and tracking control applications. As with other types of controllers, PID controllers also are expected to reduce the sensitivity of the controlled system to changes in parameters in the process being
controlled.
THE ESSENCE OF THE PID TERMS
Figure 1 shows the PID parts of a PID-controlled process (the
PID parts alone do not constitute a practical PID controller,
as will be evident later). The essence of how the PID parts
work is as follows.
The Proportional Term
First consider the proportional term kpe(t) with proportional
gain kp being a positive constant, and assume that only the
P
r(t)
e(t)
I
kpe(t)
+
ki tedt +
+
d(t)
+
u(t)
+
c(t)
Process
kdde/dt
Figure 1. A simplified version of PID control, with one degree of

control freedom and without anti-reset-windup protection.
447
proportional term is active (temporarily assume ki kd 0

and disturbance input d(t) 0). In this case, the PID controller reduces to a proportional controller (P controller). With
the system initially in equilibrium, suppose that the reference
input r(t) increases abruptly from zero to a constant steadystate level rss. The controlled output is then less than the reference input causing an increase in the error e(t). The corresponding positive control signal kpe(t) results in an increase
in the controlled output (the increase is not instantaneous
and depends on the static and dynamic characteristics of the
process). Assuming that the closed-loop system is stable, the
error signal e(t) approaches a steady-state level ess such that
ess = rss Ks kp ess
(1)
where the static process gain Ks generally is dependent on the

value of kpess. Note that Eq. (1) can be rearranged to obtain
ess as follows:
ess = rss /(1 + Ks kp )
(2)
Assuming Ks 0, it would appear that ess could be made arbitrarily small by making kp arbitrarily large; however, for any
given system, an upper limit ku on kp invariably exists such
that the closed loop system is unstable for kp ku, where the
ultimate value ku depends on the system dynamics.
The Integral Term
The steady-state error described above generally can be reduced to zero by the inclusion of the integral term. With ki
0, kp 0, and kd 0, the controller of Fig. 1 is a PI controller.
In this case, consider what would happen to the integrator
output if it started at 0 at t 0, and the error signal e was
constant at e ess 0; the integrator output would then
equal kiesst, but this increases indefinitely as t increases! The
logical conclusion is that ess must be zerothe output of the
integrator is part of the control signal u(t), and as u(t) increases, so also does the controlled output c(t) to the point
where rss css ess 0 thereby inhibiting further increases
in the output of the integrator.
The Derivative Term
In Fig. 1, if ki 0 with kp 0 and kd 0, the controller is a
PD controller. If all three gain terms are nonzero, the controller is a three-term PID controller. The kd term plays the role
of an anticipatory or predictive element. That is, even if the
e(t1) is zero, if e(t1) de(t1)/dt is large, the implication is that
the error for t t1 is going to increase. By including kde in
the control, the controlled output is forced to increase sooner
than it would otherwise, with the goal of reducing the anticipated future error. This action is useful in compensating for
dynamic lags that invariably are present in the process being
controlled. However, if significant measurement noise is present in the measured output signal, the kd term can have a
detrimental effect on performance (the derivative of rapidly
changing high-frequency noise can be extremely large even if
the magnitude of the noise is small). Because of this, and because the output of any real circuit is band-limited, practical
implementation of the kd term requires the use of a frequencydependent differentiator, as is described later in this article.
448
PID CONTROL
Performance Objectives
The values of kp, ki, and kd should be selected to provide good
system response under design conditions, and to reduce
harmful effects of process changes that lead to off-design conditions. System response is often described by step response
characteristics such as: (1) the 1090% rise time resulting
from a step change in the reference input; (2) the percent
overshoot in response to a step input; (3) the peak time associated with the peak overshoot in response to a step input; and
(4) the time required for the system output to settle to within
2% of its final value in response to a step input. Of interest
are response characteristics caused by process disturbances
(represented by d(t) in Fig. 1) in addition to responses activated by reference inputs r(t). Note that for constant disturbance values, with d(t) dss, the reason given in the previous
Subsection on the integral term for ess being 0 still holds, and
the controlled output c(t) approaches the steady-state reference input rss independent of the dss value, assuming of course
that practical upper and lower bounds on the output of the
integrator are not exceeded.
If the reference input, in addition to the error signal is
available to the controller, a modified reference input may be
beneficial. For example, if a controlled output exhibits too
much overshoot in response to a step change in a reference
input, a rate-limited version of the reference input can be generated and inserted in place of the actual reference input.
Conversely, if a controlled output reacts too slowly in response to a change in a reference input, a rate-enhanced version of the reference input can be generated and inserted in
place of the actual reference input. In some applications, future values of a reference input are available to the controller;
in machine-tool control, for example, a desired contour to be
traversed is known in advance. This look-ahead or preview
information can be used to reduce the error between the reference input and the controlled output.
ALTERNATIVE PID FORMS
The control action of Fig. 1 is expressed as
u(t) = kp e(t) + ki
t
0
e() d + kd
de(t)
dt
(3)
where kp is the proportional gain, ki is the integral gain, and

kd is the derivative gain. In terms of Laplace transforms, the
Laplace transform transfer function associated with Eq. (3) is
Gc (s)
(s)
k
= kp + i + kd s
, UE(s)
s
(4)
The above form of the PID terms is called the parallel form.
An alternative form to that of Eq. (4) is called the standard
or noninteracting form and is characterized by

1
+ Td s
Gc (s) = K 1 +
(5)
Ti s
with the obvious relationships kp K, ki K/Ti, and kd
KTd. Although these two forms are equivalent, the standard
form of Eq. (5) uses different terminology: the Ti constant is
called the integral time; and the Td constant is called the de-
rivative time. Either Eq. (4) or Eq. (5) can be arranged as a

ratio of polynomials in s. The two roots of the numerator polynomial [the zeros of Gc(s)] can be assigned to arbitrary real or
complex conjugate values by proper assignment of the PID
gains.
Another commonly used form of the basic PID is the series
or interacting form:

1

1 + sTd
Gc (s) = K 1 +
(6)

sTi
Note that the zeros of Eq. (6) are restricted to be real, and
therefore the Gc(s) of Eq. (6) is less general than those of Eqs.
(4) or (5). However, this series form can be augmented in a
simple way to counter windup (to be described in the section
on anti-reset-windup). Also, from a historical perspective, the
series form was more readily implemented in early pneumatic
and hydraulic equipment, some of which is still in use.
Because there are several alternative forms of PID, manufacturers of PID controllers do not always use the same terminology. It is important therefore to determine the terminology
and the particular PID controller forms adopted by suppliers
of PID controller equipment under consideration. For consistency in this article, however, the parallel form of Eq. (4) is
emphasized.
TWO-DEGREE-OF-FREEDOM PID
Figure 2 is a block diagram of a linear system model of a PIDcontrolled system. Anti-reset-windup and bumpless-transfer
features are not included in the linear model. In Fig. 2, G(s)
is the Laplace transform transfer function associated with a
particular operating point of the process. Note that the integral action operates on the error signal e(t) r(t) c(t) as
previously described, but that now different proportional and
derivative gains apply to the reference input r(t) and the controlled output c(t). In terms of Laplace transforms of the associated signals,

k E(s)
U (s) = kp R(s) kpC(s) + s kd R(s) kdC(s) + i
s
(7)
Whereas the feedback system of Fig. 1 has one degree of control freedom, that of Fig. 2 has two degrees of control freedom:
(1) the kp, ki, and kd gains can be selected to achieve desired
goals regarding closed-loop stability, disturbance rejection,
and sensitivity reduction; and (2) the kp and kd gains can be
selected to achieve performance goals associated with the re-
PD
R(s)
E(s)
D(s)
ki
s
+
+
U(s)
+
PD
C(s)
G(s)
+
+
N(s)
Figure 2. Linear system features of two-degree-of-freedom PID

control.
PID CONTROL
sponse of the system to the reference input. If G(s) in Fig. 2

were a second-order system, with c(t) and c(t) viewed as state
variables, the PID control form in Fig. 2 would be exactly the
control form that would be expected from a modern state-variable viewpoint. For higher-order systems, the c(t) and c(t)
states are often the most significant ones, so that PID control
in many cases can be tuned to give excellent results. Clearly,
however, there are cases where higher-order transfer functions should be used in place of the PD and PD terms of
Fig. 2.
ep
ed
449
k ps
1 + ps
+
ki
1
s
u(t)
Integrator as
part of actuator
k ps 2
(1 + ds2)
PRACTICAL DERIVATIVE ACTION

Figure 3. Linear aspects of a velocity form PID controller.
The derivative term of the PID controller is often described

as simply kds. However, this term has an output response of
A cos(t) to the sinusoidal input A sin(t). For large values
of , corresponding to high-frequency noise, the response A
cos(t) can be overly large even if the input amplitude A is
reasonable. Thus, not only is pure derivative action not
achievable, it generally is undesirable and cannot be tolerated
in practical systems. The kds term invariably is replaced by a
filtered version, as follows:
Gd (s) =
kd s
1 + s
(8)
where the time-constant is chosen such that measurement

noise [N(s) in Fig. 2] does not have a significant impact on the
control action U(s). The transfer function from N(s) to U(s) is
GNU (s)
(s)
GPID (s)
=
, UN(s)
1 + G (s)G(s)
(9)
PID
where
GPID (s) = kp +
kd s
ki
+
s
1 + s
(10)
In Eq. (9), at high frequencies where both 1 and

G( j) 0, we have

k
k
GNU ( j) GPID ( j) kp + i + d
j
(11)
The kd / term in Eq. (11) replaces a corresponding term jkd

that would have resulted from the use of the pure derivative
form. The PID transfer function of Eq. (10) has four adjustable parameters and is therefore a four-term compensator.
VELOCITY OR INCREMENTAL PID FORM

In some applications, the integral part of the PID controller
is an attribute of the actuator that drives the process. In that
case, the velocity form of PID is appropriate, as displayed
within the dashed lines of Fig. 3. Note that time constants p

and d are included to keep the high-frequency gain of the
controller bounded. At low frequencies, the output of the top
block in Fig. 3 is approximately kpep(t), where
ep (t)
, kkp r(t) c(t)
(12)
and the low-frequency output of the lower block in Fig. 3 is

approximately kded(t), where
ed (t)
, kkd r(t) c(t)
(13)
Equations (12) and (13) correspond to the two-degree-of-freedom control loop of Fig. 2.
A potential problem with the velocity PID form is that the
second-derivative of an error signal must be approximated;
for the same reasons described in the preceding section, this
process leads to difficulties in the presence of significant measurement noise. Also, with the integrator external to the controller, it is essential that ki not be zero in Fig. 3 so that no
attempt is made to cancel the 1/s of the integrator by the s
inherent in a velocity form of the PD controller. This means
that the velocity PID of Fig. 3 has the integral term as an
essential term of the controller. However, if the control signal
strom and Hagglund
u(t) is available as a measured signal, A
(2) show how to use it to obtain a valid PD form from the
velocity PID form.
The velocity form of the PID controller does have some advantages: (1) if the integrator in the actuator is implemented
in a way that limits the output of the integrator at the saturation bound (when either the upper or lower saturation bound
is intercepted), then no integrator windup can occur, and no
special anti-reset-windup strategy need be implemented; (2)
if the control is transferred from automatic to manual or visa
versa, the signal supplied to the actuator is of an incremental
control form, and the resulting transfer in control generally
will not cause a large swing in the process outputthe transfer of control will be bumpless; and (3) when the velocity PID
is implemented digitally (in which case it is called an incremental PID algorithm), the PID values being accumulated
(corresponding to those at the summation point in Fig. 3) are
incremental in nature, and often can be accommodated by a
450
PID CONTROL
digital word length that is shorter than that required by a

corresponding parallel PID algorithm.
PROPORTIONAL BAND
The control signal associated with any practical actuator has
a realistic lower bound umin and an upper bound umax. For any
u(t) umax the effective control is umax, and for any u(t) umin
the effective control is umin. When either of the preceding conditions exist, the system is said to be in saturation. The term
proportional band refers to conditions on the error signal such
that the effective control signal is not saturated.
Special Case of Proportional Band
A Conditional Integration Example
First, consider the case where only proportional control (P

control) is applied [i.e., Fig. 1 with ki kd 0, and d(t) 0];
umin /kp to emax

in this case, there will be a range from emin
umax /kp in which the control signal will not be saturated, and
the corresponding proportional band PB is
PB =
umax umin
kp
(14)
Alternatively, we say in this case that the error is in the proportional band if emin e(t) emax.
General Case of Proportional Band
When all three terms are present in the PID controller, but
d(t) 0 in Fig. 1, the control signal u(t) will be out of saturation only if the instantaneous e(t) r(t) c(t) satisfies
umin ki
e() d kd e(t)
< r(t) c(t), and

kp
t
umax ki e() d kd e(t)
r(t) c(t) <

kp
lersany controller that has a lag term may exhibit windup,

and ways of controlling the windup should be considered to
enhance performance.
Many anti-reset-windup methods have been developed.
They are based on one of two approaches: (1) conditional integration, in which integration is interrupted and/or modified
when control signal saturation is likely; or (2) integrator
tracking, whereby the output of the integrator is forced to
track a signal usually with the objective of reducing the control signal magnitude to the saturation boundary. In either
case, the anti-windup mechanism must avoid getting lockedup, such that the integrator does not perform as required under normal conditions.
Conditional integration schemes tend to be heuristic and application dependent. A given approach may work reasonably
well in one application, only to fail in another. Because of
their use of logic, conditional integration schemes are readily
incorporated in digital implementations of PID. One approach
is to condition the integration on both kpe(t) and kde(t): if
either one of these values exceeds preassigned bounds, the
integration process is suspended and the output of the integrator is reset to a desirable level (perhaps zero); on the other
hand, if both values are less then their respective bounds,
the integration process is uninhibited; and in either case, the
control signal u(t) can be formed on the basis of the three
PID terms.
Anti-reset-windup for Two-degree-of-freedom Control
A basic tracking implementation of anti-reset windup is depicted in Fig. 4. The output from the PD and PD block is
q(t) = kp r(t) kp c(t) + kd r(t)
kd c(t)
(16)
(15)
Thus, the proportional band varies in a complicated way, depending both on the integral and the derivative of the error.
In practice, both derivative terms in the above expression

would be filtered versions of the derivatives [for example,
kdc(t) would be replaced by the output of the Gd(s) filter of Eq.
(8) with the input to the filter being c(t)].
The saturation block in Fig. 4 is characterized by
w, umin < w < umax
u=
umax, w umax
u ,wu
min
min
ANTI-RESET-WINDUP
The control signal of a given system can be in saturation for
long periods of time for many reasons. For example, if a large
step increase in the reference input occurs at t 0, the error
signal for some time after t 0 also will be large, and its
direct effect on the control signal will be to force it into saturation until the system output is able to match to some degree
the reference input. Although the effective control signal is in
saturation, the integrator in the controller accumulates the
area under the large e(t) curve, unless restricted to do otherwise. If the integrator output is not restricted in some way, it
takes a relatively long time for the integrator output to reduce to a normal level; it does not even start to decrease until
after the controlled output c(t) has overshot the desired reference input value [causing e(t) to turn negative and allowing
the output of the integrator to start to decrease]. Thus it is
essential that the integrator output be managed in some effective way whenever the control signal is in saturation. It is
of interest to note that windup is not limited to PID control-
PD & PD'
e=rc
ki
(17)
q
+
+
1
s
kt
+
p +
Figure 4. A PID controller with anti-reset-windup provided by a

tracking loop. The nonlinear block has a gain of 1 for w in the range
umin w umax.
PID CONTROL
e=rc
PD
w
+
+
1
1 + sTi'
Figure 5. A series-form PID controller with anti reset windup. The

nonlinear block has the same characteristics as that in Fig. 4, and
the indicated positive feedback results in the linearized transfer function from q to u being (1 sTi )/(sTi ).
From the above equation, when w is in the range umin w

umax, u w and the tracking error v in Fig. 4 is zero, resulting
in normal operation of the PID algorithm. When w umax,
however, with the tracking gain kt 0, the additional input
to the integrator is kt(umax w) ktv 0, causing p(t) to tend
to decrease until w reduces to umax. The larger the tracking
gain kt, the faster the tracking process. In high noise environments, the tracking gain should be reduced to a reasonable
level to avoid over zealous tracking. When w umin, the
tracking error v becomes positive, and the integration process
forces w to tend to increase towards umin. Thus, whenever the
control is in saturation, the output of the integrator is automatically adjusted to a level that renders w near the saturation boundary, and any change in r(t) or c(t) that tends to
move w away from saturation results in normal PID control.
Anti-reset-windup for the Series Form of PID
In the special case of the series PID form, anti-reset-windup
can be implemented in a particularly simple way, as shown
in Fig. 5. The saturation block in Fig. 5 is characterized by
Eq. (17). Note that the feedback in the loop in Fig. 5 is positive. When the forward path is not saturated, u(t) w(t) and
linear feedback theory can be used to show under these conditions that
1
U (s) = 1 +
sTi

Q(s)
BUMPLESS TRANSFER
When a feedback loop is switched from an automatic mode of
control to a manual mode of operation or vice versa, it is often
important to avoid large instantaneous jumps in the control
action. Certain forms of PID implementation lend themselves
in a natural way to smooth transitions from one mode of control to another. Velocity (incremental) PID implementations
supply incremental values to be accumulated by an integrating actuator, so large transitions in effective control are not
incurred when the source of the incremental changes is
switched.
For a general parallel implementation of PID control,
tracking can be used to obtain bumpless transfer in addition
to anti-reset-windup. Figure 6 is a diagram of such a system.
The relationship between w and u in this diagram is that
given by Eq. (17). When the switch is in the manual mode, as
shown, the human operator supplies a value em to drive the
system: if the nonlinear block is not saturated, the control u
increases (or decreases) at a rate given by kmem; at the same
time, the signal uPID tracks um because of the upper tracking
loop, with tracking gain kt; and when the nonlinear block is
saturated, the lower tracking loop insures that the output of
the lower integrator will be maintained at either umax or umin,
as appropriate. When the switch to automatic control is made,
the uPID value starts with the most recent tracked value of
um plus any additional component supplied by the PD terms
operating on error signals involving r(t) and c(t). Similarly,
when the switch is in the PID position, the manual mode output tracks the control u(t) supplied by the PID unit because
of the presence of the lower tracking loop, also with tracking
gain kt.
Considerations of bumpless transfer also apply when parameters in the controller are changed abruptly. Such
changes are especially easy to make in digital implementations of PID control. Real-time changes in the controller parameters can be motivated by monitored changes in the dynamics of the system being controlled. When these changes
are done automatically, the resulting control system is classified as an adaptive control system. The particular PID parameter that is most readily conditioned for bumpless transfer is
(18)
r
The PD block in Fig. 5 provides (approximately)

Q(s) = K (1 + sTd )E(s)
451
PD & PD'
(19)
The combination of Eqs. (18) and (19) implements the series

PID form given in Eq. (6). When the forward path in Fig. 5 is
saturated at u umax, the contribution that u makes to w
tends to umax. Similarly, when u is saturated at u umin, the
contribution that u makes to w tends to umin. Thus the integration process is effectively bounded and windup is avoided.
An alternative and somewhat more desirable anti-resetwindup strategy for the series PID can be obtained by moving
the saturation block from the forward path in Fig. 5 to the
feedback path, but keeping the saturation block to the right
of the first-order lag block. This enables the full effect of the
PD action to be supplied to the process while yet limiting the
integral action.
(Closed-loop
feedback)
e=rc
ki
+
+
1
s
+
+
uPID
kt
(Manual)
em
km
+
+
1
s
+
um
kt
Figure 6. A general analog PID controller featuring anti-resetwindup and bumpless transfer.
452
PID CONTROL
ki. If the error signal e(t) is integrated first and then

multiplied by ki, any abrupt change in ki will result in a corresponding abrupt change in u(t). On the other hand, if the
weighted error kie(t) is integrated in real time, the effect of an
abrupt change in ki will be smoothed by the integration process. In terms of a block diagram representation, the ki block
should precede the 1/s block for bumpless transfer with respect to changes in ki.
propriate first backward differences, with a gain of Kd

kd /T. Similarly, the integration is approximated by a summation of samples, and the summation gain Ki kiT is used in
place of the ki block in Fig. 6. The upper tracking loop in the
figure can be implemented digitally, but note that the z-transform model of the 1/s for the integrator is z/(z 1). If the
output of the kt block is delayed by one sample period for convenience in the calculations for the tracking loop, the tracking
loop will have the following characteristic equation:
DIGITAL PID CONTROL
1 + [kt /(z 1)] = 0
Digital PID control often is implemented by having the system signals of interest [r(t), c(t), and e(t)] sampled periodically
with sample period T; the sampled input signals are supplied
to a microprocessor through an analog-to-digital converter
(ADC), and the microprocessor is programmed to supply the
control signal u(t) to the system using a digital-to-analog converter (DAC). Usually u(t) appears to the input of the process
being controlled as a sampled-and-held signal with sample
period T. Within the microprocessor, the linear aspects of PID
control are based on the following calculations (using a parallel form):
u(kT ) = Kp e(kT ) + Ki
k

e(mT ) + Kd {[e(kT ) e[(k 1)T]}
m=0
(20a)
To simplify notation, we use u(k) to denote u(kT), etc., thereby

replacing the above equation by
u(k) = Kp e(k) + Ki
k

e(m) + Kd [e(k) e(k 1)]
(20b)
m=0
The proportional term of course is Kpe(k); the integral term

k
is the sum Ki m0 e(m); and the derivative term is now the
first backward difference term Kd[e(k) e(k 1)]. Using z
transforms, the transfer function from U(z) to E(z) is given by
GPID (z)
z
z1
+ Kd
U (z)/E(z) = Kp + Ki
z1
z
b0 z2 + b1 z + b2
z(z 1)
which places a pole of the tracking loop at z 1 kt. For

stability of the tracking loop, this pole must have a magnitude
less than one; and a value of kt between 0.5 and 1 is recommended.
Incremental Digital PID
Equation (20b) can be used to show that
u(k) u(k 1) = Kp [e(k) e(k 1)] + Ki e(k)
+ Kd [e(k) 2e(k 1) + e(k 2)]
(24)
This falls naturally into the incremental PID form. If the actuator of the control loop contains the integrator, then during
each sample period, the digital microprocessor simply has to
perform the calculations given on the right-hand side of Eq.
(24), normalize the result by dividing by T, and send the resulting approximate derivative value through the DAC to the
integrating actuator. Of course a slight modification of the
above is required if two-degree-of-freedom control is employed; namely, the value V,
V = (Kp Kp )r(k) + (Kd Kd )[r(k) r(k 1)]
(25)
would need to be added to the right-hand side of Eq. (24).

Recursive Digital PID
(21)
which can be placed over a common denominator to obtain

the form
GPID (z) =
(23)
(22)
where b0, b1, and b2 are related to Kp, Ki, and Kd in a straightforward way (to be described). Essentially all digital PID
z-transform transfer functions can be based on Eq. (22), which
has two poles, one at z 0 and one at z 1. The two zeros
of Eq. (22) can be either two real zeros or a pair of complex
conjugate zeros located to achieve performance objectives.
Analogous Implementation of Digital PID
The general continuous PID implementation of Fig. 6 can be
converted to digital form by using the following approximations. Assume that the lower portion of the figure is left unchanged, but that uPID is supplied from the output of a DAC
and hold circuit. The derivative action is approximated by ap-
Equation (24) can be rearranged in the recursive form

u(k) = u(k 1) + (Kp + Ki + Kd )e(k)
(Kp + 2Kd )e(k 1) + Kd e(k 2)
(26a)
Also, using the z-transform relationship of Eq. (22), it can be

shown that the associated difference equation is
u(k) = u(k 1) + b0 e(k) + b1 e(k 1) + b2 e(k 2)
(26b)
By comparing coefficients in Eqs. (26a) and (26b), it follows

that b0 Kp Ki Kd, b1 (Kp 2Kd), and b2 Kd.
The recursive form of Eq. (26b) is an excellent way to
implement digital PID. It lends itself naturally to anti-resetwindup: each time u(k) is calculated, if it is outside the saturation bounds, it is reassigned the value of the nearest
saturation bound. For real-time implementation, we use the
following notation: u current value of u; u1 most recent
past value of u; e current value of e; e1 most recent past
value of e; and so on. The following sequence of events constitutes a flow diagram for computer code to implement the algorithm:
PID CONTROL
1.
2.
3.
4.
5.
6.
7.
8.
9.
Initialize e1 e2 u1 0.
Compute temp u1 b1e1 b2e2.
Wait for the sample period.
At the sample period, obtain e r c using appropriate
A/D converters.
Compute u temp b0e.
If u umax, assign u umax, or if u umin, assign u
umin.
Output u to the DAC.
Assign in proper order e2 e1, e1 e, and u1 u.
Return to step (2).
Note that the above steps are arranged to minimize the number of calculations required between the instant that e is obtained from the ADC and the instant that u is sent to the
DAC. This process minimizes the computational time delay
that is introduced into the control loop; computational time
delay can have a detrimental effect on system performance.
Also, for two-degree-of-freedom implementations, the terms
from Eq. (25) would need to be included appropriately in the
code.
Signal Property Considerations
Both control signals and measured process signals are constrained in a variety of ways. Common examples of constrained control signals are (1) signals that are continuously
adjustable between saturation levels; (2) control signals that
can be assigned only a few valuesas in the case of a relay
with dead zone, with possible output values umax, 0, or umax;
and (3) control signals that are pulse-width modulated. Even
in case (1) above, when the control is supplied by the output
of a DAC, the finite word length of the DAC may have a noticeable quantization effect for small steady-state error signals. In case (2) above, when the control is supplied by an
electromechanical relay (once a common practice, but now
largely displaced by power electronics), a correct balance of P
and D terms is needed to avoid excessive switching of the
relay; in all-electronic implementations of such systems, rapid
switching actually may be included intentionally to achieve a
sliding mode of control, and nonlinear control strategies involving nonlinear functions of e and e are appropriate. Case
(3) above is especially prevalent in electronic drives for dc motors. This is because a power amplifier dissipates the least
energy if it is either full on or full off. To reduce heat buildup
in the amplifier, the control is implemented as follows: if the
desired control action from t kT to t (k 1)T is uo where
umax uo umax, the effectively equivalent pulse-width control supplied is

umaxsign(uo ), kT t < kT + T
(27)
u(t) =
0, kT + T t < (k + 1)T
453
smaller than the dominant mechanical time constant of the

motor and its load, the above control provides smooth operation of the motor, and a conventional linear model of the
motor-and-drive generally can be used in the design of the
control system.
Measured process signals supplied to a digital controller
are quantized, sometimes as a result of ADC conversions, but
in other cases as a result of the types of measurement sensors
employed. For example, an optical rotary position sensor provides a finite number of angular positions for one complete
revolution of the sensor disk. In this case, when the magnitude of e(t) is very small, the sensor error can switch abruptly
from 0 to or , where is the effective quantization level
of the sensor. To avoid very-low-level oscillations of the control under such conditions, it may be necessary to zero the
input to the PID integrator when e(t) is small, and to effect
a small dead-zone notch in the control action.
Other Digital PID Issues
Other issues in digital controller design are sampling period
selection, antialiasing, and process modeling. Ideally, the
sampling period T should be selected to be an order of magnitude smaller than the dominant time constants of the system
being controlled. However, extremely small values of T may
lead to round-off error and computational problems on finiteword-length computers. The choice of sampling period also is
affected by the amount of process and measurement noise.
The folding frequency associated with T is 0.5/T Hz. When a
signal that has a frequency component above the folding frequency is sampled, the sampled signal exhibits an associated
aliased low-frequency component. Thus, for digital implementation, generally signals are filtered with an analog antialiasing filter prior to sampling. Often adequate for the purpose is a first-order low-pass filter with a corner frequency
near 0.5/T Hz.
When a digital controller is designed using z-transform
methods, it is helpful to have a representative z-transform
transfer-function model of the process being controlledfor
example, a model corresponding to a sample-and-hold circuit
followed by the process transfer function G(s) of Fig. 2. In the
case of a conventional zero-order hold circuit, the required
z-transform transfer function of the sample-and-hold in series
with the process is
Ghp (z) = (1 z1 )Z{G(s)/s}
(29a)
or
Ghp (z) = (1 z1 )Z
g( ) d
(29b)
The characteristic equation of the feedback loop is then

where T Tuo/umax, and
1, uo > 0
sign(uo ) = 0, uo = 0
1, u < 0
1 + GPID (z)Ghp (z) = 0
(28)
When the sample period T is larger than the major electrical

time constant of the motor drive circuit, but is substantially
(30)
with closed-loop poles of the system being roots of the characteristic equation. The PID gains can be adjusted to obtain desirable pole locations inside the unit circle of the z plane, and
a variety of other digital controller design methods can be employed.
454
PID CONTROL
Emulation Method of Digital PID Design

The process of selecting of PID gains to achieve desirable
goals is called tuning. Because most PID tuning methods have
been developed for analog PID controllers, a natural question
is can we design an analog PID using established tuning rules
and then translate the analog PID gains into meaningful digital PID gains? Of course, the answer is generally yes. However, there are several factors that should be considered in
doing this. In the first place, the sample-and-hold operation
that is inserted in the control loop has the effect of introducing a time delay, of approximately T/2 s into the loop. Also,
computational delay, denoted by T s where 0 1, is
introduced. These delays need to be taken into account when
converting from analog PID gains to digital PID gains.
Consider the following analog GPID(s):
GPID (s) = kp +
kd s
ki
+
s
1 + s
(31)
In a general emulation approach developed by Pierre and

Pierre (3), GPID(s) is replaced by digital controller transfer
function Gc(z):

2 z1
(32)
Gc (z) = Ga (z)GPID
T z+1
where the replacement of s by (2/T)(z 1)/(z 1) in GPID is
the well known Tustin approximation, and where
Ga (z)
, [0.5(3 a) + (1 a)]zz a[0.5(1 + a) + (1 a)]
(33)
in which a is an additional tuning parameter (typically

0.3 a 0.4). It is readily shown that the dc gain of Ga(z)
is Ga(1) 1 and that the zero in Eq. (33) is to the right of the
pole thereby providing phase lead. Ga(z) compensates to some
degree for sample-and-hold delay and for computational delay T in the control loop.
As a specific example of Ga(z), the case where a 0.2
and 0 in Eq. (33) gives
Ga (z) =
1.6z 0.4
z + 0.2
(34)
It is important to have a in Eq. (33) bounded away from z

1; poles of the controller near z 1 often are ringing poles,
generating oscillations of period 2T in u(t) that are not readily
observed in c(t).
From Eqs. (31) and (32), along with assigning T/2, it
follows that

k T (z + 1) kd (z 1)
Gc (z) = Ga (z) k p + i
+
(35)
2(z 1)
Tz
A controller based on the Gc(z) of Eq. (35) can be implemented
using a variety of digital control programming methods; practical implementations insure that integrator windup is
avoided.
PID TUNING
Numerous studies have been made to develop assignment
rules to specify PID parameters on the basis of characteristics
of the process being controlled. There are many representative sources that can be consulted for details on a wide variety
of alternative tuning rules (2,47). Common tuning rules are
described in this section. Most of these approaches give excellent results if the process being controlled is well damped and
is dominantly second order. Systems that are essentially first
order can often be tuned well with PI control only. Systems
with large time delay can be controlled with PID control in
combination with a Smith predictor (8). From a frequency response viewpoint, the PID integral term supplies 90 of phase
lag at low frequencies, and the practical derivative term supplies somewhat less than 90 of phase lead at some intermediate to high frequencies. Thus, if the system requires more
than 90 of phase-lead compensation over some frequency
range to achieve good performance, PID control alone will not
be adequate.
For lightly damped oscillatory open-loop systems, one
method that is tempting (but should be approached with caution) is to place the zeros of the PID controller at the pole
locations of the process, to cancel the oscillatory poles. This
approach should be avoided if the PID controller employed
has one degree of freedom only: although the response mode
of the canceled poles will not be excited by reference inputs,
disturbance inputs will excite the oscillatory mode in the controlled output, and the PID controller with its zero gain at
the oscillatory frequency will not supply damping. For twodegree-of-freedom systems, the PID gains within the loop can
be assigned to add damping to the oscillatory mode, whereas
the independent PD factors associated with the reference input can be adjusted to provide blocking zeros. Sensitivity of
oscillatory pole-zero cancellations with respect to parameter
changes also is of concern (9).
ZieglerNichols Tuning
A widely applied tuning rule, the ultimate gain rule, was developed in 1942 by Ziegler and Nichols (10). This rule is based
on experimentation with the process to be controlled. First,
with ki and kd set to zero in Eq. (3), the proportional gain kp
is increased until the system starts to oscillate; the value of
kp that starts the system oscillating is denoted as ku and is
called the ultimate gain. The period of the oscillation, Tu, also
is recorded. As an alternative procedure, rather than determining ku and Tu by forcing the actual system into instability, if the frequency response G( j) is available, it can be used
to obtain both ku and Tu analytically, as follows. First, the
frequency u at which the angle of G( ju) 180 is determined, and the corresponding value of G( ju) Au is obtained. The value of ku satisfies the characteristic equation
1 + ku G( ju ) = 0
(36)
and therefore ku 1/Au and Tu 2/u. An ultimate gain

tuning rule is then:
kp = 0.6ku
(37a)
ki = 1.2ku /Tu
(37b)
kd = 3ku Tu /40
(37c)
and
PID CONTROL
This ultimate-gain rule is but a rule-of-thumb; although it

was developed with a nominal 20% step response overshoot
goal, it is easy to find cases where it results in overshoots in
excess of 50%. It should not be surprising that the rule does
not apply well to all cases because process characteristics
vary widely.
of s3 1s2 2s 3, where the roots of this polynomial are

assigned to achieve desirable damping and natural frequency.
Explicit relations for kp, ki, and kd can be developed in terms
of 1, 2, 3, b1, b2, a1, and a2.
Tuning methods also can be based on frequency response
characteristics and the sensitivity function
Modified ZieglerNichols Methods
S( j)
Many variations of the ultimate gain rule have been developed. One popular one credited to Harriott in (4) is as follows.
First, with ki and kd set to zero in Eq. (3), kp is adjusted until
the step-response of the closed loop exhibits a decay ratio of
0.25 (the percent overshoot of the second peak in the step
response is one-fourth of the percent overshoot of the first
peak in the step response). Let kc denote the critical value of
kp corresponding to this 0.25 decay ratio. Also, let Tc denote
the difference between the time of occurrence of the second
peak and that of the first peak. Next, assign ki and kd on the
basis that
ki = 1.5kc/Tc
(38a)
kd = kc Tc /6
(38b)
455
1
, 1 + G ( j)G(
j)
(43)
Generally the values of kp, ki, and kd should be such that

max |S( j)| < 1.5
(44)
Values smaller than 1.5 in Eq. (44) usually correspond to well

damped systems.
Tuning methods based on optimization techniques can be
applied directly to a particular system if a valid G(s) repre strom and
sentation of the controlled process is available. A
Hagglund (2) do a systematic evaluation of many cases to generate tuning diagrams that can be used to obtain desirable
PID gains for a variety of G(s) forms.
and
And finally, while conducting additional closed-loop step-response tests, adjust all three gains by the same percentage
until desirable overshoot conditions are achieved.
More Advanced Tuning Methods
When the transfer function G(s) of Fig. 2 is known and is a
reasonably good model of the process being controlled, an
array of tuning methods are available, depending on the form
of G(s). For example, variations of the ZieglerNichols methods are based on cases where G(s) assumes forms such as
G(s) =
K0 esT0
0 s + 1
(39)
G(s) =
K0 esT0
s
(40)
or
A pole-placement approach has been developed (2) for those

cases where G(s) is of the form
G(s) =
b1 s + b2
s2 + a 1 s + a 2
(41)
The characteristic equation for the closed loop is

1 + Gc (s)G(s) = 0
(42)
When the PID transfer function of Eq. (4) is substituted for

Gc(s), and G(s) of Eq. (41) is substituted into Eq. (42) also, the
resulting equation reduces to a third-order polynomial having
coefficients that are functions of kp, ki, kd, b1, b2, a1, and a2. A
desirable placement of the poles of the system can be achieved
by equating coefficients of the above polynomial to coefficients
CONCLUSION
In his classical paper on governors, Maxwell (11) in 1868
clearly expressed the difference in effects produced by proportional and integral control. However, there is no one person
that can be credited with having invented PID control. Many
of the early developments, those involving anti-reset-windup
for example, were company proprietary and therefore were
not available in the open literature. With the advent of computer control in the 1960s, many of the traditional PID techniques were reassessed and translated into the digital control algorithms.
In this article, the basics of PID control have been described. Operational features of both linear PID terms and
nonlinear characteristics have been examined. Digital PID algorithms have been described and have a special significance
in the modern era of digital control.
Today, PID controllers can be purchased from many manu strom and Hagglund (2) for a recent listing of
facturers [see A
companies and features available]. In addition to being used
for SISO control, PID controllers are used in a variety of other
applications. In cascade control, one control variable is generated on the basis of several measured variables. In selector
control one actuator can be driven from a selected PID unit,
and then automatically switched to a different PID unit based
on a max/min selector to control some other process signal if
it starts to deviate from a desired range. In ratio control two
outputs are controlled with one of the outputs required to be
a fixed percentage of the other output. In split-range control
and multiactuator control there may be fewer measured signals than actuators, and the amount of control from each actuator has to be automatically balanced to achieve the desired
overall process performance goals. Feedforward control of disturbances, when they can be measured, often leads to vastly
improved disturbance rejection. Automatic gain scheduling of
PID parameters can be based on measured process conditions.
And interactions of coupled control loops can be reduced by
properly designed coupled PID controllers. In many cases, as-
456
PIECEWISE-LINEAR TECHNIQUES
pects of PID control are blended with other control concepts,

leading to higher-order controllers. PID developments in the
future will be, and already have been to some extent, coupled
with fuzzy control, neural-network control, and adaptive
control.
BIBLIOGRAPHY
1. D. P. Eckman, Principles of Industrial Process Control, 11th ed.,
New York: Wiley, 1961.
strom and T. Hagglund, PID Controllers: Theory, Design,
2. K. J. A
and Tuning, 2nd ed., Research Triangle Park, NC: Instrument
Society of America, 1995.
3. D. A. Pierre and J. W. Pierre, Digital controller design
alternative emulation approaches, ISA Trans. J. Instr. Soc. Amer.,
34: 219228, 1995.
4. B. G. Liptak, (ed.), Instrument Engineers Handbook, Vol. II, Process Control, New York: Chilton, 1970.
5. R. Isermann, Digital Control Systems, Vol. 1, 2nd ed., Berlin:
6. C. L. Smith, Digital Computer Process Control, Scranton, PA: International Textbook, 1972.
strom, and W. K. Ho, Refinements of the
7. C. C. Hang, K. J. A
ZieglerNichols tuning formula, IEE Proc., D138 (2): 111118,
1991.
8. T. Hagglund, A predictive PI controller for processes with long
dead times, IEEE Control Syst. Mag., 12 (1): 5760, 1992.
9. D. A. Pierre, Root locus near isolated pole-zero dipoles and a
counterintuitive case of digital control compensation, IEEE
Trans. Auto Control, AC-29: 439441, 1984.
10. J. G. Ziegler and N. B. Nichols, Optimum settings for automatic
controllers, Trans. ASME, 64 (8): 759768, 1942.
11. J. C. Maxwell, On governors, Proc. R. Soc. Lond., 16: 270283,
1868. Also published in R. Bellman and R. Kalaba (eds.), Mathematical Trends in Control Theory. New York: Dover Publications,
1964, pp. 317.
DONALD A. PIERRE
Montana State University

Copyright
PROCESS CONTROL
Process control was initially developed from the practices of process industries such as paper, steel, and chemical
manufacturing. Beginning in the early part of the century, it became more widely practiced starting in the
1940s. Applications now include processes in microelectronics manufacturing, such as bulk crystal growth,
chemical vapor deposition, and etching. Process control is important because it provides a means for improving
process yield and reducing variation, and also enables new processing capabilities to manufacture unique
engineered materials and structures.
Process control provides an approach to developing an automatic control system for complex, nonlinear,
distributed parameter systems involving mechanical, transport, electrical, material, and chemical attributes.
The term process control is used broadly to cover a variety of functions including supervisory control, failure
detection, and the lower level control that determines the required inputs to achieve a desired objective. It
is applicable to large plants incorporating many subprocesses and for single processes. This article primarily
focuses on the issues associated with developing the lower level of control for a single process.
A major goal of process control is to automatically determine the process input settings to achieve the
desired output condition (referred to as the command signal) while minimizing the variation of the output
from the desired level (e.g., error). Variations are caused by external inputs called disturbances and variation
of the plant, such as aging. Performance is evaluated in terms of dynamic aspects (how fast the system needs
to respond) and magnitude of the error. A powerful conceptual approach used in control system analysis is to
evaluate the output error as a function of command and disturbance signals, which provides a useful basis for
designing compensators to meet performance specifications.
Taguchis concept of the process variability loss function provides an important basis for evaluating the
value of process control (1). Given target quality objectives, deviation from that target should be characterized
by some loss function. Thus, for a quadratic loss function, the narrower the distribution, the greater value the
process provides. One way manufacturers deal with process variations is to divide (or grade) production output
into various quality classes, a practice common throughout semiconductor manufacturing. This approach,
however, actually adds cost because sorting provides no additional value. In addition, there is the loss associated
with the value that could have been attained if the production had met a narrower distribution. In this context,
process control is valuable if used to reduce variation, but the cost of the solution must be balanced against the
return.
There are two major control options: feedforward and feedback. Feedback utilizes an error signal formed
by comparing the actual output to the desired value, which is then used to determine a corrective adjustment
to the plants inputs. Feedback corrects for unknown variations, but it is reactive, acting only after an error has
occurred. Feedforward control utilizes process knowledge to determine the input values required to achieve the
desired output without process measurements. Such process knowledge can be informal, such as that of the
equipment operator, or expressed formally in a mathematical model. Feedforward is anticipatory for known
conditions, but does not compensate for unknown factors, such as disturbances or plant variations. Feedforward
control is typically used in cases where a cost-effective sensor is not available and a robust model is available.
1
PROCESS CONTROL
Such an approach has been used to control wafer temperature distribution in tube furnaces where it is not
practical to measure the wafers temperature distribution.
Feedback systems are implemented either utilizing measurements in real-time or after a process is
completed. Real time implies that the output (or related process states) are measured as the process is running
and the inputs are adjusted on the fly to compensate for errors. If the process variations or commands are slow
relative to processing time (i.e., the batch time), then real-time control is not necessarily needed. In this case,
run-to-run control is used where the results are measured after the process is completed. If the disturbances
have long-term behavior relative to process batch time, run-to-run control works well assuming that there is
only one independent disturbance. In cases where variations are fast relative to process timescales, closed-loop
control provides significant performance advantages. For example, in spot welding, the thermal dynamics of
the process vary significantly throughout the welding cycle, so that development of a dynamic measurement
controller provides significant performance improvements (2). Similarly, real-time control also works well to
eliminate time-varying disturbances. It also is advantageous if one wants to build in process flexibility (i.e.,
the ability to change process output levels), even for slower processes, when process modeling is not accurate
enough to meet the performance objectives.
It is not always possible to directly measure the variable related to the process objective. An alternative
strategy is to identify and control an intermediate or secondary process variable related to the primary variable.
In crystal growth, for example, dislocation density is an important material objective for optoelectronic devices
which is not feasible to measure in realtime. Even if it were measurable, it might not be desirable because,
once a dislocation has been introduced in the crystal matrix, it cannot be removed. However, by controlling the
crystals thermal gradients, dislocations are prevented from forming in the first place. A related strategy is to
use cascade control structures wherein disturbance paths are identified and local loops eliminate their effect
before they propagate to the primary control objectives.
Other approaches to process control include minimizing input variations [the objective of statistical
process control (SPC)], thereby eliminating external disturbances, or selecting the operating regime so that the
output is insensitive to the disturbance (the later approach is Taguchis design of experiments technique). The
disadvantage of SPC is that it requires continued effort to track down all disturbances, whereas feedback control
operates automatically. The disadvantage of the design of experiments is that it requires an extensive set of
experiments on the production equipment, which could significantly affect production. Taguchis technique,
however, utilizes existing equipment without additional capital or development costs. In some cases, it is
beneficial to combine these techniques. For example, design of experiments is used to determine the desired
operating point when number of input parameters are to be set, and local feedback is used to eliminate
disturbances and plant variation. Feedback control, however, can be implemented only if there is a manipulable
input that compensates for the specific disturbance.
Developing a process control system requires addressing a number of issues beyond the design of the
control algorithm (i.e., the equations that map measurement information to the input levels). Important
issues include specification of the relevant process objectives (both technical and economic), identifying the
constraints posed by the equipment and process physics, determining which control structure best achieves
the desired performance objectives, and considering opportunities for improving the process capabilities by
changing the system design. Answering these questions typically requires a good understanding of the relevant
process physics in addition to the relevant control principles. Modeling is useful in answering these questions,
but should be performed from a control perspective. Such a perspective seeks to understand the system by
identifying its fundamental dynamics and the limitations they pose, its disturbances, objectives, and available
manipulable inputs.
One major control structure issue stems from the fact that more than one input typically affects the
desired output(s). Thus, it is important to understand which inputs can achieve the best performance. One
way to evaluate the performance of alternatives is utilizing frequency-domain techniques to quantify gain and
bandwidth for alternative inputs. Additionally, one should evaluate if some outputs are more difficult to control
PROCESS CONTROL
independently. For example, in a welding process, it is difficult to independently control the width of the heat
affected zone and the size of the weld nugget. These problems are related to the inherent coupling of a process,
and it may not be physically possible to achieve all the objectives. Thus, an important objective of the analysis
is to determine such inherent conflicts.
A control perspective is also useful in selection of the operating regime, i.e., the specification of the nominal
operating conditions for all inputs and process settings or trajectories. Important factors include how fast the
process proceeds, the significance of different disturbances, and which aspects of the process physics dominate.
The operating regime is selected from a basic understanding of the dominant process physics and/or from
formal optimization techniques, such as design of experiments.
Consideration should also be given to redesign of the process and/or alternative control structures (i.e.,
combinations of feedback, feedforward, or cascade) to improve performance. Important examples of the benefit
of integrating system and control design include eliminating known disturbances, adding new actuators to
achieve independent control of an additional output, and sizing actuators to meet the performance specification.
Development of the control system requires identification of the fundamental dynamics of the process.
Important issues to address are factors that fundamentally limit control performance or that need to be
compensated for through system or control design. Some of these features might suggest changes in the
operating regime selected to avoid the problem, such as if there are poorly damped modes. Other features, such
as nonlinearities, need to be identified and considered in designing the control algorithm.
Process Control Analysis and Design Issues

This section provides a more formal statement of the analysis and design options for control. While this analysis
is presented in terms of linear systems, the analytical framework of control theory provides guidance and a
formal basis for developing an appropriate process control system.
Control Structures and Performance Analysis. Feedforward and feedback control is represented
by the block diagrams in Fig. 1. Each block represents the mapping of an input u to an output y. The plant (e.g.,
process) is represented by g and the controller by k. Lower case letters represent single-input, single-output
(SISO) systems whereas upper case letters represent multiple-input, multiple-output (MIMO) systems. Thus
g is a scalar operator mapping u to y (y = gu) whereas G is a matrix operator mapping the vector of inputs to
the vector of outputs (y = G u). The dynamics of linear constant coefficient systems is analyzed in the Laplace
domain with complex transform variable s in terms of transfer functions such as g(s) and k(s). (The functional
dependence on s will not be explicitly indicated henceforth.) Time-varying inputs for r, n, and d are readily
analyzed in terms of a family of inputs (impulse, step, and ramp) and in the frequency domain (see the following
section).
The input-output relationships for a SISO system are as follows (3,4):
where do is an unknown disturbance that acts on the plant and is reflected at the output, di is a disturbance
that affects a process input, and n is a noise signal that corrupts the measurement of the output. (These
PROCESS CONTROL
Fig. 1. (a) Feedforward and (b) feedback control structures.
relationships are generalized for MIMO systems by matrix algebra.) The output disturbance do indicates that
there is an effect on the output but the functional form is not known, whereas an input disturbance di represents
variations of the manipulated input or additional external inputs whose functional input/output mapping is
known.
The achievable control performance is analyzed in terms of the error where e = r y. The performance of
the two structures is expressed as
Equation (4) reveals that feedback control reduces process variation, that is |e| can be made small. In contrast,
process variation for feedforward control is reduced only by eliminating disturbances and plant variation (see
Eqs. (3) and (5)).
Analysis of the steady-state performance to step inputs provides insight into feedback control performance.
This can be done by utilizing the final value theorem, which for step inputs corresponds to evaluating the
magnitude of the corresponding closed-loop transfer functions [T = gk/(1 + gk), S = 1/(1 + gk)] at steady state
(i.e., s = 0.). In general, making |gk| 1 achieves good performance for following commands and rejecting
disturbances because this makes |S| 1. Performance with respect to insensitivity to noise, however, is poor
because |T| 1. Note that several dynamic factors limit the maximum gain that can be used, thereby limiting
achievable performance (see the following section).
One can consider the feedback controller k as an amplifier because u = kem (where em is the measured
error) and u is a power-level signal. Viewing k as an amplifier is consistent with requiring |gk| 1 to achieve
good performance. A simple version of such a controller is a proportional gain (i.e., P-type controller). More
complex forms with varying gains for different frequency ranges can achieve better performance. In cases where
PROCESS CONTROL
u is not an electrical signal, the controller can consist of two parts: the control algorithm and the actuator that
manipulates the physical input u.
The achievable performance relative to variation of the process and/or model error, g, is given by
where g = m + g and m is the nominal process or model. Thus, feedforward is sensitive to the relative model
error, but feedback control compensates for such error or process variation, provided S is made small.
The different approaches to reducing process variation are analyzed by linearizing the output about the
nominal operating point (W o , yo ) and expressing the output variation as
where the nonlinear mapping of the material inputs w and manipulable inputs u is given by y = f (w, u). Classical
SPC and Taguchi are considered feedforward approaches wherein SPC seeks to identify the assignable causes
of the variation and to eliminate them, thus making |w| small. Taguchi seeks to identify the operating regime
(i.e., values of wo , uo ) whereby the output is insensitive to the variations, that is |f /w| 1. The merit of
this method is that it requires no additional capital expense. However, because it is based on experimentally
mapping the input/output space, it might be quite expensive to run the experiments on the production line.
The required condition for implementing a feedback solution is that the plant gain g = f /u = 0. Thus, in some
cases there are disturbances or operating conditions that cannot be corrected for by feedback.
Closed-Loop Frequency and Dynamic Analysis. Frequency analysis yields important insight into
designing an appropriate process control system and into factors that pose important performance limitations.
It is also used to analyze the performance of more complex inputs and nonlinear systems linearized about a
nominal operating point. The output response of a linear system g(s) to a sinusoidal input u(t) = sin t is given
by
where |g(j)| is the magnitude of the complex transfer function evaluated at frequency and is a phase shift.
The frequency characteristics of a system can be visualized by plotting the magnitude of g(s) as a function of
frequency (s = j) on a log-log plot, known as a Bode plot (Fig. 2). An important characteristic is the range of
frequencies with uniform amplification called the bandwidth BW . The Laplace domain provides a basis for
relating the characteristics of the Bode plot to the time-domain characteristics of the process. For example,
the open-loop bandwidth approximates the open-loop dynamic response as BW dominant 1 dominant where
dominant is the dominant pole of the system, and dominant is the dominant time constant of the system (3).
Frequency analysis is useful for analyzing the error magnitude and dynamics for closed-loop systems
and for compensator design. Important design insight is obtained by comparing the magnitude plots of |gk|,
sensitivity S(s), and closed-loop T(s) transfer functions (Fig. 2). The closed-loop bandwidth of T (CLBW ) is
bounded by the frequency that |gk| 1. Since the closed-loop bandwidth characterizes the range of frequencies
PROCESS CONTROL
Fig. 2. Bode magnitude plot of open- and closed-loop transfer functions.
for good command following, disturbance rejection, insensitivity to plant variations, and speed of response, it
is used as a primary design variable. Thus K(s) is selected to achieve the desired crossover frequency for gk.
The desired performance of the system is expressed in terms of the range of frequencies where r, d, and n have
significant energy (i.e., large magnitude).
Noise and stability pose important performance limitations because S and T cannot be made small in
the same frequency range. Thus, good performance cannot be achieved if the power spectrums for desired
command following and disturbances overlap the noise spectrum. In addition, the Nyquist stability criterion
poses a conservative magnitude bound on |gk| relative to the uncertainty of the open-loop process dynamics as
|T| < |g/m|. Thus, CLBW is limited by model uncertainty, and an accurate process model is needed through the
desired closed-loop bandwidth. (Similar logic can also be used for determining the model accuracy required to
achieve performance objectives for a model-based feedforward control approach.)
The frequency-domain analysis also provides a context for selecting appropriate actuators. Each process
typically has a variety of inputs that might be used as the manipulable variable for real-time control. From a
general nonlinear representation of the process, y = f (u1 , u2 , . . .) where ui is the ith input of l different inputs,
one can linearize the system about the nominal operating point yo , uo i . Because at least p inputs are needed
to independently control p outputs, the first design decision is which p inputs achieve the best performance.
For single-input, single-output (SISO) systems, the question reduces to Which of the actuators achieve the
best performance? The comparison is made in terms of the open-loop bandwidth, reflecting how fast that input
affects output, and the open-loop gain, which indicates how large an impact input has on output.
Inputs with the greatest bandwidth and gain typically result in the best performance. However it is not
necessarily true that all high-bandwidth systems have large gain, thus possibly requiring a tradeoff analysis.
High-bandwidth systems require less gain to meet the desired closed-loop bandwidth than slower systems,
thus typically they have a larger robustness margin. In addition, they may have less model error over the
same frequency range because the dominant pole is at a higher frequency. Similarly, high-gain systems require
less controller gain to achieve a desired error performance level, thereby improving the robustness bounds.
Alternative actuators are compared by plotting the Bode plots for each input. Because the inputs are of
different units, the transfer functions should be scaled so they can be compared on a consistent basis. For
example, normalizing by the nominal value of each input expresses each input as a fractional variation of the
nominal input.
In selecting actuators, one should also evaluate limitations posed by dynamic issues such as nonminimum
phase characteristics including time delays, open-loop instabilities, and right-half-plane (RHP) zeros (4). This
factors fundamentally limit achievable closed-loop performance. For example, a RHP zero causes defective
transient responses in that the initial response is in the direction opposite to the final steady-state. In practice,
the closed-loop bandwidth is limited to the bandwidth of the RHP zero.
PROCESS CONTROL
Fig. 3. MIMO system block diagram.
MIMO Systems. Additional issues arise for MIMO systems because of cross-coupling between inputs
and outputs. This coupling causes more than one output when a single input is varied (Fig. 3). MIMO systems
are represented by y = G u, where G is a p l matrix mapping l inputs to p outputs. The frequency analysis
is extended to MIMO systems by utilizing the spectral norms and principal gains of the transfer matrix which
are calculated by a singular value decomposition. For a p p matrix, there are p singular values, i , which are
positive scalars (3,4). Thus, for |u| = 1, (G)|y| (G) where , are the maximum and minimum singular
values of G(s). (G)|y| (G)(G)|y| (G)
There are a variety of control design algorithms for MIMO systems that compensate for cross-coupling.
However, there are some systems, where the coupling makes it difficult to independently control all outputs.
Mathematically, a large
This problem is characterized a large condition numbers for G, that is, 2 (G) = /1.
condition number indicates that the matrix is close to loosing rank, that there are output directions which entail
large control efforts. Such large inputs might saturate the actuators, which could result in loss of stability or
inability to achieve the desired output value. On a practical basis, trying to design a system to operate under
these conditions requires actuators which, for the most part are not used significantly, except to reach sensitive
directions. Another analysis method that provide similar insight is the relative gain array (RGA) (3).
An important solution for poorly conditioned systems is to seek alternative system designs that improve
the ability to control the process. Determining the reason for the poor conditioning is helpful and is obtained
from the singular vectors of the process or by decomposing the transfer matrix into column vectors (15). Poor
conditioning results from several actuators with similar effects on the outputs or an input that does not have the
same magnitude of impact as the others. Alternative designs can be proposed once the nature of the limitation
is determined.
Because both i and 2 depend on the input/output scaling, all variables should be scaled consistently.
Use of dimensionless variables compensate for the different units used for each input and output. Different
normalization methods are used such as scaling about the nominal operating point, or defining perturbations
that reflect relevant engineering scales, such as tolerances, saturation limits, or error limits. For example,
chemical engineers normalize outputs by transmitter spans and inputs by appropriate valve gains (3).
Other Control Structures. Two important variations of control structure use alternative measurements of the primary process output. In processes where it is not practical or possible to directly measure the
process output, a secondary variable related to the primary objective is used for feedback. For example, in
bulk crystal growth, dislocation density is important because it affects the electro-optical characteristics of the
material, but it cannot be measured in real time. Because dislocations are related to temperature gradients,
one can instead control the temperature gradients to prevent the dislocation from being introduced into the
PROCESS CONTROL
Fig. 4. Cascade control structure.
crystal matrix. In this example, there is an added benefit because a process state upstream of the output is
controlled by feedback. Because feedback control is reactive, it only takes control action after an error has
occurred. In the case of dislocations, however, once a dislocation has been introduced into the crystal matrix, it
cannot be eliminated. Thus by controlling the upstream variable (e.g., the temperature gradients), one prevents
the defect from occurring.
A related option is to utilize a cascade control structure (Fig. 4) (3). If a disturbance or a known process
variation can be identified and measured, it is possible to close the loop around this variable to ensure that the
variation does not propagate downstream to affect the primary objective. The benefit of this practice is that
it results in better overall performance for a level of desired robustness because it significantly reduces the
action that the primary control loop needs to perform.
Control Algorithms. There are a variety of methods for designing the control algorithm, and the reader
is referred to the other related articles for specific details. For linear systems, important design methods include
classical design methods such as PID for SISO systems, and more advanced techniques such as optimal control,
robust control, H infinity, and model predictive (3) which can be used for both SISO and MIMO systems. These
methods require a dynamic process model which can be either derived from first principles or experimentally
using system identification techniques. Alternatively, adaptive control techniques do not require a system
model.
For systems that vary in time or with a changing parameter, gain scheduling is used to interpolate between
control designs developed for different conditions. Where there are significant distributions of disturbances or
noise, stochastic control design can be used. Some systems have characteristics that require special approaches.
These include nonlinearities and distributed parameter systems (which entail spatial variations). Other control
approaches provide important advantages such as neural networks for nonlinear systems, fuzzy logic that
responds to different conditions, and non-real-time techniques based on statistics and designed experiments
(4,5).
Application Examples: Electronic Materials Processing

Furnaces. Control of temperature is critical in many processes because it determines process uniformity, yield, material characteristics, and production rate. Two types of thermal systems used in the electronic
fabrication industry that illustrate interesting control issues are tube furnaces to heat many wafers at the
same time and rapid thermal processing (RTP) systems for single wafers.
Control of Tube Furnaces. Tube furnaces applications include dopant diffusion and oxidation, which
are highly sensitive to temperature variations across the wafer. In addition, temperature differences also
induce thermal stress that damage the wafer. Because wafers are loaded into a quartz boat and heated from
the edge, the rate at which the wafers are brought up to the processing temperature is limited by the radial
conduction of heat and the thermal dynamics of the furnace and load. Typically, independently actuated,
multiply segmented heaters are used to compensate for spatial thermal variations. These variations occur
PROCESS CONTROL
because of different loading (i.e., the number of wafers processed each in run), the spatial characteristics of
heat transfer modes (such as natural convection effects that differ significantly for vertical and horizontal
furnaces), and end effects. Important process objectives include the rate of heat-up/cool down (which affect
production rate without providing value), maintaining thermal uniformity, and achieving the process setpoint temperature. Because of sensor limitations, it is difficult to directly measure the wafer temperature.
Thermocouples used in the furnace include spike thermocouples, located near the heater elements, and profile
thermocouples, located near the wafer edges. Thus, tube-furnaces are controlled by secondary measurements.
The process is a coupled MIMO problem because each furnace segment affects its neighbors. Traditionally,
decoupled proportional-integral-differential (PID) loops control each zone, which does not take into account
the cross talk of neighboring elements. This coupling limits how tightly each loop is tuned, because unmodeled
dynamics cause it to become unstable or introduce a disturbance into the neighboring region.
Several model-based control schemes have been developed to overcome the measurement limitations.
These schemes are essentially a hybrid of feedback and feedforward, where the model is infers the wafer
thermal distribution based on the process model and measurement data and designs a MIMO controller.
Because many of the primary heat transfer coefficients dependent on operating temperature, wafer loading,
geometry, and material properties that are difficult to measure directly and/or highly variable, the model has
a number of coefficients that must be empirically determined with instrumented dummy wafers.
A MIMO control approach enables more aggressive control action because the otherwise ignored interactions between zones is now taken into accout. Development and application of such a process has been
undertaken by a team combining the manufacturer (Motorola), the furnace vendor (Silicon Valley Group), and
the control company (Voyan) (5). Implementation has reduced process cycle time by 18% and the fractional
variation in standard deviation (defined as the standard deviation normalized by the set-point temperature)
was reduced from 1.67 to 0.77% at high temperatures (950 C).
Rapid Thermal Processing. Rapid thermal processing (RTP) technology enables fast processing for a
variety of semiconductor manufacturing applications including diffusion, oxidation, chemical vapor deposition,
and nitridation. Tungsten-halogen lamps heat the entire wafer surface, thereby minimizing processing times
and achieving novel structures by avoiding the slow thermal relaxation that occurs with slow ramp rates.
RTPs commercial prospects are in low-volume production of application-specific integrated circuits (ASIC),
shortening the development time for new products, and possibly competing with large conventional fabricating
processes because of the reduced processing time. Typical operating characteristics include ramp rates on
the order of 50 C/s from ambient up to 1100 C, constant temperature for 15 minutes, followed by a rapid
cool down rate. Temperature uniformity across the wafer is a critical performance objective. For example, the
sensitivity of coating thickness variations to temperature variation for oxidation or CVD deposition of films
can be calculated from the deposition rate relation Rdep = k exp (E/kT) because these processes are typically
thermally activated. Normalized sensitivities for the growth of polysilicon layers are 2.5%/ C of variation.
Real-time control is critical in several aspects of RTP. Although the ramp rate is determined by the
lamp power flux, achieving uniform thermal distribution in both steady-state and during the ramps is difficult
because of heat transfer variations across the wafer. The different view factors, wafer fixturing, and the
variations in radiation and convective heat transfer over the range of temperatures and pressures prevent use
of a single open-loop thermal actuator design to achieve uniform temperature for all conditions. Even a single
closed-loop controller will not work. Process nonlinearity is characterized by gain values and time constants
that vary by an order of magnitude over the operating regime (7).
System design is important in achieving good control performance for RTP. Independently driven thermal
actuators are desirable to compensate for heat transfer variations across the wafer. In a joint TIStanford
design, three independently controllable concentric rings of actuators were used (6). The maximum input
power was 2 kW, 12 kW, and 24 kW (starting with the inner ring), consistent with the increased loss toward
the outer portion of the wafer. Even though reflectors were used, there is some actuator overlap because each
portion of the wafer sees more than one bank of lamps. Thus, the MIMO aspects of the problem must be
10
PROCESS CONTROL
Fig. 5. RTP comparison of open-loop (dashed line) and closed-loop (solid line) operation for two 24 wafer lots (7).
considered at both the system design and control design levels. At the system design level, it is appropriate to
evaluate the condition number of the system, that is, 2 for the DC transfer matrix. In the first design, 2 110,
indicating that there are some output directions that would result in large actions in one or more actuators
(1 = 12.2, 2 = 1.1, 3 = 0.11). This might not be a significant problem because the maximum singular value
corresponds to a condition of uniform temperature, which is the desired command signal. It does indicate that
the lamps could be wired in two independent banks. However, it is difficult to compensate for a disturbance
corresponding to the worst condition. To solve this problem, a baffle was designed to change the heat fluxes
of the actuators to the wafer, thereby reducing the condition number to 23. Note that a low condition number
does not necessarily mean that the actuators are decoupled.
Making accurate measurements is a significant challenge for RTP control. Thermocouples are not desirable because they require contact with the wafer and entail some amount of lag. Pyrometers are noncontact,
but are sensitive to variations in surface emissivity related to temperature and film characteristics. Acoustic
sensors, based on the temperature sensitivity of the wafers elasticity, are being developed that avoid these
problems.
One solution for optical pyrometer drift is to use run-to-run control to estimate the sensor error (7). Figure
5 compares open-loop RTP control with closed-loop. The open-loop varies widely, indicating that conventional
run-to-run control does not perform well because there are no long-term trends. The closed-loop control, in
contrast, has a ramp component, indicative of a systematic sensor error. The run-to-run measurement to detect
this drift would be film thickness, because it is strongly temperature-dependent.
A number of different design approaches have been used to develop the control algorithm (6). Utilization
of independent PID loops does not result in good performance because of the cross talk between inputs. The
large range of operating temperatures requires a design technique to handle the related nonlinearities. The
control design must also deal with the MIMO nature of the process and the significant modeling uncertainties.
One approach is to incorporate both a feedforward and feedback approach implemented with internal model
control (IMC). The process model is obtained from first principles or by system identification. To compensate
for nonlinear variations, gain scheduling is implemented by parameterizing the model coefficients as functions
PROCESS CONTROL
11
of temperature. A linear quadratic Gaussian (LQG) design was also developed on the basis of linear models
obtained from system identification. LQG easily accommodates the MIMO nature of the design problem and
can also be augmented to include integrated error states ensuring zero steady-state error. Results achieved are
control of ramp temperature distribution to 5 C and 0.5 C during hold.
Etching. Etching is a process that transfers patterns to a wafer by removing material not covered by a
mask. Researchers at the University of Michigan have applied real-time control to reactive ion etching (RIE),
which utilizes plasma ions to assist the etching process (8,9). A bias voltage accelerates the ions toward the
wafer surface, enhancing etching normal to the surface. The process chemistry is quite complex including both
gas phase and surface reactions. Etching process objectives include selectivity, the ability to etch only the
desired layer and not other material; uniformity of etch rate over the wafer; anisotropy, the ability to etch the
layer primarily in the vertical direction to prevent overhangs under the mask; and the etch depth.
The process can be divided into two coupled parts: plasma generation and etching. Inputs to the process
include power, gas flow rates, and pressure set point where the pressure is regulated by varying a throttle
valve upstream of a vacuum pump. Fluorine concentration [F] and bias voltage are outputs of the plasma
process and inputs to the etch process. The conventional control practice is to set the power input and flow
rate levels and use closed loop control to maintain chamber pressure. These variables are far away from
the physics that determine the etch characteristics, suggesting that regulating intermediate plasma variables
might yield better performance. A cascaded control structure is used with an inner loop around the plasma
generator and the outer loop using the etch rate to determine the set points of the inner loop. If no real-time etch
rate measurements are available, run-to-run control is used to adjust the set-point values of the intermediate
variables. The function of the closed-loop control of plasma variables is to reject plasma generation process
disturbances. Such disturbances include variations in the RF power and matching network, variations in
the performance of the mass flow controllers, aging of the chamber, due to polymers deposited on the inside
surfaces, and loading variations that occur when the amount of surface area varies (because of the number of
wafers or mask variation).
Experiments indicate that etching has important dynamic characteristics that could also benefit from
closed loop control. At the end of the etch, less material is left so that the etch rate increases. In some cases,
this results in etching the sidewalls, resulting in undercutting. Thus, whereas the conventional practice is to
utilize simple end-point detection by monitoring when new species are introduced into the gas phase, additional
benefits can be achieved by etch rate control. There can also be initial transients due to contaminant variations,
which change total etch time. Having dynamic control capability enables compensation for both of these factors.
Because the plasma generator is nonlinear, either local linear controllers can be developed to operate at
a set point or a strategy to compensate for the nonlinearities can be applied. Important design limitations are
imposed by both sensor and actuator considerations. Because of the poor signal-to-noise ratio of the fluorine
concentration sensor, the bandwidth is limited to less than 1 radian/second. Some of the actuators, such as the
throttle valve, are highly nonlinear, so that the regulation range is restricted for use in linear local controllers.
To minimize steady-state error, free integrator dynamics were added to the controller. Both PID and LQG/LTR
methods are used. The resulting performance of real-time control for maintaining plasma states is shown
in Fig. 6, which indicates superior rejection of load variability and a more uniform etch rate throughout the
process (8).
It is also desirable to control etch uniformity across the wafer surface, requiring a multivariable control
approach. If real-time measurement of etch rate and uniformity output is not practical, a postprocess measurement run-to-run controller is used, built from a response surface mapping of the local plasma states to
the etch outputs (9). Development of the feedback control algorithm requires a more general approach because
the MIMO controller entails significant change in the operating regime. One approach is to consider that the
process dynamics are linear and are coupled to a nonlinear static map of the inputs to output magnitudes
(e.g., a Hammerstein model 9a). One can design a linear dynamic controller that operates in conjunction with
a static controller to compensate for the nonlinearity.
12
PROCESS CONTROL
Fig. 6. RIE etch rate responses for different loading conditions: (a) open-loop, (b) closed-loop (8).
Experimental evaluation reveals that the pure run-to-run approach has results nearly comparable to
the cascade approach for uniformity. This suggests that there may be a lack of good control authority in the
selected operating regime and/or that the disturbances are more related to the etch process and, therefore, are
not compensated for by the plasma real-time controller.
Chemical Vapor Deposition. Chemical vapor deposition (CVD) enables the manufacture of coatings
with a variety of properties used in semiconductor production. Different coating structures and compositions
PROCESS CONTROL
13
are achieved by proper choice of precursors and deposition conditions which to date are primarily determined
empirically. There are numerous CVD variations in terms of different precursors, energy sources, and reactor design. Important processing objectives include composition, structure, thickness, and uniformity of the
deposited coating. Process examples used to provide insight into the application of control include control of
metallorganic CVD (MOCVD) and deposition of TiN.
MOCVD uses liquid precursors in a bubbler to bring a gas phase of the metallorganic molecules into the
reactor. Both III-V and II-VI compounds have been grown for applications, such as light emitting diodes, laser
structures, and photodiodes. Such structures need precise tailoring of the spatial composition of the coating,
which must be repeatable run-to-run. Two approaches employed to control coating composition include (1)
control of the actuators and (2) feedback control from a measurement of composition. The first approach will
work if the system is operating in a regime where mass transport limits growth but fails if chemical reactions
interfere.
Gaffney et al. (10) focus on improving the performance of the input actuators, relying on a feedforward
control approach to meet the composition objectives. This approach works if there are no process disturbances
(other than inlet material variations) and there are no significant dynamics to the process relative to the desired
deposition structures. Utilization of measurements related to the state of the bubbler enable direct feedback
control of this actuator with a simple control law. Evaluation of the performance is established by measuring
the optoelectronic characteristics of superlattices grown with five alternating layers of 3000 GaInAs with
200 layers of InP.
Warnicks et al. (11) approach is to measure the composition and thickness of the coating in real time with
an ellipsometer. Because of the range of compositions desired to grow Alx Ga1 x , the input-output mapping is
nonlinear. The dopant rate dynamics are approximated as a simple integration of feed material inputs with a
time delay to represent the transport delay, without detailed representation of process states and/or coupling. A
Hammerstein model is used because the linear dynamics are assumed to be invariant to the nonlinearities. This
could be valid for a specific operating point, for example, if the bulk pressure and temperature of the process
are held constant but there are variations in composition. A nonlinear mapping is proposed that decouples
the two outputs (composition and thickness) and linearizes the input-output mapping. Thus, linear control
techniques are used for the two resulting decoupled linear systems. Free integrators are incorporated to insure
asymptotic performance of the closed-loop system, in spite of the significant parametric modeling error that
might exist. The bound on transient errors is established by the achievable closed-loop bandwidth. The most
significant limitation is that the accuracy of the measurements is not clearly established, thereby introducing a
possible DC error in all control efforts. The transient performance characteristics of the closed-loop system are
also limited by the dead time. Because the processes are operated in the mass transport regime, the time delay
also corresponds to the rate of growth, and limits how sharp a transition is made. In contrast, the feedforward
approach of actuator control does not have such a limitation.
Major problems with each of these approaches suggest that a combination of methods could achieve better
performance. The feedforward aspect would overcome the limitations posed by the time delay, whereas feedback
would compensate for parameter uncertainties and unmodeled process physics. The combined control system
could result in tighter control of spatial composition, thereby enabling faster growth rates with acceptable
error.
To generalize control of CVD, one should consider the control structure design question. Gevelber et al.
(12,13) evaluated alternative actuator performance capabilities for CVD, where, for many applications, control
of grain size in addition to uniformity is critical. A mechanistic model of nucleation and growth suggests that
independent control of temperature and deposition rate are required to control grain size. To determine which
of the available actuators should be utilized, both the steady-state gain (Fig. 7) and dynamic characteristics
(Fig. 8) of each actuator are evaluated in terms of controlling deposition rate (12,13). This analysis is conducted
as a function of temperature because it strongly parameterizes changes in the dominant physics. Several
input variables are ruled out because they have poor features like sudden reduction of actuator effectiveness
14
PROCESS CONTROL
Fig. 7. Steady-state normalized gains of Rdep for various inputs for TiN CVD (13).
(partial pressure of TiCl4 ). Analysis of the dynamic features are obtained from an analysis of the linearized
system. One of the actuators has right-half-plane zero dynamics (pressure) that would limit the achievable
control performance. In the case presented, a clear choice is given solely in terms of gain because the open-loop
dynamics are similar. Note, however, that the dynamics vary significantly with temperature.
Selection of the appropriate operating regime must account for the variety of conflicting performance
objectives, such as maximizing growth rate while minimizing nonuniformities, while considering the variations
in the dominant process physics. Modeling analysis provides useful insight into answering these questions and
should be directed to obtaining information about the dominant phenomena scale. Thus, for this example,
because the dominant time delay scales with the reactor volume and flow rate Q, as T D /Q, one can adjust
the operating point (here the total flow rate Q), to minimize the fundamental limitation posed by the time delay
(12). In hot-wall batch reactors, the reactant concentration varies throughout the reactor, resulting in varying
growth rate from point to point. A ratio indicating the sensitivity is developed to determine which operating
regime enables one to meet uniformity requirements while maximizing growth rates.
Crystal Growth. Growth of bulk crystals by the Czochralski process is one of the first steps in wafer fabrication. The process begins by bringing a seed crystal to thermal equilibrium with the molten semiconductor,
and pulling the crystal from the melt. Heat transfer is a major phenomenon that determines growth characteristics. Important material properties that determine the crystal quality for different applications include
dislocation density and dopant distribution. Si crystals are currently grown dislocation-free for commercial use
with diameters of 200 mm. Some crystals, however, are more difficult to grow. For example, GaAs has a lower
thermal conductivity than Si and a lower critically resolved shear stress (the stress at which a dislocation is
PROCESS CONTROL
15
Fig. 8. (a) Bode and (b) 2% step response of Rdep for various inputs for TiN CVD, T = 1300K (12).
formed). Thus temperature gradients are higher, resulting in greater thermal stress and higher dislocation
densities.
Dislocations are formed at the melt-crystal interface and also formed and multiplied in the crystal. The
diameter is determined at the interface between the crystal and melt, whereas the dopant distribution is
determined by the melt flow and the interface. Because the nature of this coupling, it cannot be guaranteed
that any arbitrary set of material objectives can be achieved. For example, the interfacial shape that minimizes
dislocation formation might not be optimal for other defects and dopant distribution, suggesting that a tradeoff
16
PROCESS CONTROL
analysis is required. To the extent that the detailed physics can be modeled, the analysis can be formalized.
However, because much of the important physics is not analytically tractable, important insight is obtained
from understanding the physics and experimental investigations.
Feedback systems were developed in the 1960s for automatic diameter control. Several different diameter
measuring schemes are used including optical and weight measurement. Although there are a number of
actuators for the process including heater power, pulling rates, rotation rates of the crystal and crucible
position, and possibly an external magnetic field applied to minimize melt turbulence, typically only heater
power and pulling rate are used for diameter control. However, because the other material objectives are
coupled to the interface, they could adversely be affected by the action of the diameter servo loop.
Examination of the process indicates that several disturbances act on the growth interface (14). Because
most pullers are run with a fixed charge, the reduction in melt level changes the heat flux and, therefore, the
growth dynamics. In addition, as the crystal grows longer, it experiences a changing thermal environment.
Thus, a cascaded control structure with an inner melt temperature loop can eliminate the melt disturbance.
Direct compensation for the changing thermal environment is more difficult unless one considers designing an
independent thermal actuator. The benefit of such an actuator is even more significant if it is desired to control
the diameter and also the interface shape. Gevelbers analysis of the MIMO nature of this problem reveals that
the system is poorly conditioned without such an actuator (15).
Modeling the dominant process physics reveals a right-half-plane zero associated with the weight measurement (14). The modeling provides a parametric expression of the zeros location relative to the process poles
which is used to determine when there are significant performance limitations. The parametric description of
the pole location is also used to understand when the system is underdamped, resulting in growth variations.
This explains, for example, why it is more difficult to grow GaAs crystals and is used to help select the operating
regime. Modeling also suggests important design consequences. For example, although a bottom heater does
not achieve better control authority over the interface shape, it does result in a significantly faster system that
may yield performance advantages.
Conventional control loops on pullers have been implemented with PID logic. Rivera and Seider have
proposed a model predictive controller (MPC) and control structure to help integrate the different process
objectives (16). A cascaded control structure accounts for the different timescales. One loop is the bulk controller
that manipulates the heater power inputs. The objectives of this loop include pulling velocity, thermal stresses,
and the melt dominant distribution. The other loop controls the radius by manipulating the pulling velocity.
MPC coupled with models of the process achieves control solutions which directly take into account constraints
to meet the other objectives.
BIBLIOGRAPHY
1. M. S. Phadke, Quality Engineering Using Robust Design, Englewood Cliffs, NJ: Prentice-Hall, 1989.
2. K. Haefner et al., Real time adaptive spot welding control. In D. E. Hardt (ed.), Control Methods for Manufacturing
Processes. New York: The Dynamic Systems and Controls Division, ASME, November 27December 2, 1988, pp. 5162.
3. W. L. Luyben, Process Modeling, Simulation and Control for Chemical Engineers, 2nd ed., New York: McGraw-Hill,
1990.
4. J. M. Maciejowski, Multivariable Feedback Design, Reading, MA: Addison-Wesley, 1989.
5. C. Porter et al., Improving furnaces with model-based temperature control. Solid State Technology, 39 (11): 119131,
1996.
6. T. Kailath et al., Control for advanced semiconductor device manufacturing: A case history. In W. S. Levine (ed.), The
Control Handbook, Boca Raton, FL: CRC Press, 1996.
7. C. Schaper, T. Kailath, Thermal model validation for rapid thermal chemical vapor deposition of polysilicon. J. Electrochem. Soc., 143 (1): 374381, 1996.
PROCESS CONTROL
17
8. O. D. Patterson, P. P. Khargonekar, Reduction of loading effect in reactive ion etching using real time closed loop control,
J. Electrochem. Soc., 144 (8): 28662871, 1997.
9. M. Hankinson et al., Combined Real-Time and Run-to-Run Control of Etch Depth and Spatial Uniformity in Plasma
Etching, J. Electrochem. Soc., 144 (7): 24732479, 1997. (a) L. Ljung,, System Identification, Englewood Cliffs, NJ:
10. M. S. Gaffney et al., Control of III-V epitaxy in a metalorganic chemical vapor deposition process: impact of source flow
control on composition and thickness, J. Crystal Growth, 167 (12): 816, 1996.
11. S. C. Warnick, M. A. Dahleh, Feedback control of MOCVD growth of submicron compound semiconductor films. IEEE
Trans. Control Syst. Technol., 6: 6271, 1998.
12. M. Gevelber, M. T. Quinones,

M. L. Bufano, Towards closed-loop control of CVD coating microstructure, J. Mater. Sci.
Eng. A, 209: 377383, 1996.
13. M. A. Gevelber et al., Modelling TiN Deposition for Control of CVD. In T. M. Bessmann et al. (eds.), 96-5: Int. Conf.
CVD XIII, 189th Meet. Electrochem. Soc., Los Angeles, May 1996, pp. 157162.
14. M. A. Gevelber, G. Stephanopoulos, Dynamics and control of the Czochralski process, II. Objectives and control structure
design, J. Crystal Growth, 91: 199217, 1988.
15. M. A. Gevelber, G. Stephanopoulos, Control and system design for the Czochralski crystal growth process, J. Dynamic
Syst., Meas., Control, 115 (1): 115121, 1993.
16. R. IrizarryRivera, W. D. Seider, Model-Predictive Control of the Czochralski Crystallization Process: Part I
Conduction-Dominated Melt, J. Crystal Growth, 178: 593611, 1997.
MICHAEL A. GEVELBER
Boston University
602
ROBUST CONTROL ANALYSIS
Robust control can be generally defined as the control, by

means of fixed compensators, of uncertain plants (i.e., of systems with uncertain dynamics and unknown disturbance signals). Robustness in a control system refers to its ability to
cope with uncertainties in a satisfactory manner, maintaining
its stability and performance as desired. Uncertainty in signals and systems is inevitable, reflecting both the complexity
of the physical world and the limitation in human understanding. Uncertain signals typically arise as a result of the
randomness and unpredictability of environmental effects
and are of an unmeasurable and unpredictable nature. Uncertain system dynamics, on the other hand, can be attributed
to changes in the actual system and to modeling errors, be
they accidental or deliberate. Generally, uncertain dynamics
may come from the following sources:
1. Imperfect or incomplete knowledge of physical processes. This represents the information unattainable
because of ones limited knowledge or inadequate measurements. It can be particularly acute for complex systems and processes (e.g., those found in biomedical engineering).
2. Parameter variations. Every physical system will undergo a change in parameter values under different operating conditions. Aging itself can be a factor.
3. Neglected high-frequency dynamics, time delays, nonlinearities, and the like. It may occur as a result of a
sheer lack of knowledge, or the difficulty to model these
characteristics. It may also occur because of the desire
for models of low complexity.

Robustness is a property inherently sought after in engineering systems. The concept is directly linked to such issues
as design viability and system reliability. In broad terms, robustness can be regarded as the capability to withstand unknown, unexpected, and often hostile conditions that can adversely affect a systems behavior. A system must be
sufficiently robust in order to function properly under undesirable circumstances, conducting its task as designed. As engineering systems are becoming more and more complex and
are required to operate in increasingly more uncertain environments, robustness has become increasingly more crucial.
While robustness is a concept of universal significance, robustness analysis for control systems is the study of whether
a system, however designed, can meet specified stability and
performance goals in the presence of uncertainty within a
prescribed range.
Uncertain plants can be modeled in various ways. In particular, models can be stochastic or purely deterministic. In
robust control, uncertain systems are typically modeled deterministically, as bounded sets of system models. A property is
then said to hold robustly if it holds for every model in the
set. The simplest case is that of unstructured uncertainty: the
model set consists of all systems in a certain neighborhood
(e.g., all transfer functions lying within a certain distance
of a distinguished nominal system). Such a description is
particularly appropriate to account for unmodeled dynamics.
One rather typical example is the modeling of flexible structures. It is well known that, in general, a flexible structure
cannot be accurately represented by a finite-dimensional system. For control design purposes, however, we desire, and
most often are compelled to find, an approximate finite-dimensional model with a relatively low order. In doing so, a
common practice is to include in the nominal model a small
number of dominant modes in the low-frequency range and
to treat the high-frequency modes as modeling uncertainty.
Evidently, this description is also appropriate for modeling
errors resulting from model reduction, or from any frequency
response truncation. Moreover, it can be used to cover, albeit
in a conservative way, parameter variations. The latter
amounts to drawing up a frequency response envelope to describe the range of parameter variation in frequency domain.
Finally, in robustness analysis, it is common to introduce a

fictitious uncertainty to represent a performance objective.
A more accurate account of parametric uncertainty calls
for model sets within which individual models are uniquely
characterized by the value(s) of one or more parameter(s)
(e.g., transfer function coefficients). Typically each such parameter takes values in a known range. Such models account
for the fact that parameters in physical devices are bound to
vary with time, environment, and operating conditions. Ackermanns car steering problem (1) offers a good illustrative
example. In his study of a four-wheel steering vehicle, he
found that the vehicle mass and the adhesion between tires
and road surface are significant uncertain parameters. This
is easily understandable. The vehicle mass certainly varies
with load, and in a more subtle sense, it varies with fuel consumption. The adhesion changes with road condition, wearing
of tires, and weather condition. We can think of other uncertain parameters by considering the human-vehicle system as
a whole or by considering a whole batch of vehicles as a family
of systems. In these scenarios, differences among individual
drivers and vehicles can all constitute significant uncertain
factors. Yet a more striking example is an aircraft, whose
aerodynamic coefficients vary in large magnitudes due to
changes in altitude, maneuvering, and weather.
There is an inherent trade-off between fidelity and simplicity in modeling uncertain systems. In a sense, unstructured
and parametric uncertainties may be considered the two extremes. While an unstructured perturbation furnishes a simple characterization and is useful for simplifying robustness
analysis, it contains little information and may often be too
conservative. On the other hand, uncertain parameters often
yield a more natural and accurate model, but such elaborate
descriptions tend to complicate analysis. The process of robustness analysis, therefore, calls for a judicious balance between uncertainty description and complexity of analysis. In
its full generality, however, the description of an uncertain
system should take into account both parametric variations
and unmodeled dynamics. Uncertainty descriptions of this
kind are called structured.
To be sure, robustness is not entirely a new concept in control system analysis and design. In retrospect, the need for a
control system to tolerate unmodeled dynamics and parameter variations is precisely what motivated feedback control,
and it has been a primary goal in control system design since
its birth. This is well recognized in classical control design
and, at least implicitly, is embedded in classical loop shaping
methods. Concepts such as gain and phase margins may well
be regarded as elementary robustness measures. However, it
was not until the late 1970s that the term began to appear
routinely in the control literature, when the need for robustness was reexamined and was gaining increasing recognition. Robust control as a research direction soon thrived and
became a defining theme. After two decades of intense activity, it has evolved into a broad research area rich in theory
and potential applications. The progress has been rapid and
vast, leading to the development of a variety of key concepts
and techniques, among which notably are the H / theory,
the Kharitonov/polynomial approach, and analyses based on
state-space formulations and the Lyapunov theory.
The structured singular value (2), also known as , was
introduced in the early 1980s as a very general framework
for studying structured uncertainty in linear time-invariant
603
models. It defines an exact measure of robust stability and

performance in the frequency domain. The method is a natural progression of earlier work on using singular values to
extend the concept of stability margin to multivariable systems, an idea that was heavily influenced by operator theoretic results such as the Small Gain Theorem. The main impetus for the development of the structured singular value
theory, evidently, has been the recognition that unstructured
uncertainty is too crude a model, often leading to excessive
conservatism.
From a computational point of view, the success of
hinges critically upon whether and how it can be computed
both accurately and efficiently. Unfortunately, this is known
to be difficult. Recent studies have shown that computation of
generally amounts to solving a so-called NP-hard decision
problem, which, in the language of computing theory, is one
that suffers from an exponential growth in its computational
complexity. Although this by no means implies that every
problem is computationally difficult, it nevertheless points to
the unfortunate conclusion that the computation of in general poses an insurmountable difficulty in the worst case. In
retrospect, it thus comes as no surprise that the major progress in computing has been made by some forms of approximation, specifically, readily computable bounds. While no definitive conclusion has been drawn concerning the gap
between and such bounds, it is reassuring that the bounds
are often reasonably tight and that they have other interpretations of engineering significance.
The Kharitonov theory is another robustness analysis
approach, developed in parallel with , which deals almost
entirely with robust stability issues under parametric uncertainty; only in rare cases is unstructured uncertainty also
taken into consideration. The research in this area has a natural heritage from such a classical stability test as the RouthHurwitz criterion and was mainly triggered by a landmark
result published by V. L. Kharitonov (3) in 1978, later referred to as Kharitonovs theorem. Kharitonov considered the
question of stability of parameterized families of polynomials. Here polynomials are thought of as characteristic polynomials of systems described by rational transfer functions and
thus are stable if their zeros all lie in the open left-half of
the complex plane (continuous time) or in the open unit disk
(discrete time). In the continuous-time case, Kharitonov
showed that, for an uncertain interval polynomial whose coefficients each vary independently in a given interval, stability of the entire family can be assessed by testing merely four
simply constructed extreme polynomials. From an aesthetic
point of view, Kharitonovs theorem possesses remarkable elegance, reducing an otherwise seemingly impossible task to a
simple problem. From an engineering perspective, however,
the theorem is likely to find only limited utility because very
rarely would an uncertain system yield a family of characteristic polynomials of the form required in the problem description. Thus, Kharitonovs work triggered a flurry of activities
in the search for more realistic results, and soon came various
generalizations. Two notable features stand out from the robust stability conditions available in this category. First, they
are stated either in terms of a finite number of polynomials
or as graphical tests requiring a frequency sweep. Second, the
main success to date pertains to uncertainty descriptions in
which polynomial coefficients depend linearly on uncertain
parameters.
604
Robust stability analysis in state-space formulation often

comes under the names of stability radius, interval matrix,
and stability bound problems. The key issue here is to determine the largest uncertainty size under which stability is preserved. Unlike in the aforementioned two approaches, parametric uncertainty in state-space representation is defined in
terms of perturbations to system matrices, which can be either unstructured or structured, but most often are only allowed to be real. In a natural way, the robust stability problem translates to one of how the perturbations may affect the
eigenvalues of the system matrix, the solution of which can
draw upon rich theories from linear algebra and Lyapunov
analysis. Thus, unsurprisingly, most of the robustness conditions have this flavor. Alternatively, the problem can also be
recast as one of analysis or one of polynomial stability. In
the former case, we need to compute with respect to solely
real uncertainties, for which the computation schemes are
known to be conservative. In the latter case, the coefficients
in the resultant polynomial will depend on the unknown parameters in a multilinear or multinomial fashion. The Kharitonov approach cannot provide a satisfactory answer for such
polynomial families either. The problem thus appears to be a
very difficult one and is known to have been solved only for a
number of isolated cases. Most notably, recent progress has
made it possible to compute the stability radius in an efficient
manner when the matrix perturbation is unstructured. For
structured uncertainty, unfortunately, the problem is completely open; recent studies showed too that it is in general
NP-hard. Accordingly, the majority of results in the latter
class are sufficient conditions for robust stability.
In conclusion, robustness is a key concept, vital for engineering design in general and for control system design in
particular. Robust control has matured into a field rich in theory and potential applications. By focusing on these three selected areas, this chapter is limited only to robustness analysis of linear time-invariant systems, that is, control of linear
time-invariant plants by linear time-invariant controllers,
which itself is condensed from a vast collection of results and
techniques. Nevertheless, we should note that the concept
and theory of robust control goes far beyond the boundary of
linear time-invariant systems and, in fact, has been quickly
branching to the domains of nonlinear control, adaptive control, and the like. As a whole, it has been, and will continue
to be, a driving force behind the evolution of control theory.
THE STRUCTURED SINGULAR VALUE
Thus each plant in the model set corresponds to the selection of one element in each of the uncertain blocks. The nominal plant corresponds to the choice 0 for all elementary uncertain blocks. Note that the assumption that all the
uncertainty balls have unity radius is made at no cost: any
size information can be embedded into known blocks (e.g.,
connected in cascade with the uncertain blocks). It turns out
that, for uncertain block diagrams of this type, the transfer
function (or transfer function matrix) between any two nodes
(or tuples of nodes) in the block diagram is given by a linear
fractional transformation (LFT). Similarly, when such an uncertain plant is connected with a feedback compensator, the
transfer function between any two nodes will also be an LFT.
LFTs have attractive mathematical properties that can be
used to advantage at the modeling, analysis, and synthesis
stages. More on LFTs can be found, e.g., in Ref. 7.
Robust Stability
It should be intuitively clear that block diagrams of the type
just described can always be redrawn in the form of an
M loop as depicted in Fig. 1 (external input and outputs
have been left out). Here M corresponds to the nominal system, which is comprised of closed-loop transfer functions as
elements and has an input channel and an output channel for
each elementary uncertain block, and is a block-diagonal
matrix whose diagonal blocks are the elementary uncertain
blocks. For its generality, the M loop paradigm has found
wide acceptance in robust control (see, for example, Refs. 2
and 48).
Throughout most of this article, we will assume that the
nominal system, or equivalently M, is linear and time-invariant, as are all instances of the uncertainty blocks, equivalently, of . We will also assume that M and all instances of
are in H . In this case, an immediate payoff of the LFT
uncertainty description and the ensuing representation of the
system via the M loop is the following strong form of the
Small Gain Theorem, a necessary and sufficient condition for
well-posedness and stability of the M loop, in the case
where consists of a single uncertainty block, ranging over
the unit ball in H .
Small Gain Theorem. Let M H . Then the M loop is
well-posed and BIBO stable for all H with 1 if
and only if M 1.
As alluded to earlier, the H norm of a causal, stable, continuous-time transfer function matrix M is defined as
M = sup (M( j))
Uncertain Systems
Throughout this article, we consider model sets P with the
following property: P can be represented by a block diagram
with some of the blocks being fully known systems, and others
being elementary uncertain blocks. The latter are elementary sets, namely, unit balls in simple vector spaces. For
example, some uncertainty blocks might be the real interval
[1, 1] and others might be the unit ball of H , the set of
transfer function matrices (linear time-invariant systems)
that are causal and bounded-input bounded-output (BIBO)
stable; the size or H -norm of H is defined to be the
supremum of its largest singular value over the imaginary
axis (continuous time) or unit disk (discrete time).
where denotes the largest singular value.
Figure 1. M loop.
As an example consider a model set of the output multiplicative uncertainty type. Specifically, let
namics, consider now the model set (1 w1c)Pa with Pa explicitly given as
P = (I + w)P0
where P0 is the transfer matrix of a linear, time-invariant
nominal model, w is a scalar weighting transfer function,
and ranges over the unit ball in H . The weight w is introduced to account for the fact that the amount of uncertainty
is usually frequency-dependent; in particular, system dynamics are often poorly known at high frequencies. Suppose that
a feedback controller K has been tentatively selected to stabilize the nominal system P0. (We use the negative feedback
convention, i.e., the loop transfer function is KP0.) Isolating
from the nominal closed-loop system, we obtain an M
loop with
M = wKP0 (I + KP0 )1
Since K stabilizes P0, M is stable.
As a note of interest, we expect a keen connection between
the Small Gain Theorem and the classical Nyquist criterion.
Indeed, this can be best observed by examining single-input/
single-output systems. In such case P and K are scalar, and
thus so is M. Since both M and are stable, Nyquists criterion implies that the M loop is stable whenever the Nyquist plot of M does not encircle the critical point 1 j0.
Clearly, this will be the case for every satisfying ( jw) 1
for all w if and only if M( jw) 1 holds at all frequencies
(including ).
The Small Gain Theorem suggests that one way to obtain
a robustly stable system, or more generally to obtain a robust
design, is to make sure that the H norm of a certain system
transfer function is small enough. This has triggered an entire field of research known as H design, which is discussed
elsewhere in this encyclopedia. The focus of the present article is the case when is block-diagonal (i.e., when the uncertainty model consists of several blocks or, in other words,
when the uncertainty is structured). Typically, two types of
uncertainty blocks are considered in the literature: (i) the set
of real, constant scalar multiples of the identity, with the scalar having magnitude no larger than one, and (ii) the set of
causal and BIBO stable (H ) transfer function matrices, with
H -norm no larger than one. The latter corresponds to unmodeled dynamics. The former, on the other hand, is used to
represent parametric uncertainty, particularly when a same
uncertain parameter affects more than one coefficients in a
transfer function. For example, concurrent variation as a
function of temperature (e.g., dilation) of multiple quantities
in a mechanical system can result in such a block. This description is more general than the simpler scalar nonrepeated blocks.
Examples with structured uncertainty arise with plants
modeled as being affected by uncertainty at more than one
physical location, e.g.,
P = (I + w1 1 )P0 (I + w2 2 )
where both input and output uncertainty are accounted for.
Another instance arises in the context of the robust performance problem, discussed in a later section. For an example
including both parametric uncertainty and unmodeled dy-
605
Pa (s) =
1
sa
where a can take any value in the interval [0.9,1.1]. We may

write a 1 w2r with w2 0.1 and r 1, and Pa can be
represented as a loop with P0(s) 1/(s 1) in the forward
path and w2r in the feedback path (again using the negative feedback convention). Let K be a feedback controller (still
with the negative feedback convention) that stabilizes P0. By
extracting the uncertainty blocks c and r, we obtain an
M loop with
w2 P0 (I + P0 K)1
M=
w1 P0 (I + P0 K)1

r 0
=
0 c
w2 KP0 (I + P0 K)1
w1 KP0 (I + P0 K)1
where r is a real number ranging over [1,1] and c is a

scalar transfer function ranging over the unit ball in H .
Clearly, the condition M 1 in the Small Gain Theorem
remains sufficient for robust stability when is restricted to
be block-diagonal. However, it is in general no longer necessary. A refinement of the Small Gain Theorem for the structured uncertainty case was proposed by Doyle and Safonov in
the early 1980s (2,4). We adopt here the framework introduced by Doyle, that of the structured singular value also
known as . The Small Theorem states that, if the uncertainty is restricted to be block-diagonal, then the correct refinement is essentially (see Ref. 9 for a precise statement) to
replace M with M, where for a continuous-time transfer
function matrix M,
M = sup (M( j))
and ( ) denotes the structured singular value of its matrix

argument with respect to the block-structure under consideration. The set e is the extended real line; if no
parametric uncertainty is present, however, ( ) is continuous and e can be replaced by . Similarly, for a discrete-time
system,
M = sup (M(e j ))
[0,2 )
But what specifically is ? This is discussed next.

The Structured Singular Value
Let us denote by the set of values taken by ( j) [or (ej)]
as ranges over the set of block diagonal transfer function
matrices of interest, with the unit ball restriction lifted,
namely let
= {diag( r , c , C ) : r r , c c , C C }
606
with
tightened to the following (12):
max R (MQ) (M)
r := {diag(1r Ik , . . ., mr r Ik m ) : ir R}
1
c :=
C :=
C
{diag(1C , . . ., m
)
C
ic
ic
C}
{ 0 : MDM H + GM H MG 2 D < 0}
k m +m +i k m +m +i
r
c
r
c
}
The first and third block types, often referred to as repeated

real and full complex blocks, correspond to values of parametric and dynamic uncertainty, respectively. The second block
type, known as repeated complex, often arises in analyzing
multidimensional (10) and time-delay systems (11), and is
also used sometimes when an LFT state-space representation
of transfer functions is sought (5). It is worth noting that
while as just defined is usually adequate for representing
uncertainties frequently encountered, it can be extended further to accommodate more general situations. For example,
full real blocks (i.e., unknown real matrices) may be added
whenever desired.
The structured singular value (M) of a matrix M with respect to the block structure is defined to be 0 if there is no
such that det(I M) 0, and
;
inf
DD + ,GG
QQ
{diag(1c Ik
, . . ., mc c Ik m +m c )
m r +1
r
(M) = min{ () : det(I M) = 0}
1
otherwise. It can be checked that for structures simply consisting of one full complex block as in the Small Gain Theorem, (M(j) becomes the largest singular value of M(j),
and M is thus equal to M.
Given a matrix M and a block structure , computing
(M) is generally not an easy task. Indeed, this computation
is known to be NP-hard, even when is simplified to a structure containing only full complex blocks. Thus estimates of
(M), e.g., upper and lower bounds on (M), are often used
instead. These, as well as other properties of , are discussed next.
Let U be the set of unitary matrices in and D be the set
of nonsingular matrices that commute with every . The
latter consist of block-diagonal matrices with scalar multiples
of the identity in correspondence with full complex blocks
(C), and with arbitrary blocks in correspondence with those
constrained to be scalar multiples of the identity (r, c). Then
the following result holds:
Here the superscript H indicates complex conjugate transpose, Q is the subset of consisting of matrices whose complex blocks are unitary, D is the subset of D consisting of
Hermitian positive definite matrices, G is the subset of D
consisting of skew-Hermitian matrices (i.e., GH G) with
zero blocks in correspondence with repeated real blocks in ,
and the sign indicates that the matrix expression is constrained to be negative definite. The lower bound in condition
(2) is always equal to (M). The upper bound is never greater
than that in condition (1) (it reduces to it when G 0 is imposed) and, as was the case for condition (1), can be computed
by solving an LMI problem. See, for example, Section 8.12 in
Ref. 8.
For the class of problems where the matrix M has rank
one, Young (13) showed that the right-hand side in inequalities [Eq. (2)] is equal to (M). In that case, Chen et al. (14)
obtained an explicit formula for (M). Let M baH, where a
and b are column vectors. Let also a and b be partitioned into
subvectors ai and bi compatibly with . For i 1, . . ., mr
mc, let i aiHbi. Moreover, define
m
r +m c
|i | +
i=m r +1
Then,
(M) = inf
x

m
ai 2 bi 2
|Re(i ) + x Im(i )| +
1+x
!
2
(3)
Furthermore, if we assume with no loss of generality that for

some l mr, Im(i) 0 for 1 i l, and Im(i) 0 for i
l, and that
Re(l )
Re(1 )

Im(1 )
Im(l )
then the infimum is achieved at one of the following points:
Re(k )
,
Im(k )
l
k = 1, . . ., l
Here R is the largest absolute value of a real eigenvalue of

its matrix argument. Inequalities [see Eq. (1)] are of special
interest in the case of purely complex uncertainty structures.
In that case, (i) the lower bound is equal to (M) and R can
be replaced by the spectral radius , and (ii) the upper bound
is equal to (M) whenever mC 2mc is no greater than 3, and
extensive numerical experimentation suggests that it is never
(or at least seldom) much larger. Moreover, the upper bound
can be computed efficiently by solving a convex optimization
problem, in fact, a linear matrix inequality (LMI) problem.
LMIs define a special class of convex optimization problems
and are discussed elsewhere in this encyclopedia.
For uncertainty structures where real (scalar multiple of
the identity) blocks are present, inequalities [Eq. (1)] can be
x 0 =
;
x k =
DD
i=1
xk =
U U
m r +m c +m C
i=m r +m c +1
(1)
max R (MU ) (M) inf (DMD1 )
(2)
i=1
|Im(i )|
l
i=1
|Im(i )|
k
l
i=1 |Im(i )|
i=k+1 |Im(i )|
l

k
2
i=1 |Im(i )|
i=k+1 |Im(i )|
Finally, the infimum cannot be achieved at x0 unless x0

(, x1], and for k 1, . . ., l, it cannot be achieved at xk
unless xk [xk, xk1].
The rank-one case just alluded to is one of the rare instances for which one can obtain an explicit expression for
(M). This expression not only simplifies the computation of
the upper bound in condition (2) but also was found useful in
studying robust stability of uncertain polynomials. Indeed, as
blocks p(s) such that p 1. It thus follows that robust

performance holds if and only if
1
M
607
sup (M( j)) < 1
Figure 2. Robust performance setup.
will be discovered shortly, an important class of stability

problems for uncertain polynomials can be formulated in
terms of a rank-one problem. Consequently, the result furnishes a unifying tool for the stability problems and a link
between analysis and the Kharitonov approach to robustness analysis.
Robust Performance
A key reason for the popularity of the framework is that it
encompasses not only the robust stability problem but also
the following robust performance problem: determine
whether, for all plants in the given model set, the energy (integral of the square of the magnitude) in a specified error output signal remains below a specified threshold whenever the
disturbance inputs energy is less than a specified value.
Consider the block diagram of Fig. 2 where, as compared
to Fig. 1, external (disturbance) input and (error) output are
made explicit. Given a block diagram such as the one of Fig.
2, the input-output transfer function in the continuous-time
case is given by the linear-fractional transformation
F (M(s), (s)) = M11 (s) + M12 (s)(s)(I M22 (s)(s))1M21 (s)
where Mij(s) is the transfer function from input j to output i
of M(s), i,j 1,2, when the feedback connection through (s)
is removed. [Thus M22(s) is the transfer function matrix formerly denoted M(s).]
The issue at hand is to determine, under the assumption
that M(s) H , whether robust performance holds, that is,
whether it is the case that, for all (s) in our unit uncertainty
ball,
F (M(s), (s)) < 1
(4)
This is readily handled by noting that, in view of the Small

Gain Theorem, for any fixed (s) such that the system is stable, condition (4) is equivalent to the stability of the augmented system depicted in Fig. 3 for all fictitious uncertainty
1
M
Figure 3. Fictitious uncertainty for robust performance.
where now denotes the structured singular value corresponding to the augmented block structure diag(kpkp, )
(i.e., the block-structure corresponding to the actual uncertainty, augmented with a full complex block).
For an example of a typical robust performance problem,
consider an uncertain plant described by the multiplicative
uncertainty model set
P = (I + w1 )P0
with a fixed feedback controller K. It is desired to determine
whether w2S 1 for all possible in a possibly structured
unit uncertainty ball, where S is the sensitivity function (i.e.,
using the negative feedback convention, S (I PK)1).
Here w1 and w2 are stable transfer functions introduced for
frequency-weighting purposes. For simplicity, w1 and w2 are
assumed to be scalars. Using the transformation just outlined, we obtain
w2 (I + P0 K)1
M=
w1 (I + KP0 )1 K
w2 (I + P0 K)1 P0
w1 (I + KP0 )1 KP0

(5)
In the single-input/single-output case, M has rank one. In the

present case mr mc 0, so the right-hand side of Eq. (3) is
simply and the right-hand side of the expression defining
reduces to its second term. Thus
(M( j)) = |w2 ( j)(1 + P0 ( j)K( j))1 |

+ |w1 ( j)(1 + K( j)P0 ( j))1 K( j)P0 ( j)|
and the condition for robust performance can be expressed as
|w2 ( j)| + |w1 ( j)K( j)P0 ( j)| < |1 + P0 ( j)K( j)| Re
Extensions
The structured singular value may be further generalized in
many directions, depending on uncertainty descriptions and
characterizations. Some of these generalizations are summarized next.
Nondiagonal uncertainty structure. The uncertainty
structure need not be diagonal. It can contain unknown,
independently bounded blocks in every entry. Doyle (2)
and Kouvaritakis and Latchman (15) showed that the
analysis may be converted into one based on the standard , but this may lead to a substantial increase in
computational effort. Chen et al. (16,17) proposed a computational scheme that renders the computation growth
insignificant.
Uncertainty with phase information. Tits et al. (18)
adopted a notion of with phase, in which not only are
uncertainties known to be bounded by given quantities,
but also their phases are known to vary in given ranges.
The formulation gives a more detailed uncertainty de-
608
scription, and it requires extensions of the concept of

phase and of .
1-norm bounded uncertainty. Khammash and Pearson
(19,20) studied structured uncertainties bounded in 1
norm, which is another active research area in robust
control, concerning peak-to-peak system response. They
showed that robust stability can be assessed by computing the spectral radius of a positive matrix constructed
from the impulse response of the nominal system.
Time-varying uncertainty. Shamma (21) and Megretsky
(22) examined the robust stability problem with respect
to structured time-varying uncertainties. They showed
that if the uncertainty is allowed to vary arbitrarily over
time, robust stability holds if and only if for some D
D , DM(s)D 1. It is readily checked that the lefthand side (known as scaled H -norm), is similar to the
right-hand side in condition (1), except that here the
same D must be used at all frequencies. Subsequently,
Poolla and Tikku (23) showed that, if the time variation
of the uncertainty is arbitrarily slow, then robust stability holds if and only if the right-hand side in condition
(1) is less than 1 at all frequencies.
Finally, while may be custom made and seems to be an
all-encompassing paradigm when extended appropriately, it
cannot be applied to models in which the uncertainty block
is allowed to be unstable. An effective robustness measure for
the latter situation is furnished by the gap metric, a concept
discussed elsewhere in this encyclopedia.
where q is an unknown vector that may or may not represent

physical parameters. When q varies over a bounded set Q
m, a family of polynomials are generated:
P = { p(s, q) : q Q}
(7)
The problem of concern is to determine if the polynomial family P is robustly Hurwitz stable, by which we mean that every member in P is Hurwitz stable. We shall assume that the
coefficients ak(q) are continuous functions of q. Furthemore,
we assume that an(q) 0 for all q Q (i.e., all polynomials
in P have the same degree). For control system analysis, it is
typical to restrict the polynomial family P to the following
classes, arranged by order of increased complexity.
1. P a: the coefficients ak(q) are affine functions of q. For
example,
p(s, q) = s2 + (q1 + 2q2 + 3)s + (4q1 + 5q2 + 6)
2. P m: the coefficients ak(q) are multiaffine functions of q.
For example,
p(s, q) = s3 + (2q1q2 + 2q1 q3 + q3 + 1)s2
+ (4q2q3 + 5)s + (q1 q2 q3 + 1)
3. P p: the coefficients ak(q) are multivariate polynomials
in q. For example,
p(s, q) = s3 + (2q21 q2 + 2q1 q23 + q1 q3 + 1)s2

+ (4q2 q3 + 5)s + (q21 q22 q23 + 1)
THE KHARITONOV APPROACH

The Kharitonov approach, named after Russian mathematician V. L. Kharitonov whose celebrated 1978 theorem is often
considered to be the cornerstone of the field, is largely concerned with the issue of determining zero locations for a family of polynomials whose coefficients vary in a bounded set.
Hence, by nature, it can be best presented in a framework
different from that of the M loop, or , namely, directly as
a polynomial stability problem. This issue, however, is connected to the M loop paradigm in an intimate fashion. To
see this, simply consider a model set comprising proper real
rational functions whose coefficients take values in certain
bounded intervals. To determine robust stability of such a
plant together with any compensator will then amount to
checking whether the set of all resultant closed-loop characteristic polynomials have zeros in the stability region. For
continuous-time systems, our main focus, the stability region
of interest, is the open left half of the complex plane. Other
regions of interest include the open unit disk, a shifted left
half plane, and a sector; these regions can be imposed to
study stability of discrete-time systems or to enforce pole
placement constraints. A polynomial is generally said to be
Hurwitz stable, or is referred to as a Hurwitz polynomial, if
its zeros lie in the open left half plane.
A general description for a set of polynomials of interest is
p(s, q) =
n

k=0
ak (q)sk ,
(6)
It should be rather evident that P a P m P p and hence

that the complexity in analysis increases in that same order.
At present, the only available methods for tackling P m and
P p are largely ad hoc, via either local optimization or graphical approaches, and they are either conservative or computationally formidable. In particular, when Q is an ball [i.e., a
hyperrectangle (box) parallel to the coordinate axes], the
problem of testing the stability of P m is known to be NP-hard.
The class P a, as the sole tractable case, merits a particularly thorough study. A polynomial family P in this class consists of all polynomials of the form
p(s, q) = p(s, q0 ) +
(qk q0k )pk (s)
(8)
k=0
Here q0 belongs to Q and may be regarded as the nominal

value of uncertain parameter vector q, and the pk(s)s are fixed
polynomials. Evidently, one can assume with no loss of generality that p(s, q0) is Hurwitz stable, which is necessary for P
to be robustly Hurwitz stable as q varies over Q.
Let p [1, ], and let p be the standard p Holder norm
defined on the Euclidean space m. That is,

!1/ p
m

|qi |
, 1 p<
q p =
i=1
max |qi |,
p=
1im
Then a common description for Q adopts the notion of unit

p balls centered at q0, defined as
Q = {q : q q0 p 1}
(9)
When the coefficient ak(q) depends on qk alone [i.e., when

pk(s) is a constant multiple of sk], the p ball description gives
rise to a class of most studied polynomial families. Such families can be expressed as

n

k
0
P =
k qk s : q q p 1
(10)
k=0
for some (possibly different) q0 and some fixed scalars k. In

particular, for p , 1, 2, such P is referred to as an interval
polynomial, a diamond polynomial, and a spherical polynomial, respectively.
Arguably, the entire success of the Kharitonov approach
may be best summarized as a triumph over the polynomial
family in Eq. (8) with the p norm characterization in Eq. (9),
but not beyond, of which the most shining example is Kharitonovs celebrated theorem.
Interval Polynomials
An interval polynomial can be equivalently expressed as

n

k
(11)
P =
q k s : q k q k qk
k=0
where the qks and the qks are fixed. Kharitonovs original
treatment of the interval polynomial problem is of an algebraic nature. He constructed four extremal members of P ,
K1 (s) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 +
K2 (s) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 +
K3 (s) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 +
K4 (s) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 +
later dubbed Kharitonov polynomials. In a remarkable fashion, Kharitonov showed that the stability of the entire interval polynomial family can be ascertained by testing merely
these four.
Kharitonovs Theorem. The interval polynomial is Hurwitz
stable if and only if K1(s), K2(s), K3(s), and K4(s) are all Hurwitz stable.
Subsequent development showed that for polynomials of degree 5, 4, and 3, the test can be further simplified, requiring
checking only 3, 2, and 1 of the four Kharitonov polynomials,
respectively.
Dasgupta (24) gave a geometrical interpretation to Kharitonovs Theorem in frequency domain, which sheds light on
why such a puzzling result would hold. The interpretation
makes use of the concept of value set. The idea is to examine
the image of the polynomial family when s lies on the boundary of the stability region, namely
p( j, Q) = {p( j, q) : q Q}
609
Because p(s, q0) is stable and by assumption an(q) never vanishes on Q, we can conclude from the continuity properties of
the zeros of polynomials that if some member in P is not
Hurwitz stable, then some polynomial in P must have a zero
on the imaginary axis. Thus the entire family P is Hurwitz
stable if and only if
0
/ p( j, Q),
that is, the value set never contains the origin. This is a
rather straightforward fact known as early as in 1929 and is
generally referred to in the literature as the zero exclusion
principle. Note, in particular, that for a polynomial p(s, q) in
the family described by Eq. (11), for any q Q, the real and
imaginary parts of p( j, q) are respectively given by
Re p( j, q) = q0 q2 2 + q4 4
and
Im p( j, q) = j(q1 q3 2 + q5 4 )
Thus the real part (resp. imaginary part) of p( j, q) depends
only on the parameters with even (resp. odd) subscript. For
an interval polynomial, therefore, it becomes clear that
Re K1 ( j) = Re K2 ( j) Re p( j, q) Re K3 ( j) = Re K4 ( j)
Im K1 ( j) = Im K4 ( j) Im p( j, q) Im K3 ( j) = Im K2 ( j)
Because the four Kharitonov polynomials are themselves in
P , it follows that, at each , the value set is a rectangle in
the complex plane with vertices Ki( j), as depicted in Fig. 4.
As increases, the rectangle evolves in a continuous fashion. Its dimensions vary, but its edges remain parallel to the
axes, and the relative positions of the Ki( j)s do not change.
Now, a polynomial p(s) of degree n with positive coefficients
is Hurwitz stable if and only if the phase of p( j) monotonically increases from 0 to n/2 as goes from 0 to . This is
best seen by noting that p(s) can always be factorized as
p(s) an(s si) in terms of its zeros si, and by considering
the phase of each factor separately. When this is understood,
Kharitonovs Theorem becomes self-evident. Indeed, if the
Kis are Hurwitz stable, then the phase of each Ki( j) increases monotonically from 0 to n/2 and the entire Kharitonov rectangle rotates around the origin by a total angle of
n/2 without ever touching it. Stability of P then follows from
the zero exclusion principle.
Kharitonovs Theorem opened an era epitomized by the
search for so-called vertex results in robust stability analysis,
as manifested by the construction of the four Kharitonov polynomials at the vertices of the hyperrectangle Q. This theme
Im
K2(j)
K3(j)
K1(j)
K4(j)
Re
Figure 4. Kharitonov rectangle.
610
persisted in much of the subsequent work. Although Kharitonovs Theorem is without question a milestone in control theory, and perhaps a measured intellectual triumph in general,
we should note that the interval polynomial family is nevertheless a very special instance of an affine uncertainty class,
and hence the theorem has a rather limited scope in application. The quest thus continues.
Edge Theorem
In light of Kharitonovs Theorem, it is tempting to contemplate the possibility that a similar vertex result would hold
for the general affine polynomial family with Q a hyperrectangle. This, unfortunately, turns out to be false; a counterexample can be readily constructed to demonstrate the opposite.
What can be said about a polynomial family P in class P a?
Bartlett et al. (25) provided an answer. In what is now known
as the Edge Theorem, they took the bounding set Q to be a
convex polytope in m. Let qi, i 1, . . ., l, be the vertices of
Q. Then, it is well known that Q can be represented as a
convex hull of the vertices. That is,

Q = conv{q1 , . . ., ql } =
l

i=1
i q i :

i = 1, i 0
i=0
Because ak(q) is an affine function of q, it follows that

P = conv{ p(s, q1 ), . . ., p(s, q l )}
This implies that the value set p( j, Q) is a polygon, generated by the vertex polynomials p(s, qi). It should be rather
clear that the interval polynomial family is generated by a
polytopea hyperrectangleand so is the diamond polynomial family.
Edge Theorem. The affine polynomial family (polytope of
polynomials) P a is Hurwitz stable if and only if for each edge
point q of Q, p(s, q) is Hurwitz stable.
Let qi and qj be two vertices of Q connected by an edge. When
varies from 0 to 1, the polynomial
pij (s, ) = p(s, q i ) + (1 )p(s, q j )
defines a line segment (of polynomials) connecting p(s, qi) and
p(s, qj). The theorem shows thatquoting directly from Ref.
25it suffices to check the edges. Because an edge polynomial
involves only one parameter, its stability can be readily
tested, by resorting to either a graphical test based upon a
root locus, or the numerical solution of a generalized eigenvalue problem (see, for example, Chapter 4 in Ref. 26).
The heuristics behind the Edge Theorem are simple. Because for large values of , the value set does not contain the
origin (this is easy to see), the polynomial family will be Hurwitz stable if and only if there is no frequency at which the
origin belongs to the boundary of the (polygonal) value set. It
should be intuitively clear that every point on the boundary
of the value set must be the image of a point on an edge of
the polytope of polynomials. Thus, if for some the origin
does belong to this boundary, the corresponding edge polynomial must be unstable. Note that it is wasteful to check the
entire set of all polynomial segments joining two vertices of
the polytope because some of these segments are not edges.

In some cases, however, it is not an easy task to check which
such segments are edges and which are not.
The Edge Theorem generalizes Kharitonovs Theorem in
two important aspects. First, it is applicable to general affine
polynomial families, with Q an arbitrary polytope. Secondly,
the stability region may be any open, simply connected set
(27,28), although only stated here in terms of the open left
half plane. In contrast, Kharitonovs Theorem addresses only
Hurwitz stability. On the other hand, as the number of parameters increases, the computation effort required in the
Edge Theorem can be enormous. For example, when Q is a
box in m, with 2m vertices, the number of edges is m2m1, a
very large number even for moderate values of m. Finally, to
a lesser extent, the Edge Theorem is not applicable to situations where the boundary of Q is curved, of which the spherical polynomial family is an example.
Graphical Tests
The complexity of the Edge Theorem provides a direct motivation to search for computationally more tractable stability criteria, and graphical conditions become a natural avenue to
explore. Not only are graphical criteria time-honored tools in
classical stability analysis, but the zero exclusion principle,
with all its simplicity and transparency, also prompts a
deeper investigation of such tools. We can generally feel that
an important asset of the zero exclusion principle is its generality, both in terms of uncertain polynomial families and in
terms of stability regions.
Barmish (29) studied the issue systematically; earlier conditions had appeared sporadically on stability of perturbed
polynomials in isolated cases. Barmishs approach stems from
a geometrical argument: a convex polygon in the complex
plane does not intersect the origin as long as it can be separated from it by a straight line or, equivalently, as long as the
vertices can be separated, as a whole, from the origin by such
a line. This observation led to his construction of a testing
function, which is to be evaluated along the boundary of the
stability region. After this is accomplished, we can determine
stability by plotting the testing function. Barmishs test is certainly one step forward compared to a pure brute-force computation; however, it remains somewhat ad hoc and is computationally overwhelming. Because it requires evaluations at
all vertices, it does not clear the hurdle of exponential growth
in complexity.
On a lesser scale, Tsypkin and Polyak (30) obtained a
graphical test for a simpler problem. They examined the polynomial family in Eq. (10). Let p [1, ] be given. Furthermore, for r [1, ] such that (1/p) (1/r) 1, define
Xr () = (0r + (2 2 )r + (4 4 )r + )1/r
Yr () = (1r + (3 2 )r + (5 4 )r + )1/r
R() = 0 q00 2 q02 2 + 4 q04 4 +
I() = 1 q01 3 q03 2 + 5 q05 4 +
Note that R() Re(p( j, q0)) and I() Im(p( j, q0)).
Tsypkin-Polyak Criterion. The polynomial family P of Eq.
(10) is Hurwitz stable if and only if p(s, q0) is Hurwitz stable,
qn0 1, q00 1, and
|R()| p |I()| p
Xr ()
Yr ()
> 1,
(0, )
(12)
This condition was independently obtained by Hinrichsen and

Pritchard (31).
Up to this point, we could assert that the stability problem
for affine polynomial families remains largely unresolved.
However, as yet another observation, we find that at each ,
the zero exclusion condition defines two linear constraints in
terms of perturbed coefficients, imposed on the real and imaginary parts of p( j, q), respectively. These constraints, together with a convex bounding set Q, define in turn a convex
feasibility condition; when the parameters vary independently, it reduces further to a linear program. This is a simple
but conceptually appealing observation. It led to a reformulation via linear programming, due to Saridereli and Kern (32)
and to Tesi and Vicino (33), which can be solved readily for
each and then plotted graphically. Qiu and Davison (34)
went further to demonstrate that for very general bounding
sets it suffices to solve an optimization problem with one variable only, and the problem can be solved explicitly in special
cases. Finally, Chen et al. (14,35) recognized that the problem
can be reformulated as a special rank-one problem for each
, and showed that stability can be ascertained by evaluating
an explicit formula. These results led to the final resolution
of the affine polynomial family.
and the Kharitonov Approach

Indeed, there is an inherent linkage between analysis and
Kharitonov approach, whenever the latter applies, in that
both approaches yield necessary and sufficient conditions for
problems of the same nature. However, for a rather long time
a clear link seemed elusive. The main cause, it seems, lay in
how to reconcile an optimization-based formulation such as ,
and explicit results from the Kharitonov approach. Can one,
for example, derive Kharitonovs theorem from , or vice
versa?
The explicit formula of rank-one given earlier lends an
answer. Specifically, for a general affine polynomial family
P a [(Eq. (8)] with Q the unit ball, robust stability can be
checked by computing , with a rank-one matrix M(s) constructed as

1

pm (s)
.. p1 (s)
M(s) = .
p(s, q0 )
p(s, q0 )
1
To see this, observe that p(s, q) in Eq. (8) is the characteristic
polynomial of the M loop of Fig. 1 with diag(q1
q10, . . ., qm qm0 ). Indeed,
m

pk (s)
p(s, q)
(qk q0k ) =
det(I M(s)) = 1 +
0
p(s, q )
p(s, q0 )
k=0
Thus, stability of p(s, q) for all q Q is equivalent to stability

of the M loop for all diagonal matrices with real (parametric) entries lying in [1, 1]. The condition for this is that
M 1 where the structured singular value is computed
611
with respect to the block structure diag(1, . . ., m): i

. In light of the formula for rank-one , an explicit condition can then be stated. Such a result clearly applies to general stability regions, and it furnishes a frequency sweeping
condition for robust stability. Note that we may interpret this
result alternatively based upon the zero exclusion principle.
Indeed, under the condition that Q is the unit ball centered
at q0, all the polynomials in P will have zeros in a specified
region if and only if the zeros of p(s, q0) are in that region
and (M(s)) 1 for all s on the boundary of the region. This
follows because, according to the zero exclusion principle, it is
both necessary and sufficient that

m

min q q0 : p(s, q0 ) +
(qk q0k )pk (s) = 0 > 1
k=0
in order for the polynomial family p(s, q) in Eq. (8) to have no

zero on or exterior to the boundary of the region, for all possible q Q.
More generally, it is possible to extend the definition of
by means of more general norms and to use this extended
to study the robust stability of an affine family P with a more
general bounding set Q. Such a generalization also leads to a
similar expression when the M matrix in question has rank
one (14,35). In particular, when the stability region is restricted to the open left half plane, and Q is the unit p ball
centered at q0 with ak(q) kqk, the expression for the generalized rank-one , denoted as p( ) for purpose of distinction,
is found to be
Yr ()/|I()|
if R() = 0
if I() = 0
Xr ()/|R()|
p (M( j)) =
()Y
()
X
r
r
p
p
p
p 1/ p
(Xr ()|I()| + Yr ()|R()| )
otherwise
which leads to a similar condition for robust Hurwitz stability. This condition is slightly more general than, but essentially replicates, the graphical criterion by Tsypkin and Polyak. Note that for p , the polynomial family becomes an
interval polynomial, and the stability condition reduces to
checking whether p(s, q0) is Hurwitz stable, q0 1, and

X1 () Y1 ()
,
< 1,
(0, )
min
|R()| |I()|
A little thought reveals that the latter is equivalent to determining whether one of the four conditions Re(K1( j)) 0,
Re(K3( j)) 0, Im(K1( j)) 0, and Im(K3( j)) 0 holds.
Clearly, this is further equivalent to the requirement that the
rectangular value set in Fig. 4 never contains the origin.
Extensions
There is an immense body of literature devoted to polynomial
stability problems. Various extensions to Kharitonovs Theorem have been obtained. They generally fall into the categories of vertex results and frequency-sweeping conditions, consisting of delicate studies and intricate technical details. We
summarize some of the highlights next. A recent and comprehensive account can be found in the books by Barmish (26)
and by Bhattacharyya et al. (36).
612
Vertex and Edge Results. Much of the work in this direction

continues the thread in Kharitonovs Theorem, focusing on
simple uncertainty descriptions and leading to stability tests
based on vertex and/or edge polynomials. Some notable examples follow.
Complex Interval Polynomials. The polynomial family
has complex coefficients whose real and imaginary parts
are allowed to vary independently in given intervals.
Eight vertex polynomials need to be tested to ascertain
stability.
Diamond Polynomials. This polynomial family is described in Eq. (10), with p 1. Eight vertex polynomials
are required as well.
Stabilization of Interval Plants via First-Order Compensators. The numerator and denominator of the plant transfer function are interval polynomials, and it is to be stabilized by a first-order compensator in closed loop. It
suffices to stabilize 16 vertex plants, constructed based
upon the vertex numerator and denominator polynomials.
Generalized Kharitonov Theorem. It concerns linear combination of interval polynomials and requires checking
certain polynomial segments in addition to vertices.
Other stability conditions based on vertex polynomials are
also available. As the complexity of uncertainty structure increases slightly, they usually require testing more (e.g., 32 or
64) vertex polynomials. A clear insight concerning uncertainty structure and the required number of vertices, however, remains unavailable.
Performance Issues. The entire Kharitonov theory is largely
successful for determining stability of uncertain polynomials.
However, a number of results are also available regarding
properties of transfer functions, which have implications toward performance issues. Some examples follow.
H Norm of Interval Transfer Functions. When the numerator and denominator of a transfer function are both
interval polynomials, the H norm of the transfer function can be computed over 16 vertex transfer functions,
provided that the four Kharitonov polynomials associated with the denominator are stable.
Peak Magnitudes of Closed-Loop Transfer Functions. The
peak H norm of closed-loop transfer functions can be
computed over the edges of the plant family, when it is
an interval plant.
Nyquist and Bode Envelopes. The Nyquist and Bode plots
of open or closed-loop transfer functions associated with
an interval plant lie in envelopes determined by plots
generated by vertex and edge plants.
Other Extensions. Additional extensions may be found in
the following categories.
Schur Stability. The Kharitonov theory has been extended with varying degrees of success to other stability
regions, such as the unit circle. These results are useful
for studying stability of discrete-time systems and for addressing other performance issues.
Unstructured Uncertainty. Unmodeled dynamics may be

included along with parametric uncertainties. They may
be accommodated either in the rank-one formula or by
small gain-type conditions involving vertex and edge
plants.
Nonlinear Systems. In a system consisting of an interval
plant and a static, sector bounded nonlinear component,
stability conditions similar to the Popov and circle criteria have been obtained, which also require the testing of
vertex and edge plants.
Multilinear Uncertainty Structure. The entire success in
the Kharitonov approach relies on the key assumption
that polynomial coefficients depend linearly on uncertain
parameters, and the utility of all the results in this area
is thus measured by how much the uncertainty can deviate from this description. Little success has been
achieved in this endeavor. A fundamental barrier, as implicated by the zero exclusion principle, is that the stability problem is one of optimization subject to nonlinear,
nonconvex constraints.
STATE-SPACE APPROACH
Dynamical systems are often described by state-space equations. Accordingly, it is common to model system uncertainty
as perturbations to system matrices. An uncertain continuous-time system in this spirit may be described by
x(t)
= (A + BC)x(t)
(13)
Here A, B, and C are known matrices of appropriate dimensions. The matrix A is assumed to be stable. The system uncertainty is represented by a set of allowed values for the
real matrix , which may be unstructured or structured. Typical perturbation classes considered in the literature are as
follows, arranged in increasing order of generality. In all
cases, 0 is given.
Unstructured Perturbation. The set consist of all real
matrices with spectral norm less than a given number:
U = { real : () }
Element-by-Element Perturbation. Each element in
varies in a given interval. Let rij 0 be given. The set
is defined as
r11 11 r1m 1m
.
..
..
E = real : =
..
.
. ,
rn1 n1 rnm nm
= max{|ij | : rij > 0}
i, j
Linear Combination. The allowable set of perturbations

is described by

k

L = real: =
i Ei , = max |i |
i=1
where the Eis are given.
613
Evidently, an uncertain discrete-time system can be described

in exactly the same manner.
The problem of interest is to determine the size of the perturbation matrix, measured by a norm of choice, so that the
system remains stable. This issue naturally translates into
one concerning how the eigenvalues of a stable matrix A
would vary when it is perturbed by . More specifically, would
the eigenvalues cross the stability boundary? And if they do,
what is the minimal such that at least one of the eigenvalues leaves the stability region? These questions may be addressed by examining the characteristic polynomial
There essentially exists no result for the structured stability radius other than those already known for . For the unstructured stability radius, much of the early work was devoted to derivation of bounds. One representative example is
(s, ) = det(sI A BC)
Real Stability Radius. Let G(s) C(sI A)1B, and 2( )

be the second largest singular value. Then,
Re[G(s)]
Im[G(s)]
r(A, B, C)1 = sup inf 2 1

Im[G(s)]
Re[G(s)]
s D (0,1]
or equivalently, the characteristic equation

det(I C(sI A)1 B) = 0
Thus, it becomes clear at the outset that the problem may be
tackled in principle by using a polynomial approach. Similarly, it can also be analyzed as a problem. The latter can
be easily seen with respect to E and L, by rearranging the
elements of these sets into diagonal matrices and by defining
the M matrix appropriately. For U, we may simply adopt a
full real block structure and define accordingly. It should
be pointed out, nevertheless, that both and the polynomial
approach will lead to complications in the present context. On
the one hand, the computation of with respect to a real is
generally very difficult, and approximation by its upper bound
can be very conservative. On the other hand, the characteristic polynomial (s, ) will generally exhibit a multilinear or
multinomial dependence of its coefficients on , for which the
Kharitonov theory is ill-equipped; indeed, it is not difficult to
see that the coefficients of (s, ) are multilinear in ij if
E, and are multinomial functions of k if L. In summary,
both approaches are ineffective and conservative.
By far this uncertainty description poses the most difficult
challenge in robust stability analysis, and the state-space approach is the least developed. Results are scarce, and only in
rare cases are they nonconservative.
Stability Radius
A notion frequently encountered in studying the state-space
uncertainty description is that of stability radius. This notion
is closely related to , but it is less developed. Let be a
stability region of concern, and be its boundary. Furthermore, denote by (A) the spectrum of A. Then for any
norm of interest, the stability radius associated with the
triple (A, B, C) is defined by
r(A, B, C) = inf{ : , (A + BC) D = }
In other words, it defines the minimal perturbation size leading to instability, or the distance of A to the set of unstable
matrices. By definition, it thus follows directly that the matrix family A BC: , has all eigenvalues in
whenever r(A, B, C) . Moreover, in view of the preceding
discussion, we may regard the stability radius as the reciprocal of the maximum of a certain , with respect to an appropriate block structure and a matrix M. For further distinction,
the stability radius is said to be unstructured if is unstructured and structured otherwise.
r(A, B, C)
1
C(sI A)1 B
(14)
This, of course, is a rather straightforward consequence of the

Small Gain Theorem. Recently, however, Qiu et al. (37) obtained the following exact, readily computable formula.
The significance of this result lies in that for any s , the

function 2( ) is unimodal in over (0, 1), and hence its infimum can be computed effectively. Furthermore, when is
the open left half plane or the open unit disk, that is, when
Hurwitz or Schur stability is of concern, Sreedhar et al. (38)
developed a fast-converging algorithm for the maximization
with respect to s. Consequently, from a computational standpoint, the unstructured stability radius problem can be considered largely resolved.
Interval Matrices
An interval matrix is a family of real matrices in which all
elements are known only within certain closed intervals. In
precise terms, the interval matrix AI [A, A] is the set of
matrices defined by
AI = {A : a ij aij aij }
that is, each aij of A is confined elementwise to lie within an
interval determined by aij and aij, the corresponding elements
of A and A, respectively. An interval matrix AI is said to be
stable if every A AI is stable. Evidently, interval matrix and
set E share the same uncertainty description.
Interval matrices are direct matrix analogues of interval
polynomials, and hence there has been a lingering temptation
for extension of Kharitonovs Theorem to the former. Unfortunately, neither vertex nor edge results exist for interval matrices. In fact, more recent studies showed that in order to
determine stability of an interval matrix, we must solve an
NP-hard decision problem. This in a way explains why only
sufficient stability conditions are available.
One approach of attack is to analyze eigenvalue distribution. Heinen (39) and Argoun (40) examined the problem on
the basis of Gershgorins Theorem, and their developments
culminated in a subsequent work of Chen (41), leading to a
number of simple, albeit conservative, stability conditions. As
a representative example of these results, consider an interval matrix AI such that aii 0. Let W be constructed as
0
i= j
max{|a ij |, |aij |}
W = [wij ],
wij =
i = j
|aii |
614
Then a sufficient condition for Hurwitz stability of AI is found

by Chen (41) to be
(W ) < 1
A useful feature of this result, and more generally of conditions obtained by using Gershgorins Theorem, is that it lends
a ready characterization via the so-called M-matrices. The
latter aspect makes it possible to unify a number of sufficient
stability conditions in different forms.
Alternatively, Yedavalli and others studied interval matrices from a Lyapunov analysis standpoint. This is collectively
inspected next.
Lyapunov Analysis
The Lyapunov theory, as anticipated, is widely employed in
robust stability analysis pertaining to state-space formulation, yielding various results concerning stability radius and
interval matrices. One common thread in this approach is to
find a single quadratic Lyapunov function applicable to the
entire family of the perturbed matrices; the technique is often
referred to in the literature as quadratic stability. Another
lies in the simplicity of the stability conditions.
Let us begin with the unstructured uncertainty set U. By
constructing the usual Lyapunov function
V (x) = 12 x T Px
C(sI A)1 B < 1/

The latter condition clearly coincides with condition (14).
More results are available for the special case when B
C I. For the structured uncertainty set E, Yedavalli (43)
gave the sufficient condition
|P|R + RT |P|
2
< 1/
(16)
for the Hurwitz stability of the matrix family A :

E. Here P 0 is the unique solution to the Lyapunov equation
PA + AT P = 2I
(17)
P denotes the modulus matrix of P (i.e., each entry of P is

the absolute value of the corresponding entry of P) and R is
given by
r11
.
R = ..
rn1
..
.
r1n
..
.
rnn
k
1
(|PEi + EiT P|)
2 i=1
< 1/
(18)
where, again, P 0 is the unique solution to Eq. (17). Subsequent developments led to further extensions for problems
with even more detailed uncertainty descriptions. For example, the ks may be allowed to vary in asymmetric intervals.
Moreover, because rather obviously any interval matrix can
be represented alternatively in the form of A : E,
these conditions can be applied to determine the Hurwitz stability of an interval matrix as well.
Yet another issue clearly of interest is whether it is possible to derive vertex versions of these sufficient conditions.
Boyd and Yang (44) examined stability problems for matrix
polytopes. Specifically, they postulated the uncertainty description
A = conv{A1 , . . ., Ak }
A sufficient condition for A to be Hurwitz stable can be easily
found to be the existence of a P 0 such that
i = 1, . . ., k
(19)
Similarly, for the uncertainty set L, a vertex condition can

be obtained as
(15)
This, of course, does not come as a surprise. According to the

well-known Bounded Real Lemma (42), it is equivalent to
PAi + ATi P < 0,
we find that the entire family of matrices A BC: U

is Hurwitz stable if there exists a positive definite matrix
P 0 such that
PA + AT P + 2 BBT + CTC < 0
Furthermore, Zhou and Khargonekar (42) observed that the

uncertainty description E can be regarded as a special case
of L, for which they provided the stronger Hurwitz stability
condition
k
1
(PEi + EiT P)
2 i=1 i
< 1/
(20)
for all combinations of the i in 1, 1. It should be rather

evident that this condition improves upon inequality (18).
Both conditions (19) and (20) may be regarded as vertex results in the matrix perturbation case, and both can be posed
and solved as LMI problems.
CONCLUSION
Summary
For the past two decades, modeling uncertainty and robustness has resurfaced as a dominating theme in control theory and application and is now held unanimously by theoreticians and practitioners as the most important concern in
control system design. For both its intrinsic appeal and practical significance, robust control as a whole attracted considerable interest and underwent a period of immense development, bringing control theory to a new height. Many
important issues have been addressed. Many remain unresolved. The ultimate puzzle, it now appears, lies in the fundamental conflict between problem complexity and computational tractability.
Of the three main research areas surveyed in this article,
the structured singular value provides the most general formulation for uncertainty modeling and is the most systematically developed tool in robustness analysis. The major issues
in analysis are clearly generality of uncertainty description,

conservatism of analysis, and ease of computation. The main
success achieved with this approach, unquestionably, lies in
the progress in computing . While it cannot be computed
exactly in general, various computational schemes have been
developed for computing it approximately, and commercial
software programs are now available. This paves the way for
its application to a series of engineering design problems,
ranging from disk drive control to flight control. Successful
applications to other potential areas, including robot manipulators, flexible structures, magnetic bearings, and chemical
processes, have also been reported in laboratory experiments.
The Kharitonov approach, unlike analysis, was more restrictive in scope in its early phase of development. However,
it has undergone a bottom-up growth pattern as the uncertainty descriptions become progressively more general and sophisticated. Overall, the Kharitonov and state-space methods
may be broadly classified as a parametric approach toward
robustness analysis, originating from interval polynomials
and culminating at state-space uncertainty descriptions. The
main appeal of this approach, it appears, lies in its quest for
analytical solutions, more appealing than mere computationbased tools. The main success in the entire parametric approach, which remains the state-of-the-art today, is the resolution of the affine uncertain polynomial family case, for
which necessary and sufficient stability conditions are available, in terms of both edge tests and graphical conditions. On
the other hand, the multilinear/multinomial polynomial family and the state-space uncertainty description are the weakest link, for which only sufficient stability conditions are
available with unknown conservatism, and more systematic,
efficient, computation-based approximate tests are called for.
At present, only a few applications of the Kharitonov theory
are reported in the literature, including Ackermanns car
steering problem and an automotive engine control problem
investigated by Abate et al. (45) (see also Chapter 3 in Ref.
26). It should be rather evident that the fundamental bottleneck in all robustness analysis methods, be it analysis or
Kharitonov approach, lies in computational complexity, and
the ultimate challenge is in the conquest over the curse of
dimensionality. No matter whether this can be achieved or
not, we should be consciously aware that the dilemma is the
natural cause of problem generality and hence complexity and
results from the search of optimal solutions. In engineering
system design, we should therefore reconcile and seek a judicious trade-off between these conflicting requirements.
To Probe Further
In light of the difficulties encountered in robustness analysis
with respect to structured and/or parametric uncertainties, a
number of researchers recently examined complexity issues
from a computational standpoint, drawing upon concepts and
techniques from computing science and operation research.
The main discoveries are in the following areas.
with Real Uncertainties. Braatz et al. (46) showed that
the computation of real is NP-hard.
with Complex Uncertainties. Toker and Ozbay (47)
proved that the computation of complex is also NPhard.
615
with Real and Complex Uncertainties. Braatz et al. (46)

and Toker and Ozbay (47) both showed that the computation of is NP-hard.
The Bounds. Toker (48) and Fu (49) showed that the
problem of finding an accurate bound for is NP-hard.
Interval Matrix. Coxson and DeMarco (50) showed that
stability of interval matrices amounts to an NP-hard
problem.
These results indicate that a worst-case instance exists in
each class of the problems, for which it is rather unlikely that
computational complexity can be bounded via a polynomial
function of the problem dimension. It thus comes as no surprise that the problems are difficult, and indeed are intractable in general.
From a technical standpoint, the computational difficulty
in question may be best seen as an outcome of nonlinear, nonconvex optimization problems. Although only explored systematically in recent years, complexity issues have been under contemplation for a long time and have led to alternative,
computationally tractable approximations and formulations.
One notable remedy is to resort to formulations based upon
LMIs, and problems in this class include those that can be
described via integral quadratic constraints (IQC). Both LMIs
and IQCs offer in essence an energy-based perspective toward
system analysis, and they draw heavily upon concepts in classical passivity and dissipativity theory, leading to readily
computable, albeit only sufficient, robust stability conditions.
For a comprehensive treatment of control-relevant LMI and
convex programming problems, see Ref. 51, or the relevant
chapter in this encyclopedia. Megretsky and Rantzer (52) provided a detailed account of the IQC technique.
The computational complexity results just discussed are
strongly linked to the worst-case nature of the robustness
problems; that is, the requirement of robustness must be met
for all possible instances. Is so stringent a requirement truly
necessary? This question prompted a reexamination of robustness issues, and it led to a recent venture departing almost entirely from the worst-case formulation. A number of
researchers argued that worst-case scenarios hardly occur in
practice, that a worst-case analysis is not only overly demanding but also too pessimistic, and that, after all, worstcase analysis problems are often intractable. The argument
thus motivated the description of uncertainty via probabilistic
measures, and accordingly probabilistic approaches to robustness analysis. In this new thinking, the deterministic uncertainty description is discarded altogether and is replaced
by a probability description characterizing the likelihood that
the uncertainty may lie in a bounded set. The robustness condition then amounts to determining the probability under
which the system may become unstable. Recent studies (53
56) show that a variety of problems, which are NP-hard in
the deterministic setting, become readily solvable computationally when formulated probabilistically. The area, however, is entirely open and is not without obstacles of its own.
BIBLIOGRAPHY
1. J. Ackermann, Robust Control Systems with Uncertain Physical
Parameters, London: Springer-Verlag, 1996.
616
2. J. Doyle, Analysis of feedback systems with structured uncertainties, IEE Proc., 129-D (6): 242250, 1982.
3. V. L. Kharitonov, Asymptotic stability of an equilibrium position
of a family of systems of linear differential equations, Differentialnye Uraveniya, 14 (11): 14831485, 1978.
4. M. G. Safonov, Stability margins of diagonally perturbed multivariable feedback systems, IEE Proc., 129-D: 251256, 1982.
5. A. Packard and J. C. Doyle, The complex structured singular
value, Automatica, 29 (1): 71109, 1993.
6. W. S. Levine (ed.), The Control Handbook, Boca Raton, FL: CRC
Press, 1996.
9. A. L. Tits and V. Balakrishnan, Small- theorems with frequency-dependent uncertainty bounds, Math. Control. Signals,
Syst., in press, 1998.
10. J. M. Krause, Structured singular value analysis of multidimensional system stability, IEEE Trans. Autom. Control, 34 (6): 638
639, 1989.
11. J. Chen and H. A. Latchman, Frequency sweeping tests for stability independent of delay, IEEE Trans. Autom. Control, 40 (9):
16401645, 1995.
12. M. K. H. Fan, A. L. Tits, and J. C. Doyle, Robustness in the presence of mixed parametric uncertainty and unmodelled dynamics,
IEEE Trans. Autom. Control, 36 (1): 2538, 1991.
13. P. M. Young, The rank one mixed problem and Kharitonovtype analysis, Automatica, 30: 18991911, 1994.
14. J. Chen, M. K. H. Fan, and C. N. Nett, Structured singular value
and stability of uncertain polynomials, Part 1: The generalized ,
Syst. Control Lett., 23 (1): 5365, 1994.
15. B. Kouvaritakis and H. Latchman, Necessary and sufficient stability criterion for systems with structured uncertainties: The
major principle direction alignment principle, Int. J. Control, 42
(3): 575598, 1985.
16. J. Chen, M. K. H. Fan, and C. N. Nett, Structured singular values
with non-diagonal structures, Part 1: Characterizations, IEEE
with non-diagonal structures, Part 2: Computations, IEEE Trans.
Autom. Control, 41 (10): 15111516, 1996.
18. A. L. Tits, V. Balakrishnan, and L. Lee, Robustness under
bounded uncertainty with phase information, IEEE TAC, in press.
19. M. Khammash and J. B. Pearson, Robust disturbance rejection in
l1-optimal control systems, Syst. Control Lett., 14: 93101, 1990.
20. M. Khammash and J. B. Pearson, Performance robustness of discrete-time systems with structured uncertainty, IEEE Trans. Autom. Control, 36 (4): 398412, 1991.
21. J. Shamma, Robust stability with time-varying structured uncertainty. IEEE Trans. Autom. Control, 39 (4): 714724, 1994.
22. A. Megretski, Necessary and sufficient conditions of stability: A
multiloop generalization of the circle criterion, IEEE Trans. Autom. Control, 38 (5): 753756, 1993.
23. K. Poolla and A. Tikku, Robust performance against time-varying
structured perturbations, IEEE Trans. Autom. Control, 40 (9):
15891602, 1995.
24. S. Dasgupta, Kharitonovs theorem revisited, Syst. Control Lett.,
11: 381384, 1988.
25. A. C. Bartlett, C. V. Hollot, and L. Huang, Root locations of an
entire polytope of polynomials: It suffices to check the edges,
Math. Control, Signals, Syst., 1: 6171, 1988.
26. B. R. Barmish, New Tools for Robustness of Linear Systems, New
York: Macmillan, 1994.
27. M. Fu and B. R. Barmish, Polytopes of polynomials with zeros in

a prescribed set, IEEE Trans. Autom. Control, 34 (5): 544546,
1989.
28. A. L. Tits, Comments on polytopes of polynomials with zeros in
a prescribed set, IEEE Trans. Autom. Control, 35 (11): 1276
1277, 1990.
29. B. R. Barmish, A generalization of Kharitonovs four polynomial
concept for robust stability problems with linearly dependent coefficient perturbations, IEEE Trans. Autom. Control, 34 (2): 157
165, 1989.
30. Ya. Z. Tsypkin and B. T. Polyak, Frequency domain criteria for
lp-robust stability of continuous linear systems, IEEE Trans. Autom. Control, 36 (12): 14641469, 1991.
31. D. Hinrichsen and A. J. Pritchard, An application of state space
methods to obtain explicit formulae for robustness measures of
polynomials, in M. Milanese et al. (ed.), Robustness in Identification and Control, Boston, MA: Birkhauser, 1989.
32. M. K. Saridereli and F. J. Kern, The stability of polynomials under correlated coefficient perturbations, Proc. 26th IEEE Conf.
Decision Control, 1987, pp. 16181621.
33. A. Tesi and A. Vicino, Robustness analysis for linear dynamical
systems with linearly correlated parametric uncertainties, IEEE
34. L. Qiu and E. J. Davison, A simple procedure for the exact stability robustness computation of polynomials with affine coefficient
perturbations, Syst. Control Lett., 13: 413420, 1989.
and stability of uncertain polynomials, Part 2: A missing link,
36. S. P. Bhattacharyya, H. Chapellat, and L. H. Keel, Robust Control: The Parametric Approach, Upper Saddle River, NJ: PrenticeHall, 1995.
37. L. Qiu et al., A formula for computation of the real stability radius, Automatica, 31 (6): 879890, 1995.
38. J. Sreedhar, P. Van Dooren, and A. L. Tits, A fast algorithm to
compute the real structured stability radius, in R. Jeltsch and M.
Mansour (eds.), Stability Theory: Proc. Centenary Conf. Int. Series
Numerical Math. (ISNM), Basel: Birkhauser, 1996, vol. 232, pp.
219230.
39. J. A. Heinen, Sufficient conditions for stability of interval matrices, Int. J. Control, 39 (6): 13231328, 1984.
40. M. B. Argoun, On sufficient conditions for the stability of interval
matrices, Int. J. Control, 44 (5): 12451250, 1986.
41. J. Chen, Sufficient conditions on stability of interval matrices:
Connections and new results, IEEE Trans. Autom. Control, 37
(4): 541544, 1992.
42. K. Zhou and P. P. Khargonekar, Stability robustness bounds for
linear state-space models with structured uncertainty, IEEE
43. R. K. Yedavalli, Improved measures of stability robustness for
linear state space models, IEEE Trans. Autom. Control, 30 (6):
577579, 1985.
44. S. Boyd and Q. Yang, Structured and simultaneously Lyapunov
functions for system stability problems, Int. J. Control, 49 (6):
22152240, 1989.
45. M. Abate et al., Application of some new tools to robust stability
analysis of spark ignition engines: A case study, Proc. Amer. Control Conf., 1992, pp. 932936.
46. R. P. Braatz et al., Computational complexity of calculation,
47. O. Toker and H. Ozbay, On the complexity of purely complex
computation and related problems in multidimensional systems,
IEEE Trans. Autom. Control., 43 (3): 409414, 1998.
ROUNDOFF ERRORS
48. O. Toker, On the conservatism of upper bound tests for structured singular value analysis, Proc. 35th Conf. Decision Control,
Kobe, Japan: 1996, pp. 12951300.
49. M. Fu, The real is hardly approximable, IEEE Trans. Autom.
Control, 42 (9): 12861288, 1997.
50. G. E. Coxson and C. L. DeMarco, The computation complexity of
approximating the minimal perturbation scaling to achieve instability in an interval matrix, Math. Control, Signals Syst., 7 (4):
279292, 1994.
Theory, Studies in Applied Mathematics, Philadelphia: SIAM,
1994, vol. 15.
52. A. Megretski and A. Rantzer, System analysis via integral quadratic constraints, IEEE Trans. Autom. Control, 42 (6): 819
830, 1997.
53. R. F. Stengel and L. R. Ray, Stochastic robustness of linear timeinvariant systems, IEEE Trans. Autom. Control, 36 (1): 8287,
1992.
54. B. R. Barmish and C. M. Lagoa, The uniform distribution: A rigorous justification for its use in robustness analysis, Proc. 35th
IEEE Conf. Decision Control, 1996, pp. 34183423.
55. R. Tempo, E. W. Bai, and F. Dabbene, Probabilistic robustness
analysis: Explicit bounds for the minimum number of samples,
Proc. 35th IEEE Conf. Decision Control, 1996, pp. 34243428.
56. X. Chen and K. Zhou, On the probabilistic characterization of
model uncertainty and robustness, Proc. 36th IEEE Conf. Decision
Control, 1997, pp. 38163821.
JIE CHEN
ANDRE L. TITS
ROBUSTNESS ANALYSIS. See ROBUST CONTROL

ANALYSIS.
ROBUST SIGNAL PROCESSING. See NONLINEAR

SYSTEMS.
ROLLBACK RECOVERY. See PROGRAM DIAGNOSTICS.

ROLLING METALS. See METALS INDUSTRY.
ROTATION SPEED. See TACHOMETERS.
617
SELF-TUNING REGULATORS
A self-tuning regulator (STR) is a controller that automatically nds its parameters in the control law. Another name
or synonym is self-adjusting controller. STR is a class of
adaptive controllers used when the process to be controlled
has constant but unknown parameters. However, STR can
also be used in an adaptive context.
BASIC IDEA
Controller Design Procedure
The design of a controller contains several steps:
1. Finding specications for the closed-loop system
2. Determination of a model for the process to be controlled
3. Decision on a design method
4. Calculation of the parameters in the controller
In many cases, it is desirable to automate these steps. This
is the idea behind adaptive and self-tuning regulators.
The specications for the closed-loop system depend on
such things as quality constraints on the controlled variable, available magnitude (power) of the control signal, and
nonlinearities of the system to be controlled. This implies
that the specications are determined by the process engineer at the start of the design procedure. The specications often lead to a natural choice of the design method.
For instance, if the main specication is to keep the process output constant and if the disturbances are occasional
large disturbances, then the design procedure can be a
method that as quickly as possible eliminates the inuence of the disturbance. The choice of specications and
design method is thus usually made by the designer of the
control loop. In STRs, as well as in adaptive controllers,
steps 2 and 4 above are automatically taken care of by the
controller.
The structure of a self-tuning controller is best described
from the block diagram in Fig. 1. The self-tuning regulator
consists of two closed loops. The rst loop is a conventional
controller feedback-loop consisting of the process and the
controller where the output of the process is measured and
compared with the desired output (reference signal) of the
closed-loop system. The mismatch between the reference
and output signals is used to compute the control action
that is sent to the process. The controller has parameters
that determine its properties. These parameters are determined by the second loop in the STR, the updating loop.
In Fig. 1, the updating loop has two main blocks. The
rst block is an estimator, which determines a mathematical model of the process based on the measured inputs
and outputs. The second block carries out the design of the
controller. This block uses the process model and the specications to determine the controller parameters that then
are sent to the controller.
It is necessary that the controller feedback-loop be
closed all the time to take care of the inuence of disturbances and changes in the reference signal. The updating
loop for the controller parameters can be switched off as

soon as the estimated parameters have converged to their
nal values, that is, when the controller has tuned or adjusted itself to the specications and the process. The result
is a self-tuning regulator. However, if the process is changing over time it is necessary to update continuously the process model and the controller parameters. We then have an
adaptive controller. This implies that an STR is an adaptive controller if the parameter updating is not switched off.
The STRs are thus a special class of adaptive controllers.
One of the rst descriptions of the idea of STRs is found
in Kalman (1) where updating using parameter estimation
and design is described. The term self-tuning regulator was
om and Wittenmark (2) who gave the rst
coined by Astr
analysis of the steady-state properties of the STR based
on minimum variance control. The stability of the closedloop system and the convergence properties were analyzed in Goodwin, Ramadge and Caines (3). More details
of the properties of self-tuning and adaptive controllers
om and
can be found in Wellstread and Zarrop (4) and Astr
Wittenmark (5).
Classication of Self-Tuning Regulators
The STR in Fig. 1 contains both a block for estimation and
a block for design. An STR in this conguration is usually called an indirect self-tuning regulator. The reason is
that the controller parameters are obtained indirectly by
rst nding a process model. In many cases it is possible to make a reparameterization of the process and the
controller such that the controller parameters can be estimated directly. This leads to a direct STR. Ways to do this
reparameterization are discussed below.
Applications of Self-Tuning Regulators
The computations in an STR are quite straightforward,
but contain nonlinear and logical operations. This implies
that STRs are implemented using computers. The algorithm can be a block in a software package that is used
for larger process control applications, or the STR can be
implemented in dedicated hardware for a few control loops.
Self-tuning control has, since the mid-1970s, been used
for many applications, mainly in the process industry. Applications are found in areas of pulp and paper, chemical
reactors, autopilots, and dialysis machines.
Self-tuning regulators and adaptive controllers in general have found their main uses in three categories of applications:
When the process has long time delays

When feedforward can be used
When the disturbances acting on the process have
time-varying characteristics
The main reason self-tuning or adaptive controllers have
a great advantage in these cases is that for good control
of these types of processes it is necessary to have models
of the process and/or of the disturbances to be controlled.
The estimator part of the self-tuning controller can make
an estimate of the process and use that in the design.
Self-Tuning Regulators
Figure 1. Block diagram of a self-tuning regulator.
Linear STRs are not appropriate to use when the process is very nonlinear. The updating mechanism will then
not be sufciently fast. In such cases the nonlinearities
should be built into the process model and the controller.
the system can be written as
or
ALGORITHMS FOR SELF-TUNING CONTROL
This section describes in more detail how STRs are constructed. It also gives the main properties of STRs. To describe the algorithms we need to specify the process model,
the specications, the controller, the estimator, and the design method. We will use discrete-time models for the process and the controller since most implementations of STRs
are done using computers. It is, however, also possible to
derive continuous-time STRs.
Process Model
The process is described as a sampled-data linear system.
The process is also assumed to have a single input and a
single output. The model is given as a difference equation
where y(k) is the output signal at sampling instant k and

u(k) is the control signal. Disturbances will be introduced
below. It is assumed that the time is scaled such that the
sampling period is one time unit. The parameter d is the
time delay of the system. Equation (1) is a general description of a linear sampled-data system. To get a more compact
way of describing the system, we introduce the backwardshift operator q1 . The backward-shift operator is dened
in the following way
That is, operating on a time sequence it shifts the time

argument one step backwards. Using the backward-shift
operator and the polynomials
where H(q1 ) is called the pulse-transfer function.

Specications
The specications give the desired performance of the
closed-loop system. The specications can be given in many
different ways depending on the purpose of the closed-loop
system. It is common to distinguish between the servo and
regulator cases.
In the servo case, we give the desired performance in the
form of the time or frequency response when the reference
value is changed or when an occasional large disturbance
has inuenced the system. The typical specications are
bandwidth or response time. Further, things as overshoot
or damping can be specied. One way to give the specications is in the form of a reference model Hm dening the
desired output ym
where uc is the reference signal. Normally, dm = d, but may

also be longer.
In the regulator case, we study the closed-loop performance when disturbances essentially are acting on the
system while the reference signal is constant. The disturbance is then usually modeled as a stochastic process, in
general, ltered white noise. Typical performance indices
are to minimize the variance of the output signal around
a desired reference value or to minimize a combination of
output and input variations.
Controller
The controller is dened as
The controller has a feedback part dened by the polynomials R*(q1 ) and S*(q1 ) and a feedforward part dened
by R*(q1 ) and T*(q1 ). Using Eq. (4) on the process of Eq.
(2) gives the closed-loop system
Since the updating formulas of Eq. (7) are recursive,

they can be used also for a continuous updating of the parameters. In such cases it is, however, necessary to introduce a weighting of old inputs and outputs. The loss function of Eq. (6) puts equal weight on all data. A measurement
collected a long time ago is as important as the latest measurement. Newer measurements can be given more weight
by changing the loss function of Eq. (6) to
Estimator
Estimation of process models can be done in many different ways. Summaries of methods and their properties
can be found in Ljung (6), Soderstrom and Stoica (7), and
Johansson (8). Here only the recursive least squares (RLS)
method will be discussed. Dene the vectors
The vector contains the unknown process parameters,

while the vector contains known old inputs and outputs
of the process. The process model can now be written as
The least squares method, rst stated in Gauss (10), im which

plies that the estimate of should be chosen as ,
minimizes the loss function
Given an initial value of the parameters (0)

and the uncertainty of the parameter estimate P(0), it is possible to
derive a recursive solution to the least squares problem.
The parameter estimate can be updated recursively using
This is called the recursive least squares algorithm. The estimate at time k is obtained as an update of the estimate at
time k 1. The correction term depends on the latest process output, which is compared with the predicted output
based on the parameter estimate at time k 1. The matrix
P(k) can be interpreted as an estimate of the uncertainty of
the parameter estimate at time k. The statistical interpretation can be made rigorous by making assumptions about
the disturbances that are acting on the system.
The recursive least squares method is well suited for
process parameter estimation when there are no disturbances or when a white noise process is added to the righthand side of Eq. (2). For other noise or disturbance assumptions, there are variants of the recursive least squares
method that can be used.
where is the forgetting factor. Since the weights are exponentially decaying, the resulting algorithm is called recursive least squares with exponential forgetting. The updating formulas are only slightly modied into
In the following we will use the recursive least squares

algorithm, with or without exponential forgetting, to illustrate the properties of STRs.
Design Methods
The nal step in the construction of an STR is the design procedure. The basic STRs are based on the certainty
equivalence principle. This implies that the process parameter estimates obtained from the estimator are used as if
they are the true ones. The design principles can, however, be extended to include also the uncertainty of the
estimates, given by the P matrix. This leads to so-called
cautious or dual controllers.
Two different design principles will be discussed in detail, pole-placement design and minimum variance control.
In-depth treatments of the design methods can be found in
om and Wittenmark (9).
Astr
Pole-Placement Design. We will now discuss how the parameters in the controller in Eq. (4) can be determined
using the method of pole placement for the design of the
controller. The closed-loop system is dened by Eq. (5). The
closed-loop characteristic polynomial is thus
where A*c is given as a specication by the designer. The

key idea is now to nd the controller polynomials R* and S*
that fulll this equation. Equation (8) is called a diophantine equation. The desired closed-loop system from the reference signal to the output dened by Eq. (3) requires that
the following condition must hold
This design procedure is called model-following design, and

also pole-placement design, if the poles only are specied.
Whether model following can be obtained depends on the

model, the process, and the complexity of the controller.
The characteristic polynomial of Eq. (8) will, in general, have higher degree than the model polynomial Am .
This implies that there must be a polezero cancellation in
Eq. (9). The consequences of this will now be discussed. The
B* polynomial of the process is rst factored into
where B+ corresponds to the process zeros that can be

cancelled in the design. These zeros must be located inside
the unit circle. The zeros corresponding to B , which are
not allowed to be canceled, must then be a factor of Bm ,
which must have the form
Since B+ is canceled, it must be a factor of Ac . The closedloop characteristic polynomials is thus of the form
The polynomial Ao is called the observer polynomial, and

can be interpreted as the dynamics of a state observer. The
observer polynomial inuences, for instance, how fast the
system will recover after a disturbance. Ao is determined
by the designer and should be a stable polynomial.
Since B+ is a factor of B* and Ac , it follows from
Eq. (10) that it also is a factor of R*, which implies that
variance controller is obtained by solving the diophantine

equation
and using the control law
Also, linear quadratic gaussian controllers can be interpreted as solving a special form of the diophantine equa om and Wittenmark (9).
tion, see Astr
Design of Self-Tuning Regulators
The design of STRs can be summarized by the following
procedure:
Specications. Determine the class of controller by determining the specications on the closed-loop system.
Estimation. Estimate the process parameters using, for
instance, the recursive least squares algorithm of
Eq. (7).
Design Procedure. Determine the controller parameters using the estimated process parameters as if
they are the correct ones. The controller design is
usually reduced to the solution of an equation such
as the diophantine Eq. (8).
Control. Update the parameters of the controller, for instance, in the form of the controller in Eq. (4).
and the diophantine equation reduces to

The estimation, design, and control steps are done at each
sampling interval. In some situations, it may be sufcient
to update the estimation at a slower rate than the rate
of the control loop. The behavior of the basic indirect selftuning algorithm will be described by an example.
Finally, the polynomial T* is given by
The design procedure can now be summarized into:

+
Data. Given the process polynomials A*, B* = B B ,

and the observer polynomial Ao
Step 1. Solve the diophantine equation with respect to
R* and S*.
Step 2. The controller is given by Eq. (4) with R* =

R* B+ and T* = Ao Bm .
The diophantine equation can always be solved if there are
no common factors between the A* and B* polynomials and
if the controller polynomial has sufciently many parameters.
Minimum Variance Control. Most design procedures can
be interpreted as a pole-placement or model-following design. For instance, the minimum variance controller can
easily be formulated in this form. The minimum variance
controller is a controller that minimizes the variance of the
output from the process. In this case, we add a disturbance
term C*(q1 )e(k) on the right-hand side of Eq. (2), where C*
is a stable polynomial and e(k) is white noise. The minimum
Example 1: Indirect Deterministic Self-Tuning Regulator Assume that the open-loop process is described by
the continuous-time system
The process has an integrator and a time constant of 1 s.

There are no disturbances acting on the system and the
specications are that the controlled system should be able
to follow constant reference signals without too much overshoot. Sampling the system with the sampling interval
h = 0.5 s gives the sampled-data description
There is a process zero in b1 /b0 = 0.85. The zero is inside

the stability boundary, but it is still decided not to cancel
the zero. Let the desired closed-loop system be
Figure 2. Process output and input when an indirect self-tuning

regulator is used to control the process in Example 1. The specications are changed at time 100. The reference signal is shown as
a dashed curve.
Figure 3. Parameter estimates corresponding to the simulation

in Fig. 2. Upper diagram: a 1 (full) and a 2 (dashed), lower diagram:
b 0 (full) and b 1 (dashed). The true parameter is shown by dasheddotted lines.
This corresponds to a continuous-time system with natural

frequency = 1 and a damping of = 0.7 sampled with the
sampling period h = 0.5. The gain K is chosen such that the
steady-state gain from the reference signal to the output is
equal to one, that is, Bm (1)/Am (1) = 1. The controller solving
the design problem will have the structure
The design specication is to minimize the variance of the

output signal. Minimum variance control is equivalent to
predicting the output signal d steps ahead and choosing the
control signal such that the predicted value is equal to zero,
or any other desired set point value. The prediction horizon
should be equal to d, which is the delay in the process.
From Ref. 5 or Ref. 9 it follows that the output of
Eq. (13) can be written as
Figure 2 shows the output and the control signal when

the process is controlled by a self-tuning controller. The
reference signal is a square wave. It is seen that the output
behaves well already at the second change of the reference
signal. At time 100 the design specications are changed
and the damping is changed from = 0.7 to = 1. The
closed-loop response is immediately changed. The process
model has four unknown parameters, b0 , b1 , a1 , and a2 .
These parameters are estimated using the RLS algorithm,
and the estimated process parameters are shown in Fig. 3.
The example shows that the STR can nd good controller
parameters very quickly and that the design parameters
can be changed. The transient in the beginning depends on
the choice of initial values in the estimator.
where F* and G* are obtained from the diophantine

Eq. (11). The predicted output d steps ahead is given by
the second and third terms on the right-hand side of Eq.
(14). The prediction error is given by the rst term on the
right-hand side of Eq. (14). The prediction error is a moving average stochastic process that is independent of the
predicted output. The predicted output is zero if the control
law is chosen according to Eq. (12). Using the underlying
design principle, it has been possible to reparameterize the
model of Eq. (13) such that the reparameterized model explicitly contains the controller parameters. The controller
parameters then can be estimated directly. Using the minimum variance controller, the closed-loop system becomes
Direct Self-Tuning Regulators

The self-tuning algorithm described above relies on a separation between the estimation and the design. The design
step is repeated at each sampling instant. It can, on some
occasions, be desirable to avoid the computations done in
the design step, for instance, because of computing time
limitations. One way to do this is to convert the indirect
STR into a direct STR. This implies that the controller parameters are estimated instead of the process parameters.
How to do this reparameterization will be illustrated on
the minimum variance controller.
Let the system to be controlled be described by
The idea behind the basic direct STR is to estimate the

parameters in the prediction model
and use the controller
The estimated parameters are thus the same as the controller parameters, and the design step has been eliminated.
Elimination of Disturbances
We will now see how the inuence of disturbances can be
reduced by introducing integrators and by using feedforward.
Introduction of Integrators. Consider the process
Figure 4. The accumulated loss function V(k) when the direct
self-tuning algorithm (full) and the optimal minimum variance
controller (dashed) are used on the process in Example 2.
Figure 5. The controller gain s 0 /I 0 when the self-tuning algorithm is used (full). The gain of the optimal minimum controller
is shown as a dashed line.
Example 2: Direct Minimum Variance Self-Tuning

Algorithm Assume that the open-loop process is described by the sampled-data model
where e(k) is white noise with variance 1. The time delay

in the system is d = 1. Estimate the parameters r0 and s0
in the model
and use the controller
The optimal minimum variance controller is given by

u(k) = 0.2y(k), which is a proportional controller. Using
this controller gives the output y(k) = e(k), that is, the output should be white noise with a variance of 1. One way
to compare the optimal and the self-tuning regulators is to
compare the accumulated loss functions
which is a slight variant of Eq. (2). The signal v(k) is an

input load disturbance. If this is, for instance, a step, then
there needs to be an integrator in the controller to eliminate the inuence of this disturbance. There are several
ways to cope with this in an STR. One way is to estimate
the magnitude of the disturbance and to compensate for
it in the controller. To do so, the tuning has to be active
all the time since the disturbance may change over time.
A recommended way is to introduce an integrator directly
into the controller. This can be done by postulating that
the R* polynomial contains the factor 1 q1 . This can be
done in the direct as well as in the indirect algorithms.
In the indirect algorithm, it is necessary to modify the
estimator since the disturbance will change the relations
between the inputs and outputs. Load disturbances such
as steps have a particularly bad inuence on the estimated
model in the low-frequency range. Let the disturbance be
modeled as
where e(k) is a pulse, a set of widely separated pulses, or

white noise. For instance, a step disturbance is generated
by
The model can now be described as
Introduce the ltered signals yf (k) = Ad y(k) and uf (k) =

Ad u(k). We thus get
The new model has the equation error e(k) instead of v(k).
The process model can now be estimated from Eq. (17).
Based on the estimated model the controller design is done
by solving the diophantine equation
The slope of the accumulated loss function is an estimate

of the variance of the output.
and using the controller
Figure 4 shows the loss function when the self-tuning

algorithm and when the optimal minimum variance controller is used. After a short initial transient, the slopes of
the loss functions are the same, which indicates that the
STR has converged to the optimal minimum variance controller. This can also be seen by looking at the gain of the
controller shown in Fig. 5.
The controller now contains the factor Ad , which will eliminate the inuence of the disturbance v.
In the direct minimum variance self-tuning algorithm
an integrator can be introduced by changing the model of
Eq. (16) and estimating the controller parameters from
where u(k) = u(k) u(k 1) and using the controller
which contains an integrator.

Feedforward from Measurable Disturbances. On many occasions it is possible to measure some of the disturbances
acting on the system. A typical example is control of indoor
temperatures. By measuring the outdoor temperature also
it is possible to use this signal to compensate for changing outdoor temperatures before the disturbance has inuenced the process too much. One way to introduce feedforward in STRs is exemplied with the direct algorithm.
The estimated model is changed from Eq. (15) to
where vm (k) is the measurable disturbance. The controller

is now
results have lately been rened and extended [see Well om and Wittenmark (5)].
stead and Zarrop (4) and Astr
One important theoretical aspect is the inuence of unmodeled dynamics. Unmodeled dynamics are present if the
estimator is trying to t a too-simple model to the data. The
unmodeled dynamics may cause severe stability problems,
which must be avoided by introducing counter measures
such as careful ltering of the signals in the STR. This type
of problem has successfully been analyzed using averaging
theory.
It is important that a controller is robust against assumptions and choices of controller parameters. Much theoretical research has been devoted to make STRs and
adaptive controllers more robust. This work has resulted
in practical rules of thumb for their implementation (see
Ref. 5). Robust design methods are complementary to selftuning and adaptive control. In robust control one xed
controller is designed to cope with a variety of processes.
By using tuning and adaptation the parameters of the controller are instead tuned to adjust to the present process
dynamics.
PRACTICAL ISSUES AND IMPLEMENTATION
The rst part of the controller is the feedback from the measurement y(k) and the second is the feedforward from the
measurable disturbance vm (k). Feedforward is, in general,
very useful in STRs because to make effective feedforward,
it is necessary to have a good model of the process. By combining the measurement of the disturbance and the selftuning property of the controller it is possible to eliminate
much of the disturbance before it reaches the output of the
process.
SOME THEORETICAL PROBLEMS
The previous section described the basic ideas of STRs.
Self-tuning regulators are inherently nonlinear. The nonlinearities are due to the estimation part and the changing
parameters in the controller. This makes the analysis of
STRs very difcult. The STRs contain two feedback loops
and it is necessary to investigate the stability and convergence properties of the closed-loop systems. This is a difcult question because of the interaction between the two
feedback loops. One way to circumvent this problem is to
make a time separation between the two loops. The controller loop is assumed to be fast compared to the updating
loop. This makes it possible to use averaging theory to analyze the updating loop on a much longer time-scale. This
approach has made it possible to derive results concerning
stability and convergence of STRs.
om and Wittenmark (2) showed how to characterize
Astr
the stationary properties of STRs, that is, the properties if
and when the parameter estimation has converged. The
algorithms were used in a number of applications before
several of the theoretical problems were solved. Goodwin,
Ramadge, and Caines (3) gave the rst results showing
when the algorithm converges and that the closed-loop system remains stable during the estimation phase. These
Some problems in the implementation of STRs are discussed briey in this section. Self-tuning regulators as well
as adaptive controllers will run unattended on the processes. It is therefore very important that there be a good
safety net around the self-tuning algorithm.
There are many aspects of STR implementation that
are important for implementations of digital controllers in
om and Wittenmark (9)]. Some important
general [see Astr
issues for STRs are

Organization of the computer code

Sampling and ltering
Antireset windup
Design calculations
Excitation
Safety nets
It is important that the computer code be organized so that

as little delay as possible is introduced by the controller. In
STRs this usually implies that the estimation and the design calculations are done after the controlled signal is sent
out to the process. The latest measurement is thus used
in the computation of the control signal. The estimation
and the design are then performed, which implies that the
controller parameters are based on estimates from the previous sampling instant. This is usually no drawback since
the estimated parameters are changing very little between
samples, after the initial transient.
In all sampled-data controllers it is important that the
sampling interval be chosen properly. The sampling interval should be chosen in relation to the desired closed-loop
behavior. A common rule of thumb is that there should be
four to ten samples per rise time of the closed-loop system. It is also necessary to lter the analog signals before
they are sampled. The reason is the aliasing effect, which
implies that all frequencies over the Nyquist frequency /h,

where h is the sampling period, will be interpreted as a
lower frequency signal after the sampling. These lters are
called antialiasing lters. In the design of the controllers
it is important also to incorporate the dynamics of the antialiasing lters since they introduce a phase lag in the
system. The dynamics of the antialiasing lters will automatically be included in the estimated dynamics when
self-tuning or adaptive controllers are used. It may be necessary only to increase the order of the estimated model to
incorporate the lters into the estimated dynamics.
The indirect STRs contain a design calculation that normally involves the solution of a diophantine equation such
as Eq. (8). This equation has no solution if the A* and B*
polynomials have a common factor that is not also a factor
in Ac . This also implies that the solution of the diophantine equation is a numerically ill-conditioned problem if
there are almost common factors in A* and B*. These polynomials are obtained through estimation and there is no
guarantee that there are no common factors. The factors
that are close must thus be eliminated before solving the
diophantine equation.
Parameter estimation is a crucial element of STRs. The
estimation is relatively simple for processes with disturbances and set-point changes that excite the process all the
time. If there is not enough excitation of the process, it is
necessary to introduce a logical condition in the algorithm
that ensures that controller parameters are not changed
when there is no excitation of the process. The design of a
good safety net for an STR is a difcult task that requires
thorough knowledge of the details of the algorithms and an
understanding of where difculties may occur. Experience
shows that a good safety net normally occupies much more
code than the basic controller algorithm.
BIBLIOGRAPHY
1. R. E. Kalman, Design of self-optimizing control systems.
ASME Trans., 80: 468478, 1958.
om, B. Wittenmark, On self-tuning regulators. Auto2. K. J. Astr
matica, 9: 185199, 1973.
3. G. C. Goodwin, P. J. Ramadge, P. E. Caines, Discrete-time multivariable adaptive control, IEEE Trans. Autom. Control, AC25: 449456, 1980.
4. P. E. Wellstead, M. B. Zarrop, Selftuning Systems: Control and
Signal Processing, Chichester, U.K.: Wiley, 1991.
om, B. Wittenmark, Adaptive Control, 2nd ed. Read5. K. J. Astr
ing, MA: Addison-Wesley, 1995.
6. L. Ljung, System IdenticationTheory for the User.
7. T. Soderstrom, P. Stoica, System Identication. Hemel Hempstead, U.K.: Prentice-Hall International, 1988.
8. R. Johansson, System Modeling and Identication. Englewood
om, B. Wittenmark, Computer-Controlled Systems,
9. K. J. Astr
3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 1997.
10. K. F. Gauss, Theoria motus corposum coelestium, (1809). English translation, Theory of the Motion of the Heavenly Bodies.
New York: Dover, 1963.
BJORN
WITTENMARK
Lund University
Lund, Sweden
302
SINGULARLY PERTURBED SYSTEMS
Figure 1. RLC circuit.
to neglect it and replace the second-order RLC equation (1)

by the first-order RC equation
RC v + v = u
in Fig. 2. Neglecting several such parasitic parameters

(small time constants, masses, moments of inertia, etc.) often
leads to a significant simplification of a high-order model. To
validate such simplifications, we must examine whether, and
in what sense, a lower-order model approximates the main
phenomenon described by the original high-order model.
To see the issues involved, consider the RLC circuit in Eq.
(1) with u 0. Its free transients are due to the amount of
energy stored in C and L, that is, the initial conditions v(0)
and v(0), respectively. The simplified model in Eq. (2) disregards the transient due to v(0), that is, the dissipation of energy stored in the inductance L. When L is small, this transient is fast, and after a short initial time, the RC equation
(2) provides an adequate description of the remaining slow
transient due to the energy stored in C.
The RLC circuit in Eq. (1) with a small L is a two-timescale system, and the RC circuit in Eq. (2) is its slow timescale approximation. In higher-order models, several small
parameters may cause a multi-time-scale phenomenon, which
can be approximated by nested two-time-scale models. In
this article we consider only the two-time-scale systems.
In this example, a parameter perturbation from L 0 to
L 0 has resulted in a model order reduction. Such parameter perturbations are called singular, as opposed to regular
perturbations, which do not change the model order. For example, if instead of L, the small parameter is R, then its perturbation from R 0 to R 0 leaves the order of the RLC
equation (1) unchanged. The resulting undamped sinusoidal
oscillation is due to both v(0) and v(0).
In the engineering literature of the past 30 years, singular
perturbation techniques and their applications have been discussed in hundreds of papers and a dozen of books. This article presents only the basic singular perturbation tools for
reduced-order modeling and systematic approximation of twotime-scale systems. Our main sources are the textbook by
Kokotovic, Khalil, and OReilly (1) and the IEEE collection of
Many models of dynamic systems contain small parameters

multiplying some of the time derivatives. When such small
parameters are neglected, the dynamic order of the model is
usually reduced, as illustrated by the series RLC circuit
LCv + RCv + v = u
(1)
in Fig. 1, where v is the capacitor voltage and u is the applied

voltage. If the inductance L is very small, then it is common
(2)
+
u
+
v
Figure 2. RC circuit.
benchmark papers edited by Kokotovic and Khalil (2). These

books and the references therein are recommended for further reading.
TIME-SCALE PROPERTIES OF THE STANDARD MODEL
Several examples in the next section will show that a common
state-space model of many two-time-scale systems is
x = f (x, z, , t),
x(t0 ) = x0
(3a)
z = g(x, z, , t),
z(t0 ) = z0
(3b)
where x Rns, z Rnf, and 0 is the small singular perturbation parameter. The parameter represents the small time
constants and other parasitics to be neglected in the slow
time-scale analysis.
The rate of change of z in Eq. (3b) is of order 1/ , that is,
z O(1/ ), which means that z exhibits a fast transient.
When this fast transient settles, the longer-term behavior of
x and z is determined by the quasi-steady-state equation
g(x, z, 0, t) = 0
(4)
where the bar indicates that this equation is obtained by setting 0 in Eq. (3b). This equation will make sense only if
it has one or several distinct (isolated) roots
z = (x, t)
(5)
for all x and z of interest. If this crucial requirement is satisfied, for example, when det(g/z) 0, then we say that system (3) is a standard model.
The substitution of Eq. (5) into Eq. (3a) results in the reduced model
x = f (x, (x, t), 0, t),
x(t0 ) = x0
303
clear that we cannot expect z to converge to its quasi-steady

state z unless certain stability conditions are satisfied.
To analyze the stability of the fast transient, we perform
the change of variables y z (x, t), which shifts the quasisteady state of z to the origin. Then Eq. (3b) becomes
y = g(x, y + (x, t), , t)

f (x, y + (x, t), , t),
t
x
y(t0 ) = z0 (x0 , t0 )
(7)
Let us note that y may remain finite even when tends to

zero and y tends to infinity. We introduce a fast time variable
by setting
dy
d
1
dy
=
; hence,
=
dt
d
dt
and use 0 as the initial value at t t0. The fast time

variable (t t0)/ is stretched: if tends to zero, tends
to infinity even for finite fixed t only slightly larger than t0. In
the scale, system (7) is represented by
dy
= g(x, y + (x, t), , t)

f (x, y + (x, t), , t),
d
t
x
y(0) = z0 (x0 , t0 ) (8)
In the fast time-scale , the variables t and x are slowly
varying because t t0 and x x(t0 ). Setting 0
freezes these variables at t t0 and x x0 and reduces Eq.
(8) to the autonomous system
dy
= g(x0 , y + (x0 , t0 ), 0, t0 ),
d
y(0) = z0 (x0 , t0 )
(9)
which has equilibrium at y 0. The frozen parameters (x0,

t0) in Eq. (9) depend on the given initial state and initial time.
(6)
Tikhonovs Theorem
If Eq. (4) has several distinct roots as shown in Eq. (5), then
each of them leads to a distinct reduced model as shown in
Eq. (6). The singular perturbation analysis determines which
of these models provides an O() approximation of the slow
phenomenon in system (3).
When, and in what sense, will x(t), z(t) obtained from Eqs.
(6) and (5) be an approximation of the true solution of system
(3)? To answer this question, we examine the variable z,
which has been excluded from the reduced model in Eq. (6)
by z (x, t). In contrast to the original variable z, which
starts at t0 from a prescribed z0, the quasi-steady state z is
not free to start from a prescribed value, and there may be a
large discrepancy between z(t0) (x0, t0) and the prescribed
initial state z0. Thus, z(t) cannot be a uniform approximation
of z. The best we can expect is that the approximation z
z(t) O() will hold on an interval excluding t0, that is, for
t [tb, tf ] where tb t0. On the other hand, it is reasonable to
expect the approximation x x(t) O() to hold uniformly
for all t [t0, tf ] because x(t0) x(t0). If the error z z(t) is
indeed O() over [tb, tf ], then it must be true that during the
initial (boundary-layer) interval [t0, tb] the variable z approaches z. Let us remember that the speed of z can be large
since z g/ . In fact, having set 0 in Eq. (3b), we have
made the transient of z instantaneous whenever g 0. It is
In our investigation of the stability of the origin of the system

in Eq. (9), we should allow the frozen parameters to take any
values in the domain of interest. Therefore, we rewrite the
system in Eq. (9) as
dy
= g(x, y + (x, t), 0, t)
d
(10)
where (x, t) are treated as fixed parameters. We refer to the

system in Eq. (10) as the boundary-layer system and assume
its exponential stability, uniform in the frozen parameters;
that is,
y( ) ky(0)e
(11)
for some positive constants k and . Furthermore, we assume

that y(0) belongs to the region of attraction of the origin. Under these conditions, a fundamental result of singular perturbation theory, called Tikhonovs Theorem, guarantees that
the approximations
x = x(t) + O()
(12a)
+ O()
z = (x(t), t) + y(t/)
(12b)
304
hold uniformly for t [t0, tf ], where y() is the solution of the

system in Eq. (9). Moreover, given any tb t0, the approximation
z = (x(t), t) + O()
(13)
holds uniformly for t [tb, tf ].

Local exponential stability of the boundary-layer system
can be guaranteed with the eigenvalue condition

g
(x, (x, t), 0, t) c < 0
(14)
Re
z
for all (x, t) in the domain of interest, where denotes the
eigenvalues and c is a positive constant. Alternatively, it can
be verified by a Lyapunov analysis if there is a Lyapunov
function W(x, y, t) that depends on (x, t) as parameters and
satisfies
which (x, ) must satisfy for all x of interest and all (0,
*]. This is a partial differential equation, which, in general,
is difficult to solve. However, its solution can be approximated
by the power series
(x, ) = 0 (x) + 1 (x) + 2 2 (x) +
(21)
where the functions 0(x), 1(x), . . ., can be found by equating the terms with like powers in . To this end, we expand
f and g as power series of
f
(x) + (22)
z 1
g
g(x, 0 (x) + 1 (x) + ) = g(x, 0 (x)) + 1 (x) + (23)
z
f (x, 0 (x) + 1 (x) + ) = f (x, 0 (x)) +
c1 y2 W (x, y, t) c2 y2
(15)
where all the partial derivatives are evaluated at x and z

0(x).
We substitute Eqs. (22) and (23) into Eq. (20). The terms
with 0 yield
W
g(x, y + (x, t), 0, t) c3 y2
y
(16)
g(x, 0 (x)) = 0, that is, (x, 0) = 0 (x)
over the domain of interest, where c1 to c3 are positive constants independent of (x, t).
which is the quasi-steady-state manifold M. Equating the 1

terms, we get
g
0 (x)
(x) =
f (x, 0 (x))
z 1
x
Slow Manifold
In the state-space Rns Rnf of (x, z), the equation g(x, z, 0, t)
0 forces x and z to lie in an ns-dimensional quasi-steadystate manifold M, explicitly described by Eq. (5). It can be
shown that, under the conditions of Tikhonovs Theorem,
there exists an * 0 such that for all (0, *], the system
in Eq. (3) possesses an integral manifold M that is invariant:
whenever x(t0), z(t0) M, then x(t), z(t) M for all t [t0,
tf ]. The slow manifold M is in the -neighborhood of the quasisteady-state manifold M. We will characterize M in the special case when f and g in the system in Eq. (3) do not depend
on t and :
x = f (x, z)
(17a)
z = g(x, z)
(17b)
A derivation of slow manifolds for systems with f and g also

dependent on t and is given in Ref. (26).
We will seek the graph of M in the explicit form M : z
(x, ). The existence of M gives a clear geometric meaning
to the slow subsystem of the full-order model in Eq. (17): it is
the restriction of the model in Eq. (17) to the slow manifold
M, given by
x = f (x, (x, ))
(25)
For the standard model, det(g/z) 0, that is, g/z is nonsingular, so that
1
g
0 (x)
f (x, 0 (x))
1 (x) =
(26)
z
x
This recursive process can be repeated to find the higherorder terms in Eq. (21).
The fast off-manifold variable is
y = z (x, )
(27)
In the x,y coordinates, the system in Eq. (17) becomes
x = f (x, (x, ) + y)
(28a)
y = g(x, (x, ) + y)
f (x, (x, ) + y)
x
(28b)
In these coordinates, the slow manifold M is simply y 0,

that is, the equilibrium manifold of Eq. (28b). The geometry
of a third-order system in Eq. (17) with x R2 and z R1 is
illustrated in Fig. 3. Starting from an off-manifold initial conz(0)
(18)
To find M, we differentiate the manifold z (x, ) with

respect to t
d
(x, ) =
x
z =
dt
x
(19)
x2
and, upon the multiplication by and the substitution for x

and z, we obtain the slow manifold condition
f (x, (x, )) = g(x, (x, ))

x
(20)
(24)
x(0)
x1
Figure 3. Trajectory converging to a slow manifold.
dition (x(0), z(0)), the state trajectory rapidly converges to M

and then slowly evolves along M.
applied to Eq. (32) results in the block-diagonal system

A11 A12 L
0
=
y
0
A22 + LA12 y

B1 H(B2 + LB1 )
+
u
B2 LB1
Linear Two-Time-Scale Systems

The manifold condition in Eq. (20) is readily solvable for linear two-time-scale systems
x = A11 x + A12 z + B1 u,
x(t0 ) = x0
(29a)
z = A21 x + A22 z + B2 u,
z(t0 ) = z0
(29b)
(30)
where the nf ns matrix L() satisfies the matrix quadratic

equation
A21 A22 L + LA11 LA12L = 0
(31)
transforms the system in Eq. (29) into a block-triangular system

x
A11 A12 L
x
B1
A12
=
+
u
y
0
A22 + LA12 y
B2 LB1
(32)

x(t0 )
x0
=
y(t0 )
z0 + Lx0
2
1
2
L() = A1
22 A21 + A22 A21 (A11 A12 A22 A21 ) + O( )
(34)
We can readily verify that for 0, Eq. (31) reduces to

A21 A22 L(0) = 0
(35)
whose solution is the first term in Eq. (34). Furthermore, to

solve for the higher-order terms, an iterative scheme
1
Lk+1 = A1
22 A21 + A22 Lk (A11 A12 Lk ),
L0 = A1
22 A21
(36)
can be used.
Although the fast variable y in the triangular system in
Eq. (32) is now decoupled from the slow variable x, the slow
variable x is still driven by the fast variable y. To remove this
influence, the change of variables
= x H()y
(37)

(t0 )
x0 H(z0 + Lx0 )
=
y(t0 )
z0 + Lx0
(40)
For sufficiently small, Eq. (38) admits a unique solution

H() that can be expressed as
H() = A12 A1
22 + O()
(41)
The solution H() can also be computed iteratively as

1
Hk+1 = A12 A1
22 + ((A11 A12 L)Hk + Hk LA12 )A22 ,
H0 = A12 A1
22
(42)
If L is available from the recursive formula (36), we can use

Lk1 instead of L in Eq. (42).
From the block-diagonal form in Eq. (39), it is clear that
the slow subsystem of Eq. (29) is approximated to O() by
= A0 + B0 u,
(33)
Note that Eq. (31) is the slow manifold condition in Eq. (20)
for linear systems.
Given that A22 is nonsingular, the implicit function theorem implies that Eq. (31) admits a solution L() for sufficiently small. Furthermore, an asymptotic expansion of the
solution to Eq. (31) is given by
(39)
where A22 is nonsingular, which corresponds to det(g/z)

0, and u Rm is the control input vector. The change of variables
y = z + L()x
305
(t0 ) = x0
(43)
1
where A0 A11 A12A1
22 A21 and B0 B1 A12A22 B2. The fast
subsystem is approximated to O() by
y = A22 y + B2 u,
y(t0 ) = z0 A1
22 A21 x0
(44)
Thus as 0, the slow eigenvalues of Eq. (29) are approximated by (A0), and the fast eigenvalues are approximated
by (A22)/ . It follows that if Re(A0) 0 and Re(A22) 0,
then there exists an * 0 such that (29) is asymptotically
stable for all (0, *]. Furthermore, if the pair (A0, B0) and
the pair (A22, B2) are each completely controllable (stabilizable), then there exists an * 0 such that Eq. (29) is completely controllable (stabilizable) for all (0, *].
EXAMPLES
Example 1. An RLC Circuit
To complete our introductory example, we represent the RLC
circuit in Eq. (1) using the state variables x v and z v:
x = z
z = z
(45a)
1
(x u)
RC
(45b)
where the ns nf matrix H() satisfies the linear matrix equation
where is the small time constant L/R. The unique solution

of the quasi-steady-state equation (5) is z (x u)/(RC),
which yields the reduced-order model
(A11 A12 L)H H(A22 + LA12 ) + A12 = 0
RC x = x + u
(38)
(46)
As expected, this is the RC equation (2). The boundary-layer

system Eq. (9) for y z (x u)/(RC) is
dy
= y,
d
y(t0 ) = z(t0 ) +
1
(x(t0 ) u(t0 ))
RC
(47)
Its solution y ey(t0) e(R/L)ty(t0) approximates the fast

transient neglected in the slow subsystem in Eq. (46). Tikhonovs Theorem is satisfied, because the fast subsystem in Eq.
(47) is exponentially stable.
Example 2. A dc Motor
A common model for dc motors, shown in Fig. 4, under constant field excitation, consists of a mechanical torque equation and an equation for the electrical transient in the armature circuit, namely,
Speed (rad/s)
Current (A)
306
52.6
52.5
52.4
52.3
52.2
0.1
0.2
0.3
Time (s)
0.4
0.5
0.1
0.2
0.3
Time (s)
0.4
0.5
3.45
3.40
Figure 5. dc motor step response.
J = Ki TL
(48a)
Li = K Ri + u
(48b)
where i, u, R, and L are the armature current, voltage, resistance, and inductance, respectively, J is the combined moment of inertia of the motor and the load, is the angular
speed, TL is the load torque, and K is a motor design constant
such that Ki and K are, respectively, the motor torque and
the back emf (electromotive force).
We consider the case in which the electrical time constant
e L/R is much smaller than the mechanical time constant
m JR/K2 (3). Defining e / m, x , and z i, we rewrite
Eq. (48) as
1
R
z TL
m K
J
1
1
K
x
u
z+
z =
m R
m
m R
x =
(49a)
(49b)
The parameters of a 1 hp (746 W) dc motor with a rated

speed of 500 rpm (52.3 rad/s) are R 7.56 , L 0.055 H,
K 4.23 Vs/rad, and J 0.136 kg m2. At the rated steady
state condition, TL 14.3 N m, i 3.38 A, and u 246.8 V.
The time constants are e 0.0073 s and m 0.115 s, resulting in 0.063. The response to a 1 V step increase in u
of the full model in Eq. (48) (solid curves) and the slow subsystem in Eqs. (51) and (52) (dashed curves) is shown Fig. 5.
Note that x is a good approximation of x. Initially there is a
fast transient in z. After this boundary layer has decayed,
z becomes a good approximation of z.
The fast electrical transient is approximated by the boundary-layer system
m
dy
= y,
d
y(0) = z(0)
u(0) Kx(0)
R
(53)
which has the unique solution
Setting 0, we obtain from Eq. (49b)

0 = K x Rz + u
(50)
y = e / m y(0) = et/ e y(0)
(51)
Example 3. Multiple Slow Subsystems

To illustrate the possibility of several reduced-order models,
we consider the singularly perturbed system
(54)
Thus the quasi-steady state of z is

uKx
z=
R
which, when substituted in Eq. (49a), yields the slow mechanical subsystem
m x = x +
m
1
u
T
K
J L
(55a)
z = [z + (1 + t)x] z [z (1 + t)]
Figure 4. dc motor.
(56)
which has three distinct roots
(55b)
where the initial conditions are x(0) 1 and z(0) z0. Setting
0 results in
0 = [z + (1 + t)x] z [z (1 + t)]
(52)
x = x2 (1 + t)/z
TL
z = (1 + t)x;
z = 0;
z = 1+t
(57)
Consider first the root z (1 t)x. The boundary-layer

system in Eq. (10) is
dy
= y[y (1 + t)x][y (1 + t)x (1 + t)]
d
(58)
Taking W(y) y2 /2, it can be verified that W satisfies the

inequalities in Eqs. (15) and (16) for y (1 t) x. The reduced
system
x = x,
x(0) = 1
y(0)
= z0 + 1
ks
bs
mu
dy
= [y + (1 + t)x] y [ y (1 + t)]
d
dy
= [ y + (1 + t) + (1 + t)x][y + (1 + t)] y
d
(62)
(63)
has the unique solution x(t) 1/(1 t) for all t [0, 1).
Notice that x(t) has a finite escape time at t 1. However,
Tikhonovs Theorem still holds for t [0, tf] with tf 1. The
boundary-layer system, with t 0 and x 1,
y(0)
= z0 1
Tire
kt
dr
Road
surface
Reference
Figure 7. Quarter-car model.
As in the first case, it can be shown that the origin is exponentially stable uniformly in (x, t). The reduced system
x = x2 , x(0) = 1
du
(61)
By sketching the right-hand side function, it can be seen that

the origin is unstable. Hence, Tikhonovs Theorem does not
apply to this case and we rule out this root as a subsystem.
Furthermore, x is not defined at z 0.
Finally, the boundary-layer system for the root z 1 t
is
full model (solid curves) and two for each reduced model
(dashed curves).
Example 4. A Quarter-Car Model
A simplified quarter-car model is shown in Fig. 7, where ms
and mu are the car body and tire masses, ks and kt are the
spring constants of the strut and the tire, and bs is the
damper constant of the shock. The distances ds, du, and dr are
the elevations of the car, the tire, and the road surface, respectively. From Newtons Law, the balance of forces acting
on ms and mu results in the modeling equations
ms ds + bs (ds du ) + ks (ds du ) = 0
(65a)
mu du + bs (du ds ) + ks (du ds ) + kt (du dr ) = 0
(65b)
In a typical car, the natural frequency kt /mu of the tire

is much higher than the natural frequency ks /ms of the car
body and the strut. We therefore define the parameter

(64)
has a unique exponentially decaying solution y() for z0

a 0.
In summary, only two of the three roots in Eq. (57) give
rise to valid reduced models. Tikhonovs Theorem applies to
the root (1 t)x if z0 0 and to the root 1 t if
z0 0. Figure 6 shows z for four different values of z0 of the
2
z
1
=
ks /ms
=
kt /mu
0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2
Time (s)
Figure 6. Response of system in Eq. (55) illustrating two slow subsystems.
k s mu
k t ms
(66)
The mass-spring system in Eq. (65) is of interest because it

cannot be transformed into a standard model without an dependent scaling. From Eq. (66), the tire stiffness kt
O(1/ 2) tends to infinity as 0. For the tire potential energy kt(du dr)2 /2 to remain bounded, the displacement du
dr must be O(), that is, the scaled displacement (du dr)/
must remain finite. Thus to express Eq. (65) in the standard
singularly perturbed form, we introduce the slow and fast
variables as

ds du
,
x=
ds
2
0
ds
(60)
and has a unique exponentially decaying solution y() for z0

1 1 a, that is, for z0 a 0 where a 0 can be
arbitrarily small.
Consider next the root z 0. The boundary-layer system
in Eq. (10) is
dy
= (y + 2)(y + 1)y,
Car
body
ms
(59)
has the unique solution x(t) et for all t 0. The boundarylayer system with t 0 and x 1 is
dy
= y(
y 1)(y 2),
d
307
(du dr )/
z=
du

(67)
and u dr as the disturbance input. The resulting model is

x = A11 x + A12z + B1 u
(68a)
z = A21 x + A22z + B2 u
(68b)
308
where

0
=
0

0
B1 =
0
0
1
1
, A12
,
ks /ms bs /ms
bs /ms

0
0
0
1
, A22 =
,
A21 =
ks /ms bs /ms
ks /ms bs /ms

1
(69)
B2 =
0
A11 =
and
k s ms
k t mu
(70)
The parameters of a typical passenger car are ms 504.5

kg, mu 62 kg, bs 1,328 Ns/m, ks 13,100 N/m, kt
252,000 N/m, and 0.65. In this case, 0.08 and the
time scales of the body (slow) and the tire (fast) are well separated.
To illustrate the approximation provided by the two-timescale analysis, the slow and fast eigenvalues of the uncorrected subsystems in Eqs. (43) and (44) are found to be
2.632 j6.709 and 10.710 j62.848, respectively, which
are within 4% of the eigenvalues 2.734 j7.018 and 9.292
j60.287 of the full-order model. If a higher accuracy is desired, the series expansion in Eq. (34) can be used to add the
first-order correction terms to the diagonal blocks of Eq. (39),
resulting in the approximate slow eigenvalues 2.705
j6.982 and fast eigenvalues 9.394 j60.434, and thus reducing the errors to less than 0.5%.
Example 5. A High-Gain Power Rectifier
Many modern control systems include power electronic rectifiers as actuators. An example is a static excitation system
that controls the field voltage Efd of a synchronous machine
shown in Fig. 8 (4). The synchronous machine is modeled as
x = f (x, Efd )
(71a)
VT = h(x)
(71b)
where x is the machine state vector including the flux variables and the scalar output VT is the generator terminal voltage. Here we focus on the exciter system which, from Fig. 8,
is described by
TM E = E KM KG Efd + KMVR
Efd = VB (x)E
(72a)
(72b)
KM
1 + sTM
Efd
Synchronous
machine
VB(x)
Figure 8. Static excitation system.
VT
1
KMV B
1
(73)
where VB is a constant of the order of magnitude of VB(x), that

is, the ratio (x) VB(x)/VB O(1). Using Efd as the state
variable instead of E, Eq. (72) is rewritten as

TM
(x, Efd ) Efd

TM Efd = 1 + KG KMVB (x)
VB (x)
(74)
+ KMVB (x)VR
where (x, Efd) [VB(x)/x]x is bounded. Using , Eq. (74)
becomes

KG (x) +

(x)
(x, Efd ) Efd +

Efd =
V
(75)
TM
VB (x)
TM R
which yields, as 0, the quasi-steady state
E fd =
1
V
KG R
(76)
In a typical rectifier system, KG 1, KM 7.93, and TM

0.4 s. For VB 6, we obtain 0.021. The time-scales are
well separated, which allows us to achieve high accuracy with
singular perturbation approximations.
Example 6. Slow Manifold in a Synchronous Machine
We now proceed to illustrate the use of the slow manifold concept as a modeling tool. In most cases, the solution of the
manifold condition in Eq. (20) is evaluated approximately as
a power series in . However, for a synchronous machine
model, the slow manifold, which excludes stator circuit transients, can be calculated exactly, as shown by Kokotovic and
Sauer (5).
The synchronous machine model with one damper winding
in the quadrature axis is
d
= s
(77a)
dt

2H d
1
1
1
1
= Tm +
d q + d Ed + q Eq (77b)
s dt
Lq
Ld
Lq
Ld

Tqo
KG
=

Tdo
where TM is a time constant and KM and KG are gains.
VR
Following the input signal VR, the voltage E modulates the

supply voltage VB(x) to control Efd. The supply voltage VB(x) is
a function of x and is typically within the range of 48 per
unit on the base field voltage.
Under normal operating conditions, the product of the rectifier gain and the supply voltage KMVB(x) is very high. This
motivates us to define as
dEq
dt
Ld Ld
Ld
E
d + Efd
q
Ld
Ld
Lq Lq
dEd
Lq
= Ed
q
dt
Lq
Lq
(77c)
(77d)
Ra
Ra
1 dd
= d + Eq +
q + V sin
s dt
Ld
Ld
s
(77e)
1 dq
Ra
Ra
= q Ed
+ V cos
s dt
Lq
Lq
s d
(77f)
where , , and H are the generator rotor angle, speed, and

inertia, respectively, (Ed, Eq), (d, q), (Tdo, Tqo), (Ld, Lq), and
d = V cos ,
q = V sin
(78)
These expressions can be substituted into Eqs. (77a)(77d) to

obtain a fourth-order slow subsystem.
If the initial condition [d(0), q(0)] is not on the manifold
shown in Eq. (78), then using the fast variables
yd = d V cos ,
yq = q + V sin
(79)
we obtain the fast subsystem
dyd
=
yq ,
yd (0) = d (0) V cos (0)
dt
s
dyq
= yd , yq (0) = q (0) + V sin (0)

dt
s
(80a)
(80b)
where the state appears as a time-varying coefficient.

When the stator resistance Ra is nonzero, the slow manifold condition can no longer be solved exactly. Instead, the
leading terms in the power series expansion in Eq. (21) can
be computed to obtain any desired approximation of the slow
manifold.
Example 7. Van der Pol Oscillator
A classical use of the slow manifold concept is to demonstrate
the relaxation oscillation phenomenon in a Van der Pol oscillator, modeled in the state-space form as (6)
x = z
z = x + z
(81a)
1 3
3z
(81b)
For small, the slow manifold is approximated by

g(x, z) = x + z 13 z3 = 0
(82)
which is shown as the curve ABCD in Fig. 9. For the roots

z (x) on the branches AB and CD
g
= 1 z2 < 0
z
(83)
and the eigenvalue condition of Tikhonovs Theorem is satisfied because z2 1. Therefore, the branches AB and CD are
attractive, that is, trajectories converging to these two
branches will remain on them, moving toward either the
point B or C. However, the root on the branch BC is unstable
because z2 1; hence, this branch of the slow manifold is repulsive.
Figure 9 shows vertical trajectories converging toward AB
and CD, because 0. The mechanism of two interior (rather
3
A
2
1
z
(Ld, Lq) are the d- and q-axis voltages, flux linkages, opencircuit time constants, synchronous reactances, and transient
reactances, respectively, Ra is the stator resistance, Tm is the
input mechanical torque, Efd is the excitation voltage, and s
is the system frequency.
In the model shown in Eq. (77), the slow variables are ,
, Ed, and Eq, and the fast variables are d and q. The singular perturbation parameter can be defined as 1/s.
If the stator resistance is neglected, that is, Ra 0, it can
be readily verified that the slow manifold condition in Eq. (20)
gives the exact slow invariant manifold
309
2
3
2 1.5 1 0.5
D
0
x
0.5
D
1
1.5
Figure 9. Phase portrait of the Van der Pol oscillator.
than boundary) layers, from B to D and from C to A is clear

from this phase portrait. The relaxation oscillation forming
the limit cycle A-B-D-C consists of the slow motions from
A to B and D to C, connected with fast jumps (layers) from
B to D and from C to A. When is increased to a small
positive value, this limit cycle is somewhat deformed, but its
main character is preserved. This observation is one of the
cornerstones of the classical nonlinear oscillation theory (7),
which has many applications in engineering and biology.
STABILITY ANALYSIS
We consider the autonomous singularly perturbed system in
Eq. (17). Let the origin (x 0, z 0) be an isolated equilibrium point and the functions f and g be locally Lipschitz in a
domain that contains the origin. We want to analyze stability
of the origin by examining the reduced and boundary-layer
models. Let z (x) be an isolated root of 0 g(x, z) defined
in a domain D1 Rn that contains x 0, such that (x) is
continuous and (0) 0. With the change of variables y
z (x), the singularly perturbed system is represented in
the new coordinates as
x = f (x, y + (x))
y = g(x, y + (x))
f (x, y + (x))
x
(84a)
(84b)
The reduced system x f(x, (x)) has equilibrium at x 0,

and the boundary-layer system dy/d g(x,y (x)) has
equilibrium at y 0. The main theme of the two-time-scale
stability analysis is to assume that, for each of the two systems, the origin is asymptotically stable and that we have a
Lyapunov function that satisfies the conditions of Lyapunovs
Theorem. In the case of the boundary-layer system, we require asymptotic stability of the origin to hold uniformly in
the frozen parameter x. Viewing the full singularly perturbed
system (84) as an interconnection of the reduced and boundary-layer systems, we form a composite Lyapunov function
candidate for the full system as a linear combination of the
Lyapunov functions for the reduced and boundary-layer systems. We then proceed to calculate the derivative of the composite Lyapunov function along the trajectories of the full system and verify, under reasonable growth conditions on f and
g, that it is negative definite for sufficiently small .
310
Let V(x) be a Lyapunov function for the reduced system

such that
V
f (x, (x)) 1 12 (x),
x
x D1
(85)
where 1(x) is a positive definite function. Let W(x, y) be a

Lyapunov function for the boundary-layer system such that
W
g(x, y + (x)) 2 22 ( y),
y
(x, y) D1 D2
(86)
where D2 Rm is a domain that contains y 0, and 2(y) is a

positive definite function. We allow the Lyapunov function W
to depend on x because x is a parameter of the system and
Lyapunov functions may, in general, depend on the systems
parameters. Because x is not a true constant parameter, we
must keep track of the effect of the dependence of W on x. To
ensure that the origin of the boundary-layer system is asymptotically stable uniformly in x, we assume that W(x, y) satisfies
W1 ( y) W (x, y) W2 ( y),
(x, y) D1 D2
(87)
for some positive definite continuous functions W1 and W2.

Now consider the composite Lyapunov function candidate
(x, y) = (1 d)V (x) + dW (x, y),
0<d<1
(88)
where the constant d is to be chosen. Calculating the derivative of along the trajectories of the full system in Eq. (84),
we obtain
d W
V
f (x, (x)) +
g(x, y + (x))
= (1 d)
x
y
V
[ f (x, y + (x)) f (x, (x))]
+ (1 d)
x

W
W
+d
f (x, y + (x))
x
y x
(89)
We have represented the derivative as the sum of four

terms. The first two terms are the derivatives of V and W
along the trajectories of the reduced and boundary-layer systems. These two terms are negative definite in x and y, respectively, by the inequalities in Eqs. (85) and (86). The other
two terms represent the effect of the interconnection between
the slow and fast dynamics, which is neglected at 0. Suppose these terms satisfy the interconnection conditions
V
[ f (x, y + (x)) f (x, (x))] 1 1 (x)2 ( y)
(90)
x

W
W
f (x, y + (x)) 2 1 (x)2 (y) + 22 (y) (91)

x
y x
for some nonnegative constants 1, 2, and . Using the inequalities in Eqs. (85), (86), (90), and (91), we obtain
1 (x)
2 ( y)
T
(1 d)1
d)1 12 d2
12 (1
12 (1
d)1 12 d2

d 2
1 (x)
2 ( y)
(92)
The right-hand side of the last inequality is a quadratic form

in (1(x), (2(y)). The quadratic form is negative definite when

d(1 d)1 2 > 14 [(1 d)1 + d2 ]2
(93)

For any d, there is an d such that Eq. (93) is satisfied for
d. The maximum value of d occurs at d* 1 /(1 2)
and is given by
=
1 2
1 + 1 2
(94)
It follows that the origin is asymptotically stable for all *.

Note that the functions 1(x) and 2(x) and the interconnection
conditions in Eqs. (90) and (91) should be carefully constructed so that * may not be unnecessarily conservative.
As an illustration of the preceding Lyapunov analysis, consider the second-order system
x = f (x, z) = x x3 + z
z = g(x, z) = x z
(95a)
(95b)
which has a unique equilibrium point at the origin. Let y

z (x) z x. For the reduced system x x3, we take
V(x) x4, which satisfies Eq. (85) with 1(x) x3 and 1
1. For the boundary-layer system dy/d y, we take
W(y) y2, which satisfies Eq. (86) with 2(y) y and 2
1. As for the interconnection conditions in Eqs. (90) and (91),
we have
V
[ f (x, y + (x)) f (x, (x))] = x3 y 1 2
x
(96)
W
f (x, y + (x)) = y(x3 + y) 1 2 + 22
y
(97)
Note that W/x 0. Hence, Eqs. (90) and (91) are satisfied
with 1 2 1. Therefore, the origin is asymptotically
stable for * 0.5. Because all the conditions are satisfied
globally and (x, y) (1 d)V(x) dW(y) is radially unbounded, the origin is globally asymptotically stable for
0.5.
The preceding two-time-scale stability analysis can be extended to the nonautonomous singularly perturbed system in
Eq. (3). For sufficiently smooth f, g, and , it can be shown
that if the origin of the reduced system is exponentially stable, and the origin of the boundary-layer system is exponentially stable, uniformly in (t, x), then the origin of the full system in Eq. (3) is exponentially stable for sufficiently small .
The same conditions ensure the validity of Tikhonovs Theorem for all t t0. Note that the earlier statement of Tikhonovs Theorem, which does not require exponential stability of
the reduced system, is valid only on a compact time interval
[t0, tf] for a given tf.
COMPOSITE FEEDBACK CONTROL
The impetus for the systematic decomposition of the slow and
fast subsystems in singularly perturbed systems can be
readily extended to the separate control design of the slow
and fast dynamics. As will be shown, the crucial idea is to
compensate for the quasi-steady state in the fast variable.
Consider the nonlinear singularly perturbed system

x = f (x, z, u)
(98a)
z = g(x, z, u)
(98b)
and suppose the equation 0 g(x, z, u) has a unique root z

(x, u) in a domain D that contains the origin. A separated
slow and fast design is succinctly captured in the composite
control
u = us + uf
(99)
where us is a slow control function of x

us = s (x)
(100)
and uf is a fast control function of both x and z

uf = f (x, z)
(101)
Applying the control in Eq. (99) to the full model in Eq.

(98), we obtain
x = f (x, z, s (x) + f (x, z))
(102a)
z = g(x, z, s (x) + f (x, z))
(102b)
The fast control f (x, z) must guarantee that z (x, s(x)) is

a unique solution to the equation
0 = g(x, z, s (x) + f (x, z))
(103)
311
have the desired properties. Thus the composite control in Eq.

(99) is
u = G0 x + G2 [z + A1
22 (A21 + B2 G0 )x] = G1 x + G2 z
(111)
where
1
G1 = (Im + G2 A1
22 B2 )G0 + G2 A22 A21
(112)
When Eq. (111) is applied to the full-order model in Eq. (29),

the slow and fast dynamics of the closed-loop system are approximated to O() by the slow and fast subsystems in Eqs.
(109) and (110), respectively.
If the pairs (A0, B0) and (A22, B2) are completely controllable, the results here point readily to a two-time-scale poleplacement design in which G0 and G2 are designed separately
to place the slow eigenvalues of A0 B0G0 and the fast eigenvalues of (A22 B2G2)/ at the desired locations. Then the
eigenvalues of the closed-loop full-order system will approach
these eigenvalues as tends to zero.
The composite feedback control is also fundamental in
near-optimal control design of linear quadratic regulators for
two-time-scale systems. Consider the optimal control of the
linear singularly perturbed system in Eq. (29) to minimize
the performance index

1 T
J(u) =
(q q + uT Ru) dt, R > 0
(113)
2 0
where
in the domain D. Furthermore, we require the fast control to

be inactive on the manifold in Eq. (103), that is,
f (x, (x, s (x))) = 0
(104)
Then the slow and fast subsystems become, respectively,

x = f (x, (x, s (x)), s (x))
(105)
z = g(x, z, s (x) + f (x, z))
(106)
To obtain controllers so that the equilibrium (x 0, z 0)

is asymptotically stable, s(x) must be designed so that a Lyapunov function V(x) satisfying Eq. (85) can be found for the
slow subsystem in Eq. (105), and f (x, z) must be designed so
that a Lyapunov function W(x, z) satisfying Eq. (86) can be
found for the fast subsystem in Eq. (106). Furthermore, the
interconnection conditions corresponding to Eqs. (90) and (91)
must be satisfied, so that a composite Lyapunov function similar to Eq. (88) can be used to establish the asymptotic stability of the equilibrium.
Specializing the composite control design to the linear singularly perturbed system in Eq. (29), we design the slow and
fast controls as
s = G0 x = G0 + O()
f = G2 [z +
A1
22 (A21 x
+ B22 G0 x)]
, G2 y
q(t) = C1 x(t) + C2 z(t)
(114)
Following the slow and fast subsystem decomposition in

Eqs. (43) and (44), we separate q(t) into its slow and fast components as
q(t) = qs (t) + qf (t) + O())
(115)
qs (t) = C0 + D0 us
(116)
where
with
C0 = C1 + C2 A1
22 A21 ,
D0 = C2 A1
22 B2
(117)
and
qf (t) = C2 y
(118)
From the subsystems in Eqs. (43) and (44) and the decomposition in Eq. (115), the linear quadratic regulator problem in
Eq. (113) can be solved from two lower-order subproblems.
(107)
Slow Regulator Problem
(108)
Find the slow control us for the slow subsystem in Eqs. (43)
and (116) to minimize

1 T
Js (us ) =
(qs qs + uTs Rus ) dt, R > 0
2 0
(119)

1 T T
=
( C0 C0 + 2uTs DT0 C0T + uTs R0 us ) dt
2 0
such that the closed-loop subsystems in Eqs. (43) and (44) to

O()
= (A0 + B0 G0 )
(109)
y = (A22 + B2 G2 )y
(110)
312
applied to the system in Eq. (29) achieves an O() approximation of Jopt, that is,
where
R0 = R + DT0 D0
(120)
If the triple (C0, A0, B0) is stabilizable and detectable (observable), then there exists a unique positive-semidefinite
(positive-definite) stabilizing solution Ks of the matrix Riccati
equation
T
1 T
T
0 = Ks (A0 B0 R1
0 D0 C0 ) (A0 B0 R0 D0 C0 ) Ks
T
1 T
+ Ks B0 R1
0 B0 Ks C0 (I D0 R0 N0 )C0
(121)
and the optimal control is

us =
T
R1
0 (D0 C0
J(us ) = Jopt + O()
(129)
In Theorem 1.1, an asymptotic expansion exists for the solution to the matrix Riccati equation associated with the full
linear regulator problem. Theorem 1.3 is one of the robustness results with respect to fast unmodeled dynamics,
that is, if the fast dynamics is asymptotically stable, a feedback control containing only the slow dynamics would not destabilize the fast dynamics.
APPLICATIONS TO LARGE POWER SYSTEMS
+ BT0 Ks )
= G0
(122)
Fast Regulator Problem

Find the fast control uf for the fast subsystem in Eqs. (44)
and (118) to minimize

1 T
(qf qf + uTf Ruf ) dt, R > 0
2 0

1 T T
=
(y C2 C2 y + uTf Ruf ) dt
2 0
Jf (uf ) =
(123)
If the triple (C2, A22, B2) is stabilizable and detectable (observable), then there exists a unique positive-semidefinite
(positive-definite) stabilizing solution Kf of the matrix Riccati
equation
0 = Kf A22 AT22 Kf + Kf B2 R1 B2 Kf C2T C2
(124)
and the optimal control is

uf = R1 BT2 Kf y = G22 y
(125)
The following results are from Reference (8).
In this section, we analyze a large power system as an example of time-scales arising in an interconnected systems. A
power system dispersed over a large geographical area tends
to have dense meshes of power networks serving heavily populated areas and many fewer transmission lines interconnecting these urban centers. When such a system is subject to a
disturbance, it is observed that groups of closely located machines would swing coherently at a frequency that is lower
than the frequency of oscillation within the coherent groups.
Singular perturbation techniques have been successfully applied to these large power networks to reveal this two-timescale behavior (9).
Consider the linearized electromechanical model of an
n-machine power system in the second-order form with damping neglected
M = K
where Rn is the machine rotor angle vector, M is the diagonal matrix of machine inertias, and K is the stiffness matrix
determined by the network impedances. Assume that the system in Eq. (130) has r tightly connected areas, with the connections between the areas being relatively fewer. In this
case, we decompose K into
K = K I + K E
Theorem 1.
1. If the triples (C0, A0, B0) and (C2, A22, B2) are stabilizable
and detectable (observable), then there exists an * 0
such that for all (0, *], an optimal control exists
for the linear regulator problem (113) with an optimal
performance Jopt.
2. The composite control in Eq. (111)
1
T
T
uc = [(I R1 BT2 Kf A1
22 B2 )R0 (D0 C0 + B0 Ks )
1 T
+ R1 BT2 Kf A1
22 A21 ]x R B2 Kf z
(126)
applied to the system in Eq. (29) achieves an O( ) approximation of Jopt, that is,
2
J(uc ) = Jopt + O( 2 )
(127)
3. If A22 is stable, then the slow control in Eq. (122)

T
T
us = R1
0 (D0 C0 + B0 Ks )x
(128)
(130)
(131)
where K I is the stiffness matrix due to the impedances internal to the areas, and K E is the stiffness matrix due to the
impedances external to the areas and scaled by the small parameter that represents the ratio of the external to the internal connection strength.
For illustrative purposes, we let r 2, with r1 machines in
area 1 and r2 machines in area 2. Arranging the machines in
area 1 to appear first in , K I has a block-diagonal structure
K I = block-diag(K1I , K2I )
(132)
A particular property of the ri ri matrix K Ii , i 1, 2, is that

each of the rows in K Ii will sum to zero. If 0, this conservation property results in a slow mode in each area. When
0, the slow mode from each area will interact to form the
low-frequency interarea oscillatory mode.
To reveal this slow dynamics, we define a grouping matrix

1r 1
0
(133)
U=
0
1r 2
where 1ri is an ri 1 column vector of all ones. Using U, we

introduce the aggregate machine angles weighted according
to the inertia as the slow variables
a = (U T MU )1U T M = C
(134)
and the difference angles with respect to the first machine in

each area as the fast variables
d = G
(135)
where

1r 1
1
G=
0
Ir
1 1
0
1r 1
2
0
Ir

(136)
2 1
Noting that CM1K I 0 and K IU 0, that is, C is in the left

null space of M1K I and U is in the right null space of K I, the
system in Eq. (130) in the new variables become
a = CM 1 K EUa + CM1 K E G+ d

d = GM
K EUa + (GM
K I G+ + GM
(137a)
1
K E G+ )d (137b)
where G GT(GGT)1. The system in Eq. (137) clearly points

to the two-time-scale behavior in which the right-hand side of
Eq. (137a) is O(), indicating that a is a slow variable. The
method can readily be extended to systems with r 2 areas.
Based on this time-scale interpretation, a grouping algorithm using the slow eigensubspace has been proposed to find
the tightly connected machines if they are not known ahead
of time. Then the areas whose internal dynamics are of less
interest can be aggregated into single machine equivalents to
capture only the slow dynamics. This concept, together with
some more recently developed algorithms, has been implemented in computer software to reduce large power systems
to smaller models suitable for stability analysis and control
design (10).
FURTHER READING
Singular perturbation techniques have been successfully applied to the analysis and design of many control systems other
than those discussed in this article. For our concluding remarks, we briefly comment on some of these applications as
extensions of the results already discussed.
The two-time-scale properties can also be used to characterize transfer functions of singularly perturbed systems (11).
In a linear discrete-time two-time-scale system, the slow dynamics arise from the system eigenvalues close to the unit
circle, whereas the fast dynamics are a result of those eigenvalues close to the origin. The two-time-scale analysis for continuous-time systems can be readily extended to discrete-time
singularly perturbed systems (12).
An application of the stability results is to determine the
robustness of a control design. For example, Khalil (13) shows
a simple example where a static output feedback designed
without considering the parasitic effects would lead to an instability in the fast dynamics. Another result shows that in
adaptive control, the rate of adaptation must be sufficiently
slow so that the unmodeled fast dynamics would not cause
destabilization (14).
313
The composite control has also been applied to solve optimal regulator problems for nonlinear singularly perturbed
systems (15). Recently, composite control results for H optimal control of singularly perturbed systems have been obtained (16, 17). The composite control idea can also be used
to establish the time scales in a closed-loop system induced
by a high-gain control (18).
Singular perturbation methods also have significant applications in flight control problems (19, 20). For example, twopoint boundary-value problems arising from trajectory optimization can be solved by treating the fast maneuvers and
the slower cruise dynamics separately.
The slow coherency and aggregation technique is also applicable to other large-scale systems, such as Markov chains
(21) and multi-market economic systems (22). These systems
belong to the class of singularly perturbed systems in the nonstandard form, of which an extended treatment can be found
in Ref. 23.
A topic not covered here is the filtering and stochastic control of singularly perturbed systems with input noise. As
0, the fast dynamics will tend to a white noise. Although
the problem can be studied in two-time-scales, the convergence of the optimal solution requires that the noise input in
the fast dynamics be either colored or asymptotically small
(24).
Another topic not covered is the control of distributed parameter systems possessing two-time-scale properties. Averaging and homogenization techniques are also a class of twotime-scale methods (25). More developments are expected in
this area.
In more complex singularly perturbed systems, jump behaviors may arise not only at the end points but also in interior layers. Reference 27, beside being an introductory text to
singular perturbations, contains a detailed treatment of such
phenomena. It also contains a historical development of singular perturbations.
The singular perturbation results presented in this article
represent developments over a period of three decades and
contribute to the advances of modern control theory. As new
control problems are proposed and new applications are discovered for systems with time-scales, we expect that singular
perturbation methods will also be extended accordingly to
provide simpler models to gain useful design insights into
these new problems.
BIBLIOGRAPHY
1. P. K. Kokotovic, H. K. Khalil, and J. OReilly, Singular Perturbation Methods in Control: Analysis and Design, London: Academic
Press, 1986.
2. P. K. Kokotovic and H. K. Khalil (eds.), Singular Perturbations
in Systems and Control, New York: IEEE Press, 1986.
3. A. E. Fitzgerald, C. Kingsley, and A. Kusko, Electric Machinery,
3rd ed., New York: McGraw-Hill, 1973.
4. IEEE Committee Report, Excitation system models for power system stability studies, IEEE Trans. Power Appar. Syst., PAS-100:
494509, 1981.
5. P. V. Kokotovic and P. W. Sauer, Integral manifold as a tool for
reduced-order modeling of nonlinear systems: A synchronous machine case study, IEEE Trans. Circuits Syst., 36: 403410, 1985.
6. H. K. Khalil, Nonlinear Systems, 2nd ed., Upper Saddle River,
314
SKIN EFFECT
7. J. K. Hale, Ordinary Differential Equations, Huntington, NY:

Krieger Publisher Co., 1980.
8. J. H. Chow and P. V. Kokotovic, A decomposition of near-optimum regulators for systems with slow and fast modes, IEEE
9. J. H. Chow et al., Time-Scale Modeling of Dynamic Networks with
Applications to Power Systems, Berlin: Springer-Verlag, 1982.
10. J. H. Chow et al., Inertial and slow coherency aggregation algorithms for power system dynamic model reduction, IEEE Trans.
Power Syst., 10: 680685, 1995.
11. H. K. Khalil, Output feedback control of linear two-time-scale
12. D. Naidu and A. Rao, Singular Perturbation Analysis of Discrete
Time Control Systems, Lecture Notes in Mathematics, Berlin:
13. H. K. Khalil, On the robustness of output feedback control methods to modeling errors, IEEE Trans. Autom. Control, AC-28: 524
528, 1983.
14. B. D. Riedle and P. V. Kokotovic, Integral manifold of slow adaptation, IEEE Trans. Autom. Control, AC-31: 316323, 1986.
15. J. H. Chow and P. V. Kokotovic, A two-stage Lyapunov-Bellman
feedback design of a class of nonlinear systems, IEEE Trans. Autom. Control, AC-26: 656663, 1981.
16. Z. Pan and T. Basar, H-optimal control for singularly perturbed
systems, Part I: Perfect state measurements, Automatica, 29:
401423, 1993.
17. Z. Pan and T. Basar, H-optimal control for singularly perturbed
systems, Part II: Imperfect state measurements, IEEE Trans. Autom. Control, 39: 280299, 1994.
18. K. D. Young, P. V. Kokotovic, and V. I. Utkin, A singular perturbation analysis of high gain feedback systems, IEEE Trans. Autom. Control, 22: 931938, 1977.
19. A. J. Calise, Singular perturbation methods for variational problems in aircraft control, IEEE Trans. Autom. Control, 21: 345
353, 1976.
20. H. J. Kelley, Aircraft maneuver optimization by reduced order
approximations, In C. T. Leonides, (ed.), Control and Dynamics
Systems, New York: Academic Press, 131178, 1973.
21. F. Delebecque and J. P. Quadrat, Optimal control of Markov
chains admitting strong and weak interactions, Automatica, 17:
281296, 1981.
22. G. Peponides and P. V. Kokotovic, Weak connections, time scales
and aggregation of nonlinear systems, IEEE Trans. Autom. Control, 28: 729735, 1983.
23. H. K. Khalil, Feedback control of nonstandard singularly perturbed systems, IEEE Trans. Autom. Control, 34: 10521060,
1989.
24. A. Bensoussan, Singular perturbation results for a class of stochastic control problems, IEEE Trans. Autom. Control, 26: 1071
1080, 1981.
25. A. Bensoussan, J. L. Lions, and G. C. Papanicolaou, Asymptotic
Analysis for Periodic Structures, New York: North-Holland, 1981.
26. V. A. Sobolev, Integral manifolds and decomposition of singularly
perturbed systems, Systems and Control Letters, 5: 169179, 1984.
27. R. E. OMalley, Jr., Singular Perturbation Methods for Ordinary
Differential Equations, Berlin: Springer-Verlag, 1991.
PETAR V. KOKOTOVIC
University of California, Santa
Barbara
JOE H. CHOW
Rensselaer Polytechnic Institute
HASSAN K. KHALIL
Michigan State University
SINUSOIDAL STEADY STATE. See NETWORK ANALYSIS,

SINUSOIDAL STEADY STATE.

Copyright
SOFTWARE FOR CONTROL SYSTEM ANALYSIS AND DESIGN: SYMBOL

MANIPULATION
The computer revolution has radically changed every area of engineering, and control/systems engineering is
not an exception. Indeed, computers have become essential tools in modeling, analysis, design, and implementation of control systems. In particular, they enable the design engineer to tackle problems of ever increasing
complexity (1). The topic of this article is the effectiveness of symbolic computation software so-called computer
algebra, in the analysis and design of control systems.
Symbolic computation software should be distinguished from numerical computation software. The former
performs exact computation and can manipulate whole mathematical (symbolic) expressions, whereas the latter
is limited to approximate computation based on numerical expressions. MATLAB (The Mathworks, Inc.) and
SCILAB (Copyright INRIA; freely available via http://www-rocq.inria.fr/scilab/ are popular numerical software
packages in the control engineering community. Such software assists in a wide variety of control engineering
tasks, from modeling to real-time control implementation. The simulation facilities offered in these packages
make them very popular in educational programs. Similarly, there are a number of readily available symbolic
computation software packages. Mathematica (Wolfram Research, Inc.), Maple (Wiferloo Maple Inc.), REDUCE
c
(Anthony
C. Hearn), DERIVE (Software House Inc., Texas Instruments), Macaulay, COCOA, (freely available
via http://cocoa.dima.unige.it and MACSYMA (Macsyma, Inc.) are some of the more popular ones. Most of
these software packages incorporate numerical as well as symbolic computation so that the user can resort
to numerical analysis of symbolic expressions if required. Observe also that MATLAB, through its Symbolic
Math Toolbox, adds to its numerical core symbolic manipulation capability. In general the distinction between
purely numerical and symbolic computation software begins to blur, the trend being to offer both capabilities
in the same software environment.
In general, a particular control problem can be approached using numerical and/or symbolic software.
More precisely, all computational problems can be classified as (2 p. 275):
(1) Purely Numerical Problems Problems that can be handled only using numerical software and for which
there is no symbolic solution, such as the computation of roots of a univariate polynomial.
(2) Numerically Reducible Problems Problems for which both symbolic and numerical solutions exist, such as
finding the greatest common divisor of a set of univariate polynomials.
(3) Numerically Irreducible Problems Problems that can be handled only using symbolic software, such as the
computation of Grobner based for multivariate polynomials.
From the above classification it is clear that a familiarity with symbolic and numerical software can
benefit the control/systems engineer enormously. In this article, the power and usefulness of symbolic software
are brought to the fore. It is of course understood that numerical software is an equally indispensable tool
in the control engineers toolbox, since, in general, the control designer needs to resort to both symbolic and
numerical software to successfully carry out control design.
1
SOFTWARE FOR CONTROL SYSTEM ANALYSIS AND DESIGN: SYMBOL MANIPULATION
At first contact, symbolic software is reminiscent of university-level engineering analysis and calculus. It
offers the flexibility, excitement, and expectation of computing exactly with a completely abstract representation
of the problem at hand. Unfortunately, it shares the same drawback, in that the problems that can be dealt with
are limited in their complexity. Nevertheless, using symbolic computation, software computations that involve
cumbersome algebraic expressions that are far beyond error-prone manual manipulations become routine. As
a result, symbolic computation software has significantly pushed the boundaries of what can be regarded as an
analytically tractable mathematical problem, and so it has become an invaluable tool in the control designers
toolbox for modeling, design, and analysis of control systems.
A plethora of symbolic computation algorithms targeting control system analysis and design are available. Symbolic computation is useful in almost every step of control system modeling and the analysis and
controller design of linear and nonlinear control systems. In particular, symbolic manipulation of bond graphs
and multidimensional Laplace transforms, which are important tools in the modeling of nonlinear control
systems, are facilitated by computer algebra. The analysis of qualitative and quantitative system properties
under parameter variations is supported by symbolic computation through sensitivity and bifurcation analysis.
Robust control problems in either linear or nonlinear systems are facilitated by computer algebra software.
The important geometric approach to nonlinear control design (e.g., computing a linearizing feedback or determining zero dynamics) is enabled by commercially available symbolic software. A more elaborate overview of
other symbolic software algorithms used in modeling, analysis, and design of control systems is provided in 2.
In this article, polynomial control system analysis and design are targeted, and in particular three important methods are discussed: Grobner bases, differential algebra (in particular, Ritts algorithm), and quantifier
elimination. In essence, these methods deal with polynomials. Computation of Grobner bases is implemented in
most commercial symbolic software packages such as Maple and Mathematica. On the other hand, Ritts algorithm is not as yet commercially available, but software implementations may be obtained from the specialists
working in the field. Quantifier elimination has been implemented in Mathematica.
Polynomial systems are a natural generalization of linear systems; they possess a nice algebraic structure,
and their analysis is amenable to computer algebra. Moreover, as polynomials are universal approximators,
polynomial control models can be used as valid models for almost any physical system. Furthermore, the
restriction to polynomial systems is not severe, since any nonlinear function that satisfies a polynomial differential or algebraic equation, such as trigonometric functions, can be handled by introducing artificial states;
see 3 and 4 for more details.
The state-space model of a polynomial control system is given by
Here x n , y p , u m represent respectively the state, the observed output, and the manipulated control
input of the system. The operator is either the derivative x(t) = dx(t)/dt for continuous-time systems (when
t ) or the forward difference x(t) = x(t + 1) for discrete-time systems (when t ). The symbols , , ,
and
denote respectively the sets of real, rational, natural, and complex numbers. The vector functions f and
h are assumed to have entries that are polynomial functions in all their variables. Moreover, for computational
reasons the polynomials are assumed to have rational coefficients. Continuous-time (or discrete-time) linear
systems
form a special subclass of polynomial control systems (1). A wide range of systems can be adequately modeled
using Eq. (1). Indeed, systems as diverse as biochemical reactors, grain dryers, satellites, robots, and airplanes
have been successfully modeled using polynomial systems (1) (see, e.g., 5).
In the context of polynomial control systems, problems like determining equilibria, estimating a domain
of attraction of a stable equilibrium, and locating approximately periodic solutions, as well as testing controllability and observability properties, reduce naturally to the analysis of sets of algebraic polynomial equations.
Such problems can be addressed using the Grobner basis method. In any modeling exercise, the elimination
of variables from sets of differential and algebraic equations plays a crucial role. This problem also arises
naturally in the context of determining if two state representations are equivalent. Differential algebra and
Ritts algorithm are very useful in this context. A range of different control problems can be formulated as
questions involving quantifiers there exists and for all, and this naturally leads to the so-called quantifier
elimination algorithms.
The Grobner
Basis Method
Central objects in the Grobner basis theory are polynomial ideals and affine varieties (6). Let p1 ,. . ., ps be
multivariate polynomials in the variables x1 ,. . . , xn whose coefficients are in the field k. For the collection of
ordered n-tuples of elements of k the notation kn is used. The variables x1 ,. . . , xn are considered as place
markers in the polynomial. The notation p1 ,. . . , ps k[x1 ,. . . , xn ] is adopted. The affine variety (or variety)
defined by the s polynomials p1 ,. . . , ps is the collection of all solutions in kn of the system of equations
Formally, the variety is
For instance, a straight line, a parabola, an ellipse, a hyperbola, and a single point are all examples of varieties
in 2 . The polynomial ideal I that is generated by p1 ,. . . , ps is a set of polynomials obtained by combining these
polynomials through multiplication and addition with other polynomials: formally,
The polynomials pi , i = 1,. . . , s, form a basis for the ideal I. A very useful interpretation of a polynomial ideal
I is in terms of the equations (3). On multiplying pi by arbitrary polynomials gi k[x1 ,. . . , xn ] and adding the
products, the implication of Eq. (3) is that f = g1 p1 + + gs ps = 0, and of course f I. Hence, the ideal I =
p1 ,. . . , ps contains all the polynomial consequences of the equations (3).
A notion at the core of the Grobner basis method is that of the monomial ordering (a monomial is a
polynomial consisting of a single term), since it introduces an appropriate extension of the notion of the leading
term and the leading coefficient familiar from univariate polynomials to multivariate polynomials. One can
define many different monomial orderings [lexicographic, graded lexicographic, graded reverse lexicographic,
etc. (6)], but to be specific, consider the so-called lexicographic, or lex, ordering. Let , be two n-tuples of
integers [ = (1 ,. . ., n ), = (, . . . , n ) n . The n-tuple is said to succeed (in lex ordering), denoted as
, if in the vector difference = (1 1 ,. . ., n n ) the leftmost nonzero entry is positive. Obviously, it
is possible to define n! different lex orderings for polynomials in n variables. For the polynomial f = 2x1 3 x2 x3 3
+ 4x1 3 x3 5 , using the lex ordering x1 x2 x3 the monomial x1 3 x2 x3 3 succeeds the monomial x1 3 x3 5 , as the
multidegrees of the monomials satisfy (3,1,3) (3,0,5). With this ordering, the leading coefficient and the
leading term are respectively LC(f ) = 2 and LT(f ) = 2x1 3 x2 x3 3 . Using the alternative lex ordering x1 x2 x1 ,
the leading term becomes LT(f ) = 4x1 3 x3 5 .
In general, an ideal I does not have a unique basis, but given any two different bases p1 ,. . . , ps and
g1 ,. . . , gm of I, the varieties V(p1 ,. . . , ps ) and V(g1 ,. . . , gm ) are equal. In other words, a variety only depends
on the ideal generated by its defining equations. Some bases of an ideal may be simpler than other bases.
Intuitively, if all the polynomials in a given basis of an ideal have a degree that is lower than the degree of
any other polynomial in the ideal with respect to a particular monomial ordering, then this basis is in some
sense the simplest basis. In particular, a Grobner basis of an ideal (for a given monomial ordering) has such a
property and can be thought of as the simplest or canonical basis. Given an ideal I and a monomial ordering,
denote the set of leading terms of elements of I as LT(I). The ideal generated by elements of LT(I) is denoted
LT(I). In general, the ideal generated by the leading terms of a particular ideal I is not the same as the
ideal generated by the leading terms of the polynomials in a basis for that particular ideal I. A Grobner basis
is a special basis for which this property holds, and it is formally defined as the set of polynomials g1 ,. . . ,
gm for which LT(I) = LT(g1 ),. . . , LT(gm ). When computing Grobner bases, the user specifies a monomial
ordering; different monomial orderings produce different Grobner bases. Given a monomial ordering, the two
most important properties of Grobner bases are:
(1) Every ideal I k[x1 ,. . . , xn ], other than the trivial ideal 0, has a Grobner basis. Furthermore, any Grobner
basis of an ideal I is a basis for I.
(2) Given an ideal I k[x1 ,. . . , xn ], other than the trivial ideal 0, a Grobner basis of I can be computed in a
finite number of algebraic operations.
The first algorithm for computation of Grobner bases, published in the 1960s, is attributed to B. Buchberger (6). Since then a number of improvements have been reported and the algorithm has been implemented in most commercial symbolic software packages. Buchbergers algorithm generalizes two well-known
algorithms: Gauss elimination for sets of multivariate linear algebraic equations, and Euclids algorithm for
computing the greatest common divisor of a set of univariate polynomials.
Solving Sets of Multivariate Polynomial Equations. Grobner bases facilitate solving a set of multivariate polynomial equations (3) in the same way as the Gauss elimination algorithm facilitates solving a
set of linear algebraic equations. Indeed, in a given lex ordering the Grobner basis has a triangular structure
reminiscent of the triangular structure in Gauss elimination. If the Grobner basis is given by {1}, then the
system of polynomial equations has no solution.
Equilibria for Polynomial Systems. Consider a polynomial system without inputs x(t) = f (x(t)). A state
x n is called an equilibrium of this system if x(0) = x implies that x(t) x t. Equilibria for polynomial
systems are therefore obtained as solutions of a set of polynomial equations of the form f (x) = 0 or f (x) = x for
continuous-time or discrete-time systems respectively. Grobner bases facilitate finding all equilibria.
Example: The equilibria of the polynomial system
are obtained as solutions of the set of polynomial equations
which is hard to solve. The Grobner basis for the ideal p1 , p2 , p3 using the lexicographic ordering x1 x2 x3
is computed using Maple:
By construction the equations gi = 0, i = 1, 2, 3, 4, have the same solutions as Eq. (6). But obviously the
polynomials gi have a much simpler structure than the polynomials pi . Indeed, g4 depends only on x3 , and this
allows g4 = 0 to be solved numerically for x3 . The solutions can be substituted into g2 = 0 and g3 = 0 to obtain
polynomials in x2 only that can then be solved numerically. This process of back-substitution can be continued
until all solutions are found.
Observe that a basis for an ideal does not have a fixed number of basis polynomials, and indeed the simpler
Grobner basis contains more polynomials than the original basis. As may be observed, it even contains more
polynomials than there are variables.
Periodic Solutions for Polynomial Systems. Consider the system f (dn y/dtn , . . . , dy/dt, y, t) = 0, where
y can be thought of as the output of a closed-loop control system. Let f be a polynomial in all its variables.
To approximately compute a periodic solution y(t) of this system, the method of harmonic balancing can be
used. It can be shown that under mild conditions an approximate periodic solution implies the existence of
an exact periodic solution close to the approximate one (7). A truncated Fourier series can be considered
as a candidate approximate periodic solution y(t) = k = N N Ck e jkt , ck , where ck and c k are complex
conjugates. Postulating that y(t) is a solution leads to a set of polynomial equations, because f is polynomial.
Using the Grobner basis method with lex ordering, this set of equations can be solved for the unknowns c0 ,
c1 ,. . . , cN and to obtain y(t).
In practice the actual solution may not be so important, and it often suffices to know that a periodic
solution exists and to know its oscillation frequency . This can be achieved by finding the Grobner basis of
the polynomial equations with any lex ordering where is the lowest-ranking variable.
Example: Consider the van der Pol equation (8) y a (1 by2 ) y + y = 0, and postulate a periodic solution y(t)
= , where ck = ckr + cki j, k = 0, 1, 2, 3, and j = . Substituting y(t) into the van der Pol differential equation
and equating the coefficients on the left-hand and right-hand sides of the equation leads to
and c0 = 0, c2 = 0, and c1i = 0. The coefficients a and b are regarded as variables. To simplify the computations,
p1 is divided by c1 , and p2 by ac1 . A Grobner basis of the ideal p1 , p2 , p3 , p4 with the lex ordering c3 c3i
c3 b a contains seven polynomials of nontrivial complexity (42 pages of Maple output). It contains
one polynomial in and a only:
Obviously, for a = 0 the van der Pol oscillator is a simple harmonic oscillator with frequency = 1. In the
presence of nonlinear damping a = 0, the solution of g7 = 0, which continues = 1, can be expanded as
and using Maple, the coefficients i are obtained:
Grobner
Bases and Elimination. Grobner bases can be used to eliminate some variables of interest
from a set of polynomial equations, as illustrated by Example 1, where x1 and x2 are eliminated from g4 in Eq.
(7). The variables that are ranked higher in the lex ordering are eliminated first. This feature of Grobner bases
can be used to obtain an estimate of the domain of attraction of equilibria for polynomial systems.
Estimation of the Domain of Attraction. If the closed-loop control system is stable, it is of interest to
determine those initial conditions from which the solutions converge to a specified equilibrium. The collection
of all the initial conditions that converge to the same equilibrium is called the domain of attraction of that
particular equilibrium. Estimating domains of attractions is a hard problem. One way of obtaining a (conservative) estimate for the domain of attraction is to use Lyapunov functions. It is a standard result in Lyapunov
theory that if x = 0 is an equilibrium point for the continuous-time system d/dt x = f (x), if D n is a domain
containing x = 0, and if W : D is a continuously differentiable function such that W(0) = 0 and for all x D
{0} one has W(x) > 0 and W/x f (x) < 0, then x = 0 is asymptotically stable. Given such a Lyapunov function,
consider the sets = {x n : W/xf (x) < 0} and Bd = {x n : W(x) d}. If Bd for some d > 0, then
the set Bd is an estimate for the domain of attraction. For polynomial systems with a polynomial Lyapunov
function W, Grobner bases can be used to compute Bd systematically. Indeed, it is feasible to construct the
largest such set by finding d such that Bd is as large as possible and still inside . For polynomial systems
with polynomial Lyapunov functions, W(x) d and W/x f (x) are polynomials, and hence the boundaries of
the sets Bd and are varieties. At the points where the varieties V(W d) and V (W/xf (x)) touch each other,
the gradients of W and
variables is obtained:
are parallel. Using this information, a system of n + 2 polynomial equations in n + 2
computing a Grobner basis for the above system of equations, where the variable d has the least rank in the
lex ordering, a polynomial equation in d only is obtained. The least positive solution to this equation is the
best value of d for which Bd , and this yields in turn the best estimate for the domain of attraction that
could be obtained with the particular Lyapunov function W.
Example: Consider the system (8)
with the Lyapunov function W(x) = 3x2 1 + 4x1 x2 + 4x2 2 . The polynomials (8) are in this case
and using the lexicographic ordering x2 x1 d, the Grobner basis of the ideal p1 , p2 , p3 , p4 is computed.
It contains the polynomial g(d) = 4d4 147d3 + 786d2 + 2048d. The approximate solutions of g(d) = 0 are
1.9223, 0, 8.9657, 29.707. The smallest positive value of d for which there is a solution to the system of
equations pi = 0 is 8.9657. The corresponding estimate for the domain of attraction is therefore {x 2 : W(x)
8.9657}.
Equality of Ideals. A special type of a Grobner basis can be used to decide if two ideals are the same
or not. This is the so-called reduced Grobner basis. A reduced Grobner basis for an ideal I is a Grobner basis
G for I such that LC(p) = 1 for all p G, and for all p G, no monomial of p lies in LT(G {p}). The main
property of reduced Grobner bases is that given an arbitrary ideal I = 0 and a particular monomial ordering,
I has a unique reduced Grobner basis. Hence, two ideals J 1 and J 2 are the same if and only if their reduced
Grobner bases G1 and G2 are the same (G1 and G2 must be computed with the same monomial ordering).
Most commercial computer algebra systems, such as Maple and Mathematica, provide finite algorithms for
computing reduced Grobner bases.
Analysis of Discrete-Time Polynomial Systems. Fundamental control-theoretic concepts such as controllability or observability can be reduced to the problem of computing maximal control-invariant varieties.
This is well known for linear systems (computation of control-invariant subspaces), but it holds equally well
for discrete-time polynomial systems of the form x(t + 1) = f (x(t),u(t)). For such systems, a variety V is controlinvariant if f (V,u) V for all possible control inputs u. The computation of the maximal control-invariant
subset of a given variety V can be completed in finitely many operations. Consider the defining ideal of the
variety V; say J 1 = g1,1 ,. . ., g1,m1 , where gj,k [x]. If the variety corresponding to J 1 were control-invariant,
then g1,k f (x,u) 0 for all u and all k = 1,. . . , m1 . The polynomials g1,k f (x,u) can be viewed as polynomials
in u with coefficients in [x]. Denote the collection of all these coefficients as g2,k , k = 1, 2,. . . , m2 , and the
corresponding ideal as J 2 = g2,1 ,. . . , g2 ,m2 . Invariance would imply that J 1 = J 2 , and if this is not the case,
then obviously J 1 J 2 and the corresponding varieties satisfy V 1 V 2 . This process can be continued to
construct an ascending chain of ideals (or descending chain of varieties) J 1 J 2 J 3 . This chain must
terminate in finitely many steps (6). That is, there exists an integer N such that J N 1 J N = J N+1 =
. The variety V(J N ) is the maximal control-invariant subset of V(J 1 ). The check whether J k = J k+1 can be
completed via the computation of reduced Grobner bases for J k and J k+1 .

Observability. The discrete-time polynomial system
as said to be observable if for each pair of initial states = , there exists an integer N and an input sequence U N
= {u(0),. . . , u(N1)} such that the solutions starting at and produce different outputs after N steps, that is
h(x(N, , U N )) = h(x(N, , U N )). The polynomial h(x(N, , U N )) h(x(N, , U N )) can be regarded as a polynomial
in elements of U N with coefficients in [, ]. A chain of ideals J k is constructed using these coefficients, and
at each step the condition J k = J k+1 is tested. It can be shown that if J N for some N has a reduced Grobner
basis {1 1 ,. . . ,n n }, then J N+1 = J N and the system is observable. The above-discussed algorithm
for computing invariant sets streamlines these computations and allows a systematic determination of the
integer N.
Example: Consider the simple Wiener system (9)
The system consists of a linear dynamical system and a quadratic static output nonlinearity. The ideal J 1 =
1 2 1 2 and the output equation (12) are used to construct the following ideals:
Using the lex ordering 1 2 1 2 , the reduced Grobner basis for J 4 is G4 = {1 1 , 2 2 }, and therefore
the system (12) is observable with N = 4.
A Brief Overview of the Literature and Related Problems. Grobner bases are useful for a range of other
control problems, such as the inverse kinematic problem and motion planning in robotics (6), the computation
of the switching surfaces in the solution of time optimal control problems (10,11), identifiability, inputoutput
equivalence of different state-space realizations, normal forms and zero dynamics (12), analysis of hybrid
control systems, computation of limit cycles for discrete-time polynomial systems, observability of continuoustime polynomial systems, and forward accessibility of discrete-time polynomial systems.
In control design for linear systems, the linear functional observer problem, the model-matching problem,
the deterministic identification problem, and the disturbance decoupling problem play a central role. All these
problems can be seen as the characterization of a maximally control-invariant subspace of a given subspace
of the state space (2). In control theory this problem is known as the cover problem. The cover problem can be
solved via a computation of elementary divisors of matrix pencils, which in turn leads to multilinear equations
where the unknowns are the basis vectors of the invariant subspace. The Grobner basis method facilitates the
analytic solution of this problem. The elegance of this approach stems from the fact in one single algorithm all
possible (typically infinitely many) control-invariant subspaces can be explicitly characterized. The degrees of
freedom established in this process can then be further exploited to optimize other desirable system properties,
such as sensitivity.
We have focused on the use of Grobner bases for commutative rings over infinite fields, that is, finding
Grobner bases in the context of polynomials with rational coefficients. Two important related areas are Grobner
bases for commutative rings over finite fields and Grobner bases in the context of noncommutative rings. The
former is of interest in the context of discrete-event dynamic systems (13,14) and coding theory in communications. The latter is useful for control-theoretic problems involving polynomial matrices (15). In particular, in
15 a new extra term in the expansion of the classical state-feedback optimal control problem in the singular
perturbation form was obtained using noncommutative Grobner bases.
An important drawback of Buchbergers algorithm is that even with the best known versions of the
algorithm, it is easy to generate examples for which the computation of a Grobner basis requires inordinate
amounts of computer resources (time and/or memory). The main bottlenecks are that the total degrees of the
intermediate polynomials that the algorithm generates may be quite large and that the coefficients of the
Grobner basis may be complicated rational numbers. This may be the case even if the original ideal generators
are polynomials of small degrees with small integer coefficients. In general, the intermediate polynomials
observed in the computation of a Grobner basis can have total degrees of the order of 22d , where d is the
total degree of the ideal generators. Although this appears to be a rather negative result, it typifies the
worst-case scenario. It appears that the running time and the storage requirements seem to be much more
manageable on average. It is important to emphasize that different monomial orderings may produce very
different computational times, and some experimentation with the ordering may yield significant reductions
in computation time.
Differential Algebra In Control

In the 1980s differential algebra was applied to control problems (16). Differential algebra can be used to
transform a polynomial system from one representation to another. In general differential algebra plays an
important role in the realization theory of nonlinear control systems. In the context of control, the problems
that lend themselves to differential algebra are varied: the determination of observability, identifiability, the
calculation of zero dynamics, regulator computations, tracking control, etc.
Differential-Algebraic Tools. Differential algebra provides tools for dealing with systems of polynomial differential equations. In this algebra the derivative operation occupies center stage. A multivariate
polynomial in variables y1 ,. . . , yN and their derivatives is called a differential polynomial in y1 ,. . . , yN . For
instance, f (dn y/dtn , . . . , dy/dt, y) is a differential polynomial in y if f is a polynomial in all its variables.
The concept of ranking is introduced for differential polynomials. It is very similar to the concept of
monomial ordering for polynomials. Ranking is a total ordering of all variables and their derivatives. Examples
involving two variables are
and
10
where denotes is ranked lower than. Any ranking is possible provided it satisfies two conditions:
for all variables u and y, all nonnegative integers and v, and all positive integers . The highest-ranking
variable or derivative of a variable in a differential polynomial is called the leader. The ranking of variables
gives a ranking of differential polynomials. They are simply ranked as their leaders. If two have the same
leader, they are considered as polynomials in their leader and the one of lower degree is ranked lower.
Let A, B be two differential polynomials, and let A have the leader v. Then B is said to be reduced with
respect to A if there is no derivative of v in B and if B has lower degree than A when both are regarded as
polynomials in v. A set
of differential polynomials is called autoreduced if all the Ai are pairwise reduced with respect to each other.
Normally autoreduced sets are ordered so that A1 ,. . ., Ap are in increasing rank. Autoreduced sets are ranked
as follows. Let A = A1 ,. . . , Ar and B = B1 ,. . . , Bs be two ordered autoreduced sets. A is ranked lower if either
there is an integer k, 0 k min(s, r), such that
or else if r > s and
A characteristic set for a given set of differential polynomials is an autoreduced subset such that no other
autoreduced subset is ranked lower.
The separant SA of a differential polynomial A is the partial derivative of A with respect to the leader,
while the initial I A is the coefficient of the highest power of the leader in A. If a differential polynomial f is not
reduced with respect to another differential polynomial g, then either f contains some derivative of the leader
ug of g or else f contains ug to a higher power. In the former case one can differentiate g a suitable number (say
) of times and perform a pseudodivision to remove that derivative, giving a relation
where S is the separant of g and R does not contain the highest derivative of ug that is present in f . If f contains
ug to a higher power, a pseudodivision of f by g can be performed to obtain
where I is the initial of g, and R is reduced with respect to g.
11
Observability. In control theory a system is called observable if it is possible to compute the state from
inputs and outputs. Using differential algebra, the observability question can be answered in a constructive
manner for continuous-time polynomial systems (1).
Example: Consider the system
which is in state-space form. Suppose an inputoutput description (i.e., a description directly relating u and y)
is called for. The original set of equations is equivalent to
It is now possible to eliminate the derivative of x2 by forming
From this construction it follows that p = 0 whenever the equations of (19) are satisfied. The last equation of
(19) can be replaced by p = 0 to get the system description
From (Eq. 20) it follows that every solution of Eq. (19) also solves Eq. (21). If, moreover, it is known that x2 =
0, then the converse is also true, and Eqs. (19) and (21) are equivalent. It is now possible to form
and it readily follows that
is equivalent to (Eq. 21) if also u = 0. Finally form
to conclude that, provided ux2 = 0, Eq. (19) is equivalent to the following system description:
12
The leftmost equation is an inputoutput relation, while the middle and right equations show how x1 and x2
can be computed from the input and output. This establishes the observability, and more.
Using the terminology of differential algebra, the state-space form is an autoreduced set under the ranking
while Eq. (25) is an autoreduced set under the ranking
Autoreduced sets thus give a generalization of several of the standard forms for dynamic systems.
Global Identiability. A polynomial system that is parametrized with a (constant) parameter as
as said to be globally identifiable if, given input and output measurements u(t) and y(t) over an interval t
[0,T], there is only one constant value of that satisfies Eq. (26). Global identifiability of polynomial systems
(1) can be checked in a constructive manner using the tools from differential algebra. Moreover, conditions on
u and y under which the system is not globally identifiable can also be obtained, as is illustrated below.
Example: Consider the system
where is a constant parameter. Can be identified by observing the output y? Using the notation p = y +
2 y + 2 y, compute the expression
It follows readily that is uniquely determined by y (and its derivatives), provided that along the output
trajectory
This is thus an excitation condition that guarantees global identifiability.
The Algorithms of Ritt and Seidenberg. The concepts of the previous section were introduced in the
1930s by the American mathematician Ritt, (17). He devised an algorithm by which it is possible to start with
an arbitrary number of differential polynomials, introducing a suitable ranking of the variables and performing
successive operations of the type described by Eqs. (14), (15) to arrive at an equivalent representation in the
form of an autoreduced set of equations. This set has the property that an f belonging to the original differential
polynomials can be reduced to zero using (Eqs. 14) and (15) for different gs belonging to the set. If certain
factorization conditions are met, then Ritt showed that the generated autoreduced set is a characteristic set,
not only for the explicitly generated polynomials, but also for an infinite set of polynomials constituting the
13
so-called differential ideal. Ritt used his algorithmic procedure to produce an extensive theory for polynomial
systems of differential equations. Later a more systematic algebraic treatment was given by Kolchin (18). The
theory was also extended to partial differential equations (18) and difference equations (19).
From the viewpoint of practical calculations, a major drawback of Ritts algorithm is the factorization it
requires, since this is a task of high computational complexity. Seidenberg (20) has proposed an algorithm for
deciding the solvability of systems of equations and inequations. The algorithm uses (Eqs. 14), (15) repeatedly
to eliminate variables and reduce the problem to the single-variable case. As in the simple example above,
equivalence of the original and final sets of equations requires the separant S and initial I of (Eqs. 14), (15) to
be nonzero. To handle this, Seidenbergs algorithm splits the problem into different subproblems, considering
the cases S = 0, S = 0 (or I = 0, I = 0) together with the original equations and inequations. By repeated
splitting a tree of subproblems is generated. Each subproblem is finally reduced to a single-variable problem
for which solvability can be decided. In recent years several variations of Ritts and Seidenbergs algorithms
have been implemented in symbolic manipulation languages like Maple or Mathematica.
Other Control Problems. Differential-algebra calculations can be used to compute the zero dynamics
of a system. This dynamics is obtained by restricting the output to be zero. By adding the equation y = 0 and
using Ritts algorithm with a suitable ranking, it is possible to obtain a description of the resulting dynamics. In
a similar manner a regulator description can be obtained. Suppose a differential equation describing the desired
response from reference signal to output is given. Using this equation together with the system dynamics as
the input to Ritts algorithm, it is possible to arrive at an equation containing the control signal together with
the reference and output signals. This equation can be interpreted as a description of the regulator, although
it can be difficult to implement due to its implicit nature in the general case.
Quantier Elimination
Quantifier elimination is a method for rewriting formulae that include quantifiers such as for all () and there
exists () in an equivalent form without the quantifiers and the quantified variables. According to a theorem
by Tarski (21), it is always possible to eliminate the quantified variables in formulae consisting of logical
combinations of multivariate polynomial equations and inequalities. Tarski provided a constructive proof in
the late 1940s, but the corresponding algorithm has such complexity that it is impractical for most problems.
In the mid 1970s Collins (22) presented a new method, the so-called cylindrical algebraic decomposition, which
exhibits much better complexity. Since then the algorithmic development has made significant progress (23,24,
25). Nevertheless, cylindrical algebraic decomposition and quantifier elimination are known to be inherently
complex (26). It is therefore of importance to identify classes of problems for which the computational complexity
is much lower, and for which specialized algorithms can be developed (27,28). Implementations of cylindrical
algebraic decomposition and quantifier elimination are available in for example Mathematica (29). Depending
on the specific nature of the problem posed, different algorithms are used in order to minimize the computational
burden. The algorithmic developments are ongoing.
Introductions to cylindrical algebraic decomposition and quantifier elimination can be found in Ref. 30.
An extensive bibliography covering early papers can be found in Ref. 31, and a survey of the algorithmic
development of cylindrical algebraic decomposition is given in Ref. 32.
An early application of quantifier elimination techniques in control theory was made by Anderson et al.
(33). This contribution predates the introduction of efficient computational tools, and so it was of theoretical
interest only. With the availability of the experimental software QEPCAD (23), the number of papers related to
control increased. Now, as these methods are implemented in widely available symbolic computation software
packages like Mathematica, the control practitioner can explore their potential.
Systems of Real Algebraic Equations and Inequalities. In the context of polynomial equations
with real coefficients, inequalities arise naturally, for example to express when a quadratic polynomial has
14
real solutions. The need of using not only equations but also inequalities in describing computational problems
triggered the development of methods for doing symbolic computations with so-called semialgebraic sets. A
semialgebraic set is a generalization of an algebraic set or variety, as it is the solution set of a system of real
algebraic equations and inequalities.
An algebraic expression in variables {x1 ,. . . , xn } is an expression constructed with {x1 ,. . . , xn } and
rational numbers, using addition, multiplication, rational powers, and algebraic numbers and functions. A
system of real algebraic equations and inequalities in variables {x1 ,. . . , xn } is a logical combination of equations
and inequalities with both sides being algebraic expressions in {x1 ,. . . , xn }. The notation (and), (or),
(implies) for Boolean operators is used.
The following is an example of a system of real algebraic equations and inequalities in the variables a, b,
x, y:
Alternatively, a semialgebraic set can be characterized as a set obtained by finitely many unions, intersections,
and complementations of sets of the form {x | f (x) 0}. Here f is a multivariate polynomial with rational
coefficients. Semialgebraic sets are thus closed under projection, union, and intersection. As Grobner bases
provide a way to replace a system of multivariate polynomials by a simpler equivalent set, there are systematic
ways to simplify a system of real algebraic equations and inequalities. It can be shown that the set of solutions of
any system of real algebraic equations and inequalities in variables {x1 ,. . . , xn } can be written as a disjunction
of a finite number of cylindrical parts of the form
In the above expression, stands for one of <, , and =; and f i and gi are either , , or algebraic expressions
in the variables {x1 ,. . . , xi i } that are real-valued for all tuples of real numbers {a1 ,. . . , ai i } satisfying
The method of rewriting a real algebraic system as a disjunction of the above form is called cylindrical algebraic
decomposition; see Refs. 22,30,32. Observe the triangular nature of the resulting system.
Example: An example of a script in Mathematica for computing a cylindrical algebraic decomposition of an
ellipsoid in 3 is
15
Here the functions f i and gi can be expressed in terms of simple algebraic operations and radicals. More
typically the full generality offered by algebraic functions is required:
In Mathematica algebraic functions are represented by special Root objects. In the above case the placeholder
variable #1 is a function of x and y. The last argument enumerates the branches of the algebraic function,
which in this case are the first two roots according to Mathematicas enumeration scheme of the 12 possible
complex roots.
A quantified system of real algebraic equations and inequalities in variables {x1 ,. . . , xn } is an expression of
the form
where the Qi s are the quantifiers or , and S is a system of real algebraic equations and inequalities in
{x1 ,. . . , xn ; y1 ,. . . , ym }. According to Tarskis theorem the solution set of quantified systems of real algebraic
equations and inequalities is a semialgebraic set, which means that QS always can be rewritten as an equivalent
expression without any quantifiers and quantified variables. The process of eliminating quantified variables
from such systems is called quantifier elimination. The yi s in QS are called bounded variables, and the xi s free
variables. Problems without free variables are called decision problems.
Example: The proof of the well-known inequality between the arithmetic and the geometric mean of two real
positive numbers can be formulated as a decision problem:
The notation for quantifiers in Mathematica is
where Q is either or , var is a single variable or list of variables, and cond gives additional conditions
on the variables such as their domain and/or semialgebraic constraints. The command to perform quantifier
elimination in Mathematica is called Resolve.
16
Projection of semialgebraic sets is equivalent to quantifier elimination on existential quantifiers. Consider for
example the projection of the ellipsoid in Example 7 onto the xy plane:
Set inclusion for semialgebraic sets is another useful property that can be formulated as a decision problem: A
B is the decision problem xA B:
Sets of Equilibria. As illustrated before, Grobner bases are useful to determine equilibria for polynomial systems. In polynomial control systems it is of importance to describe the steady-state solutions: the set of
reachable equilibria, for admissible input values. Characterizing the set of reachable equilibria in state space
is a quantifier elimination problem.
Example: Add a control input to the last equation in Example 1. Let this input be constrained to the interval
1 u 1. Compute the projection of the set of corresponding equilibria onto the x1 x2 plane.
Stability. To determine the local stability of equilibria of dynamic systems the stability tests of Routh
and Hurwitz (34) may be used. These stability tests give explicit conditions on the coefficients of the characteristic equation of the Jacobian of the dynamical system at equilibrium in terms of a number of polynomial
inequalities. Hence the RouthHurwitz test can be handled using cylindrical algebraic decomposition and
quantifier elimination. A less well-known, but equivalent stability test is the LienardChipart criterion (34),
which has the advantage over the Routh criterion that it involves polynomial inequalities of lower degree.
Given a polynomial in [s] a0 sn + a1 sn 1 + + an 1 s + an , ai > 0, the LienardChipart criterion states
that all its zeros have strictly negative real parts if and only if the following inequalities holds:
17
where Di , i = 1,. . . , n, are the so-called Hurwitz determinants of order i, which are defined as follows:
where ak = 0 for k > n.

Example: Consider the following polynomial system with real parameters a and b. Determine for which values
of a and b the origin is a stable equilibrium (this is an essential step in any bifurcation analysis of a dynamical
systemthe classification of possible asymptotic behaviors of the system):
For zero input u = 0 the zero state is an equilibrium. Linearizing the system around (x,u) = (0,0) and computing
the characteristic polynomial of the resulting state transition matrix gives
Compute the solution set corresponding to the LineardChipart stability inequalities for this polynomial:
Example: Consider the problem of stabilizing in a unit-feedback scheme an unstable system G, using a lead
compensator F that is parametrized by the parameters B and b, where
The LienardChipart inequalities for the closed-loop system are
18
These can be rewritten using cylindrical algebraic decomposition to either
or
depending on the choice of the order for b and B. Using quantifier elimination, questions like For which values
of b can it be guaranteed that for all values of 5 < B < 10 the closed loop system is stable? can be answered:
Numerically this corresponds to 0.65359 < b < 1.34641.
Nonlinear Tracking. Tracking a particular output is an important control objective. There are many
control design methods that deal with this problem, mainly based on optimization techniques. However, most
(analytic) methods do not take the ever present constraints on control signals or states into account. As a
consequence the resulting control system must be validated, often through extensive simulations. For a large
class of nonlinear tracking problems, quantifier elimination can be used to decide if there exists a solution to a
given tracking problem subject to the constraints of interest.
Consider the continuous-time polynomial control system
where x n , u m and each component of f is a polynomial in all its variables. Furthermore, the input value
u(t) is required to belong to some semialgebraic set U:
called the admissible control set. Let be a rationally parametrized curve in the state space, i.e.,
where g :
is a rational function and the orientation of is defined by increasing values of s.
For the state of the system (27) to follow the curve , there has to be an admissible control u such that
the vector field f (xp , u) is tangent to at each xp on . Since the curve is parametrized in terms of s, this
19
tangency requirement can be formulated as
where > 0 ensures that the state follows in only one direction. The tracking problem is thus equivalent to
deciding if the following formula is true or false:
The above problem becomes a quantifier elimination problem if f or g include some free (design)
parameters.
Example: Consider the polynomial control system
and the curve = {x 2 |x1 = s x2 = s3 + 1.5s2 0 s 1}. Is it possible to steer the system
state along using control signals constrained to 1 u 1? Using Mathematica, this can be resolved as
follows:
The tracking problem has a solution as long as 0 a 1.12769.
20
Consider now a = 1, and compute the limits on u such that the above tracking problem has a solution:
For the tracking problem to have a solution it suffices that 0.98077 u 0.98077.
Multiobjective Feedback Design. Many objectives in linear system design are formulated in terms
of frequency-domain inequalities. Let the plant be represented by G(s, p) and the controller by F(s, q). Both G
and F are assumed to be scalar-valued rational functions of the Laplace variables s, real plant parameter p,
and real controller parameter q. It is now possible to write many robust design problems in a form suitable for
quantifier elimination.
Stability. The unit-negative-feedback closed-loop system consisting of plant and controller is asymptotically stable if and only if all zeros of the rational function 1 + G(s, p)F(s, q) have strictly negative real part.
This can be converted into polynomial inequalities in p and q using the LineardChipart criterion.
Tracking Error. The tracking error at the output of the unit-negative-feedback loop of plant and controller
is governed by the so-called sensitivity transfer function
Acceptable levels of the tracking error can be specified by the inequality |S(iw)| < T , 0 w w1 , which can be
rewritten as a real algebraic inequality in the variables w, p, q, and T . Through quantifier elimination it can
be verified that for the class of systems p P a single stabilizing controller q exists that meets the sensitivity
objective.
In essence any linear control design question formulated in frequency domain leads naturally to a quantifier elimination or cylindrical algebraic decomposition problem. More importantly, symbolic manipulation
software is up to the task of systematically approaching these control design questions in practical and moderately complex situations.
Related Problems and Other Control Applications. There is a vast array of problems in control
that can be posed as quantifier elimination problems or systems of multivariate polynomial inequalities,
whose solutions can be described by cylindrical algebraic decomposition. For example, quantifier elimination
has been used to investigate stability, stabilizability, and controllability of discrete-time polynomial systems
(35,36), stability and stabilizability of switched polynomial systems and unstable zero dynamics via switching
(30,37), frequency-domain design (38), and multiobjective robust control design (39). Quantifier elimination
has also been used for design of nonlinear control systems for nonlinear aircraft dynamics (40) and for robust
nonlinear feedback design (41).
21
The generality of the quantifier elimination method comes at the cost of very high computational complexity. Quantifier elimination software tools are not a panacea allowing one to approach every control problem
by brute force. The challenge in successfully applying quantifier elimination software tools lies in formulating
the problems in such a way that their inherent structure is maximally exploited. Most importantly, instead of
applying quantifier elimination to a complete problem, there are often simplifications that can be carried out
so that only a small problem (with far fewer variables) has to be handled by quantifier elimination. Control
designers should start focusing on providing the right heuristics and tools, built on their experience, in order to
make general-purpose quantifier elimination tools better attuned to control problems. In parallel, algorithm designers now focus on developing specialized methods as components of a general quantifier elimination package
in order to exploit structure and to handle complexity. For instance, in Mathematica (25) there are a modified
simplex linear optimization algorithm for linear systems of equations and inequalities with inexact or rational
coefficients, linear quantifier elimination for equations and inequalities with exact coefficients according to 27,
preprocessing by linear equation solvers, Grobner bases, special care and simplification for variables appearing
at most linearly in the systems, a simplified cylindrical algebraic decomposition algorithm for generic solutions
(24), and quantifier elimination by partial cylindrical algebraic decomposition (23) for the general case.
Conclusion. The possibilities offered through symbolic (and numeric) software in order to address
control problems are as varied and exciting as the control problems themselves. This article just presents the
tip of an iceberg. Dealing with complexity in control design is the main issue. For control algorithm developers
there is now more than ever a compelling need to focus on reliable software algorithms that allow nonspecialists
to approach a control problem with confidence. Software tools that provide information to the user about the
likelihood of success and the difficulties in a particular solution are called for to make this a reality.
BIBLIOGRAPHY
1. F. Cucker L. Blum S. Smale M. Shub Complexity and Real Computation, Berlin: Springer-Verlag, 1998.
2. N. Munro (ed.) Symbolic Methods in Control System Analysis and Design, Institution of Electrical Engineers, 1999.
3. L. A. Rubel M. F. Singer A differentially algebraic elimination theorem with applications to analog computability in
the calculus of variations, Proc. Amer. Math. Soc. 94: 653658, 1985.
4. P. Lindskog Methods, algorithms and tools for system identification based on prior knowledge, PhD thesis 436, Department of Electrical Engineering, Linkoping University, Sweden, 1996.
5. D. Nesic Dead-beat control for polynomial systems, PhD thesis, RSISE, Australian National University, Canberra,
Australia, 1996.
6. J. Little D. Cox D. OShea Ideals, Varieties and Algorithms, Berlin: Springer-Verlag, 1992.
7. A. Mees Dynamics of Feedback Systems, Chichester: Wiley, 1981.
8. K. Forsman Constructive commutative algebra in nonlinear control, PhD thesis, Department of Electrical Engineering,
Linkoping University, Linkoping, Sweden, 1997.
9. D. Nesic A note on observability tests for general polynomial and simple WienerHammerstein systems, Syst. Control
Lett., 35: 219227, 1998.
10. S. T. Glad An algebraic approach to bangbang control, European Control Conference, ECC 95, Rome, 1995, Vol. 4, pp.
28922895.
11. T. Georgiou U. Walther A. Tannenbaum Computational algebraic geometry and switching surfaces in optimal control,
Proc. Conf. Decision Control, Phoenix, AZ, 1999, pp. 47244729.
12. H. Fortell Algebraic approaches to normal forms and zero dynamics, PhD thesis, Department of Electrical Engineering,
Linkoping University, Linkoping, Sweden, 1995.
13. R. Germundsson Symbolic systems, PhD thesis, Department of Electrical Engineering, Linkoping University,
Linkoping, Sweden, 1995.
14. J. Gunnarsson Symbolic methods and tools for discrete event dynamic systems, PhD thesis, Department of Electrical
Engineering, Linkoping University, Linkoping, Sweden, 1997.
15. F. D. Kronewitter III Non-commutative computer algebra, PhD thesis, University of California, San Diego, 2000.
22
16. M. Fliess S. T. Glad An algebraic approach to linear and nonlinear control, in H. L. Trentelman and J. C. Willems (eds.),
Essays on Control: Perspectives in the Theory and Its Applications, Boston: Birkhauser,
1993.
17. J. F. Ritt Differential Algebra, New York: American Mathematical Society, 1950.
18. E. R. Kolchin Differential Algebra and Algebraic Groups, New York: Academic Press, 1973.
19. R. M. Cohn Difference Algebra, Huntington, NY: R. E. Krieger, 1979.
20. A. Seidenberg An elimination theory for differential algebra, University of California Publications in Mathematics,
New Series, 1956, pp. 3166.
21. A. Tarski A Decision Method for Elementary Algebra and Geometry, 2nd ed. University of California Press, 1948.
22. G. E. Collins Quantifier elimination for real closed fields by cylindrical algebraic decomposition, Second GI Conf.
Automata Theory and Formal Languages, Kaiserslauten, Lecture Notes in Comput. Sci., 33 Berlin: Springer-Verlag,
1975, pp. 134183.
23. G. E. Collins H. Hong Partial cylindrical algebraic decomposition for quantifier elimination, J. Symbolic Comput. 12:
299328, 1991.
24. S. McCallum An improved projection for cylindrical algebraic decomposition, in B. F. Caviness and J. R. Johnson
(eds.), Quantifier Elimination and Cylindrical Algebraic Decomposition, Monographs in Symbolic Computation, Berlin:
Springer-Verlag, 1998, pp. 242268.
25. A. Strzebonski Solving algebraic inequalities with Mathematica version 4, Mathematica J.7(4): 525541, 2000.
26. S. Basu R. Pollack M.-F. Roy On the combinatorial and algebraic complexity of quantifier elimination, Assoc. Comput.
Mach., 43(6): 10021045, 1996.
27. R. Loos V. Weispfenning Applying linear quantifier elimination, Comput. J. 5(36): 450461, 1993.
28. A. Strzebonski An algorithm for systems of strong polynomial inequalities, Mathematica J. 4(4): 7477, 1994.
29. S. Wolfram The Mathematica Book, 4th ed., Champaign, IL: Wolfram Media, Cambridge University Press, 1998.
30. M. Jirstrand Constructive methods for inequality constraints in control, PhD thesis, Department of Electrical Engineering, Linkoping University, Linkoping, Sweden, May 1998.
31. D. S. Arnon A bibliography of quantifier elimination for real closed fields, J. Symbolic Comput. 5(12): 267274, 1988.
32. G. E. Collins Quantifier elimination by cylindrical algebraic decompositiontwenty years of progress, in B. F. Caviness
and J. R. Johnson (eds.), Quantifier Elimination and Cylindrical Algebraic Decomposition, Berlin: Springer-Verlag,
1998.
33. B. Anderson N. Bose E. Jury Output feedback stabilization and related problemssolution via decision methods, IEEE
34. F. R. Gantmacher Matrix Theory, Vol. II, Chelsea: New York, 1960.
35. D. Nesic I. M. Y. Mareels Stabilizability and stability for implicit and explicit polynomial systems: a symbolic computation approach, Eur. J. Control 5: 3243, 1999.
36. D. Nesic I. M. Y. Mareels Dead-beat controllability of polynomial systems: symbolic computation approaches. IEEE
Trans. Autom. Control 43: 162175, 1998.
37. D. Nesic M. Jirstrand Stabilization of switched polynomial systems, in IMACS Conf. on Applications of Computer
Algebra (ACA98), Prague, 1998.
38. P. Dorato W. Yang C. T. Abdallah Quantifier elimination approach to frequency domain design, in S. Tarbouriech and G.
Garcia (eds.), Control of Uncertain Systems with Bounded Inputs, Lecture Notes in Control and Information Sciences,
227, Berlin: Springer-Verlag, 1997, pp. 165172.
39. P. Dorato W. Yang C. Abdallah Robust multi-objective feedback design by quantifier elimination, J. Symbolic Comput.
24(2): 153159, 1997.
40. M. Jirstrand Nonlinear control system design by quantifier elimination, J. Symbolic Comput. 24(2): 137152, 1997.
41. P. Dorato D. Famularo C. T. Abdallah W. Yang Robust nonlinear feedback design via quantifier elimination theory, Int.
J. Robust Nonlinear Control, 9: 817822, 1999.
D. NES IC
I. M. Y. MAREELS
The University of Melbourne
S. T. GLAD
Linkoping University
M. JIRSTRAND
MathCore AB
464
SOFTWARE FOR CONTROL SYSTEM ANALYSIS AND DESIGN, SINGULAR VALUE DECOMPOSITION
extends to two spaces A and B of dimension m spanned by

the orthonormal columns of two given matrices A and B:
A A = Im , B B = Im
The definition uses now the singular value decomposition of
the inner product
A B = UV
SOFTWARE FOR CONTROL SYSTEM

ANALYSIS AND DESIGN, SINGULAR
VALUE DECOMPOSITION
AU and B BV, then the columns

Indeed, if one takes A
of A and B span the same spaces and still are orthonormal:
The singular value decomposition (SVD) goes back to the beginning of this century. In a paper of Beltrami (1) it was
shown for the first time that any n n matrix A can be diagonalized via orthogonal row and column transformations. For
the more general case of an n n complex matrix A, the
result says that there exist unitary matrices U and V of dimension n n and a real diagonal matrix diag1, . . .,
n such that
A = UV
where 1 n 0. If A is real then U and V are also
real. This implies indeed that U and V diagonalize A since
A A = Im , B B = Im
Moreover,
A B = = diag{1 , . . ., m }
and from this diagonal form one can define the canonical
angles between the spaces A and B as cos i i, i 1,
. . ., m.
The second important property is that the singular value
decomposition yields a direct construction of best lowerrank approximation to a given matrix A. Let us indeed rewrite the SVD in its dyadic form:
U AV =
A=
The decomposition is in nature close to an eigenvalue decomposition, which was well known at the time. But this new
decomposition is also very different since singular values are
always positive real, whereas eigenvalues are in general complex. Also, the transformations are unitary in this decomposition, whereas in the eigenvalue decomposition they are just
nonsingular, and hence can be quite badly conditioned (see
the next section for a more detailed discussion). The use of
this new decomposition was not apparent from the very beginning, but nowadays it has become an invaluable tool in several application areas such as statistics, signal processing,
and control theory.
The first important property that was observed is the perturbation result for the singular values of a matrix A. If A
is a small perturbation of the matrix A, then its singular values i are perturbed by an amount that can be bounded by
the norm of the perturbation A. The fact that the sensitivity of singular values to perturbations is rather low makes
them a good candidate for measuring certain variations in an
observed phenomenon or in a model for it, and this is also its
principal use in engineering applications. More formally, one
can show that several matrix norms can actually be expressed
in terms of its singular values. The most important ones are
the 2-norm and the Frobenius norm:
Ax2
.
A2 = max
= 1 (A),
x = 0 x2
.
AF =
|ai, j |2 =
i, j
r
i2
But the singular values are also used to measure angles.

The well-known formula for the angle between two real vectors a and b of norm 1
.
aT a = 1, bT b = 1, cos = |aT b|
n
i ui vi
i=1
Then the matrices

Ar =
r
i ui vi
i=1
are rank r matrices and the error

n

.
Ar = A Ar =
i ui vi
i=r+1
has obviously norm Ar2 r1, which is the minimal norm

among all possible rank r approximations (see Ref. 2 and the
section entitled Robustness in Systems and Control). This
leads to the important concept of the A rank of a matrix, defined in terms of the machine accuracy of the computer
used, and the norm A2 of the given matrix. For A A2
one defines the A rank of A as the smallest rank r of Ar
within A distance of A. It turns out that this is the most reliable way to recognize rank deficiency of a given matrix and
hence it is an important tool in engineering. In several applications the order of the system identified from observed measurements and the minimality of a constructed model indeed
amount to rank determination problems (see the section entitled Applications in Systems and Control).
A final important feature of the SVD is that it puts the
matrix A in a diagonal form under orthogonal (or unitary)
transformations of its columns and rows, and these transformations have good numerical properties. Interpreting y Ax
as a mapping from a space X to a space Y , we have thus
transformed the coordinate systems in both spaces (by a wellbehaved transformation) such that the mapping becomes diagonal. It is obvious that this coordinate system will reveal
special properties of the original mapping since now it is decoupled in a set of scalar equations. We will see that in the
context of dynamical systems this coordinate system plays a
fundamental role in what are called balanced realizations
(see the section entitled Balanced Realization).
THE SINGULAR VALUE DECOMPOSITION

Before we give more details about its basic properties, we first
recall the singular value decomposition in its most general
form.
Theorem 1. Let A be a m n complex matrix of rank r. Then
there exist unitary matrices U and V of dimensions m m
and n n, respectively, such that
A = UV
(1)
where

0
Rmn
0

r
=
0
(2)
and r diag1, . . ., r with 1 r 0. If A is

real then U and V are also real.
NUMERICAL BACKGROUND
The importance of the SVD is strongly tied to numerical aspects. For this reason, we first give a very brief discussion of
numerical stability and conditioning, which play a very important role in the study of numerical algorithms. For more
details we refer to standard textbooks such as Refs. 3 and 4.
We also choose the example of the singular value decomposition to introduce the relevant concepts.
Let A be an arbitrary m n matrix. Then it is well known
that there always exist unitary matrices U and V such that
U A =

R
, A V = C0
0
A=
r
i ui vi
(3)
i=1
which is nothing but an alternative way of writing A

UV*. Written in block form, this becomes
A = U1 rV1
(4)
where U1 and V1 are the submatrices of U and V, respectively,

containing their first r columns. This decomposition can be
viewed as a more compact form of writing Eqs. (1) and (2). It
is also called a rank factorization of A since the factors have
the rank r of A as at least one of their dimensions.
The proof of the preceding theorem is based on the eigendecomposition of the Hermitian matrices AA* and A*A. From
the SVD one can indeed see that
AA = U TU ,
A A = V T V
where T and T are clearly diagonal. Hence the left singular vectors are the eigenvectors of AA*, the right singular vectors are the eigenvectors of A*A, and the nonzero singular
values are the square roots of the nonzero eigenvalues of both
AA* and A*A. Deriving Theorem 1 from these connections is
in fact quite simple, but we refer to Ref. 2 for the details.
(5)
where R and C have, respectively, r linearly independent

rows and columns. This implies, of course, that r is the rank
of the matrix A. We call such transformations a row and column compression of the matrix A, respectively, and R and C
are said to be of full row rank and full column rank, respectively. These decompositions can, for example, be computed
with the singular value decomposition Eqs. (1) and (2). It is
easy to verify that U*A and AV yield, respectively, a row and
a column compression of the matrix A. In this new coordinate
system, the kernel and image of the map U*AV are also simple to express. Indeed,

Ir
Im U AV = Im
,
0
The numbers 1, . . ., r together with r1 0, . . .,

min(m,n) 0 are called the singular values of A. The columns
ui, i 1, . . ., m of U are called the left singular vectors of
A and the columns vi, i 1, . . ., n of V are called the right
singular vectors of A. They also appear in the dyadic decomposition of A:
465
Ker U AV = Im
Inr
As a consequence we also have (2)

Ir
Im A = Im U
= Im U1 ,
0

Ker A = Im V
Inr

= Im V2
where U1 is the submatrix of the first r columns of U and V2

is the submatrix of the last n r columns of V. The computation of the preceding decomposition is, of course, subject to
rounding errors. Denoting computed quantities by an overbar,
we generally have, for some error matrix A,
.
A = A + A = U V
(6)
Hence, the computed decomposition does not correspond exactly to the given matrix A but rather to a perturbed version
A A. When using the SVD algorithm available in the literature (5,6), this perturbation A can be bounded by
A cA A = cA A
(7)
where is the machine accuracy and cA is some known polynomial expression in m and n (3). Very often, this is a rough
upper bound and one prefers to replace cA by some statistical
estimate cA, usually close to 1. The error A induced by this
algorithmcalled the backward error because it is interpreted as an error on the datathus has roughly the same norm
as the input error in generated when reading in the data A
in the computer. When such a bound exists for the perturbation A induced by a numerical algorithm, it is called backward stable. We can make this definition more rigorous by
considering a function X f(A) with data A and solution X.
466
If the computed solution X f(A) satisfies X f(A A)

and A A, then the algorithmic implementation f( ) of
the function f( ) is said to be backward stable.
Notice that backward stability does not warrant any
bounds on the errors in the results U, , and V. This depends
indeed on how perturbations on the data (namely A) affect
the resulting decomposition (or the differences U U U,
, and V V V). In other words, it depends on
the sensitivity of the function f( ) with input A and solution
X. This sensitivity is commonly measured by the condition of
f at A:
.
[ f (A)] = lim sup
0
AA=
X X
,
X = f (A),
X = f (A) (8)
Notice that we have not specified what norms are used in this
definition, but in principle one can use different norms in the
data and solution spaces (7). From this definition it is clear
that the condition number [f(A)] is some sort of derivative
of the function X f(A) that we want to compute. When
[f(A)] is infinite, the problem of determining X f(A) from
A is ill posed (as opposed to well posed). When [f(A)] is finite
and relatively large (or relatively small, the problem is said
to be ill conditioned (or well conditioned). Further details can
be found in Ref. 7.
It is important to note that backward stability is a property of an algorithm, while conditioning is associated with a
problem and the specific data for that problem. The errors in
the result depend on both the stability of the algorithm used
and the conditioning of the problem solved. A good algorithm
should therefore be backward stable since the size of the errors in the result is then mainly due to the condition of the
problem and not due to the algorithm. An unstable algorithm,
on the other hand, may yield a large error even when the
problem is well conditioned.
We point out that if f(A) has a Taylor expansion around
A, then we can write
f (A) = f (A) + x f (A)(X X ) + O(X X 2 )
(9)
Setting X f(A) and X f(A) and taking norms, we then

have
X X x f (A)A A + O(X X 2 ) [ f (A)]A A
This is a very powerful inequality, which indicates that forward errors X X are bounded in norm by the sensitivity
[f(A)] and the backward error A A. Forward errors depend thus on two factors: the sensitivity of the problem and
the backward error induced by the algorithm, and these two
factors multiply each other in the preceding bound.
Bounds of the type in Eq. (7) are obtained by an error analysis of the algorithm used; see, for example, Ref. 8. The condition of the problem is obtained by a sensitivity analysis; see,
for example, Refs. 4 and 8.
NUMERICAL ROBUSTNESS OF SINGULAR VALUES
One of the most important features of the singular value decomposition is that the singular values can be computed in
a numerically reliable manner. There are indeed numerical
methods available to compute the SVD in a backward stable

manner, that is, such that the backward error A satisfies
Eqs. (6) and (7). Moreover, it is known that the singular values of any matrix A are in fact well conditioned:
[ f (A)] = 1
(10)
This is derived from the variational properties of singular values (2,3) and leads to the following theorem.
Theorem 2. Let A be an arbitrary m n complex matrix
and A an arbitrary perturbation of the same dimensions.
Then the corresponding SVD of A and A A satisfy the
following strict bounds:
2 A2 ,
F AF
This result is also proven using variational inequalities

and we refer again to Ref. 2 for a proof. Suppose now that the
computed singular values i are such that
1 r > A r+1 min(m,n)
Then it is reasonable to assume that
r+1 = = min(m,n) = 0
The level of accuracy of the computed singular values is indeed A (which is a very small quantity) and hence only 1 to
r are guaranteed to be nonzero. While in general there may
be little justification for setting the small A singular values
equal to zero, it is indeed a sound choice in several applications. A typical example is the determination of Im A and
Ker A, which, for example, plays an important role in leastsquares solutions of the equation Ax b. Notice that orthonormal bases for Im A and Ker A are given by the columns
of U1 and V2 defined earlier. The condition of Im A and of Ker
A is thus connected to the sensitivity of the transformations
U and V of the SVD. Consider, for example, the computation
of Im A (where we assume m n). As the distance function
between two spaces X and Y we use the gap (X , Y )
PX PY 2, where PS is the orthogonal projector on the space
S . If A has full column rank n, then
[Im A] = n1
(11)
as illustrated by the following example (1 1, 2 a 1):
. 0
A=
0
0
0
a
;
0
0
1
0
.
A = A + A =
0
0
. 0
A =
0
0
0
c
s
0
0
s
c
0
0
0
0
1
0
0
0 0
0
1
0
a
0
0
with a a2 2, c a/a, s /a. The second basis vector

of Im A is rotated in Im A over an angle , where sin s,
and one easily checks that (Im A, Im A) s. Therefore

lim
0
(Im A, Im A)
1
= = 21
In the subsections that follow, we survey a number of problems from systems and control theory that rely heavily on
the singular value decomposition. We shall only discuss the
numerical aspects here; for the system theoretical background, we refer the reader to the systems and control literature.
Impulse Response Realization
Let H(z) be an p m transfer function of a discrete-time
causal system, and let its impulse response be given by
H(z) =
The realization problem is to find the transfer function H(z)

in state-space description,
H(z) = D + C(zIn A)1 B
Hl
..
.
..
.
..
..
.
H2
=
.
.
.
Hk
H2
H k,l
H1
..
yk = Cxk + Duk
(15)
when the impulse response sequence Hi is given. In the scalar case, this problem is related to the Pade approximation
problem, for which fast methods exist (see Refs. 16 and 17 for
a survey). In Ref. 16, it is shown that the Pade approach is in
fact unstable, and it is better to consider a more general technique based on matrix decompositions of the Hankel matrix:
(16)
Hk+l1
Here k and l are upper bounds for the minimal dimension

n of the state-space realization Eq. (15) of H(z). From the
expansion of H(z) in powers of z1 one finds that
H0 = D,
Hi = CAi1 B,
i = 1, . . ., k + l 1
and therefore Hk,l can be factorized as follows:
where
.
C l = [B AB
Al1 B],
(17)
This implies that Hk,l has at most rank n and a simple argument proves that Hk,l will have exactly rank n. Since determining the order of the system requires a rank determination, it is natural to use here the SVD Eq. (4):
H k,l = U1 nV1T
(13)
Typically the matrices A, B, C, and D are real. In case

the results for real matrices are different we will explicitly
state it.
. CA
Ok =
...
CAk1
(12)
Here, x(t) is an n-vector of states, u(t) is an m-vector of controls or inputs, and y(t) is an p-vector of outputs. The standard discrete-time analog of Eq. (12) takes the form
xk+1 = Axk + Buk
(14)
H k,l = C l O k
The problems considered in this article arise in the study of

dynamical systems that can be modeled as state-space models:
Hi zi
i=0
APPLICATIONS IN SYSTEMS AND CONTROL
x(t)
= Ax(t) + Bu(t)
..
The fact that the singular values have a low sensitivity to

perturbations does of course not mean that every algorithm
will compute them to high accuracy. The link with the eigenvalue problems AA* and A*A indicates that there can be no
finite algorithm for computing singular values. An early iterative procedure was actually based on these connected Hermitian eigenvalue problems, but they have been shown to be
unreliable because of the intermediate construction of
squared matrices (3). It was shown in Refs. 5 and 6 that the
unitary transformations U and V of the decomposition can be
constructed via an iterative procedures that works directly
on A to give the SVD. This algorithm first computes unitary
matrices U1 and V1 such that B U1*AV1 is in bidiagonal
form, that is, only the elements on its diagonal and first
superdiagonal are non-zero. Then it uses an iterative procedure to compute unitary matrices U2 and V2 such that
U2*BV2 is diagonal and non-negative. The SVD defined in
Eqs. (1) and (2) is then given by U*BV, where U U1U2
and V V1V2. The computed U and V are unitary to approximately the working precision, and the computed singular values can be shown to be the exact is for A A, where
A/A is a modest multiple of .
Other alternative methods to compute the singular values
of a matrix A were proposed later and are based on Jacobilike methods (9). They have been shown to have speed and
accuracy comparable to the Golub-Kahan algorithm (10,11).
As a consequence of the discussion in the preceding section,
the singular values are thus computed with small absolute
error. More recent results suggest that in particular cases the
singular values of matrices can sometimes be computed to
high relative accuracy as well (1214). We finally remark that
although the singular value decomposition is the most reliable method for determining the numerical rank of a given
matrix, it is considerably more expensive than, for example,
the QR factorization with column pivoting, which can usually
give equivalent information with less computation (15).
467
For the construction of the triple A, B, C, let

L = U1 n1/2 ,
R = n1/2V1T ,
H k,l = L R
(18)
468
and partition these left and right factors as follows:

L1
C1
=
,
L=
L2
C2
R = [B1
R2 ] = [R1
B2 ]
(19)
where C1 and C2 have p rows and B1 and B2 have m columns.

From the two ways [Eqs. (17) to (19)] of writing the factorization of Hk,l, one derives then that
C = C1 ,
B = B1
and that A can be solved from the overdetermined systems

L1 A = L2 ,
Realization from Input/Output Data

Very often one does not have access to the impulse response
of the system but only to a sequence of inputs ui and corresponding outputs yi. In such cases a novel algorithm was
derived in Ref. 21 based on the following Hankel matrix:
..
..
..
.
zl
..
.
..
.
z 2
=
.
.
.
zk
z2
..
H 1:k,l
z1
Tx3
y2

Txl
yl1

TAT 1 TB Tx1
=
D
u1
CT 1
Tx2
u2
Txl1
ul1

(25)
So each sequence of states

X1,l = [x1
x2
xl ]
(26)
can only be expected to be known up to an invertible row

transformation corresponding to the particular coordinate
system of the reconstructed model A, B, C, D. This row
transformation T leaves the row space of X1,l unchanged. Also
the rank condition for Eq. (22) to be solvable implies that Eq.
(23) must be full rank n since this is a submatrix of the righthand side matrix in Eq. (22). The row space of X1,l is therefore
n-dimensional. This row space can now be found with the aid
of the singular value decomposition based on the following
theorem, proved in Ref. 21.
Theorem 3. Define
.
Xk+1,l = [xk+1
zk+l1
xk+2
xk+l ]
(27)
and the Hankel matrices H1:k,l and Hk1:2k,l as before, then
where
X Tk+1,l ] = Im [H
H T1:k,l ] Im [H
H Tk+1:2k,l ]
Im [X

ui
zi =
yi
(20)
We start by noting that the problem would be much simpler if the sequence of states xk would be known as well. From
Eq. (13), rewritten as

A
xk+1
=
yk
C

xk
uk

(21)
x3
y2
xl
yl1

A
=
C

B
D
x1
u1
x2
u2
xl1
ul1

A
C

B
D
In practice, due to perturbations on the data, the row

spaces typically do not intersect. An approximate intersection, using the singular value decomposition or some rank revealing QR decomposition, has thus to be constructed. A
possible implementation of this idea is the following decomposition:
H 1:k,l
H k+1:2k,l

=
I
0
A
11
A21
Q
A12
03
01
02 V T
A33
(29)
(22)
where
Under the assumption of persistence of excitation one shows

that the right data matrix in Eq. (22) has full column rank
n m and has thus a right inverse. Equivalently, Eq. (22)
can be solved in a least-squares sense for the evolution matrix
E=
(28)
provided the input/output (I/O) data are persistently exciting

and k n, l (m p)k.
one derives immediately the concatenated form:

x2
y1
or

Tx2
y1
AR1 = R2
The particular choice of factors L and R makes the realization

unique and we shall see that it is also linked to so-called balanced realizations later on (18). This realization algorithm
based on the singular value decomposition of Hk,l was first
given in Refs. 19 and 20.
So the problem is solved as soon as the states xi are determined. But those depend on the choice of coordinates chosen
for the state-space model. Replace indeed xi by xi Txi; then
Eq. (22) becomes the related equation

x2 x3
A B
x1 x2 xl1
xl
=
(24)
y1 y2 yl1
C D u1 u2 ul1
(23)
[A11A12] has full column rank equal to the rank of H1:k,l

[which equals r mk n under the assumption
of persistence of excitation (21)]
A33 has full row rank, which must be smaller than r if
an intersection is to be detected
A21 has full column rank equal to the dimension of the
intersection, hence n
denotes an arbitrary matrix
The order in which this decomposition is constructed is as

follows. First the transformation V T is constructed to compress the columns of H1:k,l, yielding the trailing zero matrix
01. Then the rows of the trailing bottom matrix are compressed with the transformation Q, yielding 02 and a full row
rank A33. Then V T is updated to yield the full column rank
matrix A21 and the trailing zero matrix 03. Notice that all
three steps involve a rank factorization that can be done with
the singular value decomposition (or any other rank revealing
factorization). The center matrix in this decomposition has a
form that trivially displays the intersection of row spaces of
the top and bottom parts, namely,
T
AT11
A21
Im AT12 Im 0
0
0
In

= Im 0
AT33
0

In

H T1:k,l ) Im(H
H Tk+1;2k,l ) = V Im 0
Im(H
0
Balanced Realization
In the preceding section we pointed out that the realization
problem from an impulse response or from input/output data
is only defined up to a state-space transformation T, which in
principle can be chosen arbitrarily. Is there a particular coordinate system that should be chosen for some reason, and if
so, how can we construct it?
We develop here the concept of balanced realization that is
based on the singular value decomposition and has several
appealing properties. For this we first need to define the controllability Gramian Gc and observability Gramian Go of a system. For the continuous-time system equation (12) these are
defined as follows:
(e At B)(e At B)T dt,

0
.
Go (T ) =
(Ce At )T (Ce At ) dt
0
(30)
while for the discrete-time systems they are defined as follows:

K 1
. i
Gc (K) =
(A B)(Ai B)T ,
k=0
K 1
.
Go (K) =
(CAi )T (CAi )
(31)
k=0
An intuitive interpretation (18) is that the controllability

Gramian Gc measures the amount of energy needed to control
the states of the system using an input sequence of a certain
duration, while the observability Gramian Go measures the
amount of energy one can observe from the state in an output
sequence of a particular duration. Typically K and T are chosen to be infinite if the system is stable, because then these
Gramians can be computed efficiently using Lyapunov equations (22). For a given realization A, B, C, D one easily
(32)
which is exactly the balanced coordinate system. In order to

construct T one starts from the Cholesky factorization of Gc
and Go:
Gc = L1 LT1 ,
that is, the first n rows of V T are a representation of Xk1,l.

From this we can now construct A, B, C, D as explained in
Eqs. (22) to (25).
.
Gc (T ) =
shows that these matrices are positive semidefinite and that

they allow one to detect whether or not the realization is minimal by computing the rank of these matrices (18). This suggests the use of the SVD of both Gc and Go in order to find
minimal realizations of the system, provided a given system
A, B, C, D is not necessarily minimal (see also the next
section).
But here we want to focus on another application of the
SVD in this context. If one applies a state-space transforma, B
,
tion x x Tx then the system triple transforms to A
, D TAT1, TB, CT1, D and the Gramians to G
c
C
o TTGoT1. One shows then that T can be choTGcTT and G
sen such that both new Gramians are equal and diagonal:
TGc T T = G c = = G o = T T Go T 1
Because of the transformation V T in Eq. (29) one derives that
469
Go = LT2 L2
(33)
where L1 and L2 are both lower triangular. One then computes the singular value decomposition of the upper triangular matrix LT1LT2:
LT2 LT1 = UV T
(34)
using, for example, the efficient algorithm described in Ref.

1/2
23. Then, defining T 1/2UTL1
and T1 L1
1
2 V , one
checks that
T
1
1
1
T 1 T = L1
1 (V U )L2 = L1 (L1 L2 )L2 = I
(35)
and
T
T
1/2
=
G c = TGc T T = 1/2U T L1
1 (L1 L1 )L1 U
T
1
1/2
=
G o = T T Go T 1 = 1/2V T LT
2 (L2 L2 )L2 V
In this new coordinate system one can associate the diagonal

element i of with the unit vectors ei (zero everywhere except a 1 in position i): i gives then the energy needed to control the state ei as well as the energy observed from this state.
Since both these are equal the state-space system is said to
be balanced (see Ref. 18 for more details).
It turns out that in addition to an energy interpretation,
this coordinate system has useful properties. If a singleinput/single-output system is stable and if one uses the infinite-horizon Gramians, then the evolution matrix of the balanced realization
.
E =

(36)
is symmetric up to some sign changes, that is, there exists a

S SE
T. As a consediagonal matrix S of 1 such that E
quence of this, one shows that such systems have low sensitivity to roundoff propagation (24). The sign symmetry is not
preserved for multi-input/multi-output systems or when considering finite-horizon Gramians, but nevertheless the robustness properties are (25).
470
While the singular values can be interpreted as a measure

for the energy that is being transferred from the input space
to the state space as well as from the state space to the output
space, they also play an important role in constructing approximate models that preserve this energy as well as possible. Since the singular values are ordered decreasingly one
can partition the balanced evolution matrix as follows:
A
C
B
D
A 11
.
=
A21
C
1
A 12
A
22
C 2
B 1
B 2
(37)
11, B
1, D will be a good approxima 1, C
and the subsystem A
tion of the original system in the sense that it keeps the
states that were responsible for the largest part of the energy
transfer (i.e., the largest singular values). A more formal approximation measure is the so-called H norm, which can be
bounded as a function of the balanced singular values if the
balancing was based on infinite-horizon Gramians. This socalled balanced truncation problem is also related to that of
optimal approximation in the Hankel norm (26,27) and has
led to a renewed interest in the partial realization problem
and related topics (28).
Finally, we point out here that the realization algorithm
described in the section entitled Impulse Response Realization for discrete-time systems in fact constructs immediately
a balanced realization. Let the matrices Cl and Ok be defined
as in Eq. (17). Then
C l C Tl =
l1
.
(Ai B)(Ai B)T = Gc (l),
i=0
O Tk O k =
k1
(38)
.
(CAi )T (CAi ) = Go (k)
i=0
and from the choice of factorization, Eq. (18), it follows that

Cl =
1/2
n
V1T ,
O k = U1
1/2
n
(39)
Since U1 and V1 have orthonormal columns, one obtains
C lC Tl =
O Tk O k =
1/2
T
n V1 V1
1/2 T
n U1 U1
1/2
n =
1/2
=
n
n,
(40)
Controllability and Observability

The concepts of controllability and observability play a fundamental role in systems and control theory. A system in the
form of Eq. (12) is indeed a minimal representation of the
input/output behavior of the system if and only if it is both
controllable and observable. The same holds for a discretetime system, Eq. (13), except that there one talks about reachability rather then controllability. The conditions for controllability (reachability) and observability are equivalent to the
following rank conditions, respectively (29),
rank C n = n,
rank O n = n
trices since they contain powers of the matrix A, which could

lead to a considerable buildup of rounding errors (see Ref. 30).
It is well known (see e.g., Ref. 29) that the infinite-horizon
Gramians, Gc() and Go(), have the same rank as Cn and
On, respectively. Since the latter can be computed as the solution of Lyapunov equations, this seems a possible alternative,
but it turns out to be a sensitive roundabout as well (31). A
third way is to find an appropriate coordinate system for the
pair A, B or A, C such that the rank of the matrices in Eq.
(41) becomes apparent. Since observability and controllability
are dual to each other we discuss this for controllability only.
The following theorem proposes such a coordinate change,
which is orthogonal and based on a succession of singular
value decompositions (31).
Theorem 4. There always exists an orthogonal state-space
] has the form
A
transformation U such that [UTBUTAU] [B

= Bc A c
A]
[B
0
0 Ac
A1,k+1
X1 A1,1 A1,2 A1,k
..
..
0
X2
A2,2
.
.
(42)
. ..
.
.
.
.
.
..
..
..
..
..
= .
..
.
.
.
Xk Ak,k
Ak,k+1
Ak+1,k+1
where Ai,i, i 1, . . ., k, are ri ri matrices, and Xi, i 1,

. . ., k, are ri ri1 matrices of full row rank ri (with r0
m).
The matrices Xi are constructed recursively as the result
or row compressions, using, for example, the SVD (31). In this
new coordinate system, one easily sees that the controllability
n UTCn has the form
matrix C
C n = [B A B . . . A n1 B]
X1:1
..
0
X1:2
.
.
.
.
..
..
= ..
0
..
..
.
.
X1:k
0
0
..
.
..
.
..
.
..
.
(43)
where X1:i is the product X1 . . . Xi. Since these products have

n
full row rank ri by construction, the factorization Cn UC
has a second factor that is row compressed, and the result
thus follows. The controllable subspace is defined as the space
spanned by the columns of Cn. It follows from Eq. (43) that
k
this space has dimension nc i1 ri and that

In c
ImC n = Im
0
(41)
In principle one could use the SVD to check these rank

conditions, but it is not recommended to construct these ma-
n this space is spanned in the original coordiSince Cn UC

nate system A, B by the first nc columns of U. The matrix
pair Ac, Bc is shown to be controllable, and the eigenvalues
of Ac are called the uncontrollable modes of the pair A, B.

The indices ri are the so-called controllability indices of the
pair A, B (31).
A dual form of the staircase algorithm applied to A, C
yields a dual result:
V T AV
CV
Ao
= 0
0
Ao
(44)
Co
where the subsystem Ao, Co is observable. If no is the number

of rows or columns of Ao then the first no columns of V span
Ker On, which is called the unobservable subspace of the pair
A, C.
Combining the preceding decompositions, one constructs
an orthogonal state-space transformation yielding a trans, B
, D which has the form
, C
formed system A
A
C
B
D
A11
0
0
0
A12
A22
0
C2
A13
A23
A33
C3
B1
B2
0
D
(45)
and where the subsystem A22, B2, C2, D is minimal, that is,
both observable and controllable (reachable). Moreover, the
, B
, D and A22, B2, C2, D are equal.
, C
transfer functions of A
This form is closely related to the Kalman decomposition and
for its construction we refer to Ref. 31.
In addition to the controllable and unobservable subspaces
of a system, there are other spaces that play a fundamental
role in the control of systems modeled as Eqs. (12) and (13).
Two other fundamental objects in the so-called geometric system theory (32) are the supremal (A, B)-invariant and controllability subspaces contained in a given subspace. As shown in
Refs. 31 and 33 they can also be computed via a matrix recurrence based on a sequence of SVDs constructing an orthogonal basis for the relevant spaces. The role of the SVD in these
staircase algorithms is not only the reliable rank determination of the subsequent steps, but at the same time the singular values allow one to assess the sensitivity of the computed
bases (31).
Robustness in Systems and Control
In the last decade, there has been a significant growth in the
theory and techniques of robust control. These developments
mainly center around two concepts: H (34) and the structured singular value (35). They both provide a framework for
synthesizing robust controllers for linear systems, in the
sense that they achieve a desired system performance in the
presence of a significant amount of uncertainty in the system.
In this section, we first focus on H techniques. The H
norm of a stable rational transfer matrix H(s) (continuoustime) or H(z) (discrete-time) is defined as
.
H(s) = sup max[H( j)],
.
H(z) = sup max [H(e j )]
[0,2 ]
(46)
where max[ ] denotes the largest singular value of a (complex) matrix. We explain how this quantity comes about by
starting from a basic robustness problem. Consider the homo-
471
geneous systems
x = Ax,
xk+1 = Axk
(47)
which are assumed to be stable against Cs, that is, the eigenvalues of the matrix A are in a region Cs of the complex plane,
which is the open left-half plane for a continuous-time system
and the open unit disk for a discrete-time system, respectively.
The complex stability radius measures the robustness of
system stability for complex perturbations. This radius is defined as the norm of the smallest complex perturbation such
that the perturbed matrix A BC becomes unstable (where
B nm, C pn, and hence mp). For A BC to be
unstable, it must have at least one eigenvalue in the complement of Cs. It is important to note that although Cs is convex,
the set of all Cs-stable matrices St M: (M) Cs, where
(M) denotes the spectrum of M, is nonconvex, as well as its
complement Unst of Cs-unstable matrices. The stability radius r therefore measures the distance of a stable matrix A
to the nonconvex set Unst.
By continuity of the spectrum of a matrix versus perturbations on its entries, the stability radius is clearly equal to the
distance from a stable matrix A to an optimal matrix A
BC lying on Unst. Indeed, when a matrix A BC passes
the boundary Unst, at least one of its eigenvalue must also
cross Cs. The boundary Unst in the matrix space describes
matrices with at least one eigenvalue in Cs. Therefore, the
stability radius can be written as
rC (A,B,C) = inf{2 : (A + BC) Cs = }
Consider a parametrization of the boundary Cs by a real
variable , such as Cs j, or Cs ej, [0,
2]. The stability radius can then be rewritten as
rC (A,B,C) = inf [inf{2 : det(I A BC) = 0}]

C s
= inf [inf{2 : det(I C(I A)1 B) = 0}]

C s
(48)
the second equality resulting from the stability of the initial
matrix A (I A is invertible for Cs) and from the fact
that det(I XY) 0 det(I YX) 0. The following classical result allows us to simplify this expression considerably
and is based on the singular value decomposition M
UV*.
Theorem 5. One has the inequality
1
inf {2 : det(I M) = 0} max
[M]
C nn
If there are no constraints on , the bound is attained for

vmax1
max.
maxu*
Combining this with Eq. (48), it follows that
1
rC (A,B,C) = inf max
[C(I A)1 B]
C s
sup max[C(I A)1 B]
C s
1
472
which is the H norm inverse of the rational transfer function

H() C(I A)1B. This is therefore reduced to a nonconvex
optimization problem on Cs, parametrized by a real parameter .
Efficient iterative methods are available for computing this
norm (36,37), and are based on the relationship between the
singular values of H( j) and the imaginary eigenvalues of a
Hamiltonian matrix obtained from a state-space realization
of H() (38). This result is then used to develop a quadratically convergent algorithm for computing the H norm of a
transfer function.
Structured Singular Values
In Theorem 5, it was stated that the lower bound can actually
be met when there are no constraints on the perturbation .
But will be in general complex, since the matrix M is in
general complex [even for transfer functions H() with real
coefficient matrices A, B and C]. The problem becomes more
involved when one imposes to be real. It was shown in Ref.
39 that

1
r (A,B,C) = sup [H()]
(49)
C s
where, for any M pm,

(M) = inf 2
(0,1]
Re M
1 Im M
Im M
Re M
!
(50)
This becomes now an optimization problem in two real parameters and , but in Ref. 39 it is shown that the function
Eq. (50) is unimodal in . In Ref. 40 an efficient algorithm is
then derived to compute this real stability radius based on
ideas of Ref. 36.
The computation of the real stability radius can be considered a special case of the more general structured stability
radius. Structured singular values (35) have been introduced
to provide a perturbation bound for structured uncertainties
in control system analysis and design. Therefore, the structured singular value approach can be viewed as a complement
for the H approach. In a linear system with multiple independent norm-bounded perturbations, it is always possible by
rearranging the system to isolate the perturbations as a single large block-diagonal perturbation . Then, denoting the
transfer function from the collective outputs of the perturbation to their inputs by M(s), the stability problem reduces to
ensuring that det(I M) 0 at all frequencies and for all
allowable . Notice that is again not arbitrary anymore and
therefore Theorem 5 does not apply. The largest singular
value of that theorem has thus to be replaced by the so-called
largest structured singular value, which is defined as follows.
Let the set of allowable perturbations be denoted by D
nn and be defined as
D = { = block diag(1r Ik , . . ., rp Ik p , 1c Ik
1
p+1
C1 , . . ., Cr ) : ir R, ic C , Ci C c i c i }
, . . ., qc Ik p+q ,
(51)
The largest structured singular value of an n n complex

matrix M is then defined as
0 if det(I M) = 0 for all D

1
D (M) =
(52)
min
max ()
otherwise
{D : det(IM )=0}
Computing D (M) is a difficult numerical problem. It is a nonconvex optimization problem and its complexity can be nonpolynomial for certain norms (see Ref. 41). One approach,
which is computationally rather demanding, is to formulate
the problem as a nondifferentiable convex optimization problem involving the maximum singular value of a matrix obtained from M. A more efficient scheme is given in Ref. 42
and uses several smooth optimization problems that do not
involve any eigenvalue or singular value computations. The
computational complexity of the problem of computing
D (M) has prompted several researchers to look for bounds
that are easier to compute (43,44).
CONCLUDING REMARKS
In this paper we have given several uses of the singular value
decomposition in analysis and design problems of systems
and control. We have considered computational issues and
useful properties of this decomposition as well, such as diagonalization, norms, and sensitivity. The list given here is far
from complete. Closeness problems (45,46) and canonical
forms (31,47) are just a few examples. We expect the number
of applications to grow also in the future because of the serious interdisciplinary effort that is under way between the
communities of the numerical linear algebra field on the one
hand and of the systems and control field on the other hand.
BIBLIOGRAPHY
1. E. Beltrami, Sulle funzioni bilineari, Giornali di Mathematiche,
11: 98106, 1873.
2. P. Lancaster and M. Tismenetsky, The Theory of Matrices, 2nd
ed., New York: Academic Press, 1985.
3. G. H. Golub and C. F. Van Loan, Matrix Computations, 2nd ed.,
Baltimore, MD: Johns Hopkins Univ. Press, 1989.
4. N. Higham, Accuracy and Stability of Numerical Algorithms, Philadelphia, PA: SIAM, 1995.
5. G. H. Golub and W. Kahan, Calculating the singular values and
pseudo-inverse of a matrix, SIAM J. Numer. Anal., 2: 205224,
1965.
6. G. H. Golub and C. Reinsch, Singular value decomposition and
least squares solutions, Numer. Math., 14: 403420, 1970.
7. J. R. Rice, A theory of condition, SIAM J. Numer. Anal., 3: 287
310, 1966.
8. J. H. Wilkinson, The Algebraic Eigenvalue Problem, Oxford, England: Oxford University Press, 1965.
9. E. G. Kogbetliantz, Solution of linear equations by diagonalization of coefficient matrix, Quart. Appl. Math., 13: 123132, 1955.
10. C. Paige and P. Van Dooren, On the quadratic convergence of
Kogbetliatzs algorithm for computing the singular value decomposition, Lin. Alg. Appl., 77: 301313, 1986.
11. J.-P. Charlier and P. Van Dooren, On Kogbetliantzs SVD algorithm in the presence of clusters, Lin. Alg. Appl., 95: 135160,
1987.
12. J. Demmel and W. Kahan, Accurate singular values of bidiagonal
matrices, SIAM J. Sci. Stat. Comput., 11: 873912, 1990.
13. V. Fernando and B. Parlett, Accurate singular values and differential qd algorithms, Numer. Math., 67: 191229, 1994.
14. G. Golub, K. Slna, and P. Van Dooren, Computing the Singular
Values of Products and Quotients of Matrices, in M. Mooren and
B. DeMoor (eds.), SVD in Signal Processing III, Algorithms, Architectures and Applications, Amsterdam: Elsevier, 1995.
SOFTWARE HOUSES
15. T. F. Chan, Rank revealing QR factorizations, Lin. Alg. Appl., 88/
89: 6782, 1987.
16. L. S. de Jong, Towards a formal definition of numerical stability,
Numer. Math., 28: 211220, 1977.
17. A. Bultheel, Recursive algorithms for the matrix Pade problem,
Math. Comput., 35: 875892, 1980.
18. B. C. Moore, Principal component analysis in linear systems:
Controllability, observability, and model reduction, IEEE Trans.
Autom. Control., AC-26: 1731, 1981.
19. H. Zeiger and A. McEwen, Approximate linear realizations of
given dimension via Hos algorithm, IEEE Trans. Autom. Control,
AC-19: 153, 1974.
20. S. Kung, A New Identification and Model Reduction Algorithm
via Singular Value Decompositions, Proc. 12th Asilomar Conf. Circuits Syst. Comp., 1978, pp. 705714.
21. M. Moonen et al., On- and off-line identification of linear statespace models, Int. J. Control, 49: 219232, 1989.
22. R. Patel, A. Laub, and P. Van Dooren, Numerical Linear Algebra
Techniques for Systems and Control, Piscataway, NJ: IEEE, 1993.
23. M. T. Heath et al., Computing the singular value decomposition
of a product of two matrices, SIAM J. Sci. Stat. Comput., 7: 1147
1159, 1986.
24. C. Mullis and R. Roberts, Synthesis of minimum roundoff noise
fixed point digital filters, IEEE Trans. Circuit Syst., CAS-23: 551
562, 1976.
25. M. Gevers and G. Li, Parametrizations in Control, Estimation and
Filtering Problems, London: Springer, 1993.
26. V. Adamjan, D. Arov, and M. Krein, Analytic properties of
Schmidt pairs for a Hankel operator and the generalized SchurTakagi problem, Mat. USSR Sbornik, 15: 3173, 1971.
27. K. Glover, All optimal Hankel norm approximations of linear
multivariable systems, and their L error bounds, Int. J. Control,
39: 11151193, 1984.
28. A. Antoulas, New results on the algebraic theory of linear systems: The solution of the cover problems, Lin. Alg. Appl., 50: 1
45, 1983.
1980.
30. C. Paige, Properties of numerical algorithms related to computing controllability, IEEE Trans. Autom. Control, AC-26: 130
138, 1981.
31. P. Van Dooren, The generalized eigenstructure problem in linear
system theory, IEEE Trans. Autom. Control, AC-26: 111129,
1981.
32. W. M. Wonham, Linear Multivariable Control: A Geometric Approach, 2nd ed., New York: Springer, 1979.
33. V. C. Klema and A. J. Laub, The singular value decomposition:
Its computation and some applications, IEEE Trans. Autom. Control, AC-25: 164176, 1980.
34. G. Zames, Feedback and optimal sensitivity: Model reference
transformations, multiplicative semi-norms, and approximate inverses, IEEE Trans. Autom. Control, AC-26: 301320, 1981.
35. J. C. Doyle, Analysis of feedback systems with structured uncertainties, IEE Proc., Pt. D, 129: 242250, 1982.
36. S. Boyd, V. Balakrishnan, and P. Kabamba, A bisection method
for computing the H norm of a transfer matrix and related problems, Math. Control Signals Syst., 2: 207219, 1989.
37. N. A. Bruinsma and M. Steinbuch, A fast algorithm to compute
the H norm of a transfer matrix, Syst. Control Lett., 14: 287
293, 1990.
38. R. Byers, A bisection method for measuring the distance of a stable matrix to the unstable matrices, SIAM J. Sci. Stat. Comput.,
9: 875881, 1988.
473
39. L. Qiu et al., A formula for computation of the real stability radius, Automatica, 31: 879890, 1995.
40. J. Sreedhar, P. Van Dooren, and A. L. Tits, A Level-Set Idea to
Compute the Real Hurwitz-Stability Radius, Proc. 34th IEEE
Conf. Dec. Control, New Orleans, 1995, WA05.
41. O. Toker and H. Ozbay, On the NP-Hardness of the Purely Complex Computation, Analysis/Synthesis, and Some Related Problems in Multi-Dimensional Systems, Proc. 1995 Amer. Control
Conf., 1995, pp. 447451.
42. M. K. H. Fan and A. L. Tits, Characterization and efficient computation of the structured singular value, IEEE Trans. Autom.
Control, AC-31: 734743, 1986.
43. P. M. Young, M. P. Newlin, and J. C. Doyle, Practical Computation of the Mixed Problem, Proc. 1992 Amer. Control Conf.,
Chicago, 1992, pp. 21902194.
44. C. T. Lawrence, A. L. Tits, and P. Van Dooren, A Fast Algorithm
for the Computation of an Upper Bound on the -norm, Proc.
13th IFAC World Congr., San Francisco, 1996, vol. H, pp. 5964.
45. R. Eising, Between controllable and uncontrollable, Syst. Control
Lett., 4: 263264, 1984.
46. D. Boley, Computing rank-deficiency of rectangular matrix pencils, Syst. Control Lett., 9: 207214, 1987.
47. M. M. Konstantinov, P. Hr. Petkov, and N. D. Christov, Invariants and canonical forms for linear multivariable systems under
the action of orthogonal transformation groups, Kybernetika, 17:
413424, 1981.
PAUL VAN DOOREN

Universite Catholique de Louvain,
Centre for Systems Engineering
and Applied Mechanics
STABILITY THEORY, ASYMPTOTIC
341

STABILITY CRITERIA
Automatic control is an essential part of engineering and science. It finds applications in many areas from space vehicles
and missiles to industrial processes and medicine. Automatic
control devices, laboratory equipment, design and analysis
tools, and complete automatic processes and systems are offered by many companies, some of which are listed in Table
1. Basically, a control system consists of interconnected components that achieve a desired response. In order to meet the
objectives effectively, the system must be understood fully
and properly modeled mathematically. When the system is
mathematically represented, it may be designed appropriately and the performance may be examined and analyzed.
For the performance analysis, many methods are available.
For example, the classic control theory is the earliest and one
of the most established methods, mainly applied in simple
systems.
Although a nonlinear approach is available, in classic control theory, the foundations of analysis are mainly based on
linear system theory. The linear system approach assumes a
causeeffect relationship between the components of the system and expresses this relationship as differential equations.
342
Table 1. List of Manufacturers

Automated Applications Inc.
680 Flinn Ave. Unit 36
Moorpark, CA 93021
Tel: 800-893-4374
Fax: 805-529-8630
Automation Technologies International
17451 W. Dartmoor Drive
Grayslake, IL 60030-3014
Tel: 708-367-3347
Fax: 708-367-1475
Capitol Technologies, Inc.
3613 Voorde Drive
South Bend, IN 46628
Tel: 219-233-3311
Fax: 219-233-7082
CEI Automation
15250 E. 33rd Place
Aurora, CO 80011
Tel: 303-375-0050
Fax: 303-375-1112
Control Engineering Company
8212 Harbor Springs Road
Harbor Springs, MI 49740
Tel: 800-865-3591
where
FTI International, Inc.
Hashikma St. Ind. Zone
P.O.B. 87 Kfar Saba,
Israel
Tel: 052-959152-4
Fax: 052-959162
Kuntz Automation
Engineering
402 Goetz Street, Dept. 7
Santa Ana, CA 92707
Tel: 714-540-7370
Fax: 714-540-6287
Eliminating the error e(t) and rearranging Eqs. (1) and (2)
gives the closed-loop gain
The Math Works, Inc.

24 Prime Park Way
Natic, MA 01760-1500
Tel: 508-647-7000
Fax: 508-647-7001
or in the Laplace transform domain,
Munck Automation
Technology
161-T Enterprise Drive
Newport News, VA 23603
Tel: 804-887-8080
Fax: 804-887-5588
Design Technology Corporation

5 Suburban Park Drive
Billerica (Boston), MA 01821
Tel: 508-663-7000
Fax: 508-663-6841
Portech Pathfinder
Operations
1610 Fry Avenue, Dept. T
Canon City, CO 81212
Tel: 800-959-0688
Fax: 719-269-1157
Fata Automation
37655 Interchange Drive
Farmington, MI 48335
Tel: 810-478-9090
Fax: 810-478-9557
Prime Automation, Inc.

1321 Capital Drive
Rockford, IL 61109-3067
Tel: 815-229-3800
Fax: 815-229-5491
Once the system performance is stated as differential equations in the time domain, Laplace transforms are commonly,
used for frequency analysis. Some examples of Laplace transforms are given in Table 2.
A control system can be an open loop or closed loop. Figure
1 illustrates an open-loop system in which a controller controls the process without using any feedback. In this case, the
output is not compared with the input; therefore, deviations
of the output from the desired value cannot be automatically
corrected. This method finds limited application since it does
not lead to fully automatic systems.
In a closed-loop system, the actual output is compared with
a desired reference input by a suitably arranged feedback
mechanism. Figure 2 illustrates a single-inputsingle-output
feedback control system. In this system, a prescribed relationship of one system variable to another is maintained by comparing the two and using the difference as a means of control.
Using system modeling and mathematical representations, a
closed loop control system with a single-input single-output
may be represented as shown in Fig. 3. In this case, the relationship between the input and output of the single-input
single-output system can be expressed as
e(t) = r(t) c(t)H
c(t) = e(t)G
r(t) reference input

e(t) error signal
c(t) output signal
G forward path gain or process transfer function
H feedback gain
(1)
(2)
M=
M(s) =
G
c(t)
=
r(t)
1 + GH
G(s)
C(s)
=
R(s)
1 + G(s)H(s)
(3)
The main effect of the feedback is that it reduces the error

between the reference input and the system output, thus forcing the system output to follow or track the reference input.
It also affects the forward gain G by a factor of 1/(1 GH).
This effect is one of the main subjects of study in classic control theory. For example, when GH 1 the gain M will be
infinite; hence, C will increase without bound, thus leading to
unstable conditions.
In many cases, the control systems are much more complex
than a single-inputsingle-output arrangement. They may
have many inputs and outputs and controlled variables that
are interrelated as shown in Fig. 4. These systems necessitate
a multivariable control system approach for analysis and design. However, the feedback concept of the single-input
single-output linear system will be the main concern of this
article.
A control system needs to be designed carefully with a suitable configuration with clearly identified specifications to
achieve the desired performance. In this process, the identification of key parameters, mathematical representation of the
system, and system analysis play important roles. After having mathematically represented the system, it is possible to
employ analytical tools to describe the characteristics of the
feedback control system. Important characteristics, such as
the transient and steady state performance, frequency response, sensitivity, and robustness can be studied in detail.
When these characteristics are known, the desired response
can be obtained by adjusting the system parameters. Therefore, a good understanding and effective use of stability theory in control systems is very important. If the system is not
stable, it will display an erratic and destructive response and
will get out of bounds and disintegrate.
The transient response of a system is related to its stability. Typical responses times of second-order systems are illustrated in Fig. 5. In this system, the outputs are bounded by
the decaying oscillations. Therefore, a stable system may be
defined as a system with bounded response. If the oscillations
increase with time, the system is said to be unstable. The
stability of a dynamic system can be described by its response
to an input disturbance. The output response can be either
decreasing, increasing, or neutral, giving an indication of sta-
343
Table 2. Laplace Transform Table

Laplace
Transform
Time Domain Function

u(t) (unit step)
1
s
1
s2
t n, for any positive integer n
n!
s n1
eat
1
sa
eat ebt
ba
1
(s a)(s b)
n
1 2
2n
s 2 n s 2n
ent sin(n1 2 t)
sin(n t)
n
s 2 2n
cos(n t)
s
s 2 2n
1
1 2
n t
sin(n1 2 t ) where tan1
1 2
2n
s(s 2 n s 2n)
2
1 cos(n t)
2n
s(s 2n)
1 et/T
1
s(1 Ts)
1
[sin(n t) n t cos(n t)]
2n
s2
(s 2n)2
1 2T (1 2T )et/T
1
s 2(1 Ts)2
bility as defined by stability criteria. There are three types

of stability of control systems: the bounded-inputboundedoutput (BIBO) stability, asymptotic stability, and marginal
stability (stability in the sense of Lyapunov).
output relationship as
L
K i=1 (s + zi )
C(s)
= U
M(s) =
R
2
2
2
2
R(s)
k=1 (s + pk )
j=1 (s + 2 j s + a j s + j + j )
(4)
BOUNDED-INPUTBOUNDED-OUTPUT STABILITY
A system is stable if a bounded input yields to a bounded
output. When the closed-loop transfer function of a linear system is expressed as a Laplace transform, the stability may be
defined in terms of the locations of the poles of the system in
the complex plane or the s-plane. An s-plane is shown in Fig.
6 indicating the right half plane and left half plane. Take a
single-inputsingle-output system and express the input and
where the denominator of M(s) is the characteristic equation

of the system. The roots of the characteristic equation are the
poles of the closed-loop system. The time response of the output is a function of the roots of this characteristic equation.
Desired
response
Reference
input or
Controller
Process
Output
Comparator
Controller
Process
Output
Measurement
desired response
Figure 1. An open-loop control system. An open-loop control system
does not compare the output and the input; therefore, any deviations
between the two cannot be corrected automatically. This system is
applicable only in simple cases in which the process characteristics
are fully known and the outputs from the desired values are not all
very important.
Figure 2. A closed-loop control system. In a single-inputsingle-output system, the output of the system is compared with the input,
and any deviation between the two is corrected by suitably designed
controllers. However, the use of feedback can lead to instability.
Closed-loop control arrangements are used extensively in automatic
processes and devices.
344
r(t) +
e(t)
c(t)
2.0
1.8
1.6
1.4
1.2
c(t)
Figure 3. Block diagram of a closed-loop system. A closed-loop system may be represented mathematically by a forward loop transfer
function G and a feedback loop transfer function H. The relation between the input and output can be expressed in terms of these two
terms in the form of a system transfer function, which is extremely
useful in system design and analysis.
1.0
Step
input
0.8
0.6
For example, the time response of this system for an impulse

function may be written as
c(t) =
U
(Ak ePk ) +
k=1
R
B e t sin( t)
j=1
(5)
Left-Half-Plane Poles
To clarify the important concept of locations of poles on the splane, as used in stability analysis, see Eq. (4). As we can see
in this equation, the poles of the closed-loop system may be
real or complex, and simple or repeated. It is often convenient
to plot the poles and zeros of the closed-loop transfer function
on the s-plane. The s-plane can be considered to be in three
parts, the right half plane (RHP), the left half plane (LHP),
and the pure imaginary axis or j-axis. If a pole lies inside
the open LHP, then the pole has a negative real part. If it lies
inside the closed RHP, then it has a positive or zero repeated
real part.
Consider poles 1/(s p)n on the s-plane. For a real p, a
portion of the time domain response of the system will be proportional to
c(t) 1/n! t n1 e pt
(6)
If p 0, it lies on the RHP, and its response increases exponentially. If it is at the origin, p 0 and simple, its response
is a step function. When p 0 and repeated with multiplicity
n 2, then its response approaches infinity as t . If the
poles are in the LHP or p 0, then the response ept approaches zero as t .
Therefore,
1. The time response of a pole, simple or repeated, approaches zero as t if and only if the pole lies inside
the open-loop LHP or has a negative real part.
Inputs
(desired
responses)
Controllers
Process
Outputs
Measurement
Figure 4. A multivariable control system. Many control systems
have multiple inputs and multiple outputs. In these cases, using the
multivariable control theory and matrix approach is applicable. In
the design and analysis, most of the theories developed for singleinputsingle-output systems can still be used.
0.4
0.2
0
6
7
nt
10 11 12 13
Figure 5. Time response of a second-order system. The output of a

second-order system for a unit step input contains a transient and
steady state response. The sinusoidals frequency and amplitude components depend on the natural frequency and the damping ratio of
the system. If the oscillation increases without bound, the system is
said to be unstable. The stability can be related to the locations of
the poles on the s-plane.
2. The time response of a pole approaches a nonzero constant as t if and only if the pole is simple and
located at s 0.
As indicated earlier, in order to obtain a bounded response to
a bounded input, the poles of the closed-loop system must be
in the left-hand portion of the s-plane. That is, pk 0 and
j 0 so that the exponential terms epkt and ejt decay to
zero as the time goes to infinity. A necessary and sufficient
condition is that a feedback system is stable if all the poles of
the system transfer function have negative real parts. If the
characteristic equation has simple roots on the imaginary
axis ( j) with all other roots on the left half plane, the steady
state output is sustained oscillations for a bounded input. If
the input is sinusoidal with a frequency equal to the magni-
Imaginary axis
j
LHP
stable
s-plane
RHP
unstable
Real axis
Figure 6. A complex plane. The response of control system depends

on the locations of poles and zeros of the characteristic equation on
the complex plane also known as the s-plane. The poles in the closedloop transfer function on the right half plane lead to instability because the exponential term in the time domain representation increases as time increases. The poles in the LHP indicate stable conditions, whereas poles on the imaginary axis may lead to stable or
unstable conditions. In this case, repetition of roots on the imaginary
axis and the inputs must be taken into account.
tude of the j-axis pole, the output becomes unbounded. This

is called marginal stability because only certain bounded inputs cause the output to become unbounded. For an unstable
system, the characteristic equation has at least one root in
the right half of the s-plane, that is at least one of the exponential terms epkt and/or ejt will increase indefinitely as the
time increases. Repeated j-axis roots will also result in an
unbounded output for any input.
ASYMPTOTIC STABILITY
In general, the output response of a linear time-invariant system may be divided into two components.
1. The forced response is the part of the response that has
the same form as the input.
2. The natural response is the part of the response that
follows a form, which is dictated by the poles of the
characteristic equation.
In some cases, the investigation of stability by using only the
transfer function M(s) is not sufficient. Hence, the nature of
the input signal must be taken into account. For example, a
plant output c(t) is said to track or follow the reference input
r(t) asymptotically if
lim |c(t) r(t)| 0
(7)
Suppose that the transfer function of the overall control system is M(s); if M(s) is not stable the system cannot follow any
reference signals. If M(s) is stable, in order for the system to
be asymptotically stable, it is an additional requirement that
the system be capable of following all inputs. This is important because, in some cases, the output may be excited by
nonzero initial conditions such as noise or disturbance. As a
result, the stability conditions may be generalized as follows.
1. The system is stable if the natural response approaches
zero as t .
2. The system is unstable if the natural response grows
without bound as t .
3. The system is marginally stable or marginally unstable
if the natural response neither grows nor decays as t
.
4. The system is stable if bounded inputs result in
bounded outputs.
5. The system is unstable if bounded inputs result in unbounded outputs.
Here, in order to explain asymptotic stability and to lay a firm
background for the following theories, a rigorous mathematical approach may be introduced. To observe the natural response h(t) of a linear time-invariant system, a dirac delta
function (impulse) (t) may be applied to the input to give the
system internal energy upon which to act. The dirac delta
function has a rectangular shape with a height of 1/ and a
width of . is made vanishingly small so that the function
has infinite height and zero width and unit area. The ensuing
response is the natural response. Its Laplace transform is
identical to the transfer function of the system, which can be
written in the general partial fraction form as
H(s) =
n
F (s, pi , ri )
i=1
F (s, pi , ri ) =
345
K1
K2
Kr
+
+ +
(s pi )r i
(s pi )
(s pi )r i 1
(8)
where there are n sets of poles, located at s pi, each of multiple of ri.
The impulse response may be written in terms of the system poles by taking the inverse Laplace transform of H(s).
The general expression is
h(t) =
n
f (t, pi , ri )
(9)
i=1
pt
f (t, p, r) = e (k1t r1 + k2t r2 + + 1)

The behavior of h(t) is dictated by the behavior of f(t, p, r). As
t , f(t, p, r) becomes dominated by the epttr1 term, so the
behavior of h(t) as t becomes large may be investigated by the
following limit:
L = lim (t r1 |e pt |)
t
= lim
t r1
(10)
where p j. The limit is in the infinity divided by infinity indeterminate form. Applying LHopitals rule r 1 times
results in
(r 1)!
t r1 e t
(r 1)!
= lim
e t
t
r1
|L| = lim
(11)
There are several regions of interest.

1. 0. The et term forces the limit to approach infinity.
2. 0. The et term forces the limit to approach zero.
3. 0, r 1. In this case, we have a zero divided by
zero indeterminate form with three independent variables. The solution is obtained by allowing the limit (,
r 1, t) (0, 0, ) to be approached from an arbitrary
trajectory.
L=
(r 1)!
lim
r1
( , r1, t )(0, 0, )
e t
1
= ek
1
=C
(12)
where k depends of the trajectory of approach and C is

a bounded constant.
4. 0, r 1. This time we have a limit with two independent variables, (, t) (0, ).
L=
lim
(r 1)!
( , t )(0, )
= lim
r1
(r 1)!
r1
e t
ek
where k depends on the trajectory of approach.
(13)
346
We can now summarize the stability of a system depending

of the location of the poles.
Pole Locations
Stability
Poles on the left half plane

( 0) only.
Any pole on the right half
plane ( 0), or pole of
multiplicity greater than
one on the j-axis ( 0
and r 1).
Any pole on the j-axis of
multiplicity equal to one
( 0 and r 1).
The natural response approaches zero, so the system is stable.

The natural response approaches infinity, so the
system is unstable.
The natural response approaches neither zero nor
infinity. It is, however,
bounded. The system is
called marginally stable or
marginally unstable.
Consider the case of a pole of multiplicity equal to one on the

j-axis, if the input were to be a sinusoid of a frequency equal
to the distance of this pole from the origin. This would have
the same effect on the total response as if the input were zero
and the system had a pole of multiplicity equal to two on the
j-axis. The output would then approach infinity even though
the inputs were bounded. Consequently, a bounded input
function that will produce an unbounded output exists. A system classified as marginally stable under the asymptotic stability definition is, therefore, classified as unstable under the
bounded-inputbounded-output definition.
functions are listed, we are left with what is known as

Rouths array, which is the tabular technique presented next.
The RouthHurwitz criterion may be expressed as follows.
There are two necessary, but not sufficient, conditions for no
RHP poles.
1. All the polynomial coefficients ai must have the same
sign. The coefficients are determined by cross-multiplication of roots pi. If two particular coefficients were of
opposite sign, it would mean that one cross multiplication yielded a positive result whereas another yielded a
negative result. This is possible only if there exist at
least two pi of opposite sign, which means that one of
them must be on the right half plane.
2. No ai can be zero. Cancellation of terms in the crossmultiplication implies one LHP pole and one RHP pole.
If either of these two criteria is violated, it is immediately
clear that the system is unstable. Otherwise, further analysis
is required by the formation of the Rouths array.
Lets express Eq. (14) in the following form:
P(s) = an sn + an1 sn1 + an2 sn2 + + a1 s + a0
To form the Rouths array, the highest order coefficient an,

followed by every second coefficient is listed in the first row,
labeled sn. The rest of the coefficients are listed in the second
row, labeled sn1. More rows are added all the way down to s0
as illustrated.
ROUTHHURWITZ CRITERION
To determine the stability of a system, we need to know
whether any poles are located in the RHP. It is always possible to calculate these pole locations by direct computational
methods, but it is not necessary. For determining system stability, it is enough just to know whether there are any poles
on the RHP or not. This can be investigated using the Routh
Hurwitz criterion.
A generalized nth-order characteristic polynomial may be
represented as
P(s) =
n
sn1
sn2
sn3
..
.
..
.
s1
a i si
i=0
n
=k
s0
(14)
(s + pi )
i=0
where ai are the polynomial coefficients, k is a constant and

s pi are the roots of the polynomial.
The RouthHurwitz criterion is based on the Mikhailov
criterion, which states that if a system is characterized by an
nth order polynomial P(s), then it is necessary and sufficient
for stability that the following condition be satisfied.
The contour traced in the P(s) domain by P( j), 0
, must proceed counterclockwise around the origin and
limarg[P( j)] must tend toward (n/2). If the Mikhailov
criterion is applied algebraically to the generalized form of
P(s) given previously, then the RouthHurwitz criterion results are based on the determinants of the coefficients. From
these determinants, it is possible to derive a set of polynomials, known as subsidiary functions. If the coefficients of these
(15)

an
an1
an1 an2 an an3
an1
bn1an3 an1 bn3
=
bn1
bn1 =
cn1
..
.
..
.
xn1
a0
an2
an4
an3
an5
an1 an4 an an5
an1
bn1 an5 an1 bn5
=
bn1
an1 an6 an an7

an1
bn1an7 an1 bn7
=
bn1
bn3 =
bn5 =
cn3
cn5
..
.
..
.
..
.
..
.
xn3
To simplify manual calculation of the table, it is useful to note
that multiplication of any row by a positive constant will not
affect the end result.
The number of sign changes in the first column gives the

number of poles located on the right half plane.
As an example, take the polynomial
P(s) = 3s3 + s2 + 4s + 2
ficients resulting from the derivative of the auxiliary polynomial.

Consider the following polynomial:
P(s) = (s2 + 4)(s + 1)(s + 3)
(16)
We proceed to build a Routh array as

s4

s3

s2
s1
2
0
There are two changes of sign in the first column, which

means that there are two poles located on the RHP. We have
which confirms the fact that there are two poles on the right
half plane.
Special Cases
In forming Rouths array, there are three special cases that
need further consideration.
First Case. The first column of a row is zero, but the rest of
the row is not entirely zero.
Take the following polynomial as an example.
P(s) = s5 + s4 + 2s3 + 2s2 + 3s + 4
(18)
When the zero appears, replace it with a variable , to complete the table. Then take the limit as 0, both from above
and below, to determine if there are any sign changes.
0+
0
s5
s4
s2
1
2+
s1
s0
4 2
2 + 1
The table shows that there are two changes of sign in the first
column, regardless of whether approaches zero from above
or below in this case. Consequently, there are two roots in the
right half plane.
The poles are located at s1 0.6672 1.1638j, s2
0.5983 1.2632j and at s3 1.1377, confirming the
result.
Second Case. A whole row consists of zeros only.
When an entire row of zeros is encountered in row sm, an
auxiliary polynomial of order m 1 is formed by using the
sm1 row as the coefficient and by skipping every second power
of s. The row containing zeros is then replaced with the coef-
12
16
12
The auxiliary polynomial is formed and differentiated.
P(s) = 3(s + 0.476)(s 0.0712 + 1.18 j)(s 0.0712 1.18 j)

(17)
(19)
= s4 + 4s3 + 7s2 + 16s + 12
The Routh array is constructed as follows:

s3 3

s2 1

s1 2
s0 2
347
A(s) = 3s2 + 12
(20)
A (s) = 6s
We may then replace the s1 row and proceed.

s4 1
7 12
s3 4 16
0

2
s 3 12

s1 6
0

0
s
12
Because there are no changes in sign in the first column,
there are no roots on the RHP. The presence of this row of
zeros, however, indicates that the polynomial has an even
polynomial as a factor. An even polynomial has only terms
with even powers of s. It has the property that all its roots
are symmetrical about both the real and the imaginary axis.
Consequently, an even polynomial must have either roots in
both the left and right half planes or only on the j-axis. In
this case, there are no right half plane roots, so they must be
located on the j-axis. In addition, the auxiliary polynomial
A(s) is the same even polynomial that caused the row of zeros,
so we can tell that these roots are located at s 2j.
Third Case. There is a repeated root on the j-axis.
The RouthHurwitz criterion indicates the existence of
roots on the imaginary axis, but it does not indicate whether
they are of multiplicity greater than one, which is essential
knowledge if the distinction between marginal stability and
instability is required.
Take the following polynomial as an example:
P(s) = (s + 1)(s2 + 4)2
(21)
= s5 + s4 + 8s3 + 8s2 + 16s + 16
Auxillary
polynomial
s5
s4
s2
8
8
16
s0
16
16
Derivative
Final table
entry
16
16
16
16
16
16
s + 8s + 16
4
4s + 16
2
4s + 16s
3
8s
8
16
348
Even though none of the signs in the first column have

changed sign, there are two roots located at s 2j and two
at s 2j. A system having P(s) as a characteristic equation
must be considered unstable, even though the Routh
Hurwitz algorithm did not predict it.
Routh developed this criterion in 1877. In 1893, Hurwitz,
apparently unaware of Rouths work, developed a similar
technique based on determinants, from which the Routh
Hurwitz criterion is derivable. In 1892, Lyapunov developed
a more general technique that is applicable to both linear and
nonlinear systems, called the direct method of Lyapunov.
Im
=
R=
Re
NYQUIST CRITERION
Given the characteristic equation of a system, the Routh

Hurwitz criterion enables a system analyst to determine
whether or not the system is stable without actually solving
the roots of the characteristic equation. Unfortunately, the
method still requires the characteristic equation, which may
be somewhat cumbersome, to be derived. The Nyquist criterion offers a graphical method of solution based on the openloop transfer function, thereby saving some algebraic manipulation. In addition, Nyquists method is quite capable of handling pure time delays, which Rouths method and the root
locus method can handle only clumsily, at best.
The Nyquist criterion is based on the following principal
argument. Suppose a contour 1 is traced arbitrarily in the splane as shown in Fig. 7(a). If each point s, comprising 1
were to be transformed by a polynomial function of s, L(s),
then a new contour 2 would result in the L(s)-plane, as illustrated in Fig. 7(b). Provided that 1 does not pass through any
poles or zeros of L(s), the contour 2 does not encircle the origin. The principal argument relates the number of times that
the new contour 2 encircles the origin to the number of poles
and zeros of L(s) encircled by 1. In other words, 2 encircles
the origin by the difference between the number of poles and
number of zeros in contour 1
N = ZP
s-plane
(22)
jv G(s)-plane
Figure 8. The Nyquist contour. If a contour is traced on the s-plane

covering the entire RHP in the clockwise direction and if the number
of zeros of G(s)H(s) are greater than number of poles then the corresponding contour on the G(s)H(s) will encircle the origin at least once
in the same direction. The poles of G(s)H(s) can usually be determined
easily from the mathematical model. The number of zeros of
G(s)H(s) can be determined by the Nyquist plot.
where
N is the number of encirclements of the origin by 2,
Z is the number of zeros of L(s) encircled by 1, and
P is the number of poles of L(s) encircled by 1.
A positive N indicates that 2 and 1 both travel in the same
direction (i.e., clockwise or counterclockwise), whereas negative N indicates opposite directions. Some examples of contours 2 and 1 and encirclements are given in the section
dedicated for Nyquist. Interested readers should refer to this
section.
The Nyquist Contour
Consider the transfer function of a closed-loop system
C(s)
G(s)
=
R(s)
1 + G(s)H(s)
G(s)
=
1 + L(s)
(23)
From this equation, the following points should be clear.
(a)
s-plane
(b)
Figure 7. Contours in the s-plane and G(s)H(s)-plane. Every closed

contour on the s-plane traces a closed contour on the G(s)H(s)-plane.
If there are any poles or zeros (but not equal in numbers) of
G(s)H(s) in the contour in the s-plane, the contour in the G(s)H(s) will
encircle the origin at least once. If the number of poles of G(s)H(s)
inside the contour in the s-plane is greater than zero, the contour in
the G(s)H(s)-plane goes in the opposite direction of the contour on the
s-plane. If the zeros are greater than poles, the contours are in the
same direction.
1. The poles of 1 L(s) are the poles of L(s), the open-loop

transfer function. This makes identification of the poles
of 1 L(s) possible simply by inspection in most cases.
2. Most importantly, the zeros of 1 L(s) are the poles of
C(s)/R(s), the closed-loop transfer function.
The problem of determining the closed-loop pole locations
may then be reduced to determining the zeros of 1 L(s). To
do this, the Nyquist contour, which covers the entire right
half s-plane, as shown in Fig. 8, is used. It consists of a section along the j-axis covering and a semicircle
described by s , /2 arg(s) /2. The diagram also
shows how the contour skips around simple (not repeated)
poles of L(s) located on the j-axis to avoid discontinuities
when the contour is mapped onto the L(s) plane. The direction
of the contour has been arbitrarily drawn as clockwise.
If the Nyquist contour is mapped onto the 1 L(s)-plane,
we would find that the resulting contour would encircle the
origin N Z P times (in the clockwise sense). It should be
emphasized that Z is the variable under investigation because it concerns the poles of the closed-loop system. The requirement for stability is that 1 L(s) contain no zeros on
the right half plane, or Z 0. That is to say, if the (clockwise)
Nyquist contour were mapped onto the 1 L(s)-plane, it is a
requirement for closed loop stability that the resulting contour encircle the origin counterclockwise exactly the same
number times as the number of poles of L(s) in the RHP.
The contour shown in Fig. 8 skips to the right around jaxis poles. Consequently, these j-axis poles are not considered to be right-half-plane poles. It is perfectly feasible for the
contour to skip to the left around these poles, in which case
they should be included in the count of right-half-plane poles.
It is emphasized that the poles of L(s) are easily obtainable.
A further refinement of the Nyquist criterion is that it is
unnecessary to plot the contour on the 1 L(s)-plane and
observe the number of encirclements about the origin. The
plot on the L(s) plane is in fact identical to that of the 1
L(s)-plane, except that it is shifted left by one unit. It will
therefore suffice to plot the contour on the L(s)-plane and observe the number of encirclements about the Cartesian point
(1, 0).
Simplified Nyquist Plot
Suppose that the function 1 L(s) contains P poles on the
RHP, P poles on the j-axis, and Z zeros on the right half
plane and that two contours are to be mapped onto the L(s)plane.
1. The Nyquist contour skips to the right around the P
poles on the j-axis. When mapped on the L(s)-plane, it
is found to encircle the Cartesian point (1, 0) point
N1 times.
2. The Nyquist contour skips to the left around the P
poles on the j-axis. When mapped on the L(s)-plane, it
is found to encircle the Cartesian point (1, 0) point
N2 times.
Each contour may be considered to consist of three sections,
each contributing a certain number of turns about the
Cartesian point (1, 0).
1. The section consisting of , excluding the
skips around j-axis poles. Because of symmetry about
the real axis in both the contour and the location of
poles and zeros, this section may be divided into two
halvesthe positive imaginary axis and the negative
imaginary axis, each contributing NA turns.
2. The section consisting of the infinite semicircle, contributing NB turns. If the order of the numerator of L(s) is
less than or equal to the denominator, then as s ,
L(s) corresponds to a point on the real axis or an encirclement of the origin. The contribution to the number of
turns about the Cartesian point (1, 0) in either case
is, therefore, NB 0.
349
3. The skips around the j-axis poles. Because the two

contours skip around these poles in opposite directions,
if contour 1 were to contribute NC turns, then contour 2
would contribute NC turns.
Combining all these sections,
N1 = 2NA + NB + NC
(24)
N2 = 2NA + NB NC
(25)
From the principal argument, it is also known that N1 Z

P; therefore,
N2 = Z P P
(26)
Eliminating N1, N2, and NC and realizing that NB 0, we find

that
2Z 2P P
4

P
= ZP
NA =
or
(27)
where is the angle of rotation (in the clockwise sense) about

the point (1, 0) when the line 0, 0 is mapped onto
the L(s)-plane, where s j.
For stability, we require that Z 0, from which the modified form of the Nyquist stability criterion may be expressed
as
= P+
P
2
(28)
That is to say, if the open-loop transfer functions frequency

response is plotted on polar coordinates and is found to encircle the Cartesian point (1, 0) in a counterclockwise direction
by an angle of exactly (P P/2) radians. In this case, where
P is the number of open-loop transfer function poles on the
right half plane and P is the number of open-loop transfer
function poles on the imaginary axis, the closed-loop system
is stable.
For the illustration of Nyquist stability criterion, lets take
an example of a system having a open-loop transfer function
G(s)H(s) of
G(s)H(s) = L(s)
30
(s + 1)(s + 2)(s + 3)
30
= 3
s + 6s2 + 11s + 6
=
(29)
The Nyquist plot of L(s) of Eq. (29) can be obtained in a number of ways (e.g., polar plots) by substituting s j. By calculating the real and imaginary components of L(j), the Ny-
350
NG /DG and H(s) NH /DH. Equation (30) then reduces to
kNG DH
C(s)
=
R(s)
DG DH + kNG NH
Imag axis
2
1
0
1
2
3
4
2
Real axis
Figure 9. A typical Nyquist plot. This is the plot of a third-order
system and hence it traces three quadrants. The curve cuts the real
axis on the negative side. If the gain is increased sufficiently, the
curve will encircle the 1 point hence indicating instability. This
means that at least one of the roots of the characteristic equation,
poles of the closed loop system, will be on the right half of the s-plane.
quist plot of Eq. (29) may be plotted as shown in Fig. 9. It can

be seen that the contour does not encircle the point (1, 0),
so the system is stable.
Further examples of Nyquist plots are given in Fig. 10.
From the Nyquist plots, it is possible to find phase and
gain margins of the system. The gain margin is defined to be
the amount of gain that can be allowed before the system becomes unstable, and the phase margin is the angle at unity
gain. It is also possible to find the phase crossover frequency
c and the gain crossover frequency g either from the graph
or mathematically. From the graph the phase and gain margins of the preceding example are 25 and 6 dB, respectively.
It is also possible to design the system to obtain desired responses by varying the margins.
THE ROOT LOCUS
Often engineers want to see how changes in some parameters
such as loop gain will affect the performance and the stability
of a system. The root locus is a widely practiced method in
this regard. It gives information about how the closed-loop
poles of the system vary as the parameter in question is
changed. This is particularly useful in determining the range
the parameter may cover while keeping the system stable. As
discussed previously, the relative stability of a system is
largely determined by the location of poles, which the root
locus approach clearly confirms.
Equation (31) reveals that the zeros of the closed-loop system

are independent of k and correspond to the zeros of G(s) and
the poles of H(s). However, as k 0, there is pole/zero cancellation of the H(s) pole term DH, and as k there is pole/
zero cancellation of the G(s) zero term NG. The location of the
closed-loop poles, or the roots of the characteristic equation,
is the subject of the remaining discussion.
The root locus is a plot in the s-plane of the poles of the
closed-loop transfer function as the parameter k varies from
0 to . From Eq. (1), it should be clear that these poles correspond to the zeros of the 1 kG(s)H(s) denominator. The root
locus is therefore a plot in the s-plane of Eq. (31)
kG(s)H(s) = 1
Equation (32) may be expressed in its polar form as
u
A
1
vi=1 i =
k
B
i=1 i
u
v

i
i = (1 + 2n)
i=1
(32)
(33a)
(33b)
i=1
where Ai is the distance between a point on the root locus and

the ith zero of the loop transfer function L(s); Bi is the distance between a point on the root locus and the ith pole; i is
the angle about the ith zero from the positive real axis to a
point on the root locus; i is the angle about the ith pole from
the positive real axis to a point on the root locus; u is the
number of zeros in the loop transfer function; v is the number
of poles; k is the loop gain; and n is any integer. It should be
evident that Eq. (33b) determines whether a point is on the
root locus, and Eq. (33a) just determines the value of k corresponding to that point.
It is always possible to solve Eq. (32) for an array of values
for k, but that would be too time consuming. Evans developed
a set of rules for sketching the root locus, reducing the problem to a few simple calculations.
Figure 12 shows the root locus of a typical system with an
open-loop transfer function given by
k
s(s + 2)(s + 5)
k
= 3
s + 7s2 + 10s
T (s) =
(34)
Further examples of root locus are given in Fig. 10.

The Root Locus Method of Evans
Formulation of Root Locus

Figure 11 shows a block diagram of a system with a variable
loop gain. It has a closed-loop transfer function given by
kG(s)
C(s)
=
R(s)
1 + kG(s)H(s)
(31)
A number of rules may be applied to sketch the root locus.

The root locus starts with k 0 at the poles of G(s)H(s)
and finishes with k at the zeros of G(s)H(s).
(30)
In order to investigate how G(s) and H(s) contribute poles and

zeros to the closed loop system, it is informative to let G(s)
This can be seen from the magnitude condition

|G(s)H(s)| =
1
k
(35)
Transfer function
Nyquist plot
351
Root locus
Imag axis
Imag axis
K
(s 1 + 1)
0
1

1
0
Real axis
Imag axis
Imag axis
Real axis
K
(s 1 + 1)(s 2+ 1)
0
Real axis
Imag axis
K
(s 1 + 1)(s 2+ 1) (s 3 + 1)
Imag axis
Real axis
1

0
0
Real axis
Imag axis
K
s(s 1 + 1)
Imag axis
Real axis
1

1
0
Real axis
Imag axis
K
s(s 1 + 1)(s 2 + 1)
Imag axis
Real axis
As k approaches zero, the magnitude of the loop transfer function becomes infinite, corresponding to a pole. For k becoming
infinitely large, the loop transfer function becomes infinitesimally small, corresponding to a zero.
Actually, inspection of Eq. (31) reveals that the poles of
H(s) and zeros of G(s) never actually appear as poles of the
+
G(s)
0
Real axis
Real axis
R(s)
1
1

C(s)
H(s)
Figure 11. Block diagram of a closed-loop system with variable k. In
many systems, one of the parameters of the system is varied to
achieve the desired response. In this case, the variation of k in the
forward path will relocate the roots of the characteristic equation on
the s-plane. The suitable locations of the roots lead to appropriate
system design.
Figure 10. Examples of Nyquist and root

locus plots. The stability of control systems can be determined by various methods as exemplified here. In obtaining these
examples, a popular software called MATLAB was used.
closed-loop transfer function because of pole/zero cancellation

for k 0 and k . This point should be kept in mind when
designing systems with the root locus; for very high and very
low gains, there may be significant pole/zero cancellation.
The zeros of G(s)H(s) include both the finite zeros found in
the denominator terms and the infinite zeros at s
caused by a denominator of higher order than the numerator.
The result is that there are always the same number poles
and zeros and that the root locus will always have enough
zeros at which to terminate, be they finite or infinite.
The root locus plot is symmetrical about the real axis. All
physically realizable transfer functions have real coefficients. Transfer functions with real coefficients always
produce complex poles and zeros in conjugate pairs,
which means that if Eq. (32) locates a point s j
as being on the root locus, then s j must also be
on the root locus.
352
shown in Fig. 13, plotted for G(s)H(s) (s 1)(s 3)(s2

4s 8) . All breakaway points s must satisfy the following conditions:
10
8
6
d
G(s)H(s) = 0
ds
arg[G(s)H(s)] = (1 + 2n)
Imag axis
4
2
0
2
4
6
8
10
10
0
2
Real axis
10
Figure 12. The root locus of a system with a characteristic equation 1 ks(s 2)(s 5). This is a typical example of root locus.
Roots start from poles of the characteristic equation when k 0 and
approaches zeros as k . In this example, all three zeros are at .
At some value of k, the root loci crosses the imaginary axis to the
RHP, thus indicating unstable conditions. With the aid of root locus
a suitable value of k can be determined to locate the roots at the
desired points on the s-plane.
A point on the real axis is on the root locus if and only if

there is an odd number of poles and zeros on the righthand side of it. The angular contribution to Eq. (33b) of
a pole or zero on the real axis to the left of a point on the
real axis is always zero. In the case of a complex conjugate pair of poles or zeros, if one member of the pair contributes an angle of then the other will contribute 2
. The total contribution from the pair is then 2 0
rad. Similarly, any pole or zero to the right of a point on
the real axis will end up contributing rad to the angular equation. Consequently, an odd number of poles or
zeros is required to satisfy Eq. (33).
(38)
Angle of departure from a complex root. The angle at

which a branch leaves a pole or arrives at a zero may be
determined by assuming a point s infinitesimally close to
the singularity. Because s is infinitesimally close to the
pole or zero, the angular contributions to the angle Eq.
(33b) from all the other poles and zeros are known, and
the only unknown quantity is the contribution from the
pole or zero in question. This angle, the angle of departure, is easily found by Eq. (33b).
Imaginary axis intersection. This is a very important
point to know because it reveals the value of k that will
result in a marginally stable closed loop system. Forming
a Routh table with the unknown k as a parameter and
then solving for k to give a row of zeros in the table is
one of the most common methods. For higher-order systems, the table may become too cumbersome. In such a
situation, it may be more desirable to solve Eq. (32) as
G( j)H( j) =
1
k
(39)
(36a)
(36b)
where n is the angle between the positive real axis and

the nth asymptote; (A, 0) is the point at which the asymptotes intersect the real axis; pi is the ith open-loop
transfer function pole location; zi is the ith open-loop
transfer function zero location; v is the number of openloop transfer function poles; and u is the number of zeros.
Imag axis
(2n + 1)
vu
v
u
i=1 pi
i=1 zi
A =
vu
(37b)
For real values of s, Eq. (37a) implies Eq. (37b), but for
complex s, there is no such implication.
If there are n poles involved in a breakaway point,
then there are always n branches entering and n
branches leaving. The angle between the entering and
leaving branches is given by
Branches terminating at infinite zeros approach an

asymptotic line that is described by
n =
(37a)
2
0
2
4
6
6
Applying these rules will provide a reasonable sketch of

the root locus. There are several significant points on the
sketch that may be of interest to locate in terms of their precise location and the value of k required to achieve them.
Breakaway points. These are the points at which multiple roots meet and then diverge. This most commonly occurs on the real axis, but it may occur anywhere, as
0
Real axis
Figure 13. Breakaway points. The root locus plotted for G(s)H(s)
(s 1)(s 3)(s2 4s 8) shows typical breakaway points at which
multiple roots meet and then diverge. The breakaways generally occur on the real axis, but they may occur anywhere. In this example,
a breakaway has happened on the real axis where as two others have
taken place on the s-plane, in which the corners of the two root loci
have met.
R(s)
1
(s + 1)(s + k)
353
ing that there is a known solution x1, then the equation may
be reduced to a first-order linear equation by letting
C(s)
x = x1 +
Figure 14. A representation of a variable component other than the

root gain. In this case, one of the open-loop poles is the variable. This
can be handled by forming an equivalent loop transfer function to
construct the root locus.
1
u
(44)
Taking the derivative with respect to t results in

dx
dx1
1 du
=
2
dt
dt
u dt
(45)
Substituting Eqs. (44) and (45) into Eq. (41) gives

In this case, the root locus is solved for an imaginary axis
intercept, s j.
dx1
1 du
1
2
+ P(t) x1 +
dt
u dt
u
= Q(t) x21 +
1
2x1
+ 2
u
u
(46)
System Parameters Other Than Gain

In many situations the loop gain is not the parameter that is
variable. It may be that the position of the open loop poles is
the variable, as in Fig. 14. The way to handle this is to form
an equivalent loop transfer function for the purpose of constructing the root locus as
1
C(s)
=
R(s)
(s + 1)(s + k) + 1
(40)
After some algebraic manipulation, Eq. (40) may be expressed

in the form of
1
s2 + s + 1
s+1
1+k 2
s +s+1
G(s)
=
1 + kG(s)H(s)
C(s)
=
R(s)
(41)
THE RICCATI EQUATION

Many matrix equations naturally arise in linear control system theory. One of the most applied equations is the Riccati
equation, which can be expressed as
(42)
The equation was first developed and applied by Count Riccati and Jacopo Francesco in the early eighteenth century.
In recent times, the Riccati equation finds wide application,
particularly in the area of optimal control and filtering. In
these applications, the matrix Riccati equation depicts a system of Riccati equations given by
X(t)
+ X(t)A(t) D(t)X(t) B(t)X(t) + C(t) = 0
But since it is known that

dx1
+ P(t)x1 = Q(t)x21 + R(t)
dt
(47)
Eq. (46) reduces to

du
+ [2x1 (t)Q(t) P(t)]u = Q(t)
dt
(48)
which is a linear first-order differential equation and is simple to solve particularly if P(t), Q(t), and R(t) are constants,
as would the case if Eq. (41) were to describe a time-invariant system.
The Matrix Riccati Differential Equation
Consider the dynamic system given by the state-space description as
where G(s) 1/(s2 s 1) and H(s) s 1. The root locus

may now be constructed in the normal manner.
It may also occur that there are two parameters that are
variable. Then the root locus may be represented by a set of
contours or a surface plots.
dx
+ P(t)x = Q(t)x2 + R(t)
dt
+ R(t)
(43)
Solution of the Riccati Equation

A closed form solution to Eq. (42) cannot be guaranteed depending of the functions P(t), Q(t), and R(t). However, assum-
x(t)
= Fx(t) + Gu(t)
y(t) = Hx(t)
(49)
where x(t) is the state matrix (n by 1), u(t) is the control matrix (q by 1), and y(t) is the matrix (p by 1) of output variables
to be controlled and F, G, and H are matrices of appropriate
dimensions that characterize the system.
In the optimal control applications, the object of optimal
control is to find u(t) over an interval t [t1, t2] such that
some cost function is optimized. One of the popularly used
cost function is the quadratic cost function that can be generalized by

F (t1 , t2 , T ) =
t2
t1
[y (t)y(t) + u (t)u(t)] dt + x (t2 )Tx(t2 ) (50)
where T is a constant, real, symmetric (T T) and nonnegative definite (T 0) matrix.

It can be proven that there exists a unique optimal control
for finite t2 t1 0, which has the form of
u(t) = G P(t1 , t2 , T )x(t)
(51)
where P(t1, t2, T) can be described by
P(t)
+ P(t)F + F P(t) P(t)GG P(t) + H H = 0
with the terminal condition that P(t2) T.
(52)
354
Correlation of Eq. (52) together with Eq. (43) reveals that

the optimum control problem does indeed reduce to the problem of solving the Riccati matrix equation with constant, real
A, B, and C matrices and D A. Furthermore, because any
matrix UU is a symmetric, nonnegative definite matrix for
U with real elements, it follows that B and C in are symmetric and nonnegative definite matrices, which are necessary
conditions for solutions.
which corresponds to Eq. (55). Applying Eq. (56), the final

solution to Eq. (57) is found to be
Solution of the Riccati Matrix Differential Equation
Riccati Algebraic Equation and Infinite Horizon
The following equation gives the form of the Riccati equation

of interest applied in optimal control problems:
In regulating control systems, it is not convenient to restrict

control to a finite time period, t [t1, t2]. Even though it is
possible to let t1 approach negative infinity, it is customary to
let t2 approach infinity. Assuming that the system is time invariant [i.e., the matrices A, B and C in Eq. (53) are constant], the two intervals yield the same result.
Suppose that (A, B) are stabilizable. Then there must exist
some stabilizing control that is not necessarily optimal. The
resulting cost function associated with this stabilizing control
will then dominate the optimal cost function, and it must be
finite. Consequently, the solution of the Riccati equation with
infinite horizon must be bounded.
The solution X(t), as t approaches infinity, will either approach a constant value or become periodic, depending on the
system and the value of T chosen. The value of X(t) for t
in the case where it is constant may be found by substituting
(t) 0 into Eq. (53) to give the Riccati algebraic equation.
X
X(t)
+ X(t)A A X(t) X(t)BX(t) + C = 0
(53)
with B and C matrices being symmetric and nonnegative

definite.
The solution of Eq. (53) may be found by setting up the
linear Hamiltonian matrix differential system as

U(t)
A
=
C
V(t)
B
A

U(t)
V(t)

(54)
where V(t2) TU(t2), and T is as defined in Eq. (50).

Then if the solution of Eq. (54) may be found by using Eq.
(53)

U(t)
w11 w12 U(t2 )
=
(55)
V(t)
w21 w22 V(t2 )
and w11 w12T is invertible, then the solution of Eq. (53) is
given by
X(t) = (w21 + w22 T)(w11 + w12 T)1
(56)
As an example, consider
dx
+ 2x x2 + 1 = 0
dt
As expected, x(t2) x(0) T.
XA + A X XBX + C = 0
Hxj = xj + xj1
(57)
v(0) = 2.414a 0.414b

Also, Eq. (59) may be expressed as

u
0.8536e1.414t + 0.1464e1.414t
=
v
0.3534e1.414t + 0.3534e1.414t

0.3536e1.414t + 0.3536e1.414t u(0)
0.1464e1.414t + 0.8536e1.414t
v(0)
(60)
B
A
given by x1, x2, . . ., xn. If we let

xi =

ui
vi
and
U = [u1 un ]
and
(61)
(64)
where is an eigenvalue of H and x1 is the associated eigenvector.

Equation (63) has a solution for X if and only if there exists
a set of Jordan chains of
A
H=
C
(59)
(63)
If A, B, and C are 1 by 1 matrices (i.e., there are just scalar

numbers), the solution of Eq. (63) is a trivial task. The general solution is a little more complex, involving Jordan chains.
A Jordan chain of the matrix H is a set of vectors, x1, x2,
. . ., xn such that
For some arbitrary constants a and b, the solution becomes

u(0) = a + b
(0.3534 + 0.1464T )e1.414t + (0.3534 + 0.8536T )e1.414t

(0.8536 0.3536T )e1.414t + (0.1464 + 0.3536Te1.414t
(62)
Hx1 = x1
which corresponds to A [1], B [1], and C [1] in Eq. (53).

The associated linear Hamiltonian matrix is then given by

u
1 1 u
=
(58)
v
1 1 v
which has the solution

1
u
1
e1.414t
=a
e1.414t + b
0.414
v
2.414
x(t) =
V = [v1 vn ]
then the solutions of Eq. (63) are given by X VU1.
We can verify this by using it to derive the quadratic formula. Let

ax2 + bx + c = 0
b2
c

a
b
2
(66)
which has two eigenvalues given by

b
ac
2
b2 4ac
=
2
(67)
The eigenvector associated with the eigenvalue of 0 is
1
w = b
(68)
2a
a
The solution of Eq. (65) is then found as
2a 2a
b b2 4ac
=
2a
x=
which is the familiar quadratic equation.

For 0, there exists one Jordan chain,
1
x1 = b
2a
0
x2 = 1
values, hence achieved an equilibrium state. If the system is

in the equilibrium state, that is, no states are varying in time,
the equilibrium state may be described by
(65)
where A b/2, C c, and B a, to form the Hamiltonian matrix,
H=
355
(69)
(70)
e = f(xe ) = 0
X
(72)
In order to seek solutions to Eq. (71), Lyapunov introduced a

continuously differentiable scalar function V(x) with the following properties.
1. Positive definite if V(0) 0 for all t t0 and V(x) 0
for all t t0.
2. Positive semidefinite if V(x) 0 for all x.
3. Negative definite or negative semidefinite if V(x) positive definite or positive semidefinite.
These conditions ensure that V is positive if any state is different from zero but equals zero when the state is zero. These
conditions ensure that V is a smooth function and the trajectory does not expand indefinitely but rather is drawn to the
origin. This can be explained with the aid of Fig. 15. Lyapunov stability states that an equilibrium state xe of a dynamic
system is stable if for every 0, there exists a 0, where
depends only on , such that (x0 xe) results (x(t; x0)
xe) for all t t0.
This statement of stability in the sense of Lyapunov indicates that if an equilibrium state is stable, the trajectory will
remain within a given neighborhood of the equilibrium point
if the initial state is close to the equilibrium point. Likewise,
an equilibrium state xe of a dynamic system is unstable if
there exists an , such that a corresponding value cannot
be found.
From the preceding explanations, asymptotic stability may
be defined. An equilibrium state xe of a dynamic system is
asymptotically stable if
a. it is Lyapunov stable.
b. there is a number a 0 such that every motion starting within a in the neighborhood of xe converges to xe
as t .
giving one solution, x b/2a.
LYAPUNOV STABILITY
Lyapunov studied the question of motion, basing his theories
on the nonlinear differential equations of motion. His equations for linear motion are equivalent to Rouths criterion.
The Lyapunovs theorem determines the stability in the small
region about an equilibrium point. The stability in the large
may be determined by the global stability techniques. In this
article, stability in the small will be introduced, and some references will be made to global stability methods.
Lyapunov considers the stability of general systems described by ordinary differential equations expressed in the
state-variable form as
= f(x)
X
(71)
Suppose that in a system all states have settled to constant
Unstable
x=0
xo
Asymptotically
stable
Stable
Figure 15. Lyapunov stability criteria. The equilibrium state xe is
stable if for every 0, there exists a 0, where depends only
on , such that the trajectory remains within a given neighborhood of
the equilibrium point. Likewise, an equilibrium state xe of a dynamic
system is unstable if there exists an , such that a corresponding
value cannot be found.
356
A simplified version of Lyapunovs first theorem of stability,

Ax is a valid
Eq. (71), may be explained. Suppose that X
model about the equilibrium point xe and the roots of the
characteristic equation may be expressed in matrix form as
sI A = 0
(73)
All eigenvalues of A have negative real parts if for any symmetric positive definite matrix N, the Lyapunov equation
AT M + AM = N
(74)
has a symmetric positive definite solution.

For example, suppose that Eq. (71) can be expressed as
= Ax(t)
X
(75)
Its solution is x(t) eAt x(0). If the eigenvalues of A are k1,

k2, and k3, then every component of x(t) is a linear combination of ek1t, ek2t, and ek3t. These time functions will approach
zero as t if and only if ki has negative real parts. Thus
it can be concluded that any nonzero initial state will approach to zero only if A is stable. This can be generalized
as follows.
1. If the characteristic values all have negative real parts,
the equilibrium point is asymptotically stable.
2. If at least one of the values has a positive real part, the
equilibrium point is unstable.
3. If one or more characteristic values have zero real
parts, with all other values having negative real parts,
the system stability cannot be determined with the current method.
Testing the stability by considering the linear part is referred
to as Lyapunovs first or indirect method. Using the Lyapunov
function directly on the nonlinear equations themselves is
called the second or direct method. The argument is as follows. Lyapunov showed that in the case of nonlinear systems
Eq. (71) may be extended as
= Fx + g(x)
X
(76)
where g(x) contains all the higher powers of x. If g(x) goes to

zero faster than x does, then the system is stable if all the
roots of F are strictly inside the left half plane and will be
unstable if at least one root is in the right-half plane. For the
system with roots in the left half plane and on the imaginary
axis, the stability depends on the terms in the function g.
For global stability analysis of linear constant systems,
quadratic functions are often used. Consider the function
V xT Px, where P is a symmetric positive matrix. The V is
the sum of squares of xi. In general, if P is positive, we can
find a matrix T such that P TTT and V zi, where z
Tx. For the derivative of V the chain rule can be used as
V = dxT Px/dt
= xT Px + xT Px
= xT (FT P + PF)x
= xT Qx
(77)
where
Q = (FT P + PF)
(78)
For any positive Q, the solution of P of the Lyapunov equation is positive if and only if all the characteristic roots of F
have negative real parts. That is, if a system matrix F is
given, it is possible to select a positive Q, solve the Lyapunov
equation in n(n 1)/2 unknowns, and test to see if P is positive by looking at the determinants of the n principal minors.
From this, the stability may be determined from the equations without either solving them or finding the characteristic roots.
The study of nonlinear systems is vast, here only the basic
principles of the methods have been discussed. Also, Lyapunov methods are applied in many diverse areas of control engineering; therefore, it is impossible to cover them all here.
Interested readers should refer to the reading list given at
the end of this article.
ROBUST STABILITY
Consider a linear time-invariant feedback system with a
plant transfer function G(s) and a compensator with Gc(s) cascaded as shown in Fig. 16. In many applications, the plant
model will not accurately represent the actual physical system because of (1) unmodeled dynamics and time delays, (2)
changes in equilibrium points, (3) nonlinear characteristics of
the plant, (4) noise and other disturbance inputs, and (5) parameter drift. The aim of a robust system is to assure that
performance is maintained in spite of model inaccuracies and
parameter changes.
The closed-loop transfer function of the system in Fig. 16
may be written as
M(s) =
G(s)Gc (s)
C(s)
=
R(s)
1 + G(s)Gc (s)
(79)
The sensitivity of the system to changes in G(s) or Gc(s) can

be expressed by the sensitivity function
S=
R(s) +
1
M/M
=
G/G
1 + G(s)Gc (s)
(80)
C(s)
Gc(s)
G(s)
H(s)
Figure 16. Block diagram of a closed-loop control system for robust
stability analysis. In the many mathematical representation of systems, a full account of all affected parameters may not be taken into
consideration because of unmodeled dynamics and time delays. Also,
during the operations, the equilibrium points may change, parameters may drift, and noise and disturbances may become significant.
The aim of a robust system is to assure that performance is maintained in spite of model inaccuracies and parameter changes.
STABILITY THEORY: INCLUDING SATURATION EFFECTS
G(s) H(s)-plane
EXPONENTIAL STABILITY
Imaginary
The study of exponential signals (eat) is important in linear

system analysis. They contain a variety of signals such as
constants, sinusoids, or exponentially decaying or increasing
sinusoids.
A system with an n-dimensional state model is said to be
an exponential system if its state-transition matrix (t, ) can
be written in matrix exponential form
0
1
Real
(t, ) = e (t, )
Figure 17. An example of a closed-loop system resulting from parameter drifts in Nyquist plots. This diagram indicates that because
of uncertainties in modeling and changes in parameters the gain and
phase margins may be altered. These alterations may lead to unstable conditions if these margins are close to critical values.
As can be seen from Eq. (80), the sensitivity function has the
same characteristic equation [1 G(s)Gc(s)] as the closed-loop
transfer function M(s). For sensitivity to be small, it is necessary to have a high value for loop gain L(s) G(s) Gc(s). The
high gain is obtained at high frequencies of L( j). But as we
know, high gain could cause instability and poor responsiveness of M(s). Now, the design problem becomes a matter
of selecting Gc(s) such that the closed-loop sensitivity is small,
and the closed-loop transfer function has a wide bandwidth.
At the same time, the desired gain and phase margins must
be achieved.
The stability of the control system depends on the openloop transfer function L(s) G(s)Gc(s). Because of the uncertainties outlined here, the transfer function may be written
as
L(s) = Gc (s)[G(s) + G(s)]
357
(81)
For system stability, Nyquists stability condition must always be satisfied. That is the 1 point must not be encircled
by the L( j) under any circumstances. An example of uncertainty in a typical Nyquist plot resulting from G(s) is illustrated in Fig. 17. To guarantee stability, a safe gain and
phase margin must be ensured. Many methods are available
to deal with the robustness of the system including classical
methods linked to the use various compensators and PID controllers. The H technique is one method that finds extensive
application in robust control design and analysis.
In general, feedback reduces the effect of disturbances and
moderate modeling errors or parameter changes in the control
system. In the presence of disturbances and sensor noises,
systems are designed such that they keep the tracking errors
and outputs small for disturbance inputs. In order to achieve
this, the sensitivity to modeling errors and sensor noise must
be made small, thus making the system robust. In this case,
the plant output will follow any reference input asymptotically even if there are variations in the parameters of disturbance and noise. Briefly, it can be said that the system is
more robust if it can tolerate larger perturbations in its parameters.
(82)
where (t, ) is an n n matrix function of t and .

A sufficient condition for the system to be uniformly exponentially stable is that the eigenvalues of the of the n n
matrix (1/t) (t, ) be bounded as functions of t and have real
parts v for all t and for some v 0 and .
In many applications the stochastic components and random noises are included in the dynamical system models. The
stochastic aspects of the model are used to capture the uncertainty about the environment in which the system is operating. The analysis and control of such systems involve
evaluating the stability properties of the random dynamical
systems. The stability of the system can be studied by stochastic stability approaches.
BIBLIOGRAPHY
1. W. S. Levine, The Control Handbook, Boca Raton, FL: CRC
Press, 1996.
2. K. Ogata, Modern Control Engineering, 3rd ed., Upper Saddle
River, NJ: Prentice-Hall, 1997.
3. B. J. Kuo, Automatic Control Systems, 6th ed., Englewood Cliffs,
4. K. Watanabe, Adaptive Estimation and Control: Partitioning Approach, Hertfordshire, UK: Prentice-Hall, 1992.
HALIT EREN
ALEXANDER LOWE
Curtin University of Technology,
Australia
390
STABILITY THEORY, NONLINEAR
as time becomes large. This article deals with the latter notion of stability. An asymptotically stable response is the basis of a steady-state behavior whereby other responses asymptotically approach the steady-state response. A steady-state
response can be as simple as a constant (time-invariant) response, or it can be a periodic one. These, as well as other
more complicated steady-state responses, are described in the
first section.
In our study of dynamical systems, we model the system
by a finite number of coupled first-order ordinary differential
equations
x1 = f 1 (t, x1 , . . ., xn )
x2 = f 2 (t, x1 , . . ., xn )
..
.
..
.
xn = f n (t, x1 , . . ., xn )
where xi denotes the derivative of xi with respect to the time
variable t. We call the variables x1, x2, . . ., xn the state variables. They represent the memory that the dynamical system
has of its past. They are usually chosen as physical variables
that represent the energy-storing elements. For example, in
an RLC electrical circuit, the state variables could be voltages
across capacitors and currents through inductors, while in a
spring-mass-damper mechanical system, the state variables
could be positions and velocities of moving masses. We usually use vector notation to write the above equations in a compact form. Define

x1
f 1 (t, x)

x2
f 2 (t, x)

x = . , f (t, x) = .
..
..

xn
f n (t, x)
and rewrite the n first-order differential equations as an ndimensional first-order vector differential equation
x = f (t, x)

Stability analysis plays a central role in systems engineering.
There are two general notions of stability that arise in the
study of dynamical systems: input-output stability and stability of a particular response or a set of responses. In the first
notion, the system is viewed as a map from the space of input
signals to the space of output signals. It is said to be stable if
an input that is well behaved in some sense (e.g., signals with
finite amplitude or energy) will always produce an output
that is well behaved in the same sense. In the second notion,
the input to the system is fixed so that the response of the
system over time is determined solely by the initial state of
the system at the initial time. Such a response is said to be
stable if other responses starting at nearby initial states stay
nearby; otherwise it is unstable. It is said to be asymptotically
stable if all responses starting at nearby initial states not
only stay nearby, but also approach this particular response
which we call the state equation, and x is referred to as the

state. The response of the system due to initial state x0 at
initial time t0 is the solution of the differential equation x
f(t, x) subject to the initial condition x(t0) x0. This solution
is unique provided the function f is locally of Lipschitz character in the domain of interest D; that is, every point in D has
a neighborhood D0 and a nonnegative constant L such that
the Lipschitz condition
f (t, x) f (t, y) Lx y
is satisfied for all x and y in D0. x is a measure of the length
of the vector x in the n-dimensional state space (the space of
n
n-dimensional real vectors), and is defined by x2 i xi2. The
locus of the solution x(t) in the state space is usually referred
to as a trajectory. A special case of the state equation arises
when the function f does not depend explicitly on t; that is,
x f(x), in which case the system is said to be autonomous;
otherwise it is said to be nonautonomous.
In the first section, we introduce the most common forms of
steady-state responses, namely, equilibrium points (constant
solutions), limit cycles (periodic solutions), tori, and strange

attractors (chaos). The most effective stability analysis tools
are available for the case of equilibrium points, which we
treat in the second section. We make the notions of stability,
instability, and asymptotic stability, introduced earlier, precise and present Lyapunovs method and illustrate it by examples. We treat the special case of linear systems and show
how stability of an equilibrium point of a nonlinear system
can be studied by linearizing the system about this point. We
use the center-manifold theorem to treat the critical case
when linearization fails and end the section with an extension
of Lyapunovs method to nonautonomous systems. An important issue in the analysis of dynamical systems is the effect of
changes in the systems parameters on its behavior. Smooth
changes in the systems behavior are usually studied via sensitivity analysis tools, but when the change in the parameters
results in a change in the qualitative behavior of the system,
like the disappearance of an equilibrium point or a limit cycle,
it is studied via bifurcation theory, which we introduce in the
final section.
STEADY-STATE BEHAVIOR
The steady-state behavior of a system is described by the asymptotic nature of solutions as time becomes large. For nonlinear systems, this will depend on the system as well as on
the initial conditions provided. The possible types of steadystate behavior are more varied than one might think; they
include the well-known constant time behavior (asymptotically stable equilibria) and periodic time behavior (asymptotically stable limit cycles), as well as more complicated behaviors, such as multiperiodic behavior (asymptotically stable
tori) and chaos (strange attractors). We begin with some
definitions that are general enough to capture this range of
possibilities, and then provide some examples.
The steady-state behavior of a system takes place on a subset of the state space called an attractor. The key ingredients
for defining an attractor A are the following: (1) If a solution
is started in A , it never leaves A . That is, A is an invariant
set, defined by saying that for each x(0) x0 A , x(t) A
t. (2) Solutions started sufficiently close to A will approach
A as t . That is, A is locally attractive. (3) A feature of
an attractor A is that it contains a solution that comes arbitrarily close to every point in A at some time. This implies
that A is minimal in the sense that there are no subsets of
A that satisfy conditions (1) and (2).
The domain of attraction (or region, or basin, of attraction)
for an attractor A is defined to be the set of initial conditions
in the state space that are asymptotic to A as t . It can
(at least formally) be constructed by considering a neighborhood UA of A that is used in proving its asymptotic stability,
and taking t0x(t) x(0) UA . This simply starts solutions
in a neighborhood in which one knows they will approach A
in forward time and runs time backward. In this way all solutions that will approach A are collected. Except in simple
problems it is impossible to determine the domain of attraction, although parts of it can often be estimated using Lyapunov methods, as described in the next section. It is important to realize that a nonlinear system may possess multiple
attractors of various types. The domains of attraction for different attractors must, of course, be distinct. Typically, differ-
391
ent domains of attraction are separated by solutions that are

asymptotic to saddle-type invariant sets (that is, invariant
sets that are generally unstable but that have some stable directions).
The most well-known type of attractor is an asymptotically
stable equilibrium point of an autonomous system, x f(x).
An equilibrium point for this system is a point x such that
f(x) 0, representing a constant, time-invariant, steady-state
response. Obviously the solution for the system initiated at
such a point is simply x(t) x, and x is invariant and minimal. The matter of its stability is considered in detail in the
following section.
The next simplest type of attractor is represented by a stable closed trajectory in the state space, called a limit cycle, .
A solution x(t) that lies on is necessarily periodic, since by
starting from a point on , the solution takes time T to traverse the trajectory and return to the starting point, after
which the motion continually repeats. Thus, x(t T) x(t)
for all points on , where T is the period of the limit cycle.
The frequency content of a limit cycle is composed of a fundamental harmonic plus multiples of the fundamental. Limit cycles can arise in autonomous or nonautonomous systems, examples of which follow. A simple autonomous system that
possesses a limit cycle is the following:
r = r(1 r 2 ),
(1)
which is expressed in polar coordinate form. This two-dimensional system has an invariant set (r, ): r 1, [0, 2),
a circle, which attracts all solutions except for the unstable
equilibrium at r 0, and on which solutions wind around
with constant speed leading to a period T 2/. Furthermore, since all points on the circle are visited by every solution started on it, it satisfies all the conditions for an attractor. Such a closed trajectory can, of course, exist in
higher-order systems as well. Another example of an asymptotically stable periodic attractor is offered by the simple linear equation
x = x + cos(t)
(2)
which has a known steady-state solution of the form xss(t)

A cos(t ), which is obviously asymptotically stable for
0 since the transient decays to zero. In order to view this
steady state as a limit cycle, one considers the extended state
space, constructed by supplementing the preceding equation
with the trivial equation and replacing t by in Eq.
(2). (This renders the system autonomous.) The extended
state space, (x, ), is a cylinder, shown in Fig. 1, on which the
=
Figure 1. Limit cycle in a cylindrical state space.
392
steady-state solution is a closed trajectory with period T

2/. While both of these examples have circular limit cycles
around which solutions move at constant speed, this, of
course, is not generally the case.
A steady-state response that is composed of multiple, noncommensurate frequencies corresponds to an asymptotically
stable torus in the state space. Roughly speaking, each angular coordinate on the torus has an associated frequency. (Note
that a limit cycle can be viewed as a one-dimensional torus.)
A simple example of an asymptotically stable torus is given
by a state model that is a simple generalization of that used
for the first limit-cycle example,
r = r(1 r 2 ),
1 = 1 ,
2 = 2
(3)
Tori can exist in Euclidean spaces of dimension three and

higher. It is easy to show that the r 0 solution is unstable
and the solution with r 1, 1 1t 10, 2 2t 20 is
asymptotically stable. Here 1 and 2 represent the frequencies of the steady-state response. Note that if 1 /2 is rational,
then every solution on the torus is periodic and closed, representing a one-parameter family of periodic responses. In contrast, when 1 /2 is irrational, the torus will be covered by a
single solution (a dense solution) for any initial condition.
Thus, the torus satisfies condition (3) for an attractor only in
the incommensurable case. Also, note that in more general
examples, the various rotation speeds are not constant on the
torus, and the tori can be highly distorted. The response of a
torus is generally composed of a set of discrete frequencies
that include 1 and 2, as well as various linear combinations
of them that result from nonlinear interactions. Also, one can
encounter tori with more than two frequencies.
A chaotic steady-state response corresponds to a complicated set in the state space known as a strange attractor.
While chaos is observed in many simulations and experiments, it is virtually impossible to prove the existence of a
strange attractor for a given system model. The essence of
these difficulties lies in the fact that there exist extremely
complicated invariant sets in such systems, and it is not possible to prove that an asymptotically stable periodic solution
does not exist nearby. However, these subtle issues fall outside the main topic of this article. The response of a strange
attractor has a broadband frequency content, which is rather
unexpected for a deterministic system.
Note that the complexity of these attractors is related to
their dimensionality. The simplest, an equilibrium, has dimension 0, the limit cycle has dimension 1, and a torus with
N frequencies has dimension N. It is interesting to note that
a chaotic attractor, if it exists, will have a noninteger, or fractal, dimension due to the rich structure of the invariant manifold on which it exists. The difficulties associated with determining the stability of various types of invariant sets is
similarly related to their dimensionality. For equilibria many
analysis techniques exist, as described in detail in the following. Techniques also exist for limit cycles, tori, and chaos, but
can rarely be applied without computational tools.
functions of x, defined for all x in a domain D n that contains the origin x 0. Suppose the origin is an equilibrium
point of x f(x); that is, f(0) 0. Our goal is to characterize
and study the stability of the origin. There is no loss of generality in taking the equilibrium point at the origin, for any
equilibrium point x 0 can be shifted to the origin via the
change of variables y x x.
The equilibrium point x 0 of x f(x) is stable, if for each
0, there is () 0 such that x(0) implies that
x(t) , for all t 0. It is said to be asymptotically stable if
it is stable and can be chosen such that x(0) implies
that x(t) approaches the origin as t tends to infinity. When
the origin is asymptotically stable, the domain of attraction is
defined as the set of all points x such that the solution of x
f(x) that starts from x at time t 0 approaches the origin as
t tends to . When the domain of attraction is the whole
space n, we say that the origin is globally asymptotically
stable.
Lyapunovs Method
In 1892, Lyapunov introduced a method to determine the stability of equilibrium points without solving the state equation.
Let V(x) be a continuously differentiable scalar function defined in D. A function V(x) is said to be positive definite if
V(0) 0 and V(x) 0 for every x 0. It is said to be positive
semidefinite if V(x) 0 for all x. A function V(x) is said to be
negative definite or negative semidefinite if V(x) is positive
definite or positive semidefinite, respectively. The derivative
(x)
of V along the trajectories of x f(x) is given by V
n
i1(V/xi)xi (V/x)f(x), where V/x is a row vector whose
ith component is V/xi.
Lyapunovs stability theorem states that the origin is stable
if there is a continuously differentiable positive definite function
(x) is negative semidefinite, and it is asymptotiV(x) so that V
(x) is negative definite. A function V(x) satiscally stable if V
fying the conditions for stability is called a Lyapunov function.
The surface V(x) c, for some c 0, is called a Lyapunov
surface or a level surface. Using Lyapunov surfaces, Fig. 2
makes the theorem intuitively clear. It shows Lyapunov
surfaces for decreasing constants c3 c2 c1 0. The condi 0 implies that V(x(t)) decreases along the trajectory
tion V
x(t). Therefore, when a trajectory crosses a Lyapunov surface
V(x) c, it moves inside the set V(x) c and can never come
0, the trajectory moves from one Lyapuout again. When V
nov surface to an inner Lyapunov surface with a smaller c.
As c decreases, the Lyapunov surface V(x) c shrinks to the
origin, showing that the trajectory approaches the origin as
0, we cannot be
time progresses. If we only know that V
c3
c2
V(x) = c1
STABILITY OF EQUILIBRIUM POINTS

We consider the autonomous system x f(x), where the components of the n-dimensional vector f(x) are locally Lipschitz
c1 < c2 < c3
Figure 2. Level surfaces of a Lyapunov function.
sure that the trajectory will approach the origin, but we can
conclude that the origin is stable since the trajectory can be
contained inside any neighborhood of the origin by requiring
the initial state x(0) to lie inside a Lyapunov surface contained in that neighborhood.
(x) is only negative semidefinite, we may still be
When V
able to conclude asymptotic stability of the origin if we can
(x)
show that no solution can stay identically in the set V
0, other than the trivial solution x(t) 0. Under this condition, V(x(t)) must decrease toward 0, and consequently x(t)
0 as t . This extension of the basic theorem is usually
referred to as the invariance principle.
Lyapunov functions can be used to estimate the domain of
attraction of an asymptotically stable origin, that is, to find
sets contained in the domain of attraction. If there is a Lyapunov function that satisfies the conditions of asymptotic stability over a domain D and if V(x) c is bounded and contained
in D, then every trajectory starting in V(x) c remains in
V(x) c and approaches the origin as t . Thus, V(x)
c is an estimate of the domain of attraction. If the Lyapunov
function V(x) is radially unbounded that is, x implies
that V(x) , then any point x n can be included in the
bounded set V(x) c. Therefore, the origin is globally asymptotically stable if there is a continuously differentiable, radially
unbounded function V(x) such that for all x n, V(x) is posi (x) is either negative definite or negative
tive definite and V
semidefinite but no solution can stay identically in the set
(x) 0 other than the trivial solution x(t) 0.
V
Lyapunovs method is a very powerful tool for studying the
stability of equilibrium points. However, there are two drawbacks to the method of which the reader should be aware.
First, there is no systematic method for finding a Lyapunov
function for a given system. In some cases, there are natural
Lyapunov function candidates like energy functions in electrical or mechanical systems (see Example 2). In other cases, it
is basically a matter of trial and error. Second, the conditions
of the method are only sufficient; they are not necessary. Failure of a Lyapunov function candidate to satisfy the conditions
for stability or asymptotic stability does not mean that the
origin is not stable or asymptotically stable.
Example 1. Consider the second-order system
x1 = x1 + x21 x2 ,
x2 = x1 x2
The system has three equilibrium points at (0,0), (1,1), and

(1,1). We want to study the stability of the origin (0,0). We
take the quadratic function
V (x) = ax21 + 2bx1 x2 + dx22 = xT Px
where
P=

a
b
b
d
as a Lyapunov-function candidate. For V(x) to be positive

definite, we must have a 0, d 0, and ad b2 0. The
derivative of V(x) along the trajectories of the system is given
393
by
V (x) = 2(ax1 + bx2 )x1 + 2(bx1 + dx2 )x2

= 2(ax1 + bx2 )(x1 + x21 x2 ) + 2(bx1 + dx2 )(x1 x2 )
Choosing b 0 yields
V (x) = xT Qx + 2ax31 x2
where
a
Q=2
0.5d
0.5d
d
The matrix Q is positive definite when ad d2 /4 0. Choose

a d 1. Near the origin, the quadratic term xTQx domi (x) is negative definates the fourth-order term 2x13x2. Thus, V
nite and the origin is asymptotically stable. Notice that the
origin is not globally asymptotically stable since there are
other equilibrium points. We can use V(x) to estimate the domain of attraction of the origin. The function V(x) is positive
definite for all x. We need to determine a domain D about the
(x) is negative definite and a set V(x) c
origin where V
D, which is bounded. We are interested in the largest set
V(x) c that we can determine, that is, the largest value for
the constant c. Using the inequalities xTQx min(Q)x2
(x) x2
x2 and 2x13x2 x122x1x2 x4, we see that V
4
x . Hence V(x) is negative definite in the domain x 1.

We would like to choose a positive constant c such that
V(x) c is a subset of this domain. Since xTPx
min(P)x2 x2, we can choose c 1. Thus, the set V(x)
1 is an estimate of the domain of attraction.
Example 2. A simple pendulum moving in a vertical plane
can be modeled by the second-order differential equation
ml = mg sin kl
where l is the length of the rod, m is the mass of the bob, is
the angle subtended by the rod and the vertical line through
the pivot point, g is the acceleration due to gravity, and k is
a coefficient of friction. Taking x1 and x2 as the state
variables, we obtain the state equation
x1 = x2 ,
x2 = a sin x1 bx2
where a g/l 0 and b k/m 0. The case b 0 is an

idealized frictionless pendulum. To find the equilibrium
points, we set x1 x2 0 and solve for x1 and x2. The first
equation gives x2 0 and the second one gives sin x1 0.
Thus, the equilibrium points are located at (n, 0), for n 0,
1, 2, . . . . From the physical description of the pendulum
it is clear that the pendulum has only two equilibrium positions corresponding to the equilibrium points (0,0) and (,0).
Other equilibrium points are repetitions of these two positions that correspond to the number of full swings the pendulum would make before it rests at one of the two equilibrium
positions. Let us use Lyapunovs method to study the stability
of the equilibrium point at the origin. As a Lyapunov-function
candidate, we use the energy of the pendulum, which is de-
394
fined as the sum of its potential and kinetic energies, namely,

x1
V (x) =
a sin y dy +
1
2
x22 = a(1 cos x1 ) +
1
2
x22
The reference of the potential energy is chosen such that

V(0) 0. The function V(x) is positive definite over the domain 2 x1 2. The derivative of V(x) along the trajectories of the system is given by
V (x) = ax1 sin x1 + x2 x2 = bx22
(x) 0 and we can conWhen friction is neglected (b 0), V
clude that the origin is stable. Moreover, V(x) is constant
during the motion of the system. Since V(x) c forms a closed
contour around x 0 for small c 0, we see that the trajectory will be confined to one such contour and will not
approach the origin. Hence the origin is not asymptotically
stable. On the other hand, in the case with friction (b 0),
(x) bx22 0 is negative semidefinite and we can conclude
V
(x) is only negative semthat the origin is stable. Notice that V
(x) 0 for x2 0
idefinite and not negative definite because V
irrespective of the value of x1. Therefore, we cannot conclude
asymptotic stability using Lyapunovs stability theorem. Here
comes the role of the invariance principle. Consider the set
(x) 0 x2 0. Suppose that a solution of the state
V
equation stays identically in this set. Then
x2 (t) 0 x2 (t) 0 sin x1 (t) 0
Hence, on the segment x1 of the x2 0 line, the
(x) 0 condition only at the origin
system can maintain the V
x 0. Noting that the solution is confined to a set V(x) c
and, for sufficiently small c, V(x) c x1 , we
conclude that no solution can stay identically in the set
V(x) c x2 0 other than the trivial solution x(t) 0.
Hence, the origin is asymptotically stable. We can also estimate the domain of attraction by the set V(x) c where
c minx1 V(x) 2a is chosen such V(x) c is a closed
contour contained in the strip x1 .
Example 3. Consider the system
x1 = x2 ,
x2 = g1 (x1 ) g2 (x2 )
Linear Systems
The linear time-invariant system x Ax has an equilibrium
point at the origin. The equilibrium point is isolated if and
only if det(A) 0. Stability properties of the origin can be
characterized by the locations of the eigenvalues of the matrix A. Recall from linear system theory that the solution of
x Ax for a given initial state x(0) is given by x(t)
exp(At)x(0) and that for any matrix A there is a nonsingular
matrix P (possibly complex) that transforms A into its Jordan
form; that is,
P1 AP = J = block diag[J1 , J2 , . . ., Jr ]
where Ji is the Jordan block associated with the eigenvalue
i of A. Therefore,
exp(At) = P exp(Jt)P1 =
mi
r
t k1 exp(it)Rik
(4)
i=1 k=1
where mi is the order of the Jordan block associated with the

eigenvalue i. If one of the eigenvalues of A is in the open
right-half complex plane, the corresponding exponential term
exp(it) in Eq. (4) will grow unbounded as t . Therefore,
to guarantee stability we must restrict the eigenvalues to be
in the closed left-half complex plane. But those eigenvalues
on the imaginary axis (if any) could give rise to unbounded
terms if the order of the associated Jordan block is higher
than 1, due to the term tk1 in Eq. (4). Therefore, we must
restrict eigenvalues on the imaginary axis to have Jordan
blocks of order 1. For asymptotic stability of the origin,
exp(At) must approach 0 as t . From Eq. (4), this is the
case if and only if Re(i) 0, i.
When all eigenvalues of A satisfy Re(i) 0, A is called a
Hurwitz matrix. The origin of x Ax is asymptotically stable
if and only if A is a Hurwitz matrix. The asymptotic stability
of the origin can be also investigated using Lyapunovs
method. Consider a quadratic Lyapunov-function candidate
V(x) xTPx where P is a real symmetric positive-definite matrix. The derivative of V along the trajectories of x Ax is
given by
V (x) = xT Px + xT Px = xT (PA + AT P)x = xT Qx
where Q is a symmetric matrix defined by
where g1( ) and g2( ) are locally Lipschitz functions and saty
isfy gi(0) 0, ygi(y) 0 y 0, i 1, 2, and 0 g1(z) dz
, as y . The system has an isolated equilibrium point
at the origin. The equation of this system can be viewed as a
generalized pendulum equation with g2(x2) as the friction
term. Therefore, a Lyapunov-function candidate may be taken
x
as the energylike function V(x) 01 g1(y) dy x22, which is
2
positive definite in and radially unbounded. The derivative
of V(x) along the trajectories of the system is given by
V (x) = g1 (x1 )x2 + x2 [g1 (x1 ) g2 (x2 )] = x2 g2 (x2 ) 0
(x) is negative semidefinite. Note that V
(x) 0 imThus, V
plies x2g2(x2) 0, which implies x2 0. The only solution that
can stay identically in the set x 2x2 0 is the trivial
solution x(t) 0. Thus, by the invariance principle, the origin
is globally asymptotically stable.
PA + AT P = Q
(5)
If Q is positive definite, we can conclude that the origin is

asymptotically stable; that is, A is a Hurwitz matrix. Suppose
we start by choosing Q as a real symmetric positive definite
matrix, and then solve Eq. (5) for P. If Eq. (5) has a positive
definite solution, then again we can conclude that the origin
is asymptotically stable. Equation (5) is called the Lyapunov
equation. It turns out that A is a Hurwitz matrix if and only
if for any given positive definite symmetric matrix Q there exists
a positive definite symmetric matrix P that satisfies the Lyapunov equation (5). Moreover, if A is a Hurwitz matrix, then P is
the unique solution of Eq. (5).
Equation (5) is a linear algebraic equation that can be
solved by rearranging it in the form Mx y, where x and y
are defined by stacking the elements of P and Q in vectors.
Almost all commercial software programs for control systems

include commands for solving the Lyapunov equation.
Linearization
Consider the nonlinear system x f(x) and suppose that f(x)
is continuously differentiable for all x D n. The Jacobian
matrix f /x is an n n matrix whose (i, j) element is
f i /xj. Let A be the Jacobian matrix evaluated at the origin
x 0. By applying the mean-value theorem to each component of f, it can be shown that f(x) Ax g(x), where
g(x)/x 0 as x 0. Suppose A is a Hurwitz matrix and
let P be the solution of the Lyapunov equation (5) for some
positive definite Q. Taking V(x) xTPx, it can be shown that
(x) is negative definite in the neighborhood of the origin.
V
Hence, the origin is asymptotically stable if all the eigenvalues
of A have negative real parts. Using some advanced results of
Lyapunov stability, it can be also shown that the origin is
unstable if one (or more) of the eigenvalues of A has a positive
real part. This provides us with a simple procedure for determining stability of the origin of a nonlinear system by calculating the eigenvalues of its linearization about the origin.
Note, however, that linearization fails when Re(i) 0 for all
i, with Re(i) 0 for some i.
Example 4. The pendulum equation has two equilibrium
points at (0,0) and (,0). Let us investigate stability of each
point using linearization. The Jacobian matrix is given by
f
1
x1
f
=
f
x
2
x1
f1

0
x2
=
a
cos
x1
f2
x2

1
b
To determine stability of the origin we evaluate the Jacobian

at x 0, to obtain

0
f
A=
=
x x=0
a
1
b
The eigenvalues of A are 1,2 b/2 b2 4a. For all

positive values of a and b, the eigenvalues satisfy Re(i) 0.
Hence, the equilibrium point at the origin is asymptotically
stable. In the absence of friction (b 0), both eigenvalues are
on the imaginary axis. In this case we cannot determine the
stability of the origin through linearization. We have seen
before that in this case the origin is a stable equilibrium
point, as determined by an energy Lyapunov function. To determine the stability of the equilibrium point at (,0), we evaluate the Jacobian at this point. This is equivalent to performing a change of variables z1 x1 , z2 x2 to shift the
equilibrium to the origin, and then evaluating the Jacobian
f /z at z 0.

f
A =
x x
1 = , x 2 =0
0
=
a
1
b
are 1,2 b/2 b2 4a. For all a

The eigenvalues of A
0 and b 0, there is one eigenvalue in the open right-half
plane. Hence, the equilibrium point at (,0) is unstable.
395
The Center-Manifold Theorem

When the Jacobian of an equilibrium point has one or more
eigenvalues with zero real parts and all other eigenvalues
with negative real parts, the stability of the equilibrium cannot be ascertained from linearization. In these cases, the local
nonlinear nature of the system will dictate the stability of the
equilibrium, and the center-manifold theorem allows one to
determine precisely the nature of the nonlinear terms that
determine the stability. The main idea behind this technique
is that the critical behavior occurs in a low-dimensional invariant manifold of the state space, one with dimension equal
to the number of eigenvalues with zero real parts. The stability in the other dimensions is dominated by the exponential
behavior associated with the eigenvalues that have negative
real parts, but the nonlinear coupling between the marginal
and asymptotically stable modes can play a critical role in
determining stability. The center-manifold theorem makes
these ideas precise.
We begin with the following motivating example:
y = zy y 3 ,
z = z + ay 2
(6)
Here, from the linearization point of view, the z dynamics are

asymptotically stable and y is neutral. Based on this, one
might be tempted to make the assumption that z 0, and
therefore y is governed by y y3, and thus the origin is
asymptotically stable. However, as is shown in an example
below, this is incorrect; the stability of the origin is dictated
by the sign of (a 1). The problem with the naive assumption
made previously is that z approaches something small, but
nonzero, and the correction, which stems from the nonlinear
coupling terms and is captured by the center manifold, is crucial for determining stability.
The development of the center manifold technique begins
with the autonomous system x f(x), which has an equilibrium at x 0. The Jacobian A is defined as before and the
state equation is written as x Ax g(x), where g(x)
f(x) Ax contains terms that are essentially nonlinear about
the origin. In order to split the dynamics into linearly asymptotically stable and neutral parts, the linear part of the equation is put into real Jordan form via a matrix P, as follows:

A1 0
1
(7)
J = P AP =
0 A2
where all eigenvalues of A1 have zero real parts and all eigenvalues of A2 have negative real parts. The coordinate transformation

y
P
=x
z
then puts the state equation into the split form
y = A1 y + g1 ( y, z),
z = A2 z + g2 ( y, z)
(8)
The z component of the dynamics of this system is dominated

by the relatively fast linear system z A2z, whereas the y
dynamics are slower than any exponential order. The key to
the center-manifold technique is to capture the small, but crucial, coupling effects correctly in the nonlinear terms.
396
A center manifold for this system is simply a smooth invariant manifold of the form z h(y) with h(0) 0 and
(h/y)(0) 0. Under some smoothness conditions on the nonlinear terms in Eq. (8) (that are inherited from the original
equation), a local center manifold exists, although it is not in
general unique. The power of the center manifold is that it
can be used to reduce the dimensionality of the problem, as
follows. By restricting the system dynamics to the center
manifold one obtains y A1y g1(y, h(y)), referred to as the
reduced system. The center manifold theorem states that if the
origin of the reduced system is asymptotically stable (unstable),
then the origin of the original system x f(x) is likewise asymptotically stable (unstable).
The construction of the center manifold can be carried
out as follows. We take the time derivative of z h(y) to obtain z [h(y)/y]y. Equation (8) is used to substitute in for
z and y, and z is replaced everywhere by h(y). This leads to
A2h(y) g2(y, h(y)) [h(y)/y][A1y g1(y, h(y))]. This equation for h(y), which must satisfy the conditions h(0) 0 and
h/y(0) 0, is generally impossible to solve. However, since
only local information is needed for stability considerations,
an approximation for h(y) can be obtained by assuming a series expansion for h(y), substituting it into the equation, and
matching coefficients, as demonstrated in the forthcoming Example 5. Once the expansion form for h(y) is determined and
the expanded version of the reduced equation is in hand, various techniques can be employed for determining the stability
of the reduced system. In general, this task is made much
easier due to the lower dimensionality of the reduced system.
Example 5. Consider the system given in Eq. (6). Here A1
0, A2 1, g1(y,z) yz y3, and g2(y,z) ay2. The center
manifold is assumed to be of the form h(y) c1y2 c2y3
. This is substituted into the equation for h(y) and expanded in powers of y, and the coefficients of y2, y3, etc., are
gathered and solved. This leads to the result that c1 a and
c2 0. Therefore, h(y) ay2 O(y4). [We use the notation
f(y) O(yp) when f(y) kyp for sufficiently small y.] The
reduced system is given by taking the equation for y and
simply replacing z by the expansion for h(y), resulting in y
(a 1)y3 O(y5). Thus, the conclusion is reached that for
a 1 0, x 0 is an asymptotically stable equilibrium point
for system Eq. (6), while for a 1 0 the origin is unstable.
For a 1 0 no conclusions regarding stability can be drawn
without considering higher-order expansions.
Nonautonomous Systems
Suppose the origin x 0 is an equilibrium point of the nonautonomous system x f(t, x); that is, f(t, 0) 0 for all t 0.
For nonautonomous systems we allow the Lyapunov-function
candidate V to depend on t. Let V(t, x) be a continuously differentiable function defined for all t 0 and all x D. The
derivative of V along the trajectories of x f(t, x) is given by
(t, x) V/t (V/x) f(t, x). If there are positive-definite
V
functions W1(x), W2(x), and W3(x) such that
W1 (x) V (t, x) W2 (x),
V (t, x) W3 (x)
(9)
for all t 0 and all x D, then the origin is uniformly asymptotically stable, where uniformly indicates that the
definition of stability and the convergence of x(t) to zero are
independent of the initial time t0. Such uniformity annotation

is not needed with autonomous systems since the solution of
an autonomous state equation starting at time t0 depends
only on the difference t t0, which is not the case for nonautonomous systems. If inequalities in Eq. (9) hold globally and
W1(x) is radially unbounded, then the origin is globally uniformly asymptotically stable.
Example 6. Consider the nonautonomous system
x1 = x1 g(t)x2 ,
x2 = x1 x2
where g(t) is continuously differentiable and satisfies 0

g(t) k and g g(t) for all t 0. The system has an equilibrium point at the origin. Consider a Lyapunov-function candidate V(t, x) x12 [1 g(t)]x22. The function V satisfies the
inequalities
x 21 + x 22 V (t, x) x 21 + (1 + k)x 22
The derivative of V along the trajectories of the system is
given by
V = 2 x21 + 2 x1 x2 [2 + 2 g(t) g(t)]

x22
Using the bound on g(t), we have 2 2g(t) g(t) 2
2g(t) g(t) 2. Therefore

V 2 x 21 + 2 x1 x2 2 x 22 = xT
2
1

1
x = xT Qx
2
The matrix Q is positive definite. Hence the origin is uniformly asymptotically stable. Since all inequalities are satisfied globally and x12 x22 is radially unbounded, the origin is
globally uniformly asymptotically stable.
BIFURCATION THEORY
The term bifurcation, strictly speaking, refers to the splitting
of a whole into two parts. While this is relevant to its meaning in dynamical systems, it has taken on a much broader
definition. The general bifurcation problem deals with qualitative changes in system behavior as parameters are varied
in a quasistatic manner. The simplest example is the case
when a parameter is varied in such a manner that the real
part of an eigenvalue of the Jacobian at an equilibrium point
changes sign, corresponding to the change in stability of the
equilibrium. There are two generic ways in which such transitions can occur. The first is a real eigenvalue passing through
zero; the second is a complex conjugate pair of eigenvalues
passing through the imaginary axis. While linearization and
the center-manifold theory allow one to determine the stability of the equilibrium point, the larger question looms as to
what changes take place near the equilibrium through such a
transition. Questions such as these are the basic motivation
behind bifurcation theory.
Consider the system x f(x, ), where represents a system parameter. The technical definition of a bifurcation is as
follows: A bifurcation is said to occur at 0 if the state space
for 0 is not topologically equivalent to that for 0. It
is said that 0 is the bifurcation value of the parameter. The
reason for emphasizing topological equivalence in the state

space is that one is interested in transitions that cause qualitative changes in the structure of the system response. For
example, a smooth increase in the amplitude of a limit cycle
is not a bifurcation, but the disappearance of a limit cycle is.
A very important feature of bifurcation theory is that it
allows for one to build a catalog of the generic qualitative
changes systems may undergo as parameters are varied. For
some simple, but widely encountered cases, this classification
is complete. However, there are many important issues that
remain unresolved. At the end of this section we will further
address these classifications. The first convenient classification of bifurcations is to separate them into local and global.
Local bifurcations describe changes in a neighborhood of an
equilibrium, whereas global bifurcations involve nonlocal
changes. (It is interesting to note that the study of local bifurcations in which two parameters are varied requires knowledge of global bifurcations.) We focus our attention on the
analysis of local bifurcations and demonstrate some global bifurcations at the end of the discussion.
Center-manifold theory is a powerful tool for analyzing
these local bifurcations. It, along with normal-form theory (described briefly later), allows one to reduce local bifurcations
to their simplest possible forms, facilitating the classification
mentioned above. To fix the ideas, consider the original system x f(x, ) augmented by the trivial dynamics of the parameter 0. Clearly the dynamics are linearly neutral
and have a zero eigenvalue (the case of multiple parameters
goes through in the same manner). The center-manifold theory described earlier is redeveloped by carrying along the
equation 0. The result is that the slow dynamics now
contain y and and the center manifold is of the form z
h(y, ). The procedure is carried out as before, with the key
observation that is considered as a state variable, that is,
terms such as y are taken to be nonlinear. [The equation for
h(y, ) simplifies to that of the original case since 0, with
carried along in a straightforward manner.] This results in
a reduced system describing the y dynamics that depends on
the parameter, along with 0. The equations are valid for
a range of values near the bifurcation value, and therefore
this process allows for the unfolding of the behavior of the
system about the bifurcation.
Example 7. Consider the following parameterized version of
the system given in Eq. (6):
= 0,
y = y + zy y 3 ,
z = z + ay2
(10)
Here A1 0 (the 2 2 zero matrix), A2 1, g1(, y, z) (0,

y yz y3)T, and g2(, y, z) ay2. Note that is treated as
a state variable. The center manifold is assumed to be of the
form h(, y) b1y2 b2y b32 . This is substituted
into the equation for h(, y) and expanded in powers of y and
and the coefficients of y2, y, 2, etc., are gathered and
solved. This leads to the result that b1 a, b2 0, and b3
0. Therefore h(, y) ay2 , and the reduced system is
given by taking the equation for y and simply replacing z by
the expansion for h(, y), resulting in y y (a 1)y3
, which is valid in some neighborhood of (, y) (0,0).
Clearly, for 0 the stability of the origin of the original
system is dictated by the sign of ; this is known from linearization. The center-manifold results confirm this, but offer
397
y
a >1
a <1
Figure 3. Subcritical pitchfork bifurcation (left); supercritical pitchfork bifurcation (right).
much more. It is seen that the approximate reduced equation

has equilibria at y 0 and /(1 a). It is not difficult to
show that the nonzero equilibria have the following properties: for a 1 they exist for 0 and are unstable when
they exist, while for a 1 they exist for 0 and are asymptotically stable when they exist. Thus, for a 1 there exist
two unstable equilibria near the asymptotically stable origin
for 0, and they collapse onto the origin and disappear at
0, leaving the origin unstable for 0. This subcritical
pitchfork bifurcation is shown in Fig. 3. The case for a 1, a
supercritical pitchfork bifurcation, is shown in the same figure, from which the reader can infer the dynamic behavior as
is varied through zero.
This example points out some important features of local
bifurcations. First, bifurcations that involve n eigenvalues
with zero real parts require analysis of an nth-order dynamical system, obtained by reduction to the parametrized center
manifold. Second, one can easily extend the ideas to include
more than one parameter. The selection of the proper parameters to uncover all critical behavior near a bifurcation is a
subtle matter that involves techniques beyond the scope of
this article. Third, the analysis of this example is very
straightforward, as there is only one nonlinear term in the
approximate reduced equation. In more complicated problems
several nonlinear terms may occur, rendering the stability or
bifurcation analysis much more difficult. In such cases one
can use normal-form theory to simplify the problem. This
technique, which shares many similarities with feedback linearization, involves a systematic sequence of coordinate
changes that remove as many nonlinear terms as is possible
for the given system. This technique, when the bifurcation
parameters are included, allows one to systematically reduce
entire classes of problems into generic forms. This produces
the classification scheme described above. Here we offer the
results of this classification for the simplest bifurcations. As
an eigenvalue of an equilibrium passes through zero, there
are three generic things that can occur. The most general is
the saddle-node bifurcation, in which a pair of equilibria
merge and annihilate one another. If all other eigenvalues
of the equilibrium of interest are stable, then the bifurcation
diagram as the parameter is varied is as shown in Fig. 4. If
Figure 4. Saddle-node bifurcation (left); transcritical bifurcation

(right).
398
STACK SIMULATION
the equilibrium persists through the instability, a transcritical bifurcation, shown in Fig. 4, generally occurs. When a special symmetry in the problem exists, the pitchfork bifurcation
encountered in the preceding example can occur. These cover
the generic bifurcations involving a single eigenvalue going
through zero. When a complex conjugate pair of eigenvalues
passes through the imaginary axis, something more interesting happens. First, the center-manifold reduction for this
problem leads to a dynamical system consisting of two equations. When expressed in polar coordinates these are r
r ar3 and br2 . The radial variable,
representing the amplitude of the oscillation, undergoes a
pitchfork bifurcation (although r 0 is meaningless) that is
super- (a 0) or sub- (a 0) critical as is increased through
zero. The angular variable simply describes rotation at a
nominal frequency of with a small amplitude-dependent
shift arising from nonlinear effects. This is the Hopf bifurcation. The result of this bifurcation is shown in Fig. 5, and it
results in the birth of stable limit cycles as an asymptotically
stable equilibrium goes unstable (for a 0) or in the merging
of an asymptotically stable equilibrium with an unstable limit
cycle (for a 0). This completes the list of bifurcations that
generically occur as a single parameter is varied and an equilibrium point changes stability.
Center-manifold and normal-form methods also exist for
analyzing the bifurcations that occur when limit cycles
change stability. These problems can be handled by defining
and using a Poincare map near the limit cycle and studying
the generic bifurcations of fixed points of maps. It is found
that limit cycles can undergo saddle-node, transcritical, and
pitchfork bifurcations that are essentially the same as shown
in Figs. 3 and 4. The analogy to the Hopf bifurcation, the
Neimark-Sacker bifurcation, involves a limit cycle changing
stability and merging with a two-dimensional torus around it
(in a super- or subcritical manner). However, this situation is
complicated by the fact that resonances can occur between the
two frequencies of the torus, and this can lead to secondary
nonlinear phenomena. In addition, limit cycles have an additional bifurcation, the period-doubling bifurcation, in which a
limit cycle changes stability and merges with another limit
cycle that has a period twice that of the original. Again, this
can be super- or subcritical in nature.
Before leaving the topic of local bifurcations it should be
noted that if several parameters are being simultaneously
varied, one can encounter situations in which several eigenvalues simultaneously have zero real parts. Similarly, one
may find that some critical terms in a given normal form
change sign as parameters are varied (for example, the parameter a in the Hopf bifurcation). The bifurcation analysis
of such problems is extremely rich and often exceedingly complicated, but some classifications along these lines have been
carried out for two and three parameter systems.
Global bifurcations result in qualitative changes in the
state space that cannot be described as local to any equilib-
Figure 5. Hopf bifurcation.
Figure 6. Saddle connection global bifurcation.
rium. Perhaps the simplest interesting example of a global

bifurcation is the saddle connection shown in Fig. 6. Here, as
a parameter is varied in the two-dimensional state space, a
stable limit cycle moves toward and eventually merges with
a saddle point, forming a saddle loop. As the parameter is
pushed beyond this point, the limit cycle disappears. Note
that this occurs without any changes to the stability types of
the saddle point or the limit cycle. Global bifurcations involving saddle connections in systems of dimension three and
higher are the source of chaos in dynamical systems.
FURTHER READING
For further reading on the Lyapunov stability and its applications, we refer the reader to Refs. 1 and 2. As for the steadystate behavior of dynamical systems, chaos, and bifurcations,
we recommend Refs. 35.
BIBLIOGRAPHY
1. H. K. Khalil, Nonlinear Systems, 2nd ed., Upper Saddle River, NJ:
Prentice Hall, 1996.
2. M. Vidyasagar, Nonlinear Systems Analysis, 2nd ed., Englewood
3. S. H. Strogatz, Nonlinear Dynamics and Chaos, Reading, MA: Addison-Wesley, 1994.
4. J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, 2nd ed., New York:
5. D. K. Arrowsmith and C. M. Place, An Introduction to Dynamical
Systems, Cambridge, UK: Cambridge University Press, 1990.
HASSAN K. KHALIL
STEVEN W. SHAW
Michigan State University
STABILIZATION. See BILINEAR SYSTEMS.
357
STABILITY THEORY: INCLUDING

SATURATION EFFECTS
The basic concept of the stability of dynamical systems is
carefully developed, and its application to the design of feedback control systems is presented. Classical stability theories
for linear and nonlinear feedback control systems are discussed and the Lyapunov and RouthHurwitz stability criteria are developed. Next, attention is given to conventional
control design and to frequency-domain methods: The root locus and Nyquist stability criteria are presented, and the Bode
plot and Nichols chart-based methods for stability and for degree of stability determination are discussed. The emphasis
then shifts to the stability of control systems comprising linear plants and actuation elements which are subject to saturation. Then, nonlinear controllers are called for. Moreover,
the broader question of bounded input and bounded output
stability is addressed. Thus, a comprehensive time-domain
358
design methodology for nonlinear tracking controllers for actuator saturation effects mitigation, is presented.
LYAPUNOV STABILITY CRITERION
Stability is a very important characteristic of a system. If a
system is linear and time-invariant, several criteria such as
the Nyquist and Routhian stability criteria may be applied to
determine stability. However, for nonlinear and time-varying
systems, the Lyapunov direct method is applicable. Its advantage is that it is not necessary to solve the system equation,
which is generally very difficult.
The system with zero input is defined by
x = f(x, t)
(1)
and contains the n-dimensional response vector x x1, x2,

. . ., xn. The vector f(x, t) is a function of x and t. The solution of Eq. (1) is a function of the initial condition x(0) and
the initial time t0 and is given by
x = (t, x(0), t0 )
x = f(x, u, t)
(2)
The length of the vector x can be measured by the Euclidean

norm (1)
x = (x21 + x22 + + x2n )1/2
(3)
The system of Eq. (1) is uniformly stable (2) when there is a

finite positive constant such that, for any x(0) and t0, the
solution of Eq. (2) satisfies the condition
x x0 ,
region . Note that with this definition it is not necessary for

the trajectory to approach the equilibrium point. It is necessary only for the trajectory to stay within the region . This
permits the existence of a continuous oscillation about the
equilibrium point. The state-space trajectory for such an oscillation is a closed path called a limit cycle. The performance
specifications for a control system must be used to determine
whether or not a limit cycle can be permitted. The amplitude
and frequency of the oscillation may influence whether it represents acceptable performance. When limit cycles are not acceptable, more stringent restraints must be chosen to exclude
their possible existence (25).
Limit cycles are usually characterized as stable or unstable. If a limit cycle is stable, it means that trajectories in the
state space on either side will approach the limit cycle. An
unstable limit cycle is one in which the trajectories on either
side diverge from the limit cycle. These trajectories may approach other limit cycles or equilibrium points.
In the case that there is an input u(t), the system is defined by
t t0
(4)
where x0 represents the equilibrium solution as t . The

vector x0 may be a point or it may be a continuous oscillation
(limit cycle). Figure 1 represents the graphical plots of several
possible solutions of Eq. (2) for an initial condition x(0) and a
finite final value x0.
Stability in the Sense of Lyapunov
Consider a region in the state space enclosing an equilibrium point x0. This equilibrium point is stable provided that
there is a region (), which is contained within such that
any trajectory (3) starting in the region does not leave the
(5)
In linear systems, stability is a system characteristic which is

independent of any initial condition x(0) and/or the input
magnitude u(t). However, in nonlinear systems the stability
of the system may depend on the initial condition and/or the
magnitude of the input (4). This case is typical for control
systems and is considered later in this article.
An important case is a system described by bounded-inputbounded-output (BIBO) stability. In such a system, either linear or nonlinear, the output response to any bounded
input is also bounded. For a linear time-invariant (LTI) system, the necessary and sufficient condition for stability is that
(5,6) the integral of the system weighting function or impulse
response must be finite. This is expressed by

|h(t)| dt <
(6)
An equivalent condition for an LTI system is that all eigenvalues (3) are located in the left-half plane. In that case, the
transient response associated with those eigenvalues decrease with time, and the output response therefore approaches the particular solution which is determined by the
input.
Asymptotic Stability
b
c
x(0)
x0
Figure 1. State-plane trajectories indicating (a) Lyapunov stability;

(b) asymptotic stability; and (c) instability.
An equilibrium point is asymptotically stable if, in addition to

being stable in the sense of Lyapunov, all trajectories approach the equilibrium point. This means that the perturbation solution x*(t) approaches 0 as time t approaches infinity.
This is the stability definition usually used in control-system
design.
Figure 1 shows trajectories in the state plane illustrating
the general principle of Lyapunov stability (3), asymptotic
stability, and instability. The trajectory a starting at the initial state point x(0) and remaining within the region meets
the conditions for Lyapunov stability. This trajectory is
closed, indicating a continuous output oscillation or limit cycle. Trajectory b terminates at the equilibrium point x0; thus
it represents asymptotic stability. Trajectory c leaves the region and therefore indicates instability.
When the region includes the entire state space, the
definitions of Lyapunov and asymptotic stability are said to
apply in a global sense. The stability or instability of a linear
system is global because any initial state yields the same
stability determination. A stable linear system is globally
asymptotically stable.
The technique for linerarizing nonlinear differential equations in the neighborhood of their singularities or equilibrium
points is presented in the section entitled Linearization (Jacobian Matrix). The validity of determining stability of the
unperturbed solution near the singular points from the linearized equations was developed independently by Poincare
and Lyapunov in 1892. Lyapunov (4,7) designated this as the
first method. This stability determination is applicable only in
a small region near the singularity and results in stability in
the small. The section on linearization considers Lyapunovs
second method, which is used to determine stability in the
large. This larger region may include a finite portion, or sometimes the whole region, of the state space.
359
This can be written more compactly as
f (x, y) =
n
n

i=1 j=1
a i j xi y j
a11
a
= [x1 x2 xn ] 21

an1
a12
a22
an2

y1
a1n
y
a2n 2

ann
yn
(11)
= xT Ay = x, Ay
The matrix A is called the coefficient matrix of the bilinear

form, and the rank of A is called the rank of the bilinear form.
A bilinear form is called symmetric if the matrix A is symmetric.
Quadratic Form. A quadratic form V is a real homogeneous
polynomial in the real variables x1, x2, . . ., xn of the form
V =
n
n
a i j xi x j
(12)
i=1 j=1
where all aij are real. This is the special case of Eq. (11)
where x y. The quadratic form V can therefore be expressed
as the inner product
QUADRATIC FORMS
Some of the techniques used in determining stability of control systems and for optimizing their response utilize scalar
functions expressed in quadratic form. The necessary background for expressing functions in quadratic form is developed in this section. Then some important properties of quadratic forms are presented.
Conjugate Matrix. The elements of a matrix may be complex quantities. For example, a matrix A may have the elements aij ij jij. A conjugate matrix B has elements with
the same real component and with imaginary components of
the opposite sign; that is, bij ij jij. This conjugate property is expressed by
B = A
(7)
Inner Product. The inner product is also called a scalar (or

dot) product since it yields a scalar function. The scalar product of vectors x and y is defined by
x, y
= (x )T y = yT x = x1 y1 + x2 y2 + + xn yn
(8)
When x and y are real vectors, the inner product becomes

x, y
= x1 y1 + x2 y2 + + xn yn
(9)
Bilinear Form. A scalar homogeneous expression containing

the product of the elements of vectors x and y is called a bilinear form in the variables xij and yij. When they are both of
order n, the most general bilinear form in x and y is
f (x, y) = a11 x1 y1 + a12 x1 y2 + + a1n x1 yn

+ a21 x2 y1 + a22 x2 y2 + + a2n x2 yn
+
+ an1 xn y1 + an2 xn y2 + + ann xn yn
V (x) = xT Ax = x, Ax
A homogeneous polynomial can always be expressed in terms

of a symmetric matrix A. In the expansion of Eq. (12) the
cross-product terms (i j) all have the form (aij aji)xixj.
Choosing aij aji makes the matrix A for the quadratic form
symmetric. This is illustrated in Eq. (14).
V (x) = x21 4x22 + 5x23 + 6x1 x2 20x2 x3
1
3
0
= x T 3
4 10 x = xT Ax
0 10
5
(14)
The rank of the matrix A is called the rank of the quadratic

form. If the rank of A is r n, the quadratic form is singular.
If the rank of A is n, the quadratic form is nonsingular. A
nonsymmetric matrix can be converted into an equivlent symmetric matrix by replacing all sets of elements aij and aji by
the average value (aij aji)/2. The principal diagonal will be
preserved.
Length of a Vector. The length of a vector x is called the
Euclidean norm and is denoted by x. It is defined as the
square root of the inner product x, x. For real vectors it is
given by

x = x, x
= x21 + x22 + + x2n
(15)
A vector can be normalized so that its length is unity. In that
case it is called a unit vector. The unit vector may be denoted
by x and is obtained by dividing each element of x by x:
x =
(10)
(13)
x
x
(16)
Principal Minor. A principal minor of a matrix A is obtained by deleting any row(s) and the same numbered col-
360
umn(s). The diagonal elements of a principal minor of A are

therefore also diagonal elements of A. The determinant A is
classified as a principal minor, with no rows and columns deleted. The number of principal minors can be determined
from a Pascal triangle. The number of principal minors for a
matrix A of order n is: 1 for n 1, 3 for n 2, 7 for n 3,
and 15 for n 4, . . ..
Leading Principal Minor. There are n leading principal minors, formed by all the square arrays within A that contain
a11. Starting with [a11], the next square array is formed by
including the next row and column. This process is continued
until the matrix A is obtained. The leading principal minors
are given by
1 = |a11 |
123

a
11

= a21

a31
12
a12
a22
a32

a

= 11
a21

a13

a23 12n = |A|

a33

a12

a22
(17)
The subscripts of are the rows and columns of A used to

form the minor. The minor contains the common elements of
these rows and columns.
Definiteness and Semidefiniteness. The sign definiteness for
a scalar function of a vector, such as V(x), is defined for a
spherical region S about the origin described by x K (a
constant equal to the radius of S). The function V(x) and all
V(x)/xi for i 1, 2, . . ., n must be continuous within S.
The definiteness of a quadratic form is determined by analyzing only the symmetric A matrix. If A is given as a nonsymmetric matrix, it must first be converted to a symmetric
matrix.
Positive Definite (PD). A scalar function, such as the quadratic form V(x) x, Ax, is called positive definite when
V(x) 0 for x 0 and V(x) 0 for all other x K. The
positive definite condition requires that A 0; thus, the
rank of A is equal to n. The rank of A may be determined by
putting it in hermite normal form. When a real quadratic
form V(x) xTAx is positive definite, then the matrix A is
also called positive definite. The matrix A has the property
that it is positive definite iff there exists a nonsingular matrix H such that A HTH.
The definiteness of the matrix A can be determined by reducing this matrix to diagonal form by means of a congruent
transformation such that the diagonal matrix B PTAP. The
matrix P may be formed by using the following elementary
transformations:
1. Interchanging the ith and jth rows and interchanging
the ith and jth columns.
2. Multiplying the ith row and column by a nonzero scalar
k.
3. Addition to the elements of the ith row of the corresponding elements of the jth row multiplied by a scalar
k and addition to the ith column of the jth column
multiplied by k.
If the principal diagonal of the matrix B contains only positive, nonzero elements, then the matrix A is positive definite.
Another method of diagonalizing the A matrix is to determine a modal matrix T (8). In that case the diagonalized ma-
trix contains the eigenvalues obtained from I A 0. In

order for the matrix A to be positive definite, all the eigenvalues must be positive.
An alternate method of determining positive definiteness
is to calculate all the leading principal minors of the matrix
A. If all the leading principal minors of A are positive, the
real quadratic form is positive definite. (This is sometimes
called the Sylvester theorem.) Note that when the leading
principal minors are all positive, then all the other principal
minors are also positive.
Positive Semidefinite (PSD). The quadratic form V(x) is
called positive semidefinite when V(x) 0 for x 0 and
V(x) 0 for all x K. The function V(x) is permitted to
equal zero at points in S other than the origin (i.e., for some
x 0), but it may not be negative. In this case A 0, the
rank of A is less than n, and all of the eigenvalues of A are
positive. The positive semidefinite quadratic form can therefore be reduced to the form y 12 y 22 y r2, where r
n. When V(x) is positive semidefinite, then the matrix A is
also called positive semidefinite. The matrix A is positive
semidefinite iff all its characteristic values are 0. In order
to be positive semidefinite, all the principal minors (9) must
be nonnegative.
Negative Definite (ND) and Negative Semidefinite (NSD). The
definitions of negative definite and negative semidefinite follow directly from the definitions above when the inequalities
are applied to A. If a matrix A does not satisfy the conditions for positive definiteness or positive semidefiniteness,
then A is checked for these conditions. If A satisfies the
positive definite or positive semidefinite conditions, then A is
said to be negative definite or negative semidefinite, respectively.
Indefinite. A scalar function V(x) is indefinite if it assumes
both positive and negative values within the region S described by x K. If A is not positive definite, positive semidefinite, negative definite, or negative semidefinite, then it is
said to be indefinite. Note that when A, in general form, has
both positive and negative elements along the principal diagonal, it is indefinite. When A is transformed to a diagonal form,
it is indefinite if some of the diagonal elements are positive
and some are negative. This means that some of the eigenvalues are positive and some are negative.
Example 1. Use the principal
definiteness of
1 1
A = 1 1
1 1
minors to determine the
1
0
(18)
The leading principal minors are evaluated to check for positive definiteness:

1 1 1

1 1

= 0,
1 = 1,
12 =

=
1
1
1

==0
123

1 1
1 1 0
The conditions for positive definiteness are not satisfied. Next
the remaining principal minors are evaluated to check for
positive semidefiniteness.

1 1
1 1
= 1,

13 =

=
23
1 0 = 1
1 0
2 = 1,
3 = 0
x2
Since the principal minors are not all nonnegative, the matrix A is not positive semidefinite. Similarly, A is not negative
definite or negative semidefinite; therefore A is indefinite.
Example 2. Transform A in Eq. (18) to the diagonal Jordan
form and check its definiteness. The characteristic equation
is

1
1
1

|II A| = 1
1 1 = 3 22 2 = 0
(19)

1
1

The eigenvalues are 1 2.732, 2 0.732, and 3 0. Thus
the matrix A is indefinite since
2.732
= 0
0
0
0.732
0
0
0
A linear system with no forcing function (an autonomous system) and with A 0 has only one equilibrium point x0. A
nonlinear system, on the other hand, may have more than
one equilibrium point. This is easily illustrated by considering
the unity-feedback angular position-control system shown in
Fig. 2. The feedback action is provided by synchros which generate the actuating signal e sin(i o). With no input,
i 0, the differential equation of the system is
o + ao + K sin o = 0
(20)
x2 = K sin x1 ax2
(21)
The slope of the trajectories in the phase plane (25) is obtained from
x2
K sin x1 ax2
=
x1
x2
(22)
In the general case the unforced state equation is x f(x).

Since equilibrium points x0 exist at x f(x0) 0, the singularities are x2 0 and x1 k, where k is an integer. The
system therefore has multiple equilibrium points.
In a small neighborhood about each of the equilibrium
points, a nonlinear system behaves like a linear system. The
states can therefore be written as x x0 x*, where x* represents the perturbation or state deviation from the equilibrium point x0. Each of the elements of f(x) can be expanded
f( e)
K
s(s + a)
Saddle point
Saddle point
x1
Separatix
in a Taylor series about one of the equilibrium points x0. Assuming that x* is restricted to a small neighborhood of the
equilibrium point, the higher-order terms in the Taylor series
may be neglected. Thus, the resulting linear variational state
equation is
f1
f1 f1
x
xn
1 x2
f2 f2
f
2
x1 x2
xn
x =
x = Jx x
(23)
fn
fn fn
x1 x2
xn
x= x 0
This is obviously a nonlinear differential equation because of

the term sin o. With the phase variables x1 o and x2
x1 o, the corresponding state equations are
N=
Node
Figure 3. Phase portrait for Eq. (20).
LINEARIZATION (JACOBIAN MATRIX) (35,7,10)
x1 = x2 ,
361
Synchro
Figure 2. A nonlinear feedback control system.
where f i is the ith row of f(x) and Jx f /xT is called the

Jacobian matrix and is evaluated at x0.
For the system of Eq. (21) the motion about the equilibrium point x1 x2 0 is represented by

0
1 x1
x1
= Jx x
x = =
(24)
K a x1
x2
For these linearized equations the eigenvalues are 1,2
a/2 (a/2)2 K. This equilibrium point is stable and is
either a node or a focus, depending upon the magnitudes of a
and K. A node denotes an overdamped response and a focus
denotes an underdamped response about the equilibrium
point. For motion about the equilibrium point x1 , x2 0,
the state equation is

0
1
x =
x
(25)
K a
The eigenvalues of Jx are 1,2 a/2 (a/2)2 K. Thus, one
eigenvalue is positive and the other is negative, and the equilibrium point represents an unstable saddle point. The motion
around the saddle point is considered unstable since every
point on all trajectories, except on the two separatrices, moves
away from this equilibrium point. A phase-plane portrait for
this system can be obtained by the method of isoclines (25).
From Eq. (22) the isocline equation is x2 K sin x1 /(N a).
The phase portrait is shown in Fig. 3. Note that the linearized
362
equations are applicable only in the neighborhood of the singular points. Thus, they describe stability in the small.
In analyzing the system performance in the vicinity of
equilibrium, it is usually convenient to translate the origin of
the state space to that point. This is done by inserting x
x0 x* into the original equations. With the origin at x0, the
variations from this point are described by x*.
SECOND METHOD OF LYAPUNOV
The second, or direct, method of Lyapunov provides a means
for determining the stability of a system without explicitly
solving for the trajectories in the state space. This is in contrast to the first method of Lyapunov, which requires the determination of the eigenvalues from the linearized equations
about an equilibrium point. The second method is applicble
for determining the behavior of higher-order systems which
may be forced or unforced, linear or nonlinear, time-invariant
or time-varying, and deterministic or stochastic. Solution of
the differential equation is not required. The procedure requires the selection of a scalar energy function V(x), which
is tested for the conditions that indicate stability. When V(x)
successfully meets these conditions, it is called a Lyapunov
function. The principal difficulty in applying the method is in
formulating a correct Lyapunov function because, for asymptotically stable systems, the failure of one function to meet
the stability conditions does not mean that a true Lyapunov
function does not exist. This difficulty is compounded by the
fact that the Lyapunov function is not unique. Nevertheless,
there is much interest in the second method (10,11).
In order to show a simple example of a Lyapunov function,
consider the system of Fig. 2 represented by Eq. (20) which
has multiple equilibrium points at o 0, o n, where n is
an integer. The proposed function is the sum of the kinetic
and potential stored energies, given by
V (o , o ) = 12 o2 + K(1 cos o )
(26)
This function is positive for all values of o and o, except at

the equilibrium points, where it is equal to zero. The rate of
change of this energy function along any phase-plane trajectory is obtained by differentiating Eq. (26) and using Eq. (20):
V (o , o ) = o o + K(sin o )o = ao2
ellipsoid, in the state space with its center at the origin. The
entire state space is filled with such nonintersecting closed
surfaces, each representing a different positive value of V(x).
(x) is negative for all points in the state space, it
When V
means that all trajectories cross the closed surfaces from the
outside to the inside and eventually converge at the equilibrium point at the origin.
A qualitatively correct Lyapunov function is shown in Fig.
4 for a second-order system. The function V(x1, x2) x12 x22
is positive definite and is represented by the paraboloid surface shown. The value V(x1, x2) ki (a constant) is represented
by the intersection of the surface V(x1, x2) and the plane z
ki. The projection of this intersection on the x1, x2 plane is a
closed curve, an oval, around the origin. There is a family of
such closed curves in the x1, x2 plane for different values of
ki. The value V(x1, x2) 0 is the point at the origin; it is the
innermost curve of the family of curves representing different
levels on the paraboloid.
The gradient vector of V(x) is defined by
V (x)
x1
V (x)
gradient V (x) = V (x) = x2

(28)
..
.
V (x)
xn
(x) along any trajectory is
The time derivative V
V (x) dx1
V (x) dx2
V (x) dxn
V (x) =
+
+ +
x1 dt
x2 dt
xn dt
(29)
= [V (x)]T x = V, x
(x) can be evaluated without

It is important to note that V
knowing the solution of the system state equation x f(x).
The gradient V(x1, x2) describes the steepness between adjacent levels of V(x1, x2). The function V(x1, x2) is negative definite in Fig. 4except at the origin, where it is equal to zero.
The state-plane trajectory crosses the ovals for successively
smaller values of V(x). Therefore, the system is asymptoti-
(27)
is negative along any trajectory for all values

The value of V
of o, except o 0. Note that the slope d o /do of the phaseplane trajectories [see Eq. (22)] is infinite along the line represented by o 0, except at the equilibrium points o n.
Thus, the line o 0 does not represent an equilibrium, except at o n, and it is not a trajectory in the state plane.
Since the energy stored in the system is continuously decreasing at all points except at the equilibrium points, the equilibrium at the origin and at even multiples of o n is asymp is NSD.
totically stable. This V
This example demonstrates that the total system energy
may be used as the Lyapunov function. When the equations
of a large system are given in mathematical form, it is usually
difficult to define the energy of the system. Thus, alternate
Lyapunov functions must be obtained. For any system of order n, a positive, constant value of the proper positive definite
Lyapunov function V(x) represents a closed surface, a hyper-
z = V(x)
x2
V(x) = const = k1
x1
0
V(x)
State plane trajectory
Figure 4. A positive definite function V(x1, x2) and projections on the

x1, x2 plane.
cally stable, and V(x) is a proper Lyapunov function. The concepts described above may be summarized in the following
theorem, which provides sufficient, but not necessary, conditions for stability.
Theorem 1. Lyapunov asymptotic stability. A system x
f(x, t), where f(0, t) 0 for all t, is asymptotically stable in
the vicinity of the equilibrium point at the origin if there exists a scalar function V(x) such that:
1. V(x) is continuous and has continuous first partial derivatives in a region S around the origin.
2. V(x) 0 for x 0. [V(x) is PD.]
3. V(0) 0.
(x) 0 for x 0. [V
(x) is ND.]
4. V
Conditions 13 ensure that V(x) is positive definite. Therefore, V(x) k is a closed surface within the region S. Condi (x) is negative definite, and thus any tration 4 means that V
jectory in S crosses through the surface V(x) k from the
outside to the inside for all values of k. Therefore, the trajectory converges on the origin where V(0) 0.
(x) 0 for x 0 can be relaxed in
The condition that V
Theorem 1 under the proper conditions. Condition 4 can be
(x) 0; that is, V
(x) is negative semidefinite.
changed to V
(x) is not
This relaxed condition is sufficient, provided that V
equal to zero at any solution of the original differential equation except at the equilibrium point at the origin. A test for
(x) 0 into the
this condition is to insert the solution of V
state equation x f(x) to verify that it is satisfied only at the
equilibrium point. Also, if it can be shown that no trajectory
can stay forever at the points or on the line, other than the
0, then the origin is asymptotically staorigin, at which V
ble. This is the case for the system of Fig. 2 as described at
the beginning of this section. For linear systems there is only
one equilibrium point which is at the origin; therefore it is
(x) to be negative semidefinite. Theorem 1 may
sufficient for V
also be extended so that it is applicable to the entire state
space. In that case the system is said to have global stability,
or stability in the large. Including these conditions results in
the following theorem:
Theorem 2. Lyapunov global asymptotic stability. A system
is globally asymptotically stable if there is only one stable
equilibrium point and there exists a scalar function V(x)
such that:
1. V(x) is continuous and has continuous first partial derivatives in the entire state space.
2. V(x) 0 for x 0.
3. V(0) 0.
4. V(x) at x .
(x) 0.
5. V
(x) 0, except at x 0, or any locus in the
6. Either V
(x) 0 is not a trajectory of the
state space where V
system.
any xi , then V(x) ki is a closed curve for any ki. Conditions 5 and 6 mean that V(x) is continuously decreasing along
any trajectory in the entire plane and ensures that the system
is asymptotically stable. In order to check for global stability,
it is necessary to select V(x) so that conditions 14 are all satisfied.
Finding a proper Lyapunov function V(x) means that the
system is stable, but this is just a sufficient and not a necessary condition for stability. The fact that a function V(x) has
not been found does not mean that it does not exist and that
the system is not stable.
Application of the Lyapunov Method to Linear Systems
The second method of Lyapunov is applicable to time-varying
and nonlinear systems. However, there is no simple general
method of developing the Lyapunov function. Methods have
been developed for many such systems, and the literature in
this area (4) is extensive. The remaining applications in this
chapter are restricted to linear systems. Since linear systems
may have only one equilibrium point, the stability or instability is necessarily global in nature. The RouthHurwitz stability criterion is available for determining the stability of linear
systems. However, the necessity of obtaining the characteristic polynomial can be a disadvantage for higher-order systems. Thus, the following material is presented to develop familiarity with the more general second method of viewing
stability. Confidence in the second method of Lyapunov can
be developed by showing that the results are identical to
those obtained with the RouthHurwitz method.
The approach presented is to select a V(x) which is positive
(x). The coefficients of V(x) and
definite and to evaluate V
those restraints on the system parameters are then deter (x) negative definite or negative semimined which make V
definite. Consider the linear, unity feedback system presented
in Fig. 5 with r(t) 0.
Example
G(s) =
K
,
s(s + a)
a>0
(30)
The differential equation for the actuating signal, when r

0, is
e + ae + Ke = 0
(31)
When phase variables with x1 e are used, the state equations are
x1 = x2 ,
x2 = Kx1 ax2
(32)
A simple Lyapunov function which is positive definite is

V (x) =
R(s) +
1
1
1
p x2 + p x2 = xT Px
2 1 1 2 2 2
2
E(s)
G(s)
Y(s)
Conditions 13 ensure that V(x) is positive definite. Condition 4 is satisfied when V(x) is positive definite (i.e., it is
closed) in the entire state space. When V(x) goes to infinity as
363
Figure 5. Unity-feedback linear system.
(33)
364
where p1 0 and p2 0. Its derivative is
V (x) = p1 x1 x1 + p2 x2 x2 = p1 x1 x2 p2 Kx1 x2 ap2 x22
p1 p2 K
0
T
2
= xT p p K
x = x Nx
1
2
ap2
2
(34)
(x) is always negative semidefinite if N is

The function V
negative semidefinite. The matrix N is negative semidefinite if all the principal minors of N are positive or zero.
2 = ap2 0
12 =
(35)
(p1 p2 K)2
0
4
(36)
Equation (35) is satisfied since it is required that both a and

p2 be greater than zero. Since the left side of Eq. (36) cannot
be positive, only the equality condition p1 p2K 0 can be
satisfied, which yields p1 p2K. Since p1 0 and p2 0, this
(x) ap2x22,
requires that K 0. Then Eq. (34) yields V
which is negative semidefinite. The Lyapunov global asymptotic stability theorem (Theorem 2) is satisfied since, by condi (x) 0 holds
tion 4, V(x) as x . The condition V
along the x1 axis, where x2 0 and x1 has any value. A way
(x) being negative semidefinite is sufficient
of showing that V
for global asymptotic stability is to show that the x1 axis is
not a trajectory of the system state equations [see Eq. (32)].
The first equation yields x1 0 or x1 c. The x1 axis can be
a trajectory only if x2 0 and x2 0. But the second state
equation yields x2 Kx1 Kc 0. This is a contradiction
since it requires x1 c 0. Therefore, the x1 axis is not a
trajectory, and the system is asymptotically stable. Using
Lyapunovs second method, the necessary condition for stability is shown above to be K 0. This result is readily recognized as being correct, either from the Routh stability criterion or from the root locus.
For example, a simple diagonal matrix such as N I

or N 2I is positive definite. A positive semidefinite
matrix N can be chosen which contains all zero elements except one positive element in any position along
the principal diagonal.
Step 2. Determine the elements of P by equating terms in
Eq. (40). Since P is symmetric, this requires the solution of n(n 1)/2 equations.
Step 3. Use the Sylvester theorem to determine the definiteness of P.
Step 4. Since N is selected as positive definite (or positive
semidefinite), the necessary and sufficient condition for
asymptotic stability (instability) is that P be positive
definite (negative definite).
The Sylvester conditions for the positive definiteness of P are
the same as the RouthHurwitz stability conditions. The
equivalence is derived in Ref. 12. Linear systems which are
asymptotically stable are globally asymptotically stable.
Example. Examine the conditions of asymptotic stability for
the second-order dynamic system
x + a1 x + a0 x = 0
(41)
With phase variables the coefficient matrix is
0
A=
a0
1
a1
(42)
The symmetric matrix P and a chosen positive semidefinite

matrix N are
p12
p22 ,
p11
P= p
12
2
N=
0
0
0
(43)
The identity in Eq. (40) yields three simultaneous equations:

The solution of these equations yields
Lure Method
Another design approach is the use of a procedure developed
by Lure (cited in Ref. 12). Consider the unexcited system represented by the state equation in which A is of order n:
x = Ax
(37)
The quadratic Lyapunov function V(x) is expressed in terms

of the symmetric matrix P by
V (x) = x Px
T
(38)
The time derivative of V(x) is

V (x) = x T Px + xT Px = xT (AT P + PA)x = xT Nx
n12 = 0 = p11 + a1 p12 + a0 p22

n22 = 0 = 2p12 + 2a1 p22
p12
1
=
,
a0
p22
1
=
,
a0 a1
(44)
p11
a + a21
= 0
a0 a1
The necessary conditions for P to be positive definite are obtained by applying the Sylvester theorem:
p11 =
a0 + a21
> 0,
a0 a1
p11 p22 p212 =
a0
>0
a20 a21
(45)
(39)
where the symmetric matrix N is given by

N = (AT P + PA) = 2(PA)sym
n11 = 2 = 2a0 p12
(40)
The following procedure is used:

Step 1. Select an arbitrary symmetric matrix N of order n
which is either positive definite or positive semidefinite.
The second equation requires that a0 0. Using this condition in the first equation produces the necessary condition
a1 0. These are obviously the same conditions that are obtained from the RouthHurwitz conditions for stability, as
shown in a later section.
Krasovskii (cited in Ref. 4) has shown that a similar approach may be used with a nonlinear system. However, only
sufficient conditions for local asymptotic stability in the vicinity of an equilibrium point may be obtained. For the Lyapu-
j
s = jd
s = f + jg
3
s = b + jc
2
s=a
s=0
3
1
2*
s = b jc
s = jd
Poles of the response transform located at the origin or on

the imaginary axis that are not contributed by the forcing
function R(s) are undesirable in a control system. Poles in the
right half-plane result in transient terms that increase with
time. Such performance characterizes an unstable system.
Therefore, poles in the right-half s plane are not permissible.
s plane
s=e
Routh Stability Criterion
6*
s = f jg
4*
Figure 6. Location of poles in the s plane. (Numbers are used to

identify the poles.)
(x)
nov function V(x) xTPx, the time derivative is V
xTNx, where
N = JxT P + PJx
(46)
The matrix Jx is the Jacobian evaluated at the equilibrium

point. Selecting P I may often lead to a successful determination of the conditions for asymptotic stability in the vicinity
of the equilibrium point. This extension to nonlinear systems
shows the generality of the method.
LOCATION OF POLES AND STABILITY
The dynamic equations of a system are often developed in
linearized form. However, if they are nonlinear, it is possible
to obtain a linearized approximation as described in the section entitled Linearization (Jacobian Matrix). In terms of
the Laplace transform variable s, the response transform
Y(s) for the linear representation can be expressed, in general, as the ratio of two polynomials. The response transform
Y(s) has the form
aw sw + aw1 sw1 + + a1 s + a0
P(s)
R(s) =
R(s)
Q(s)
sn + bn1sn1 + + b1 s + b0
P(s)
(47)
R(s)
=
(s s1 )(s s2 ) (s sk ) (s sn )
Y (s) =
where R(s) is the system input and the values of the poles s1,
s2, . . ., sn may be real or occur in conjugate complex pairs.
The stability and the corresponding response of a system
can be determined from the locations of the poles of the response transform Y(s) in the s plane. The possible positions of
the poles are shown in Fig. 6, and the responses associated
with the poles are given in Table 1. These poles are the roots
of the characteristic equation Q(s) 0.
The stability characteristic of a linear time-invariant system

is determined from the systems characteristic equation (13).
A system is asymptotically stable if and only if all roots of the
characteristic equation lie to the left of the imaginary axis.
Rouths stability criterion provides a means for determining
stability without evaluating the roots of this equation. The
response transform Y(s) has the general form given by Eq.
(47), where R(s) is the driving transform. Although the inverse Laplace transform of Y(s) can be performed, the polynomial Q(s) must first be factored. Computer and calculator programs are readily available for obtaining the roots of a
polynomial (18). Figure 6 and Table 1 show that stability of
the response y(t) requires that all zeros of Q(s) have negative
real parts. Since it is usually not necessary to find the exact
solution when the response is unstable, a simple procedure to
determine the existence of zeros with positive real parts is
needed. If such zeros of Q(s) with positive real parts are
found, the system is unstable and must be modified. Rouths
criterion is a simple method of determining the number of
zeros with positive real parts without actually solving for the
zeros of Q(s). Note that zeros of Q(s) are poles of Y(s) (15,16).
The characteristic equation is
Q(s) = bn sn + bn1 sn1 + bn2 sn2 + + b1 s + b0 = 0
1
22*
3
44*
5
66*
Form of
Response
Ae at
Ae bt sin(ct )
A
A sin(dt )
Ae et
Ae ft sin(gt )
Characteristics
Damped exponential
Exponentially damped sinusoid
Constant
Constant-sinusoid
Increasing exponential (unstable)
Exponentially increasing sinusoid
(unstable)
(48)
If the b0 term is zero, divide by s to obtain the equation in the

form of Eq. (48). The bs are real coefficients, and all powers
of s from sn to s0 must be present in the characteristic equation. A necessary but not sufficient condition for stable roots
is that all the coefficients in Eq. (48) be positive. If any coefficients other than b0 are zero or if the coefficients do not all
have the same sign, then there are pure imaginary roots or
roots with positive real parts and the system is unstable. It is
therefore unnecessary to continue if only stability or instability is to be determined. When all the coefficients are present
and positive, the system may or may not be stable because
there still may be roots on the imaginary axis or in the righthalf s plane. Rouths criterion is mainly used to determine
stability. In special situations it may be necessary to determine the actual number of roots in the right-half s plane. For
these situations the procedure described in this section can
be used.
The coefficients of the characteristic equation are arranged
in the pattern shown in the first two rows of the following
Routhian array. These coefficients are then used to evaluate
the rest of the constants to complete the array.
Table 1. Relation of Response to Location of Poles

Position
of Pole
365
sn | b n
bn2
bn4
bn6
| bn1
bn3
bn5
bn7
| c1
c2
c3
| d1
d2
n1
n2
n3
s1 | j 1
s0 | k 1
(49)
366
The constants c1, c2, c3, and so on, in the third row are evaluated as follows:
bn1 bn2 bn bn3
bn1
(50)
bn bn5
b b
c2 = n1 n4
bn1
(51)
c1 =
This pattern is continued until the rest of the cs are all equal
to zero. Then the d row is formed by using the sn1 and sn2
rows. These contants are
c1 bn3 bn1 c2
c1
(52)
bn1 c3
c b
d2 = 1 n5
c1
(53)
d1 =
Q(s) = s + s + 10s + 72s + 152s + 240

4
1. Substitute s 1/x in the original equation; then solve

for the roots of x with positive real parts. The number
of roots x with positive real parts will be the same as
the number of s roots with positive real parts.
2. Multiply the original polynomial by the factor (s 1)
which introduces an additional negative root. Then
form the Routhian array for the new polynomial.
The first method is illustrated in the following example:
Q(s) = s4 + s3 + 2s2 + 2s + 5
(54)
10
152
s4 |
72
240
s3 |
62
88
s2 |
70.6
240
s1 |
122.6
s0 |
240
Theorem 1. Division of a row: The coefficients of any row

may be multiplied or divided by a positive number without
changing the signs of the first column. The labor of evaluating
the coefficients in Rouths array can therefore be reduced by
multiplying or dividing any row by a constant. This may result, for example, in reducing the size of the coefficients and
therefore simplifying the evaluation of the remaining coefficients.
s |
s |
5
(57)
The zero in the first column prevents completion of the array.

The following methods overcome this problem.
Method 1. Letting s 1/x and rearranging the polynomial
gives
Q(x) = 5x4 + 2x3 + 2x2 + x + 1
(58)
The new Routhian array is
(55)
In the first column there are two changes of sign, from 1 to

62 and from 62 to 70.6; therefore Q(s) has two roots in the
right-half s plane. Note that this criterion gives the number
of roots with positive real parts but does not tell the values of
the roots. If Eq. (54) is factored, the roots are s 3, s2,3
1 j3, and s4,5 2 j4. This confirms that there are
two roots with positive real parts. The Routh criterion does
not distinguish between real and complex roots.
s4 |
2
The Routhian array is formed by using the procedure described above:
s5 |
(56)
The Routhian array is
This is continued until no more d terms are present. The rest

of the rows are formed in this way, down to the s0 row. The
complete array is triangular, ending with the s0 row. Notice
that the s1 and s0 rows contain only one term each. Once the
array has been found, Rouths criterion states that the number of roots of the characteristic equation with position real
parts is equal to the number of changes of sign of the coefficients in the first column. Therefore the system is stable if all
terms in the first column have the same sign. Reference 17
shows that a system is unstable if there is a negative element
in any position in any row.
The following example illustrates this criterion:
5
Theorem 2. A zero coefficient in the first column: When the

first term in a row is zero but not all the other terms in that
row are zero, the following methods (14) can be used:
s4 |
s3 |
s2 |
s1 |
s |
(59)
There are two changes of sign; therefore there are two roots
of x in the right-half s plane. The number of roots of s with
positive real parts is also two. This method does not work
when the coefficients of Q(s) and of Q(x) are identical.
Method 2
Q1 (s) = Q(s)(s + 1) = s5 + 2s4 + 3s3 + 4s2 + 7s + 5
(60)
The reader may obtain the Routhian array which has two
changes of sign in the first column, so there are two zeros of
Q(s) with positive real parts. Thus, the same result is obtained by both methods. An additional method is described in
Ref. 20.
Theorem 3. When all the coefficients of one entire row are
zero, the procedure is as follows:
1. The auxiliary equation can be formed from the preceding row, as shown below.
2. The Routhian array can be completed by replacing the

all-zero row with the coefficients obtained by differentiating the auxiliary equation.
3. The roots of the auxiliary equation are also roots of the
original equation. These roots occur in pairs and are the
negative of each other. Therefore, these roots may be
imaginary (complex conjugates) or real (one positive
and one negative), may lie in quadruplets (two pairs of
complex-conjugate roots), and so on.
367
characteristic equation, which is

Q(s) = s4 + 7s3 + 15s2 + (25 + K)s + 2K
(66)
The coefficients must all be positive in order for the zeros of

Q(s) to lie in the left half of the s plane, but this is not a
sufficient conditon for stability. The first column of the Routhian array permits evaluation of precise boundaries for K. It
is left as an exercise for the reader to show that the closedloop system is stable for 0 K 28.1.
Consider the system which has the characteristic equation

q(s) = s4 + 2s3 + 11s2 + 18s + 18 = 0
s4 |
11
s3 |
18
s |
s2 |
18
s2 |
s1 |
(61)
18
(After dividing the s3 row by 2)
(62)
(After dividing the s2 row by 2)
The presence of a zero row for s1 indicates that there are roots
that are the negatives of each other. The next step is to form
the auxiliary equation from the preceding row, which is the
s2 row. The highest power of s is s2, and only even powers of s
appear. Therefore the auxiliary equation is s2 9 0. The
roots of this equation are s j3. These are also roots of the
original equation. The presence of imaginary roots indicates
that the output includes a sinusoidally oscillating component.
For a sinusoidal input with the frequency corresponding to
the imaginary root, the response is unbounded. Thus, the system with imaginary roots is considered unstable.
To complete the Routhian array, the auxiliary equation is
differentiated and is
2s + 0 = 0
(63)
The coefficients of this equation are inserted in the s1 row,

and the array is then completed:
s1 |
s0 |
(64)
Since there are no changes of sign in the first column, there

are no roots with positive real parts.
In feedback systems, the ratio of the output to the input
does not have an explicitly factored denominator (see the section entitled Design Using the Root Locus). An example of
such a function is
P(s)
K(s + 2)
Y (s)
=
=
R(s)
Q(s)
s(s + 5)(s2 + 2s + 5) + K(s + 2)
(65)
The value of K is an adjustable parameter in the system and

may be positive or negative. The value of K determines the
location of the poles and therefore the stability of the system.
It is important to know the range of values of K for which the
system is stable. This information must be obtained from the
DESIGN USING THE ROOT LOCUS

As described in the section entitled Location of Poles and
Stability, the stability of linear time-invariant (LTI) systems
is determined by the locations of the system poles. Any pole
in the right-half plane leads to an exponentially increasing
transient which therefore means that the system is unstable.
An effective method for designing a feedback control is by use
of the root locus. Figure 7 shows a block diagram of a singleinput single-output feedback system expressed in terms of the
Laplace transform variable s. G(s) is the forward transfer
function representing the dynamics of the plant. Also, H(s)
is the feedback transfer function representing the dynamics
in that path. The feedback signal B(s) is compared with the
command input signal R(s), and their difference is called the
actuating signal. The overall closed-loop transfer function is
C(s)/R(s), where R(s) is the control system input and C(s) is
the controlled output signal.
For the root-locus method the open-loop transfer function
can be put in the form
G(s)H(s) =
K(s + a1 ) (s + ah ) (s + aw )
sm (s + b1 )(s + b2 ) (s + bc ) (s + bu )
(67)
where ah and bc may be real or complex numbers and may lie

in either the left-half or right-half s plane. The value of K
may be either positive or negative. Equation (67) can be rewritten as
G(s)H(s) =
w
K h=1 (s zh )
K(s z1 ) (s zw )
=
u
sm (s p1 ) (s pu )
sm c=1 (s pc )
(68)
where indicates a product of terms. The degree of the numerator is w and that of the denominator is m u n. The
zh are the zeros and the pc are the poles of G(s)H(s). In the
form shown in Eq. (68), with the coefficients of s all equal to
unity, the K is defined as the loop sensitivity.
The underlying principle of the root locus is that the poles
of the control ratio C(s)/R(s) are related to the zeros and poles
of the open-loop transfer function G(s)H(s) and to the loop
R(s) +
E(s)
G(s)
C(s)
B(s)
H(s)
Figure 7. Block diagram of a feedback system.
368
sensitivity K. This is shown as follows. Let

G(s) =
N1 (s)
,
D1 (s)
H(s) =
N2 (s)
D2 (s)
B(s)H(s) given in Eq. (67) as

|K| =
(69)
Thus
C(s)
A(s)
G(s)
= M(s) =
=
R(s)
B(s)
1 + G(s)H(s)
(70)
|sm | |s p1 | |s pu |
|s z1 | |s zw |
(angles of denominator terms)

(angles of numerator terms)

(1 + 2h)180 for K > 0
=
for K < 0
h360
(79)
(80)
where
D D + N1 N2
B(s) 1 + G(s)H(s) = 1 2
D1 D2
Design Application: Root Locus Procedures

(71)
Rationalizing Eq. (70) gives

C(s)
N1 D2
P(s)
= M(s) =
=
R(s)
D1 D2 + N1 N2
Q(s)
(72)
From Eqs. (71) and (72) it is seen that the zeros of B(s) are
equal to the poles of M(s), and they determine the form of the
closed-loop system response. The degree of the numerator of
B(s) is equal to m u. Therefore B(s) has n m u finite
zeros. The roots of B(s) 0, which is the characteristic equation of the closed-loop system, must satisfy the equation
B(s) 1 + G(s)H(s) = 0
(73)
The procedure for obtaining the root locus is to first put the
open-loop transfer function G(s)H(s) into the form shown in
Eq. (68). The poles and zeros are then plotted in the s
j plane. Then the angle condition given in Eq. (80), for both
K 0 and K 0, is used to obtain the complete root locus.
Then the root locus can be calibrated in terms of the loop
sensitivity K by using the magnitude condition given in Eq.
(79). The dominant roots are those that contribute the most to
the overshoot and the settling time of the closed-loop system
response. Selection of the dominant complex roots on the root
locus is based on the characteristics of a simple second-order
system with a unit step input (8). The peak overshoot Mp is
related to the damping ratio by
M p = 1 + exp
1 2
(81)
These roots must satisfy the equation

G(s)H(s) =
K(s z1 ) (s zw )
= 1
sm (s p1 ) (s pu )
(74)
Thus, as the loop sensitivity K assumes values from zero to

infinity, the open-loop transfer function must always be equal
to 1. The corresponding values of s which satisfy Eq. (74)
for any value of K are the poles of M(s). The plots of these
values of s are defined as the root locus of M(s). They satisfy
the following conditions:
For K 0:
|G(s)H(s)| = 1
h = 0, 1, 2, . . .
(76)
For K 0:
|G(s)H(s)| = 1
(77)
h = 0, 1, 2, . . .
(82)
An alternate specification is based on the desired settling

time Ts. The real part of the desired complex root is given by
=
number of time constants

Ts
(83)
The transient coefficient et decays to, and stays within a

specified percentage of, the final value, in the time Ts. The
latter is expressed as the number of time constants obtained
from Eq. (83). For example, using 2% of the final value as the
criterion requires four time constants for the selected root.
Root Locus Construction Rules
The following properties facilitate the drawing of the root
locus:
Angle condition:
\G(s)H(s) = h360,
= cos1
(75)
Angle condition:
\G(s)H(s) = (1 + 2h)180,
A specification based on Mp yields a required value of . Roots

with a specified value of occur at the intersection of the root
locus with a straight line drawn at an angle drawn from
the negative real axis, where
(78)
Thus, the root locus method provides a plot of the variation

of each of the poles of C(s)/R(s) in the complex s plane as the
loop sensitivity K is varied from zero to infinity. All the angles
are measured as positive in the counterclockwise sense. Since
G(s)H(s) usually has more poles than zeros, it is convenient
to express the angle condition for the generalized form of
Rule 1. The number of branches of the root locus is equal

to the number of poles of the open-loop transfer function.
Rule 2. For positive values of K, the root locus exists on
those portions of the real axis for which the total number of real poles and zeros to the right is an odd number.
For negative values of K, the root locus exists on those
portions of the real axis for which the total number of
real poles and zeros to the right is an even number (including zero).
Rule 3. The root locus starts (K 0) at the open-loop poles
and terminates (K ) at the open-loop zeros or at
infinity.
Rule 4. The angles of the asymptotes of the root locus
branches that end at infinity are determined by a
h360 for K < 0

(1 + 2h)180 for K > 0
[Number of poles of [Number of zeros of
G(s)H(s)]
G(s)H(s)]
A root-locus digital-computer program (8,21) will produce

an accurate calibrated root locus. This considerably simplifies
the work required for the system design. By specifying for
the dominant roots or Km, a computer program can determine
all the roots of the characteristic equation.
Root Locus Example. Given here is the unity feedback system with
K1
s(s/25 + 1)(s2 /2600 + s/26 + 1)
65,000K1
=
s(s + 25)(s2 + 100s + 2600)
K
= 4
s + 125s3 + 5,000s2 + 65,000s
(84)
G(s) =
Rule 5. The real-axis intercept of the asymptotes is

w
h=1 Re(zh )
nw
n
o =
c=1 Re(pc )
j=1
pj =
n
where K 65,000K1.
Specification: Find C(s)/R(s) with 0.5 for the dominant
roots (roots closest to the imaginary axis).
1. The poles of G(s)/H(s) are plotted on the s plane in Fig.
8. The values of the poles are: s 0, 25, 50 j10,
50 j10. The system is completely unstable for K
0. Therefore, this example is designed only for the condition K 0.
2. There are four branches of the root locus. Since there
are no zeros, all branches end at infinity for K .
3. The locus exists on the real axis between 0 and 25.
4. The angles of the asymptotes are
=
(1 + 2h)180
= 45 , 135
4
5. The real-axis intercept of the asymptotes is

o =
0 25 50 50
= 31.25
4
jw
K
K
s3
55.9 + j18
50 + j10
= 0.5
s1
K=0
K=0
o
50 j10
25
sa
60
K=0
s2
K=0
s4
(86)
j=1
Figure 8. Root locus for
can be used to find one real or two complex roots. Factoring known roots from the characteristic equation can
also simplify the work of finding the remaining roots.
s plane
6.6 + j11.4
rj
(87)
(85)
Rule 6. The breakaway point for the locus between two

poles on the real axis (or the break-in point for the locus
between two zeros on the real axis) can be determined
by taking the derivative of the loop sensitivity K with
respect to s. Equate this derivative to zero and find the
roots of the resulting equation. The root that occurs between the poles (or the zeros) is the breakaway (or
break-in) point.
Rule 7. For K 0 the angle of departure from a complex
pole is equal to 180 minus the sum of the angles from
the other poles plus the sum of the angles from the
zeros. Any of these angles may be positive or negative.
For K 0 the departure angle is 180 from that obtained for K 0. For K 0 the angle of approach to a
complex zero is equal to the sum of the angles from the
poles minus the sum of the angles from the other zeros
minus 180. For K 0 the approach angle is 180 from
that obtained from K 0.
Rule 8. The imaginary-axis crossing of the root locus can
be determined by forming the Routhian array for the
closed-loop characteristic equation. Equate the s1 row to
zero and form the auxiliary equation from the s2 row.
The roots of the auxiliary equation are the imaginaryaxis crossover points.
Rule 9. The selection of the dominant roots of the characteristic equation is based on the specifications that give
the required system performance; that is, it is possible
to evaluate , d, and , which are used to select the
location of the desired dominant roots. The loop sensitivity for these roots is determined by means of the
magnitude condition. The remaining roots are then determined to satisfy the same magnitude condition.
Rule 10. For those open-loop transfer functions for which
w n 2, the sum of the closed-loop roots is equal to
the sum of the open-loop poles. Thus, once the dominant
roots have been located,
n
369
G(s)H(s) =
65, 000K1
s(s + 25)(s2 + 100s + 2600)
370
6. The breakaway point sa on the real axis between 0 and

25 is found from the derivative dK/ds 0, using Eq.
(79) to obtain K(s). The result is
14. With a unit step input, R(s) 1/s, the output response
can be obtained from a CAD program (8,21):
c(t) = 1 + 1.4e6.6t sin(11.4t 143.8 )
dK
= 4s3 + 375s2 + 10,200s + 65,000 = 0
ds
+ 0.2e55.9t sin(18.0t 123.7 )
This yields sa 9.15.

7. The angle of departure from the pole 50 j10 is obtained from
0 + 1 + 2 + 3D = (1 + 2h)180
where the angles are measured from each open-loop
pole to the pole 50 j10. This yields 3D 123.1.
8. Using Eq. (72), the closed-loop transfer function is
(90)
The figures of merit which describe the significant step

response characteristics are ts 0.64 s, tp 0.31 s,
Mp 1.56, and c(t)ss 1.00.
The root locus design method ensures more than system stability. It also permits selection of the closed-loop system poles
that best satisfy the desired performance specifications.
C(s)
65,000K1
FREQUENCY RESPONSE
= 4
(88)
3
R(s)
s + 125s + 5,100s2 + 65,000s + 65,000K1
The objective in using feedback is threefold. The first purpose
The Routhian array for the denominator of C(s)/R(s), is to achieve stability and/or a specified degree of stability
which is the characteristic polynomial, is
margin. The second goal is to reduce the sensitivity to parameter variation. The third goal is disturbance rejection. The
s4 |
1
5100
65,000K1
previous section introduces the root locus method of design.
The closed-loop poles are assigned in the left-half plane so
s3 |
1
520
(After division by 125)
that the performance is stable, and they are located on the
1
14.2K1 (After division by 4580)
s2 |
root locus in positions that best achieve the desired performance criteria. This section introduces the frequency res1 | 520 14.2K1
sponse methods that ensure closed-loop system stability by
0
s |
14.2K1
application of the Nyquist criterion and meet desired figures
of merit. The gain of the forward transfer function is the adPure imaginary roots exist when the s1 row is zero.
justable parameter which is used in the design. The feedback
This occurs when K1 520/14.2 36.6. The auxiliary
control system is shown in Fig. 7. For a linear time-invariant
2
equation is s 14.2K1 0. The imaginary roots are
(LTI) system, the open-loop transfer function in the s domain
s j14.2K1 j22.8.
is shown in Eq. (68), and the closed-loop transfer function is
9. Additional points on the root locus are found by locat- represented by Eqs. (70) and (72).
ing points that satisfy the angle condition
The forward frequency transfer function is written in the
generalized
form:
\s + \(s + 25) + \(s + 50 j10)
+ \(s + 50 + j10) = (1 + 2m)180
The root locus is shown in Fig. 8.

10. The radial line for 0.5 is drawn on the graph of
Fig. 8 at the angle cos1 cos1 0.5 60. The
dominant complex roots obtained from the graph are
s1,2 = 6.6 j11.4
11. Applying the magnitude condition of Eq. (79) yields
K = 65, 000K1 = |s| |s + 25| |s + 50 j10| |s + 50 + j10|
For s 6.6 j11.4, K 598,800 and K1 9.25.
12. The other roots are evaluated to satisfy the magnitude
conditon K 598,800. The other roots of the characteristic equation are
s = 55.9 j18.0
13. The overall closed-loop transfer function is
C(s)
598,800
=
R(s)
(s + 6.6 j11.4)(s + 55.9 j18)
(89)
G( j) =
=
Km (1 + jT1 )(1 + jT2 ) (1 + jTw )

m
( j) (1 + jTa )[1 + (2 /n ) j + (1/n2 )( j)2 ]
(91)
Km G ( j)
where m defines the system type, Km is the gain constant,

and G( j) is the forward transfer function with unity gain.
There are two forms typically used for the plot of G( j). In
the first category is the Nyquist plot which is the output
input ratio in polar coordinates. The Nyquist stability criterion is applied to this plot to determine the closed-loop system
stability. The second category is the pair of Bode plots. One
plot is the magnitude in decibels versus the logarithm of frequency, and the second plot is the angle versus the logarithm
of frequency. Typically, semilog graph paper is used so that it
is not necessary to obtain the logarithm of frequency. The
data from the Bode plot is then plotted on a Nichols plot in
rectangular coordinates: log magnitude versus angle. The
Nichols chart is a set of contours which are drawn on the
Nichols plot and is used to both ensure stability and to adjust
the gain in order to meet the closed-loop specifications. A digital computer CAD program (8,21) provides considerable assistance in obtaining these plots and performing the design.
Direct Polar-Plot Characteristics

To obtain the direct polar plot of a systems forward transfer
function, the following characteristics are used to determine
the key parts of the curve.
Step 1. The forward transfer function has the general form
shown in Eq. (91). For this transfer function the system
type is equal to the value of m and determines the portion of the polar plot representing the lim0G( j)
0(w m u)90. The low-frequency polar-plot characteristics (as 0) of the different system types are
summarized in Fig. 9. The angle at 0 is m(90).
The arrow on the polar plots indicates the direction of
increasing frequency.
Step 2. The high-frequency end of the polar plot can be
determined as follows:
lim G( j) = 0\(w m u)90
(92)
Note that since the degree of the denominator of Eq.

(91) is usually greater than the degree of the numerator, the high-frequency point ( ) is approached (i.e.,
the angular condition) in the clockwise sense. The plot
ends at the origin, tangent to the axis determined by
Eq. (92). Tangency may occur on either side of the axis.
Step 3. The asymptote that the low-frequency end approaches, for a Type 1 system, is determined by taking
the limit as 0 of the real part of the transfer function.
Step 4. The frequencies at the points of intersection of the
polar plot with the negative real axis and the imaginary
axis are determined, respectively, by setting
Im[G( j)] = 0
(93)
Re[G( j)] = 0
Step 5. If there are no frequency-dependent terms in the

numerator of the transfer function, the curve is a
270
Type 3
0+
K0
1 + j0
180
=+
0+
=0
Type 2
smooth one in which the angle of G( j) continuously

decreases as goes from 0 to . With time constants in
the numerator, and depending upon their values, the
angle may not continuously vary in the same direction,
thus creating dents in the polar plot.
Step 6. As is seen later in this chapter, it is important to
know the exact shape of the polar plot of G( j) in the
vicinity of the 1 j0 point and the crossing point on
the negative real axis.
Nyquists Stability Criterion
A system designer must be sure that the closed-loop system
he designs is stable. The Nyquist stability criterion (22,23)
provides a simple graphical procedure for determining closedloop stability from the frequencyresponse curves of the openloop transfer function G( j)H( j). The application of this
method in terms of the polar plot is covered in this section;
application in term of the log magnitudeangle (Nichols) diagram is covered in a later section. The closed-loop transfer
function of the system is given in Eqs. (70) and (72). The characteristic equation formed from the denominator of this
closed-loop transfer function is
B(s) = 1 + G(s)H(s) =
B(s) =
0+
Type 1
90
Figure 9. A summary of direct polar plots of different types of
systems.
D1 D2 + N1 N2
=0
D1 D2
(94)
For a stable system the roots of the characteristic equation

must not lie in the right-half s plane or on the j axis. Note
that the numerator and denominator of B(s) have the same
degree. The poles of the open-loop transfer function G(s)H(s)
are the poles of B(s). Since the denominator of C(s)/R(s) is the
same as the numerator of B(s), the condition for stability may
therefore be restated as: For a stable system, none of the
zeros of B(s) can lie in the right-half s plane or on the imaginary axis. Nyquists stability criterion relates the number of
zeros and poles of B(s) that lie in the right-half s plane to the
polar plot of G(s)H(s).
In this analysis it is assumed that the control systems
range of operation is confined to the linear region. This yields
a set of linear differential equations which describe the dynamic performance of the systems. Because of the physical
nature of feedback control systems, the order of the denominator D1D2 is equal to or greater than the order of the numerator N1N2 of the open-loop transfer function G(s)H(s). This
means that limsG(s)H(s) 0 or a constant.
A rigorous mathematical derivation of Nyquists stability
criterion is based on complex variable theory. A qualitative
approach to Nyquists stability criterion is presented for the
special case that B(s) is a rational fraction. The characteristic
function B(s) given by Eq. (94) can be rationalized, factored,
and then written in the form
Type 0
371
P(s)
(s Z1 )(s Z2 ) (s Zn )
=
(s p1 )(s p2 ) (s pn )
Q(s)
(95)
where Z1, Z2, . . ., Zn are the zeros and p1, p2, . . ., pn are the
poles. The poles pi are the same as the poles of the open-loop
transfer function G(s)H(s) and include the s term for which
p 0, if it is present.
372
Generalizing Nyquists Stability Criterion
Consider now a closed contour Q which encloses the whole

right-half s plane (see Fig. 11), thus enclosing all zeros and
poles of B(s) that have positive real parts. As a consequence
of the theory of complex variables used in the formal derivation, the contour Q must not pass through any poles or zeros
of B(s). When the results of the preceding discussion are applied to the contour Q, the following properties are noted:
O
Q
s plane
Z1
p2
(s Z1)
(s p
2)
(s)
Z4
p4
p1
(s
p3
(s Z3)
Z3
)
p3
(s Z2)
p3
Q
Z2
Figure 10. A plot of some poles and zeros of Eq. (95).
In Fig. 10 some poles and zeros of a generalized function

B(s) are drawn on the s plane. Also, an arbitrary closed contour Q is drawn which encloses the zero Z1. To the point O
on Q, whose coordinates are s j, are drawn directed
line segments from all the poles and zeros. The lengths of
these directed line segments are given by s Z1, s Z2, s
p1, s p2, and so on. Not all the directed segments from the
poles and zeros are indicated in the figure, because they are
not necessary to proceed with this development. As the point
O is rotated clockwise once around the closed contour Q, the
directed segment s Z1 rotates through a net clockwise angle
of 360. All the other directed segments rotate through a net
angle of 0. Thus, referring to Eq. (95), it is seen that the
clockwise rotation of 360 for s Z1 is simultaneously realized
by the function B(s) for the enclosure of the zero Z1 by the
path Q.
Consider now a larger closed contour Q which includes the
zeros Z1, Z2, and Z3 and the pole p5. As a point O is rotated
clockwise once around the closed curve Q, each of the directed line segments from the enclosed pole and zeros rotates
through a net clockwise angle of 360. Since the angular rotation of the pole is experienced by the characteristic function
B(s) in its denominator, the net angular rotation realized by
Eq. (95) must be equal to the net angular rotations due to the
pole p5 minus the net angular rotations due to the zeros Z1,
Z2, and Z3. Therefore, for this case, the net number of rotations N experienced by B(s) 1 G(s)H(s) for the clockwise
movement of point O once about the closed contour Q is
equal to 2; that is,
1. The total number of clockwise rotations of B(s) due to

its zeros is equal to the total number of zeros ZR in the
right-half s plane.
2. The total number of counterclockwise rotations of B(s)
due to its poles is equal to the total number of poles PR
in the right-half s plane.
3. The net number of rotations N of B(s) 1 G(s)H(s)
about the origin is equal to its total number of poles PR
minus its total number of zeros ZR in the right-half s
plane. N may be positive (ccw), negative (cw), or zero.
The essence of these three properties can be represented by
the equation
N = Change in phase of [1 + G(s)H(s)]/2 = PR ZR
where counterclockwise rotation is defined as being positive

and clockwise rotation is negative. In order for the characteristic function B(s) to realize a net rotation N, the directed line
segment representing B(s) (see Fig. 12) must rotate about the
origin 360N degrees, or N complete revolutions. Solving Eq.
(96) for ZR yields
ZR = PR N
(97)
For a stable system, B(s) can have no zeros ZR in the righthalf s plane. It is therefore concluded that, for a stable system, the net number of rotations of B(s) about the origin must
be counterclockwise and equal to the number of poles PR that
lie in the right-half plane. In other words, if B(s) experiences
a net clockwise rotation (i.e., if N is negative), this indicates
that ZR PR, where PR 0, and thus the closed-loop system
is unstable. If there are zero net rotations, then ZR PR and
the system may or may not be stable, according to whether
PR 0 or PR 0.
j
s=+j
O
s = re j
N = (number of poles enclosed)
s plane
Q
+
(number of zeros enclosed) = 1 3 = 2

where the minus sign denotes clockwise (cw) rotation. Also,
for any closed path that may be chosen, all the poles and zeros
that lie outside the closed path each contribute a net angular
rotation of 0 to B(s) as a point is moved once around this
contour.
(96)
s=j
Figure 11. The contour that encloses the entire right-half s plane.
270
373
270
B(s) = [1+ G(s) H(s)]

B(s)
180
1 + j0
180
0
1 + j0
B(s)
G(s) H(s)
90
90
(a)
(b)
Obtaining a Plot of B(s). Figures 12(a) and 12(b) show a plot

of B(s) and a plot of G(s)H(s), respectively. By moving the origin of Fig. 12(b) to the 1 j0 point, the curve is now equal
to 1 G(s)H(s), which is B(s). Since G(s)H(s) is known, this
function is easily plotted, and then the origin is moved to the
1 point to obtain B(s). In general, the open-loop transfer
functions of many physical systems do not have any poles PR
in the right-half s plane. In this case, ZR N. Thus, for a
stable system the net number of rotations about the 1 j0
point must be zero when there are no poles of G(s)H(s) in the
right-half s plane. If the function G(s)H(s) has some poles in
the right-half s plane and the denominator is not in factored
form, then the number PR can be determined by applying
Rouths criteiron to D1D2. The Routhian array gives the number of roots in the right-half s plane by the number of sign
changes in the first column.
Analysis of Path Q. Nyquists criterion requires that the
whole right-half s plane must be encircled to ensure the inclusion of all poles or zeros in this portion of the plane. In Fig.
11 the entire right-half s plane is included by the closed path
Q which is composed of the following two segments:
1. One segment is the imaginary axis from j to j.
2. The other segment is a semicircle of infinite radius that
encircles the entire right-half s plane.
The portion of the path along the imaginary axis is represented mathematically by s j. Thus, replacing s by j in
Eq. (95) and letting take on all values from to gives
the portion of the B(s) plot corresponding to that portion of
the closed contour Q which lies on the imaginary axis.
A requirement of the Nyquist criterion is that
limsG(s)H(s) 0 or a constant. Therefore, as the point O
moves along the segment of the closed contour represented by
the semicircle of infinite radius, the corresponding portion of
the B(s) plot is a fixed point. As a result, the movement of
point O along only the imaginary axis from j to j results in the same net rotation of B(s) as if the whole contour
Q were considered. In other words, all the rotation of B(s)
occurs while the point O goes from j to j along the
imaginary axis.
Effect of Poles at the Origin on the Rotation of B(s). Some
transfer functions G(s)H(s) have an sm in the denominator.
Since no poles or zeros can lie on the contour Q, the contour
shown in Fig. 11 must be modified to the contour shown in
Figure 12. A change of reference for B(s).
Fig. 13(a). Consider the transfer function

G(s)H(s) =
K1
s(1 + T1 s)(1 + T2 s)
(98)
The point O is first moved along the negative imaginary axis

from s j to a point where s j 0 90 becomes
very small; that is, s j. Then the point O moves along a
semicircular path of radius s ej in the right-half s plane,
with a very small radius until it reaches the positive imaginary axis at s j j0 0 90. From here the point O
proceeds along the positive imaginary axis to s j. Letting the radius approach zero, 0, for the semicircle
around the origin, ensures the inclusion of all poles and zeros
in the right-half s plane. To complete the plot of B(s), the effect of moving point O on this semicircle around the origin
must be investigated.
For the semicircular portion of the path Q represented by
s ej, where 0 and 90 90, Eq. (98) becomes
G(s)H(s) =
K
K1
K
K
= 1j = 1 e j = 1 e j
s
e

(99)
where K1 / as 0, and goes from 90 to 90

as the directed line segment s goes counterclockwise from
90 to 90. Thus, in Fig. 13(b), the points G(s)H(s) for
0 and 0 are joined by a semicircle of infinite radius
in the first and fourth quadrants. Figure 13(b) shows the completed contour of G(s)H(s) as the point O moves along the
= 0
1
2
7 6
j0 +
s = e j
5
4
3
12
Q
j0
1 + j0
B(s)
G(s)H(s)
j
(a)
=+
=
= 0+
7
(b)
Figure 13. (a) The contour Q which encircles the right-half s plane.
(b) Complete plot for Eq. (98).
374
modified contour Q of Fig. 13(a) in the s plane in the clockwise

direction. When the origin is moved to the 1 j0 point, the
curve becomes B(s). The plot of B(s) in Fig. 13(b) does not
encircle the 1 j0 point; therefore N 0. Also, from Eq.
(98), PR 0. Thus, using Eq. (97), ZR 0 and the system is
stable. If the gain K1 is increased so that the intersection of
G(s)H(s) with the negative real axis occurs to the left of the
1 point, then N 2 and Z 2, with the result that the
closed-loop system is unstable.
Nyquist Stability: Type 2 System. Transfer functions that
have the term sm in the denominator have the general form,
as 0,
G(s)H(s) =
Km
Km
Km
Km
= m jm = m e jm = m e jm
sm
e

(100)
where m 1, 2, 3, 4, . . .. With the reasoning used in the

preceding example, it is seen from Eq. (100) that, as s moves
from 0 to 0, the plot of G(s)H(s) traces m clockwise semicircles of infinite radius about the origin. For example, if m 2,
then, as goes from /2 to /2 in the s plane with radius,
, G(s)H(s) experiences a net rotation of (2)(180) 360.
Since the polar plots are symmetrical about the real axis, it
is only necessary to determine the shape of the plot of
G(s)H(s) for a range of values of 0 . The net rotation
N of the plot for the range of is twice that of
the plot for the range of 0 . Consider the system
K2 (1 + T4 s)
s2 (1 + T1 s)(1 + T2 s)(1 + T3 s)
G(s)H(s) =
1 0
G(s)H(s)
=+
B(s)
Radial
line
PR = 0
N=0
ZR = 0
Stable
system
Figure 14. The complete polar plot for Eq. (101).
Nichols Plot and Stability

The log magnitude of the open-loop transfer function
G( j)H( j), abbreviated as Lm, is defined as 20 times the
logarithm to the base 10:
Lm G( j) = 20 log10 |G( j)|
1 + j0
When G( j)H( j) Passes Through the 1 j0 Point. When

the curve of G( j)H( j) passes through the 1 j0 point,
the number of encirclements N is indeterminate. This corresponds to the condition where B(s) has zeros on the imaginary
axis. A necessary condition for applying the Nyquist criterion
is that the path encircling the specified area must not pass
through any poles or zeros of B(s). When this condition is violated, the value for N becomes indeterminate and the Nyquist
stability criterion cannot be applied. Simple imaginary zeros
of B(s) mean that the closed-loop system will have a continuous steady-state simusoidal component in its output which is
independent of the form of the input. In addition, a sinusoidal
input with the frequency equal to that of the imaginary zero
produces an unbounded output. Therefore, this condition is
considered unstable.
dB
(102)
(101)
where T4 T1 T2 T3. Figure 14 shows the mapping of

G(s)H(s) for the contour Q of the s plane. The word mapping,
as used here, means that for a given point in the s plane there
corresponds a given value of G(s)H(s) or B(s). The presence of
the s2 term in the denominator of Eq. (101) results in a net
rotation of 360 in the vicinity of s 0, as shown in Fig. 14.
For the complete range of frequencies the net rotation is zero;
thus, since PR 0, the system is stable. The value of N can
be determined by drawing the line radiating from the 1

j0 point (see Fig. 14) and noting one cw and one ccw crossing;
thus, the sum of these crossings is N 0. Like the previous
example, this system can be made unstable by increasing the
gain sufficiently for the G(s)H(s) plot to cross the negative
real axis to the left of the 1 j0 point.
where the Lm has the units of decibels (dB).

The Nichols plot is defined for the open-loop transfer function G( j)H( j) as having the Lm on the vertical axis and
the angle on the horizontal axis. This is an alternate to the
polar plot and can be used to determine closed-loop system
stability by use of the Nyquist criterion. The corresponding
polar and Nichols plots for the transfer function of Eq. (98)
are shown in Figs. 15(a) and 15(b). The data for plotting these
curves can be obtained by use of a computer-aided design
(CAD) program (8,21).
The log-magnitudeangle diagram is drawn by picking for
each frequency the values of log magnitude and angle. The
resultant curve has frequency as a parameter. The curve for
the Eq. (98) is drawn in Fig. 15(b). Note that the point 1
j0 on the polar plot becomes a series of points on the Nichols
plot, having the values (0 dB, k), where k takes on all odd
integer values. Changing the gain raises or lowers the
Lm G( j) curve without changing the angle characteristics.
Increasing the gain raises the curve, and analysis of the polar
plot has shown that this reduces stability.
The log-magnitudeangle diagram for G(s)H(s) can be
drawn for all values of s on the contour Q of Fig. 13(a). The
resultant curve for minimum-phase systems is a closed contour. Nyquists criterion can be applied to this contour by determining the number of points having the values 0 dB and
odd multiples of 180, which are enclosed by the curve of
G(s)H(s). This number is the value of N which is used in the
equation ZR N PR to determine the value of ZR. As an
example, consider a control system whose transfer function is
given by Eq. (98). Its log-magnitudeangle diagram, for the
contour Q, is shown in Fig. 16. From this figure it is seen that
the value of N is zero. Thus ZR N PR 0, and the system
375
Phase
margin
(+)
Lm G ( j ) (dB)
Lm (dB)
1
a
1
0 dB
(+)
360
270
Gain
margin (+)
0 Angle
90
180
c
Gain
margin, Lma
(+)
90
135
180
G( j)
225
270
Phase
margin,
(+)
=
(a)
(b)
(c)
Figure 15. Frequency response plots: (a) Polar plot, (b) Nichols plot, and (c) Bode plots of
G( j).
is stable. The log-magnitudeangle contour for a non-minimum-phase system does not close; thus it is difficult to determine the value of N. For these cases the polar plot is easier
to use to determine stability.
It is not necessary for minimum-phase systems to obtain
the complete log-magnitudeangle contour to determine stability. Only that portion of the contour is drawn representing
G( j)H( j) for the range of values 0 . The stability
is then determined from the position of the curve of
G( j)H( j) relative to the (0 dB, 180) point. In other
words, the curve is traced in the direction of increasing frequencythat is, walking along the curve in the direction of
increasing frequency. The system is stable if the (0 dB, 180)
point is to the right of the curve. This is a simplified rule of
thumb which is based on Nyquists stability criterion for a
minimum-phase system.
A conditionally stable system is one in which the curve
crosses the 180 axis at more than one point. Figure 17
shows the transfer-function plot for such a system with two
stable and two unstable regions. The gain determines
whether the system is stable or unstable.
Gain Margin and Phase Margin Stability from the Nichols Plot
The absolute stability of an LTI closed-loop system can be
determined by applying the Nyquist stability theorem, using
the representation of the open-loop transfer function G( j) as
a polar plot or as Nichols plot (log magnitude versus angle).
Some measures of degree of stability can be expressed in
terms of gain margin and phase margin. The following quantities are used to express these stability measures:
Gain Crossover. This is the point on the plot of the transfer function at which the magnitude of G( j) is unity
[Lm G( j) 0 dB]. The frequency at gain crossover is
called the phase-margin frequency .
Phase Margin. This is 180 plus the negative trigonometrically considered angle of the transfer function at the
gain-crossover point. It is designated as the angle ,
which can be expressed as 180 , where
G( j ) is negative.
Phase Crossover. This is the point on the plot of the transfer function at which the phase angle is 180. The fre-
This portion of
contour represents
the semicircle of
radius of contour Q
= 0+
= 0
0+
+ dB
+ dB
0 dB
270
180
90
+90
+180
+270
dB
= +180
G(s) Contour dB
This portion of contour represents the semicircle
of infinite radius r of contour Q
=
Figure 16. The log-magnitudeangle contour for
the minimum-phase system of Eq. (98).
(a)
Lm (dB)
Lm (dB)
0
180
Lm (dB)
Lm (dB)
376
to the gain margin. However, the phase margin gives a better

estimate of damping ratio, and therefore of the transient overshoot of the closed-loop system, than the gain margin.
The phase-margin frequency , phase margin , crossover
frequency c, and the gain margin Lm a are readily identified
on the Nichols plot, as shown in Fig. 15(b).
Bode Plots (Logarithmic Plots)
180
180
(b)
(c)
180
(d)
Figure 17. Log-magnitudeangle diagram for a conditionally stable

system: (a, c) stable; (b, d) unstable.
quency at which phase crossover occurs is called the

gain-margin frequency c.
Gain Margin. The gain margin is the factor a by which the
gain must be changed in order to produce instability.
Expressed in terms of the transfer function at the frequency c, it is
|G(c )|a = 1
(103)
On the polar plot of G( j), the value at c, as shown in

Fig. 15(a), is G( jc) 1/a. In terms of the Lm in dB,
the gain margin is
Lm a = Lm G( jc )
(104)
This is shown in Fig. 15(b) as the amount by which the

plot of G( j) must be raised so that it goes through the
0 dB, 180 (1 j0) point.
These quantities are illustrated in Fig. 15 on both the polar
and the Nichols plots. Note the algebraic sign associated with
these quantities as marked on the curves. Both cases shown
on Figs. 15(a) and 15(b) represent stable systems.
The phase margin angle is the amount of phase shift at
the frequency that would just produce instability. The
phase margin for minimum-phase systems must be positive
for a stable system, whereas a negative phase margin means
that the system is unstable.
It can be shown (8) that the phase margin angle is related to the effective damping ratio of the system. Satisfactory response is usually obtained with a phase margin of 45
to 60. As an individual gains experience and develops his
own particular technique, the desirable value of to be used
for a particular system becomes more evident. This guideline
for system performance applies only to those systems where
closed-loop behavior is that of an equivalent second-order system. The gain margin must be positive when expressed in
decibels (greater than unity as a numeric) for a stable system.
A negative gain margin means that the system is unstable.
The damping ratio of the closed-loop system is also related
The Bode plots consist of two components. One plot is for the
log magnitude [Lm G( j) 20 log G( j)] versus log , and
the second plot is for the angle of G( j) versus log . The
Bode plots are often used to represent the open-loop transfer
function. The log magnitude of G( j) converts the operations
of multiplication and divisoin to addition and subtraction, respectively. Each factor of a transfer function [see Eq. (91)],
plotted versus log , has a distinctive characteristic which is
readily recognized.
Constant Km. The Lm Km is a constant which is independent of frequency. Thus, it is a horizontal line and
serves to raise or lower Lm G( j), when it is larger than
or smaller than unity, respectively.
j Factor. The Lm G( j) has a positive slope of 20 dB per
decade when this factor appears in the numerator. Its
slope is negative when it appears in the denominator of
G( j). The transfer function type is related to the low
frequency slope of the Lm plot. A zero slope indicates a
Type 0 system, a slope of 20 dB/decade indicates a
Type 1 system, and so on.
1 jT Factor. The Lm(1 jT) has a corner frequency
at cf 1/T. At frequencies below cf the asymptote is
the 0 dB line. Above cf the asymptote is a straight line
that passes through 1/T at zero dB, with a slope of
20 dB/decade. When (1 jT) appears in the numerator of G( j), the slope of the asymptote is positive.
When it appears in the denominator, the slope of the
asymptote is negative.
Quadratic Factor [1 2j/n ( j/n)2]. The log magnitude of the quadratic factor has a corner frequency at
cf n. At frequencies below cf the asymptote is the 0
dB line. Above cf the asymptote is a straight line that
passes through cf at 0 dB with a slope of 40 dB/decade.
When the quadratic factor is in the denominator, the
slope of the asymptote is negative. When it appears in
the numerator, the slope of the asymptote is positive.
Because this quadratic factor has the damping ratio
as an additional variable, there is a family of log magnitude plots which depend on the value of .
The gain margin and the phase margin angle can be determined from the Bode plots, as shown in Fig. 15(c). Adjusting
the gain results in raising or lowering the log magnitude
curve, without changing the angle curve of G( j). This permits the designer to change the phase margin frequency ,
the phase margin angle , and the gain margin. For a stable
closed-loop system, it is necessary for both the phase margin
angle and the gain margin to be positive.
Experimental Determination of Transfer Function
The log-magnitudephase-angle diagram is of great value
when the mathematical expression for the transfer function
of a given system is not known. The magnitude and angle of

the ratio of the output to the input can be obtained experimentally for a steady-state sinusoidal input signal at a number of frequencies. These data values are used to obtain the
exact log-magnitudeangle diagram. Asymptotes are drawn
on the exact log magnitude curve, using the fact that their
slopes must be multiples of 20 dB/decade. From these asymptotes, the system type and the approximate time constants
are determined. Thus, in this manner, the transfer function
of the system can be synthesized (8, 24).
Care must be exercised in determining whether any zeros
of the transfer function are in the right-half s plane. A system
that has no open-loop zeros in the right-half s plane is defined
as a minimum-phase system (8). A system that has open-loop
zeros in the right-half s plane is a non-minimum-phase system. The stability is determined by the location of the poles
and does not affect the designation of minimum or non-minimum phase.
The angular variation for poles or zeros in the right-half s
plane is different from those in the left-half plane. For this
situation, one or more terms in the transfer function have the
form 1 Ts and/or 1 As Bs2. As an example, consider
the functions 1 jT and 1 jT. The log magnitude plots
of these functions are identical, but the angle plot for the former, as varies from 0 to , goes from 0 to 90, whereas for
the latter it goes from 0 to 90. Therefore, care must be
exercised in interpreting the angle plot to determine whether
any factors of the transfer function lie in the right-half plane.
CLOSED-LOOP FREQUENCY RESPONSE
Specifications for closed-loop system performance are often
stated either in terms of the time response characteristics
with a specified input or in terms of the frequency response.
The time response figures of merit with a unit step input may
include: peak overshoot, Mp; peak time, tp; final value, yss; and
settling time, ts. Additional figures of merit that may be specified include: system type, m; gain constant Km; and duplicating time, tD (time to first reach the final value). For systems
of higher than second order, an effective damping ratio eff
may be used to compare the response to that of a simple second-order system. The settling time ts is affected principally
by the dominant closed-loop poles, with modifying effects due
to the other poles and zeros. There is, therefore, an effective
damping ratio eff .
An alternate or supplementary way of identifying performance specifications is in terms of frequency response characteristics. For a simple second-order system, frequency response characteristics consist of a set of figures of merit which
are the maximum value Mm and the frequency m at which
the maximum value occurs. The relationships for a simple
second-order system are
Mm =
1
,
2 1 2

m = n 1 2 2
(105)
These values are correlated to the peak value Mp and time

response frequency of oscillation d in the time domain by

M p = 1 + exp
,
d = n 1 2
(106)
1 2
377
Therefore, designing for given values of Mm and m results in

effective values of eff and n. These, in turn, yield corresponding values of the time domain figures of merit which include
Mp, d, and, for a 2% settling time,
ts =
4
,
n
tp =
n 1 2
(107)
When G( j) is plotted in a polar (Nyquist) plot, there exists a family of constant M closed-loop contours, where M
C( j)/R( j). These contours are a set of circles with specified
centers and radii (8):
x0 =
M2
,
M2 1
y0 = 0,

M

r0 = 2
M 1
(108)
Therefore, the gain constant Km can be selected which makes

the plot of G( j) tangent to a desired value Mm. In accordance
with Eq. (105), this results in a corresponding set of values of
and n. This produces a set of affiliated values of Mp, d, ts,
and tp, as given by Eqs. (106) and (107). The polar Nyquist
plot therefore not only provides information on closed-loop
stability, but also provides a design process for determining
degree of stability. This degree of stability provides information on the time response characteristics.
The constant M circles on the polar plot can be transformed to the Nichols (Lm versus angle plot) plot. Then, adjusting the gain Km raises or lowers the plot of G( j). This
procedure can be used to make the plot of Lm G( j) tangent
to the desired M curve on the Nichols chart. The frequency at
the point of tangency determines the value of m. Equations
(105)(107) determine the time response characteristics.
NONLINEAR STABILITY AND CONTROL
In this article the stability of dynamical systems is treated in
a control theoretic context. Thus, the stability of feedback control systems is discussed. This is in the tradition of the classical Nyquist stability method in linear control theory covered
in the preceding sections, where, based on information about
the open-loop plant, the stability of the closed-loop feedback
control system is being ascertained. In the remaining sections
of this article the more realistic problem of the stability of
constrained feedback control systems is considered. Specifically, the stability of closed-loop feedback control systems
which consist of linear plants driven by actuation elements
which are subject to hard saturation constraints is addressed.
Obviously, under conditions of small perturbations, the
hard actuator constraints are not active and stability is then
determined according to the first method of Lyapunov using
conventional linear analysis methods. Attention is now given
to the treatment of high amplitude maneuvers and large perturbations away from trim, namely, the treatment of the stability in the large of control systems comprised of linear
plants and actuators which are subject to saturation.
Even though the primary interest is in stability, a complete analysis and a mathematically rigorous treatment of the
stability problem in a control context is undertaken by embedding the stability problem into the more general framework of tracking control. Thus, the problem of tracking a zero
exogeneous command signal naturally reduces to the investi-
378
gation of the feedback control systems stability. Moreover,

actuator saturation is addressed by utilizing a nonlinear
dual-loop control architecture, where (internally modified)
reference signals are generated by the feedback controller,
even in the absence of a nonzero exogeneous command signal.
Indeed, the feedback controller responds to the exogeneous
reference signal and to the current state measurement; and
the ill effects of saturation caused by large excursions in the
state can be mitigated by injecting into an inner-loop linear
controller a modified (nonzero) reference signal that tries to
pull the control system out of saturation. This situation,
where the state excursions are large, can arise even in the
case where the exogeneous reference signal vanishes for a period of timein which case the feedback control system is,
strictly speaking, a regulator. Finally, in the case where the
feedback control system is nonlinear, the question of input/
output [viz., bounded input, bounded output (BIBO)] stability
becomes dominant. Whereas it is well known that asymptotically stable linear control systems are BIBO stablethat is,
stability with respect to perturbations in the initial state also
guarantees the input/output stability of the control system,
and vice versathe same cannot be said about nonlinear control systems. Hence, the response of the nonlinear control system to exogeneous reference signals needs to be considered
and the broader issue of BIBO stability of tracking control
systems must be addressed.
In the remaining sections of this article, the design of nonlinear tracking controllers for the mitigation of actuator saturation effects is discussed. Evidently, the development of a
nonlinear controller synthesis methodology which addresses
the conflicting requirements of high-amplitude dynamic reference signal tracking and regulation in the face of hard actuator displacement and rate constraints, actuator dynamics,
and unstable open-loop plants is sorely needed. At the same
time, in many applications (e.g., in flight control), full-state
feedback can be assumed. Current analysis and design methods can address one, or some, of the above requirements, but,
taken together, these requirements represent, at best, a difficult task, in particular with respect to satisfying the inherent
tradeoffs between tracking control/regulation performance
and closed-loop nonlinear system input/output and asymptotic stability. The objective here is to critically examine the
state of the art and point the reader to a workable tracking
control paradigm which, from the outset, also acknowledges
actuator constraints.
The adverse effects of controller-induced state and/or control constraint violation on closed-loop system performance
and stability are well documented for both stable and unstable open-loop plants, namely, degraded system performance,
limit cycling, and unstable system response. In addition, in
feedback control systems with open-loop unstable plants (e.g.,
in modern flight control) and where feedback is used for stabilization, saturation can cause the feedback loop to be opened
and immediately induce instabilityas opposed to more benign open-loop stable plants, where feedback does not play
such a critical role. Moreover, in flight control, and for properly designed aircraft, actuator rate saturation usually occurs
before actuator deflection saturation.
There is much interest with regard to constrained control
in the recent literature, and actuator saturation is a topic of
active research in control theory. Thus, numerous controller
design and modification methodologies exist which address
linear time-invariant (LTI) systems with state and/or control

constraints (11,2448). Much attention has been given to regulation and set-point control, as opposed to tracking control.
Also, most of the work fails to address actuator rate saturation or, for that matter, actuator dynamics. Notable exceptions concerning tracking control are Refs. 30, 33, 34, 46, and
51, and rate saturation is explicitly considered in Refs. 30, 38,
and 46. In most cases, open-loop stable plants are assumed
and conditions are devised for global stability. In these investigations the compensator is allowed to send infeasible control
signals to the actuator. In the landmark treatise (50), feedback control systems with amplitude constrained actuators
and stable open-loop plants are considered. Necessary and
sufficient frequency-domain conditions, which are a generalization of the Nyquist criterion, are given for global stability
in the face of windup. Many of the proposed methodologies
can be classified as anti-windup methods (26,35,44): The goal
of anti-windup controllers is to avoid or delay actuator saturation-induced windup of linear dynamic compensation. While
these methods may prevent or reduce windup and improve
closed-loop system performance, they do not fully prevent constraint violation, namely, actuator saturation, which is the
root cause of windup. Ad hoc anti-windup compensators
(26,33,35,37,39,44,52), dont perform well in the case of openloop unstable plants with actuator displacement and rate constraints. Finally, unstable open-loop plants are specifically
addressed in Refs. 28, 30, 34, 46, and 51.
State and control constraint mitigation strategies based on
saturation avoidance focus on constraint violation prevention.
Recent work on saturation avoidance methods has produced
several important results based on the concepts of output admissibility and static admissibility which are applicable to the
problem of tracking control (11,29,30,3449). Gilberts discrete-time reference governor (DTRG) (3132) is representative of these methods. Another interesting method is the linear quadratic tracking (LQT) concept (4042), which uses a
receding horizon optimal control paradigm to avoid downstream state and control constraint violations.
In this article, and motivated by problems in modern flight
control, the synergetic consideration of stability and tracking
control of open-loop unstable plants, with actuators subject to
rate and amplitude saturation, using state feedback, is undertaken. Since Refs. 31, 32, and 4042 are representative of the
state of the art with respect to constrained tracking control,
their results are reviewed here. A viable feedback control concept is presented in the section entitled Control Concepts.
The section entitled Linear Quadratic Tracking (LQT) discusses the LQT methodology for saturation avoidance and improved tracking performance developed in Refs. 4042. The
section entitled Maximal Output Admissible Sets discusses
the concept of maximal output admissible sets which play a
crucial role in the synthesis of BIBO tracking control systems.
The section entitled Discrete-Time Reference Governor is
devoted to Gilberts DTRG, which is illustrated using a scalar
example. In the section entitled Static Admissibility and Invariance the same scalar example is used to investigate more
general invariance-based control methods with guaranteed
BIBO stability and also yield improved tracking performance;
some fine points are illustrated. A suboptimal approach is
presented in the section entitled Suboptimal Approach,
which is followed by concluding remarks in the section entitled Conclusion.
379
Limiter
r
kr
uc
Plant = (A, B)
kx
Figure 18. Current control system with open-loop limiter.
CONTROL CONCEPT
Figures 1821 show examples of nonlinear control system architectures for linear systems with control constraints.
Figure 18 shows the typical nonlinear control system architecture currently used in (flight control) practice: A simple
open-loop limiter is used to restrict the reference signal and
help mitigate the adverse effects of actuator saturationcaused windup. Figure 19 is a general nonlinear feedback
control system architecture where the nonlinear controller,
G, computes the commanded control signal based on the current plant state and the current, and possibly past, values of
the reference signal. The controller is cognizant of the saturation level of the downstream actuator which is located at the
plant input. Hence, the controller-generated signal, uc, may
attain its limits, and by doing so the plant is driven to its
full capacity. Figure 20 shows a specific example of a general
nonlinear controllers architecture. A nonlinear dual-loop control system is envisaged where the controllers nonlinearity is
confined to a feedback limiter, N, which performs a nonlinear
scaling of the reference signal. The inner loop controller is
linear and the nonlinear element N is contained in the outer
loop, as illustrated in Fig. 20. Furthermore, the inner-loop linear controller is the result of a linear design, wherein it is
assumed that the actuator operates in the linear regime.
Hence, the inner loop is a linear control system. The nonlinear element N in the outer loop is a (pilot) command limiter
which employs feedback. The nonlinear element N scales the
reference signal such that the commanded control signal uc
does not violate the actuator imposed control constraints in
the inner loop: The feedback (pilot) command limiter generates a modified reference signal r which drives the inner loop
in such a way that saturation in the downstream actuator is
precluded, and strictly linear action ensues in the inner loop.
The outer loops nonlinearity N renders transparent the actuators nonlinearity in the inner loop, and therefore the feedback control system shown in Fig. 20 is equivalent to the simpler feedback control system of Fig. 21, where the inner loop
is linear. In conclusion, in Figs. 20 and 21 the linear inner
loop controller is designed to meet small signal tracking and
stability specifications, and the nonlinear element N in the
outer loop is designed to yield good tracking performance and
uc
Plant = (A, B)
BIBO stability. Obviously, the design challenge is to maintain

tracking performance and the BIBO stability guarantee of the
dual loop nonlinear control system, for a relatively large class
of exogeneous reference signals, and for a sufficiently large
set of initial states.
Clearly, the control schemes shown in Figs. 20 and 21 are
more advanced than the simple control scheme shown in Fig.
18, and are discussed in this article.
LINEAR QUADRATIC TRACKING (LQT)
It is convenient to address the actuator constrained tracking
control problem using a time-domain, receding-horizon, optimal control formulation. Now, application of optimal control
methods to the tracking problem requires a priori knowledge
of the dynamic reference signal for all time. This also applies
to linear quadratic (LQ) optimal control. Also, knowledge of
the reference signal ahead in time is required, irrespective of
whether the control signal is, or is not, constrained. Of course,
this a priori knowledge does not exist, and in Refs. 4042,
tracking control is addressed using reference signal extrapolation and implementing a receding-horizon optimal control
methodology in which the exogeneous (pilot) reference signal
is predicted over the optimization horizon, based on current
and near past pilot reference signal values. Specifically, polynomial extrapolation and interpolation is used to obtain the
reference signal over the predetermined optimization horizon.
Then, within each receding horizon window (RHW), the reference signal vector, r, is known, so that the LQ optimal control
methodology may be applied to each of the finite-horizon control problems. The resulting finite-horizon LQ optimal control
problem is solved within each RHW. In this way the indefinite-horizon control problem is broken up into an open-ended
sequence of finite-horizon control problems. In a discrete-time
formulation the optimization window is of length N ( the
number of samples within each RHW).
Specifically, an inner-loop linear quadratic controller, designed for good small-signal performance, minimizes a quadratic cost functional over the optimization horizon N. Thus,
suppose that the bare plant is
x(k + 1) = Ax(k) + Bu(k),
x(0) = x0 ,
k = 0, 1, . . ., N 1
x
Figure 19. Nonlinear control system.
380
kr
uc
Plant = (A, B)
x
kx
Figure 20. Two-loop control concept.
and the tracked output signal is
manded reference signal at time now, and is not a predicted

value. Hence, in the current optimization window, LQ optimal
control returns the optimal control time history
y(k + 1) = Cx(k + 1)
u (i) = ki x x0 + ki r r1 ,
The cost functional is
J(u) =
N

[Q(r(k + 1) y(k + 1))2 + Ru2 (k)]
k=0
The Q and R weights establish the tracking performance/control effort tradeoff in the inner (linear) loop and determine
the small signal performance of the control system. Fullstate feedback is assumed.
For a given reference sequence r, LQ optimal control returns the optimal control time history u* [u*0 , u*1 , . . .,
u*N1], obtained within each RHW as a linear function of the
complete reference vector r [r1, r2, . . ., r N], and the initial
plant state, x0, where r1 is the currently commanded reference
signal. Now, polynomial extrapolation and interpolation
yields r2, . . ., rN, the reference signals prediction, linear in
the data. Specifically, if, for example, a simple ZOH extrapolation strategy is used, it is shown in Ref. 40 that the predicted
reference vector [r2, . . ., r N] is linear in r1; that is, r is linear
in r1, which ultimately yields the optimal control sequence
[u*0 , u*1 , . . ., u*N1] linear in r1, the current pilot-demanded
reference signal, and x0. Note that r1 is the actual pilot de-
i = 0, 1, . . ., N 111.1
In particular, at time 0, at the start of the window, u*(0)

k0xx0 k0rr1. In the sequel, the notation used is
kx k0 x ,
kr k0 r
so that
u (0) = kx x0 + kr r1
(110)
Both kx and kr are provided by the solution of the finite horizon LQ optimal control problem. The row vector kTx Rn corresponds to the gain derived from the solution, over the finite
optimization horizon, of the Riccati difference equation; this
same Riccati equation is also associated with the solution of
the finite horizon regulation problem. Moreover, one can deviate from the optimal tracking control solution in the RHW
and instead, once kx has been determined, choose the gain kr
such that asymptotic tracking of a step reference command r
is enforced. Of course, if integral action is used to obtain a
type-one system, then asymptotic tracking is achieved for
any kr.
kr
(109)
uc = u
Plant = (A, B)
x
kx
Figure 21. Saturation mitigation concept.
Furthermore, since the optimal control time history u* is

a linear function of r1 and x0, the actuator amplitude constraints of, say, umax, can be readily transformed into constraints on the reference signal at time now, r1: In the RHW
umax ki x x0 + ki r r1 umax ,
i = 0, 1, . . ., N 1
there is nothing that can be done about the plants initial

state x0. Hence, for example, for kr 0 the above inequalities
yield the explicit constraints on the exogeneous reference signal:

umax ki x x0
umax ki x x0
r1 min
max
0iN1
0iN1
ki r
ki r
Next, differencing Eq. (109) yields an explicit expression for
the actuator rate, namely,
1
(u ui1 )
T i
1
(k x + ki r r1 ki1 x x0 ki1 r r1 )
=
T i x 0
1
[(ki x ki1 x )x0 + (ki r ki1 r )r1 ]
=
T
u i
Similar to the derivation for actuator amplitude constraints,

the above equations are manipulated to yield an additional
explicit bound on the reference signal at time now, r1,

T u max (ki x ki1 x )x0
,
r1
max
1iN1
ki r ki1 r

u1 kx x0 T u max
kr

r1
min
1iN1
T u max (ki x ki1 x )x0

ki r ki1 r

,
u1 kx x0 + T u max
kr
(111)
where u*1 is the control signal from the last window. Hence,
actuator constraints are easily transformed into constraints
on the current reference signal. Saturation avoidance is guaranteed, provided that the reference signal at time now, r1,
satisfies the above inequalities.
Actuator displacement and rate constraints impose constraints on the inner closed-loops exogenous reference signal.
Thus, an additional outer control loop is employed which performs a nonlinear modification of the reference signal from
the pilot, such that the downstream actuator constraints in
the inner loop are satisfied. Now, if r1 causes the violation of
any of the constraints, it may be optimally scaled such that
the modified reference signal, r1, satisfies all constraints;
here, optimally scaled means that in the outer loop r1 is
chosen so that, for example, r1 r1 is minimized, subject to
r1 satisfying the actuator constraints.
In accordance with the receding-horizon modus operandi,
at each time step the optimization window is shifted forward
in time, a new state is attained, and upon receipt of a new
reference signal from the pilot the reference signal extrapola-
381
tion algorithm is reapplied and the open-loop LQ optimal control problem is solved from the beginning. Hence, r1 (or r1) is
the only reference value actually tracked; and within each
RHW, u*0 is the only control which is actually applied to the
system, so that feedback action is achieved. Now, the windows local time instant 0 corresponds to the current time
k. Thus, at time k, the optimal control signal u*k satisfies
u (k) = kx x(k) + kr r(k + 1)
and since the nonlinear element N in the outer loop causes
the exogeneous reference signal rk1 to be replaced by rk1,
which is then sent to the inner loop, the actual optimal control signal is
u (k) = kx x(k) + kr r (k + 1)
(112)
Many of the constraints on rk1 are induced by constraints on

subsequent elements of the predicted reference vector (rk2,
rk3, . . ., rkN). But these reference signals are not actually
realized, so enforcement of these downstream constraints
may lead to an overly conservative scaling of rk1. Thus, the
designer may wish to specify feasibility criteria which do not
necessarily include all the constraints in the RHW. An important special case entails the enforcement of actuator constraints at time now only, whereupon the inequalities from
above are reduced to

umax ki x x0 u1 kx x0 T u max
r1 max
,
ki r
kr

(113)
umax ki x x0 u1 kx x0 T u max
,
r1 min
ki r
kr
Finally, if rk1 satisfies the applicable constraints, then N
is transparent and clearly rk1 rk1, that is, tracking performance during small-signal operation is not sacrificed. Then,
the closed-loop linear system, which translates the exogeneous r command into aircraft responses, is given by
x(k + 1) = Acl x(k) + Bcl r(k + 1)
y(k + 1) = Cx(k + 1)
where Acl A Bkx, Bcl Bkr, and the actuator displacement
is
u(k) = kx x(k) + kr r(k + 1)
In general, the nonlinear LQT control law depends on (1) the
choice of the Q and R weights for small-signal performance,
(2) the reference vector prediction method, (3) the selection of
the applicable feasibility criteria for saturation avoidance,
and (4) the modified reference signal optimization criterion.
So far, only saturation avoidance was considered for modified reference signal feasibility determination. BIBO stability
is only guaranteed under certain additional restrictive conditions. Thus, the design for guaranteed BIBO stability of the
control system shown in Figs. 20 and 21 is now undertaken.
MAXIMAL OUTPUT ADMISSIBLE SETS
In Ref. 32 the concept of maximal output admissible set is
employed to develop reference governors N for discrete-time
382
systems. The discrete-time reference governor (DTRG) developed in Ref. 32 is a nonlinear dynamical element, that is, it
is a first-order lag filter with a variable bandwidth parameter, [0, 1], which scales the reference signals increments
so that the controlled systems constraints are not violated.
These constraints characterize the maximal output admissible set. By forming a modified reference signal such that the
discrete-time systems state update satisfies these constraints, both saturation avoidance and BIBO stability are
enforced: BIBO stability also requires that the maximal output admissible set be bounded. In the sequel, the DTRGbased approach to tracking control is discussed. More importantly, the construction of the maximal output admissible set
is presented. This is a critical step in the development of more
general BIBO tracking control systems, in both the discretetime and the continuous-time settings.
The LTI discrete-time system
x(0) = x n ,
u(t) ,
(114)
t = 0, 1, 2, . . . (= I + )
is considered. Initially, the regulation problem is exclusively

analyzed and the prespecified small-signal linear-state feedback control law is
u(t) = kx x(t)
Also, assume there are constraints on both the state and control vectors, including linear combinations of them. Then,
with appropriate choices of matrices C and D, along with a
set Y, these rather general constraints may be represented as
y(t) = Cx(t) + Du(t) Y p
(115)
Let Acl A Bkx and Ccl C Dkx. Then, Eqs. (114) and
(115) become
x(t + 1) = Acl x(t),
x(0) = x,
y(t) = Ccl x(t) Y
t I+
(116)
Hence, the saturation avoidance problem has been transformed into a feasibility problem consisting of an unforced
LTI discrete-time system with an output constraint. In Ref.
31 the maximal output admissible set associated with system
(116) is defined as the set of all initial states x n such that
the unforced closed-loop linear systems response does not violate the system output constraints for all t I. Thus, the
maximal output admissible set is the largest set of initial
states, x n, that is positively invariant w.r.t. system [see
Eq. (116)]. Evidently, the maximal output admissible set is
O (Acl , Ccl , Y ) = {x n : Ccl Atcl x Y
t I+ }
(117)
Example 1. The Output Constraint Set Y 0. The maximal output admissible set is then the subspace of unobservable states that corresponds to the pair (Acl, Ccl), namely,
O = (N(Ccl ))A
O (Acl , Ccl , Y ) = {0}

iff
1. The pair (Acl, S) is observable, where the n-column matrix S is such that Sx 0 implies that Cclx Y, and
2. Acl does not have an eigenvector v which corresponds to
a nonnegative eigenvalue of Acl such that Cclv Y.
For a proof, see, for example, Ref. 45. Of major interest is the
case where the constraint set Y is a polyhedron, namely,
Y = {y p : f i (y) 0,
x(t + 1) = Ax(t) + Bu(t),

m
Example 2. The Set Y is a Cone. This is the situation

where there are one-sided control constraints, for example,
0 u(t)in which case C nonnegative orthant in m set
of vectors in m with non-negative entries. Then
cl
This is the largest subspace contained in the null space of the

matrix Ccl which is invariant under Acl.
i = 1, 2, . . ., s}
(118)
That is, the s functions f i(y), f i : p are linear in y, with

f i(0) 0. Obviously
O (Acl , Ccl , Y ) = {x n : f i (Ccl Atcl x) 0,
i = 1, 2, . . ., s, and t I + }
(119)
Although Y is a polyhedron, Eqs. (117) and (119) each represent an infinite number of constraints. However, if there exists a finite t*i I for each inequality in (119) for which the
constraints associated with the ith inequality constraint are
inactive for t t*i , let t* 1 i s maxt*i . Then O(Acl, Ccl,
Y) is characterized by a finite number (st*) of inequality constraintsin this case, O(Acl, Ccl, Y) is said to be finitely determinedand determination of whether a particular initial
state vector, x, is an element of O(Acl, Ccl, Y) involves evaluation of a finite set of linear inequalities. Moreover, it may
transpire that the constraints associated with one or more of
the inequalities, f i, that define Y are inactive for all t I.
Thus, let S* denote the set of inequality constraints that are
active for some t I, namely, S* 1, 2, . . ., s. Then,
O(Acl, Ccl, Y) may be written as
O (Acl , Ccl , Y ) = {x n : f i (Ccl Atcl x) 0,
t = 0, 1, . . ., ti , and i S }
(120)
In the discrete-time case under consideration, sufficient conditions for the existence of a finite t*, namely, sufficient conditions for the finite determination of O, are as follows: (1) Acl
is asymptotically stable, (2) 0 int(Y), (3) Y is bounded, and
(4) the pair (Acl, Ccl) is observable. Here, int( ) denotes the
interior of the set.
The concept of maximal output admissible sets is only applicable to unforced systems. Thus, it is not directly applicable to the tracking problem without some modification. In Ref.
32 the concept of maximal output admissible sets is adapted
for use with the tracking problem by using a nonlinear element in the DTRG which has first-order dynamics; that is, a
first-order lag filter with a variable bandwidth parameter
[0, 1] is used to prefilter the exogenous reference signal. The
closed-loop system state vector is then augmented with the
prefilter state, namely, the modified reference signal, r. The
end result is an augmented system whose exogenous input
can be turned off by setting 0. Thus, consider the tracking

control problem
x(t + 1) = Ax(t) + Bu(t),
x(t) n , u(t) m
u(t) = kx x(t) + kr r(t)
(121)
with state and control constraints

y(t) = Cx(t) + Du(t) Y p
(122)
and where the control constraint set Y is given by Eq. (118).

The ensuing closed-loop system is
x(t + 1) = Acl x(t) + Bcl r(t)
(123)
y(t) = Ccl x(t) + Dcl r(t) Y p
where, as before, Acl A Bkx, Ccl C Dkx, and Bcl

Bkr, Dcl Dkr. To transform this problem into one which
allows use of the concept of maximal output admissible sets,
the exogenous reference signal, r, is prefiltered by the firstorder lag filter given by

r (t + 1) = r (t) + (r(t), xg (t)) r(t) r (t)
(124)
where (r(t), xg(t)) [0, 1]; and in Eq. (119) r(t) is replaced by
the filters output r(t) (the modified reference signal). Then,
the augmented state is
r
xg =
x
(125)
inequality constraintsthen at each time-step the upper

limit imposed on the scalar (r(t), xg(t)) by each linear inequality constraint is given by a simple algebraic formula. Thus, at
each time-step (r(t), xg(t)) [0, 1] may be chosen so that it
satisfies the minimum of the upper limits and is easily computed on-line. Hence, a tracking controller could easily be
synthesize; moreover, the output vector is guaranteed to be
bounded.
Unfortunately, application of the concept of maximal output admissible sets to the tracking control problem generally
results in an augmented dynamics matrix Ag which is only
Lyapunov stable. Thus, O(Ag, Cg, Y) is generally not finitely
determined. The reason O(Ag, Cg, Y) is not finitely determined stems from the fact that the unforced response of a
Lyapunov stable linear system does not decay to the origin,
unlike that of an asymptotically stable linear system. In fact,
the unforced response of the Lyapunov stable linear system
does converge to an a priori unknown equilibrium point. In
general, the equilibrium point will not be reached in a finite
number of time-steps, and thus t* cannot be bounded. The
following will be useful in resolving this problem.
The set of statically admissible inputs, Rs, for the system
defined by Eq. (123), is the set of constant inputs for which
the associated equilibrium point is admissible. That is,
Rs = {r m : yss Y }
where yss is the steady-state response. yss Hor, where Ho
Ccl(I Acl)1 Bcl Dcl. Hence, an equivalent expression for Rs
is
Rs = {r m : Ho r Y }
and the augmented system dynamics and the output constraints are given by

xg (t + 1) = Ag xg (t) + Bg (r(t), xg (t)) r(t) [I
y(t) = Cg xg (t) Y
0]xg (t)
(126)
where

Ag =
I
Bcl
0
,
Acl
I
Bg =
,
0
Ccl ]
From Eq. (124) notice that if (r(t), xg(t)) 1, the exogenous

reference signal, r(t), is passed through unmodified, but with
a one time-step delay. More importantly, if (r(t), xg(t)) 0
the current modified reference signal, r(t), remains unchanged, and Eq. (126) becomes an unforced system with output constraints. Thus, if at time t and for all initial
states xg() O(Ag, Cg, Y), (r(), xg()) can be chosen such
that the updated state xg( 1) O(Ag, Cg, Y), it is possible
to guarantee that the controlled systems state and control
constraints will not be violated in the future. Moreover, assume that O(Ag, Cg, Y) is bounded. Then because 0 is
always an option and because the new reference signal is chosen such that it does not remove the state of the plant from
the compact set of admissible states, the BIBO stability of
tracking control systems which employ the DTRG is guaranteed.
If Y is given by Eq. (118) and O(Ag, Cg, Y) is finitely determinedthat is, it is characterized by a finite set of linear
(127)
The set of statically admissible states, Xs, for the closedloop system defined by Eq. (123), is the set of initial states for
which there exists a statically admissible reference signal
such that the ensuing trajectory does not violate the systems
state and control constraints for all time. That is,
Xs = {x n : r Rs
Cg = [Dcl
383
s.t.
y(t) Y
t I+ }
(128)
For the augmented system defined by Eq. (126), the set of

statically admissible states, Xgs, is
Xgs = O (Ag , Cg , Y )
(129)
Similar to the maximal output admissible set, both Xs and

Xgs are positively invariant sets. Xs is the projection of Xgs onto
the plane r 0. Now, recall that the reason O(Ag, Cg, Y) is
not finitely determined is that an infinite number of timesteps is required to reach the unknown equilibrium point.
However, if it is somehow known that the unforced system
given by Eq. (126) will converge to an equilibrium point xg
int(Xgs), then only the transient response for a finite number
of time-steps needs to be considered, until the peak-to-peak
magnitude of constrained quantity oscillations decay sufficiently. If r is restricted to values in int(Rs) and if x(0) Xs,
then x(t) Xs and xg(t) int(Xgs) for all t I. Moreover, the
set int(Xgs) int( O(Ag, Cg, Y)) is finitely determined.
Now, define Y() Y by Y() y : f i(y) , i 1, ,
s, where 0 minf j(0): i 1, , s. Then, to restrict
r to values in int(Rs), which results in xg int(Xgs), append
384
the additional constraints

f i ([Ho
cable constraints are
0]xg ) ,
i = 1, . . ., s
x(k + 1) = ad x(k) + bd r (k)
(130)
to those that define O(Ag, Cg, Y). Denoting int(Xgs) by Xgs

yields

Xgs
= {xg n+m : f i (Cg Atg xg ) 0, t = 0, . . ., ti , i S ,
and f j ([Ho
0]xg ) , j = 1, . . ., s}
(131)
Now, Xgs is characterized by a finite set of inequality constraints. Moreover, the upper limit imposed on (r(t), xg(t))
[0, 1] by each inequality, so that xg(t 1) Xgs, is given by a
simple formula. Thus, if xg(0) Xgs, and at each time-step we
choose (r(t), xg(t)) [0, 1] such that it satisfies the minimum
of the upper limits imposed by all inequality constraints in
Eq. (131), then xg(t) Xgs for all t I.
The finite determination of the maximal statically admissible (invariant) set is exclusively an artifact of discrete-time
dynamics. The maximal output admissible invariant sets of
continuous-time systems are not polyhedral.
y(k) = u(k) = cx(k) + dr (k) Y R
(138)
Y = {y : f i (y) 0, i = 1, 2}
(139)
where ad 0.999, bd 9.995 104, c kx 3, d kr

1, f 1(y) y 1, and f 2(y) y 1. r(k) is the feasible reference signal.
The reference governor is
r (k + 1) = r (k) + (k)[r(k) r (k)]
xg (k + 1) = Ag xg (k) + Bg (k)(r(k) [1 0]xg (k))

y(k) = Cg xg (k) Y
(132)
and the small-signal, linear control law is

u = k x x + k r r
(133)
Combining Eqs. (132) and (133) gives

x = acl x + bkr r ,
acl = a + bkx
(134)
Now, choose kr such that asymptotic tracking is enforced,

namely,
kr =
a
a
kx = cl
b
b
1
0
Ag =
9.995 104 0.999

1
Bg =
,
Cg = [1 3]
0

Now the maximal output admissible set for the augmented

system is concerned with the homogeneous system [(k) 0]
and is defined as
O = {x : f i (Cg Akg xg ) 0, k = 0, . . ., ki , i S }
x = acl (x r )
r W0 = {r : H0 r Y ()}
(144)
H0 = d + c(1 ad )1 bd = 1.9985
(145)
(135)
where
and
a
u = kx x cl r
b
(143)
where the constraints, f i(CgAgkxg) 0, are inactive for k k*i

and i S* 1, 2. Since the augmented system is only Lyapunov stable, k*i may be unbounded, so a finitely determined
approximation, O, to the maximal output admissible set is
needed. As noted in the section entitled Maximal Output Admissible Sets, O is obtained by restricting the feasible input, r, to values that are statically admissible with respect
to the reduced constraint set Y() y : f i(y) , i 1, 2,
where 0 minf i(0): i 1, 2. Thus, the additional constraint is added:
Then, the inner loop in Figs. 20 and 21 is

(142)

r (k)
xg (k) =
x(k)
and
1 u 1
(141)
where the augmented state is
The concept of a statically admissible set and the discretetime reference governor (DTRG) are illustrated. Specifically,
a tracking controller is synthesized for a constrained scalar
control system. The continuous-time control system is given
by
x(0) = xo ,
(140)
Combining the closed-loop system and reference governor dynamics results in the augmented second-order system
DISCRETE-TIME REFERENCE GOVERNOR
x = ax + bu,
(137)
(136)
Application of the DTRG to the system represented by Eqs.

(132) and (133) requires the consideration of the equivalent
discrete-time closed-loop system. For example, with a 2,
b 1, kx 3, kr 1, and a sampling interval of T 0.001 s,
the equivalent discrete-time closed-loop system and the appli-
Then,
O = {x : f i (Cg Akg xg ) 0, k = 0, . . ., ki , i S ,
f j ([H0
0]xg ) , j = 1, 2}
(146)
and k*i is guaranteed to be bounded (32). Notice that O is

composed of two sets of constraints. The first set ( fi(CgAgkxg)
0, k 0, . . ., k*i , i S*) deals with saturation of the transient response, and the second set ( fj([H0, 0]xg) , j 1, 2)
deals with saturation in steady state. Now, Algorithm 3.2 of
Ref. 31 may be used to determine k*i and S*. In this case S*
is empty. That is, the inequalities associated with the transient saturations are inactive. This should be expected because the closed-loop system of Eqs. (137) and (138) is a stable
first-order (over damped) system. Thus, Eq. (146) becomes
O = { f j ([H0
0]xg ) , j = 1, 2}
(148)
where

min 1, 0.95 H0 r (k)
H0 (r(k) r (k))
1 (k) =
1,
if H0 (r(k) r (k)) > 0

if H0 (r(k) r (k)) 0
(149)
and

min 1, 0.95 + H0 r (k)
H0 (r (k) r(k))
2 (k) =
if H0 (r (k) r(k)) > 0

if H0 (r (k) r(k)) 0
(150)
In this case, the end result of Eqs. (148)(150) is to limit the

feasible reference signal such that
0.47536 r 0.47536
over, the DTRG is now replaced by a static nonlinearity N

whose synthesis is outlined. The flexibility afforded by this
less conservative and more general approach makes it possible to consider two separate tracking control concepts. The
first tracking control concept is minrr r subject to the control constraints. The second control concept for the selection
of r attempts to drive the output x to r as quickly as possible,
subject to the control constraints. A continuous-time derivation is given.
(147)
Now, with 0.05, Eqs. (3.10) and (3.15) of Ref. 32 may

be used to generate (k), namely,
(k) = min{1 (k), 2 (k)}
385
(151)
While the above reference governor avoids saturation and affords BIBO stability, it is somewhat conservative in that it
restricts the feasible reference signal, r, such that at all
times it is always statically admissible. The BIBO stable
tracking controller developed in Refs. 4042 employs a static
nonlinearity N; it is nevertheless related to Gilberts DTRG
(32) in that it also includes static admissibility of the modified reference signal in the feasibility criteria. However, there
are certainly cases where a modified exogenous reference input that is not statically admissible over a finite time interval
would not necessarily result in saturation. Thus, as discussed
in the next section, the achievable tracking performance can
be enhanced, while at the same time the constraint violation
is avoided and BIBO stability is guaranteed.
STATIC ADMISSIBILITY AND INVARIANCE
The scalar constrained control system is revisited and the application of statically admissible and more general (maximal
output admissible) invariant sets to achieve saturation avoidance and BIBO stability enforcement is demonstrated. The
construction of these invariant sets is a crucial step in the
design of tracking control laws. Insights into the critical role
played by the above-mentioned invariant sets permit focusing
on design for tracking performance, while at the same time
guaranteeing the BIBO stability of the control system. More-
Control Concept 1: Choose r such that r r is minimized subject to the control constraints. Assuming kx 0,
acl 0, and b 0 results in the explicit (nonlinear) control
law
b
bkx
b
bkx
r
x+
r
x
if
a
a
a
a
cl
cl
cl
cl
bk
b
bkx
b
x

x
if
r
x
r (x, r) =
acl
acl
acl
acl
b
bkx
b
bkx
if
r
x+
a x+ a
a
a
cl
cl
cl
cl
(152)
and substituting Eqs. (152) into (135) results in the closedloop system
bkx
b
bkx
b
x+
r
x
acl (x r) if
a
a
a
a
cl
cl
cl
cl
bkx
b
if
r
x
x = ax + b
(153)
acl
acl
bkx
b
if
r
x+
ax b
acl
acl
The saturation avoidance control law (152) does not guarantee BIBO stability in the case of an open-loop unstable plant.
However, the desired BIBO stability can be obtained through
an additional invariance requirement.
A bounded set XI X x: x is invariant with respect
to the system given by Eqs. (132), (133), and (153) if and only
if on the boundary of XI, xx 0. Thus, if a bounded invariant
set is characterized as
!
a
XI = x : r such that 1 u = kx x cl r 1 and xx 0
b
then, by restricting x to x XI, BIBO stability is guaranteed.
First, consider the case of an open-loop unstable plant, a 0.
For (bkx /acl)x b/acl r (bkx /acl)x b/acl, the result is r
r, and x acl(x r). Hence, r x results in x 0, and r x
results in x 0. Second, for r (bkx /acl)x b/acl we have
x ax b. In this case x b/a results in x 0, and x
b/a results in x 0. Finally, for r (bkx /acl)x b/acl we
have x ax b. Therefore, x b/a results in x 0, and
x b/a results in x 0. This is summarized in Fig. 22, where
the directions of the arrows represent the sign of x in the
Cartesian product space, V, defined by
V = {v 2 : v = [r, x]T , r, x }
From Fig. 22 and the above discussion it is clear that if the
control law allows x b/a the system will diverge due to the
constraint on the control signal, u. Also, if the system ever
386
kx x
b
acl
acl
r=1
b
subject to the control constraints in Eq. (133). Thus,

a
kx x cl r = 1
b
If r > x, then r max(x)
If r < x, then r min(x)

If r = x, then r min(x2 )
b
a
From Eq. (135) and the assumption that acl 0, this results in
x=r
b
a
1
kx
1
kx
b
a
If r > x, then max r

If r < x, then min r
If r = x, then min (x r )2
subject to
a
b
acl
Figure 22. Cartesian product space, V, showing invariant region for

the scalar system.
achieves either of the equilibrium points v [b/a, b/a]T,

it becomes stuck. Thus, in addition to control law (152),
there must be a limit on x such that

b
b
x XI = x : + x
a
a
(154)
then saturation will be avoided, and the system output will

be bounded. Hence, the control law [see Eq. (152)] is modified
as follows:
a

r (x, r) =
b
+
bk
bkcl
a x+
cl
bkx
b
bkx
b
x+
r
x
acl
acl
acl
acl
b
b
and + x
a
a
b
b
if x > and r
a
a
b
b
if x < + and r +
a
a
bkx
b
b
if r >
x
and x
acl
acl
a
bkx
b
b
if r
x+
and x +
acl
acl
a
(155)
if
b
acl
b
acl
Now, under the explicit (nonlinear) control law [see Eq.

(155)], x XI is enforced, the system will not become stuck
at x b/a, and saturation avoidance plus invariance in a
bounded subset of the state space yield the BIBO stability
guarantee.
A similar analysis for the case of an open-loop stable plant
(a 0) shows that the saturation avoidance set is XI 1.
Hence, the simpler control law [see Eq. (152)] yields global
BIBO stability in this case.
Control Concept 2: Choose r so as to either maximize or
minimize x, based on the sign of r x, and subject to the
control constraints, therefore maximizing the instantaneous
reduction in tracking error. Also, if r x, then minimize x2,
bkx
b
bkx
b
x+
r
x
acl
acl
acl
acl
Hence, the explicit control law is finally obtained
bkx
b
x
, if r > x
a
a
cl
cl
b
bkx
x+
, if r < x
a
a
cl
cl
b
b
if r = x and x
r = r
a
a
bkx
b
b
if r = x and x <
a x a
cl
cl
bkx
b
b
if r = x and x >
a x+ a
a
cl
(156)
(157)
cl
and substituting Eq.

closed-loop system,
ax + b
ax b
x = 0
ax + b
ax b
(157) into Eq. (135) results in the
if r > x
if r < x
b
b
x
a
a
b
if r = x, and x <
a
b
if r = x, and x >
a
if r = x, and
(158)
If the open-loop system is stable (i.e., a 0), then control

law (157) results in a globally BIBO stable closed-loop system.
However, if the open-loop system is unstable (a 0), then, as
before, x must be restricted such that x XI, where XI is given
by Eq. (154). In this case, control law [see Eq. (157)] is modified as follows:
bkx
b
b
x
if r > x, and x
acl
acl
a
b
b
bkx
if r < x, and x +
a x+ a
a
cl
cl
b
b
(159)
r = r, r = x
if + x
a
a
b
b

if r > x, and x >
a
a
b
b
+
if r < x, and x < +
a
a
Now, x XI is enforced, the system will not become stuck
at x b/a, and BIBO stability is achieved.
The scalar controlled output variable y must track the exogeneous reference signal r. The point of departure is LQT.
Hence, a tracking linear control law is specified; the simplest
such control law is u kxx krr, where kr 1/cA1
cl b and
Acl A bkx. Hence,
0.5
DTRG
CC1
CC2
0.45
0.4
387
0.35
r(t) = h(t 0.5) h(t 1.5)
u = kx x
0.3
1
r
cA1
b
cl
x 0.25
and the linear inner-loop is

0.2
x = Acl x
0.15
1
br
cA1
b
cl
0.1
The set of rest points of the closed-loop system is the one1

dimensional subspace x (1/cA1
cl b)Acl br, r . Now,
1 u 1 yields the constraint on statically admissible r:
0.05
0
0.5
1.5
2
2.5
Time (sec)
3.5
Figure 23. Comparison of responses to a statically inadmissible

input.
Notice that while it is necessary to enforce x XI for the

above control laws when the open-loop plant is unstable, neither Eq. (155) nor Eq. (159) requires that r be statically admissible; that is, r b/a is allowed when x int(XI). This
is in contrast to Gilberts DTRG, and the globally BIBO stable
LQT controller constructed in Refs. 32 and 40, which restrict
the feasible reference signal to statically admissible values.
By relaxing the requirement that the modified reference signal be statically admissible, the tracking performance is enhanced. Simulation results are shown in Figs. 23 and 24.
These concepts are applied to higher-order plants and flight
control in Refs. 42, 43, and 49.
SUBOPTIMAL APPROACH
A single-input continuous-time control-constrained plant is
considered:
x = Ax + bu,
cA1
b
cl
r1
Thus, the set of statically admissible reference signals is

1
rs r rs, where rs cA1
cl b/(1 kxAcl b), and the set of
admissible rest points is the segment
Xr = {x | x =
1
A1 br, rs r rs }
cA1
b cl
cl
BIBO stabilty enforcement requires the construction of certain invariant sets in the state space.
The maximal statically admissible set Xs is characterized
as follows: The rest state which corresponds to the constant
reference signal r is
xr =
1
A1
cl br,
cA1
b
cl
rs r rs
It is convenient to use the perturbation state x : e; x xr and

scale r : e r/rs whereupon the dynamics are transformed into
x = Acl x
1 u 1
y = cx
1 + r kx x 1 + r
r(t) = 0.3h(t 0.5) 0.3h(t 1.5)
0.4
(160)
and the saturation avoidance constraint is transformed into
0.45
(161)
Now, fix r, rs r rs, and determine the largest set contained in the slab in Eq. (161) which is invariant under Eq.
(160); denote this maximal output admissible set by Xs(r). Finally, the maximal statically admissible set Xs is
DTRG
CC1
CC2
0.35
0.3
Xs = 1r1 Xs (r)
x 0.25
0.2
Proposition. The set Xs Rn is bounded iff the pair (A, kx)

is observable.
0.15
0.1
0.05
0
b1
kx A1
cl
0.5
1.5
Time (sec)
2.5
Figure 24. Comparison of responses to a statically admissible input.
Remark. The solution of well-posed inner-loop optimal control problems automatically renders the pair (A, kx) observable. Hence, the set Xs will be bounded and BIBO stability
can be enforced.
In general, the maximal statically admissible set Xs is compact and convex. However, Xs is not polyhedral; that is, it is
388
not finitely determined and, moreover, is rather difficult to

construct.
In the two-dimensional case (n 2), it is possible to obtain
a closed-form representation of Xs: The boundaries of the r
cross-sections Xs(r) are trajectories of the unforced system; the
boundaries of Xs are the envelopes of the above mentioned
family of trajectories. For the control constrained system,

0 1
0
x(t)
=
x(t) +
u(t)
2 1
2
It is now fairly easy to synthesize the outer-loop nonlinearity N, namely, the nonlinear control law r r (x, r). For
example, consider the Control Strategy 1: r is chosen to minimize r r, subject to the saturation avoidance constraint
1 kx x
provided that
y(t) = [1 0]x(t)
1 u(t) = [9
2.5]x(t) + 8r(t) 1
Xs is shown in Fig. 25.

In higher dimensions, suboptimal (i.e., smaller statically
admissible or invariant) sets, which are easy to characterize,
are certainly useful. In engineering applications, guaranteeing BIBO stability for a reasonably large but bounded set of
initial states which contains the origin is often sufficient. In
this respect, the following holds.
Let P be the real symmetric positive definite solution of
the Lyapunov equation
ATcl P
+ PAcl = Q
where Q is a somewhat arbitrary, real, symmetric, and positive definite matrix. The ellipsoid

1
T
Es = x | x Px
kx P1 kTx
is statically admissible. Furthermore, Xr Es, provided that
2
1 T
T
1 T
(cA1
cl b) (kxP kx )(b (Acl ) b).
1
r 1
cA1
b
cl
xT Px <
1
kx P1 kTx
xT Px =
1
kx P1 kTx
For x such that
r must satisfy the additional invariance-enforcing inequality

r
2
bT Px xT Qx
cA1
b
cl
Feasibility is guaranteed, by construction.

Hence, a dual-loop tracking controller as shown in Fig. 20
can be constructed. For all measurable exogeneous reference
signals r and for all initial states in the ellipsoid Es, good
small-signal performance is achieved, high-amplitude signals
are optimally tracked, and BIBO stability is guaranteed.
CONCLUSION
The basic concept of the stability of dynamical systems is
carefully developed, and its application to the design of feed-
Xs = U(Xs (r)) for rs r rs

1
0.5
1.5
1
X2
0.5
X2
0.5
0.5
1
0.5
0.5
1.5
1.5
X1
0.5
0
X1
(a)
(b)
Figure 25. (a) Maximal statically admissible set. (b) Comparison of Xs and the suboptimal
ellipse.
0.5
1.5
back control systems is presented. Classical stability theories

for linear and nonlinear feedback control systems are discussed and the Lyapunov and RouthHurwitz stability criteria are developed. Linearization of nonlinear dynamics in
terms of the Jacobian matrix is presented. Attention is given
to conventional control design and to frequency-domain methods: The root locus and Nyquist stability criteria are developed and, in addition, the Bode plot and Nichols chart-based
methods for stability, and degree of stability determination,
are discussed. The emphasis then shifts to the stability of control systems comprised of linear plants and actuators which
are subject to saturation. The state of the art with respect to
nonlinear controller synthesis methods that address tracking
control in the presence of actuator constraints is examined.
The saturation avoidance approach is pursued. Special attention is given to tracking performance and to the BIBO stability of the closed-loop nonlinear control system. Results based
on the concept of invariant setsnamely, maximal output admissible sets, maximal statically admissible sets, and the linear quadratic tracking (LQT) control methodare presented.
These results are applied to the synthesis of a dual-loop
nonlinear control structure. The on-line computational requirements are modest. Good tracking performance of highamplitude command signals is achievable, small-signal performance is preserved, and BIBO stability is guaranteed
provided that the command signals are sufficiently smooth,
and the initial state is contained in a predetermined invariant set.
BIBLIOGRAPHY
1. B. Porter, Synthesis of Dynamical Systems, London: Thomas Nelson, 1969.
2. W. J. Rugh, Linear System Theory, Englewood Cliffs, NJ: Prentice-Hall, 1993.
3. J. J. DAzzo and C. H. Houpis, Linear Control System Analysis
and Design, 3rd ed., New York: McGraw-Hill, 1988.
4. F. Csaki, Modern Control Theories, Budapest: Akademiai Kiado,
1972.
1980.
6. H. M. James, N. B. Nichols, and R. S. Phillips, Theory of Servomechanisms, Radiation Laboratory Series, Vol. 25, New York:
McGraw-Hill, 1974.
7. J. E. Gibson, Nonlinear Automatic Control, New York: McGrawHill, 1963.
8. J. J. DAzzo and C. H. Houpis, Linear Control System Analysis
and Design, 4th ed., New York: McGraw-Hill, 1995.
9. K. N. Swamy, On Sylvesters criterion for positive semidefinite
matrices, IEEE Trans. Autom. Control, AC-18: 306, 1973.
10. K. Ogata, Modern Control Engineering, Englewood Cliffs, NJ:
11. N. Minorsky, Theory of Nonlinear Systems, New York: McGrawHill, 1969.
12. R. E. Kalman and J. E. Bertram, Control system analysis and
design via the second method of Liapunov, J. Basic Eng., 80: 371
400, 1960.
13. E. A. Guillemin, The Mathematics of Circuit Analysis, New York:
Wiley, 1949.
14. V. Singh, Comments on the Routh-Hurwitz Criterion, IEEE
389
15. B. Porter, Stability Criteria for Linear Dynamical Systems, New

York: Academic, 1968.
16. K. J. Khatwani, On Routh-Hurwitz criterion, IEEE Trans. Autom.
Control, AC-26: 583584, 1981.
17. S. K. Pillai, On the -method of the RouthHurwitz criterion,
18. D. M. Etter, Engineering Problem Solving with MATLAB, Engelwood Cliffs, NJ: Prentice-Hall, 1993.
19. V. Krishnamurthi, Implications of Routh stability criteria, IEEE
20. F. R. Gantmacher, Applications of the Theory of Matrices, New
York: Wiley-Interscience, 1959.
21. MATLAB Version 4, Natic, MA: The MathWorks, Inc., 1993.
22. H. Nyquist, Regeneration Theory, Bell Syst. Tech. J., 11: 126
147, 1932.
23. H. W. Bode, Network Analysis and Feedback Amplifier Design,
Princeton, NJ: Van Nostrand, 1945, Chapter 8.
24. T. F. Schubert, Jr., A quantitative comparison of three Bode
straight-line phase approximations for second-order, underdamped systems, IEEE Trans. Educ., 40: 135138, 1997.
25. A. A. Andronow, A. A. Vitt, and S. E. Chaikin, Theory of Oscillations, Reading, MA: Addison-Wesley, 1966.
26. R. S. Baheti, Simple anti-windup controllers, in Proc. 1989 Amer.
Control Conf., Vol. 2, 1989, pp. 16841686.
27. D. S. Bernstein, Optimal nonlinear, but continuous feedback control of systems with saturating actuators, in Proc. 32rd Conf. Decision Control, December 1993, pp. 25332537.
28. D. S. Bernstein, Actuator saturation and control system performance, in Proc. 32rd Conf. Decision Control, 1993, pp. 24202421.
29. P. J. Campo and M. Morari, Robust control of processes subject
to saturation nonlinearities, Comput. Chem. Eng., 14 (4/5): 343
358, 1990.
30. P. R. Chandler, M. J. Mears, and M. Pachter, A hybrid LQR/LP
approach for addressing actuator saturation in feedback control,
in 1994 Conf. Decision Control, Dec. 1416, 1994, pp. 38603867.
31. E. G. Gilbert and K. T. Tan, Linear systems with state and control constraints: The theory and application of maximal output
admissible sets, IEEE Trans. Autom. Control, AC-36: 1008
1020, 1991.
32. E. G. Gilbert, I. Kolmanovsky, and K. T. Tan, Nonlinear control
of discrete-time linear systems with state and control constraints:
A reference governor with global convergence properties, in Proc.
33rd Conf. Decision Control, December 1994, pp. 144149.
33. R. Hanus, M. Kinnaert, and J. L. Henrotte, Conditioning technique, a general anti-windup and bumpless transfer method, Automatica, 729739, 1987.
34. R. A. Hess and S. A. Snell, Flight control design with actuator
saturation: SISO systems with unstable plants, in 33rd Aerospace
Sciences Meeting and Exhibit, AIAA, paper No 95-0336, January
1995; also published in the AIAA J. Guidance and Control, 19 (1):
191197, 1996, and see also the Errata in AIAA J. Guidance and
Control, 19 (6): 1200.
35. I. Horowitz, Feedback systems with rate and amplitude limiting,
Int. J. Control, 40: 12151229, 1984.
36. P. Kapasouris, M. Athans, and G. Stein, Design of feedback control systems for unstable plants with saturating actuators, in
IFAC Symp. Nonlinear Control Syst. Design, June 1989, pp.
302307.
37. P. Kapasouris and M. Athans, Multivariable control systems with
saturating actuators antireset windup strategies, in Proc. 1985
Amer. Control Conf., 1985, pp. 15791584.
38. P. Kapasouris and M. Athans, Control systems with rate and
magnitude saturation for neutrally stable uncertain plants, in
Proc. 29th Conf. Decision Control, Vol. 6, 1990, pp. 34073409.
390
39. N. J. Krikelis and S. K. Barkas, Design of tracking systems subject to actuator saturation and integrator windup, Int. J. Control,
39: 667683, 1984.
40. R. B. Miller, A New Approach to Manual Tracking Flight Control
with Amplitude and Rate Constrained Dynamic Actuators, Ph.D.
dissertation, Air Force Institute of Technology, March 1997.
41. R. B. Miller and M. Pachter, Manual tracking control with amplitude and rate constrained actuators, in Proc. 1996 Conf. Decision
Control, December 1996, pp. 31593164.
42. R. B. Miller and M. Pachter, Manual control with saturating actuators, in Proc. 1997 Eur. Control Conf., July 14, 1997.
43. R. B. Miller and M. Pachter, Maneuvering flight control with actuator constraints, AIAA J. Guidance, Control Dynamics, 20 (4):
729734, 1997.
44. C. Moussas and G. Bitsoris, Adaptive constrained control subject
to both input-amplitude and input-velocity constraints, in Proc.
IEEE Int. Conf. Syst., Man Cybern., Vol. 4, 1993, pp. 601606.
45. M. Pachter and D. H. Jacobson, Observability with a conic observation set, IEEE Trans. Autom. Control, AC-24: 632633, 1979.
46. M. Pachter, P. R. Chandler, and M. Mears, Control reconfiguration with actuator rate saturation, in Proc. Amer. Control Conf.,
June 1995, pp. 34953499.
47. M. Pachter, P. R. Chandler, and L. Smith, Velocity vector roll
control, in 1996 AIAA Guidance, Navigation Control Conf., San
Diego, CA, July 1996, AIAA paper No. 96-3867.
48. M. Pachter, P. R. Chandler, and L. Smith, Maneuvering flight
control, in 1997 Amer. Control Conf., Albuquerque, NM, June
46, 1997.
49. M. Pachter and R. B. Miller, Manual flight control with saturating actuators, IEEE Control Syst. Magazine, 18: 1019, 1997.
50. V. M. Popov, The Hyperstability of Control Systems, New York:
Springer, 1973.
51. A. A. Rodriguez and Y. Wang, Saturation Prevention Strategies
for an Unstable Bank-To-Turn (BTT) Missile: Full Information,
Ph.D. dissertation, Department of Electrical Engineering, Arizona State University, Tempe, AZ.
52. A. G. Tsirukis and M. Morari, Controller design with actuator
constraints, in Proc. Conf. Decision Control, 1992, pp. 26232628.
J. J. DAZZO
M. PACHTER
Air Force Institute of Technology
(AFIT/ENG)
STOCHASTIC OPTIMIZATION, STOCHASTIC APPROXIMATION AND SIMULATED ANNEALING
529
STOCHASTIC OPTIMIZATION, STOCHASTIC

APPROXIMATION AND SIMULATED ANNEALING
Optimization problems are central to human existence. Individuals and organizations are often faced with making tradeoffs between different factors in order to obtain desirable outcomes, and the problem of choosing these factors in some
best way is the essence of the optimization problem. Some
of the earliest manifestations occurred at the dawn of civilization when humans developed strategies for societal organization and for obtaining food and shelter. Optimization continues today throughout society in the design, operation, control,
and evaluation of modern systems.
Formal optimization is associated with the specification of
a mathematical objective function and a collection of factors
(or parameters) that can be adjusted to optimize the objective
function. In particular, one can formulate an optimization
problem as follows:
Find that solves min L( )
C
(1)
where L : Rp R1 represents some loss function to be minimized, represents the p-dimensional vector of adjustable parameters, and C Rp represents a constraint set defining the
allowable values for the parameters . (Note that a maximization problem can be trivially recast as the minimization problem in Eq. (1) by applying a minus sign to the objective function.) Our focus in this article will be problems for which
represents a vector of continuous parameters; this is in contrast to discrete problems such as how many items X do we
need to optimize performance? Discrete optimization is a
large subject unto itself, and will not be considered in any
detail here (see, e.g., Ref. 1 for a detailed discussion of this
subject from a deterministic perspective). Further, we are inJ. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering. Copyright # 1999 John Wiley & Sons, Inc.
530
terested in the case where L is sufficiently complicated that

it is not possible to obtain a closed-form analytical solution to
Eq. (1). This is far and away the most common setting for
large-scale optimization problems encountered in practice.
Hence to solve for * in Eq. (1), one uses an iterative algorithm: a step-by-step method for moving from an initial guess
at * to a final value that is expected to be closer to the true
* than the initial guess.
One of the major distinctions in optimization is between
global and local optimization. All other factors being equal,
one would always want the global optimal solution to Eq. (1),
that is, the * that provides a lower value of L than any other
value of C. However, in practice this solution is often not
available and one must be satisfied with obtaining a local solution. For example, L may be shaped so that there is a
clearly defined (local) minimum point over a broad region of
the allowable space C while there is a very narrow spike at
a distant point such that the trough of this spike is lower
than any point in the broad region. Hence, the local solution
is better than any nearby , but may not be the best possible
. Because of the inherent limitations of the vast majority of
optimization algorithms, it is usually only possible to ensure
that an algorithm will approach a local minimum with a finite
amount of resources being put into the optimization process.
However, since the local minimum may still yield a significantly improved solution (relative to no formal optimization
process at all), the local minimum may be a fully acceptable
solution for the resources available (human time, money, computer time, etc.) to be spent on the optimization. Most of this
article will focus on algorithms that are only guaranteed to
yield a local optimum; however, we will also consider one type
of algorithm (simulated annealing) that aims to find a global
solution from among multiple local solutions.
Our focus in this article is the study of iterative algorithms where
1. there is random noise in the measurements of L (and
its derivatives if relevant) and/or
2. there is a random choice in computing the search direction as the optimization algorithm iterates toward a solution.
The above two characteristics contrast with classical deterministic optimization, where it is assumed that one has perfect information about the loss function (and its derivatives,
if relevant) and that this information is used to determine the
search direction in a deterministic manner at every step of
the algorithm. In many practical problems one will not have
perfect information about the loss function due to inevitable
noise effects. A common example is where it is desired to minimize some mean squared error in the performance of some
system (e.g., the tracking error in a robot manipulation problem). Only rarely will one be able to compute the mean
squared error (or its derivatives); rather, one might be able to
get a specific observation of the squared error, but this differs
(sometimes very significantly) from the mean squared error.
Relative to point 2 above, it is sometimes beneficial to deliberately introduce randomness into the search process as a
means of speeding convergence and making the algorithm
less sensitive to modeling errors. Although the introduction
of randomness may seem counterproductive, it is well known
to have beneficial effects in some settings (another case where
this is true is in numerical integration, where Monte Carlo

methods can be much more efficient in high-dimensional
problems than deterministic quadrature approaches).
However, deterministic optimizationwhich includes linear and nonlinear programming and such well-known methods as steepest descent, NewtonRaphson, and conjugate gradientprovides a useful starting point for the study of
stochastic methods. In particular, many (though certainly not
all) of the methods in both deterministic and stochastic optimization for the continuous problems of interest in this article
rely in some manner on the gradient vector of the loss function with respect to the parameters:
L

1
.
.
g ( )
.
L
p
Then for local unconstrained optimization (i.e., C Rp), a
necessary condition for optimization when L is a continuously
differentiable nonlinear function is that * satisfies
g ( ) = 0
(2)
(constrained problems can also be formulated in this way

through, e.g., the use of a penalty function added to the basic loss function to penalize violations of the constraints).
The proof of this result follows from simple Taylor series arguments showing that if Eq. (2) is not true then one can move
in some direction that reduces the value of L. Many optimization algorithms are based on Eq. (2), thereby converting the
general optimization setting of Eq. (1) to the problem of finding a root to the equation g() 0.
This article will focus on two broadand popularclasses
of stochastic optimization algorithms: stochastic approximation and simulated annealing. There are many other stochastic optimization algorithms that we are not considering: notably, genetic algorithms, evolutionary strategies, evolutionary
programming, and various types of iterative random search.
Many references on these other approaches are available to
the interested reader. Among these are Ref. 2 for genetic algorithms, Ref. 3 or the journal Evolutionary Computation for
genetic algorithms and other evolutionary methods, and Refs.
4, 5, or 6 for random search.
The next section of this article reviews the core stochastic
approximation (SA) algorithm that is based on direct (but
usually noisy) measurements of the gradient vector g(); this
is the well-known RobbinsMonro SA algorithm. Then follows
an overview of SA when only measurements of the loss function L() are available [not measurements of the gradient
g()]. The subsequent section analyzes in greater detail one of
the gradient-free SA methodssimultaneous perturbation
SAintroduced previously. Then a review of simulated annealing is given. The final section is a brief summary putting
these algorithms into perspective.
ROBBINSMONRO STOCHASTIC APPROXIMATION
Background and Algorithm Form
We now discuss the well-known RobbinsMonro stochastic
approximation (RMSA) algorithm, which is a gradient-based
stochastic optimization algorithm (sometimes referred to as a

stochastic gradient method) for a wide variety of nonlinear
problems. This subsection will introduce the basic algorithm
form. The following two subsections discuss some of the theoretical properties related to convergence and asymptotic distributions. Then we will summarize three extensions to the
basic algorithm form, and finally summarize how the RMSA
algorithm is implemented in several different nonlinear applications.
The prototype stochastic optimization application for the
RMSA algorithm is the problem of finding a root to the equation g() 0 based on (noisy) measurements of g(). Let Y()
represent the measurement of g(). [Although this article is
written in the language of optimization, many of the ideas
carry over directly to the general root-finding context as well,
where g() represents the function for which a zero is to be
found.]
RMSA was introduced in a famous 1951 paper (10) and
has spawned a large number of follow-on papers. The algorithm has the form
k+1 = k akY (k )
(3)
where ak is a nonnegative gain sequence that must satisfy

certain conditions (discussed below) and k represents the estimate of * at the kth iteration. Since the deterministic term
L/ does not equal the stochastic term Y, the SA algorithm
is fundamentally different from the well-known deterministic
steepest descent algorithm. However, there is an intuitive
connection, since E(Y) L/ under standard RMSA conditions [typically under the relatively modest regularity conditions justifying the interchange of a derivative and an (expectation) integral] (see, e.g., Ref. 7 or 8).
A variation on the basic form in Eq. (3) is to include a
projection operator, say C, that automatically maps solutions
outside the constraint set C back to the nearest point
within C. Kushner and Yin (98) treat this approach extensively. In such a case, Eq. (3) becomes
k+1 = C [k akY (k )]
We will not treat this form further in this article.
Convergence of the RobbinsMonro
Stochastic Approximation Algorithm
As with any optimization algorithm, it is of interest to know
whether the iterate k will converge to * as k gets large. In
fact, one of the strongest aspects of SA is the rich convergence
theory that has been developed over many years. Researchers
and analysts in many fields have noted that if they can show
that a particular stochastic optimization algorithm is a form
of SA algorithm, then it may be possible to establish formal
convergence where otherwise that might have remained an
open question. In neural networks, for example, White (9) was
apparently the first to use this idea to show convergence of
certain forms of the well-known backpropagation algorithm.
Note that since we are in a stochastic context, convergence is
in a probabilistic sense. In particular, the most common form
of convergence established for SA is in the almost sure (a.s.)
(or with probability one) sense. [A historical note: Robbins
and Monro (10) showed conditions for mean-square conver-
531
gence of the iterate, which implies convergence in probability.

Blum (11) was apparently the first to give conditions for a.s.
convergence. Neither of a.s. and mean-square convergence is
implied by the other, but a.s. is stronger than in probability.]
Many sufficient conditions have been given over the years
for a.s. convergence of the SA algorithm in Eq. (3). Ruppert
(12) and Rustagi (13, Chap. 9), for example, discuss conditions
that have largely evolved out of the statistics perspective.
Ljung (14), Kushner and Clark (15), Kushner and Yin (98),
and Benveniste et al. (16, Chap. I.2) discuss a somewhat different set of conditions that have largely grown out of the
engineering perspective. Central to the latter approach is the
definition of an underlying ordinary differential equation
(ODE) that roughly emulates the SA algorithm in Eq. (3) for
large k and as the random effects disappear. It turns out that
the convergence properties of this deterministic differential
equation are closely related to the stochastic convergence
properties of Eq. (3). Lai (17, Sec. 2) provides a nice intuitive
explanation of this differential-equation-based approach.
Probably the most famous of the convergence conditions
for RMSA are those on the gain sequence ak. The conditions
provide a careful balancing between wanting to damp out the
noise effects as we get near the solution * (ak 0) and
avoiding premature (false) convergence of the algorithm
(k0ak ). The scaled harmonic sequence a/(k 1),

a 0, is the best-known example of a gain sequence that
satisfies the gain conditions (and, as discussed in the next
subsection, is an optimal choice with respect to the theoretical
rate of convergence, although in practice other decay rates
may be superior in finite samples). Usually some numerical
experimentation is required to choose the best value of the
scale factor that appears in the decaying gain sequence. Other
conditions important for convergence relate to the smoothness
of g(), the relative magnitude of the noise, and the position
of the initial condition.
Asymptotic Normality of RobbinsMonro Stochastic
Approximation and Choice of Gain Sequence
We discussed above the issue of convergence of the iterate
k. This is of central importance in any optimization algorithm. Also of importance is the probability distribution of the
iterate (which, recall, is a random vector in our stochastic optimization context). Having knowledge of the distribution provides key insight into two main aspects of the algorithm: (1)
error bounds for the iterate and (2) guidance in the choice of
the optimal gain ak so as to minimize the likely deviation of
k from *.
Unfortunately, the theory governing the asymptotic distribution of the SA iterate is rather difficult. This is to be expected, given the nonlinear transformations arising from the
recursion (3): a value Y(k) forms the basis for k1, which in
turn is the point of evaluation for Y(k1) (generally a nonlinear mapping) in the next iteration.
General results on the asymptotic distribution of SA iterates are given by Fabian (18). His work is a generalization of
previous asymptotic distribution results for SA by Sacks (19).
He shows that, under appropriate regularity conditions,
k/2 (k ) N(0, )
dist
(4)
532

dist
as k , where denotes convergence in distribution (see

Ref. 20, Chap. 3, or any other graduate-level probability text
for a formal definition of this type of convergence), is some
covariance matrix that depends on the Hessian matrix of
L() (at *), N(0, ) represents a multivariate normal
distribution with mean 0 and covariance , and governs the
decay rate for the SA gain sequence ak [e.g., ak a/(k 1)].
Ruppert (12) also discusses this result. Various special cases
of this result dealing with the situation 1 are presented
in Refs. 13 (pp. 258259), 21 (pp. 7178), and 12. The intuitive interpretation of Eq. (4) is that k * will be approximately normally distributed with mean 0 and covariance matrix /k for k reasonably large.
Equation (4) implies that the rate at which the iterate k
approaches * is k/2. This follows because a random vector
with the distribution N(0, ) on the right-hand side of Eq. (4)
is well behaved (i.e., not degenerate 0 or in magnitude),
and k * must be decaying at a rate k/2 to balance the
k/2 blowup factor on the left-hand side of Eq. (4). Under
standard conditions on ak (see preceding subsection), the
rate of convergence of k to * is maximized at 1.
In practical finite-sample problems, however, the choice of
1 is not generally recommended. Most analysts and researchers find a lower value of yields superior finite-sample
behavior. (This is a fact that is well known, but not widely
documented in the literature because it contradicts the theoretical result suggesting the optimality of 1; nevertheless,
in the authors experience and in the experience of all SA implementers he has consulted, a lower value of is generally
preferable. Othereffectively equivalentways exist for
slowing down the decay rate of ak so that it acts like 1 in
finite samples; see, e.g., the practically oriented paper of Okamura et al. (22) for estimation in finite-impulse-response
adaptive filters.) The intuitive reason for the desirability of
1 is that a slower decay provides a larger step size in the
iterations with large k, allowing the algorithm to move in bigger steps towards the solution. This observation is a practical
finite-sample result, as the asymptotic theory showing optimality of 1 is unassailable.
In fact, in many applications, a constant step size ( 0) is
used. Typical applications involve adaptive tracking or control
problems where * is changing in time. The constant gain provides enough impetus to the algorithm to keep up with the
variation in *, whereas if a decaying gain were used, too little weight would be applied to the current input information
to allow for the algorithm to track the solution. Such constant-gain algorithms are also frequently used in neural network training even when there is no variation in the underlying * (9,23). Constant-step-size SA algorithms will generally
not formally converge. (However, a partial convergence theory
is possible for constant gains. This is typically based on limiting arguments as the gain magnitude gets small. Essentially, one is able to show that the iterate from a constantgain algorithm will approach the optimal to within some
error that decreases as the gain magnitude is made smaller.
See, e.g., Refs. 2428.) Also note that the limiting distribution
for the standardized SA iterate [analogous to the left-hand
side of Eq. (4)] is not generally normal with constant step
size (29,26).
Extensions of Standard RMSA
This section discusses some extensions to the basic RMSA
framework presented in Eqs. (3)(4). In particular, we con-
sider: (1) the setting where the observed gradient Y includes

a state vector that evolves as is being updated, (2) methods
of algorithm acceleration by improved choice of the gain sequence, (3) iterate averaging for SA as a means for accelerating convergence, and (4) the setting where the loss function L
may change with time.
Joint Parameter and State Evolution. In our first generalization, we replace Y(k) by Y(k, xk), where xk represents a state
vector related to the system being optimized. The book by
Benveniste et al. (16) is devoted to this framework, which was
apparently first considered by Ljung (14) and extended by
Metivier and Priouret (30). It is typically assumed that xk
evolves online according to a set of certain Markov transition
probabilities. The convergence properties of k [as given by
Eq. (3)] then depend on the properties of these transition
probabilities. Benveniste et al. (16) discuss several applications of this framework in the context of telecommunications,
fault detection, and signal processing. One of the common
Markov representations of the evolution of the state is via the
linear state equation xk1 A(k)xk B(k)wk, where A and B
are appropriately dimensioned matrices and wk is a sequence
of independent, identically distributed random vectors (see,
e.g., Ref. 14). Then the behavior of the Markov chain, and the
associated convergence of k, can be tied directly to the stability of this linear process using well-known results in linear
systems theory.
A specific application of this form is in temporal difference
learning for function approximation (99). The goal here is to
approximate the cost-to-go function associated with many
time-series prediction and control problems. The cost-to-go
function measures the expected future cost associated with
specific policies (e.g., control strategies) for a dynamic process,
given that the process is currently in a particular state. When
the state evolves according to a Markov chain, temporal difference learning can be cast as an RMSA form with joint parameter and state evolution.
Adaptive Estimation and Second-Order Algorithms. There are
a large number of methods for adaptively estimating the gain
sequence (or a multivariate analog of the gain) to enhance the
convergence rate of RM-type SAs. Some of these are built on
stochastic approximation analogs of the famous Newton
Raphson deterministic optimization algorithm (e.g., Ref. 31).
It is knownsee, e.g., Ref. 16, Sec. 3.2that the asymptotically optimal gain for RMSA is a matrix given by H(*)1 /k,
where H() represents the Hessian matrix of L(). This is
identical to deterministic optimization, except that we now
have the decay factor k included to cope with the stochastic
effects. Unfortunately, this is largely of theoretical interest
only, since in practice one does not know either * or the Hessian as a function of . It also is an asymptotic result, and, as
discussed in the preceding subsection, optimality for practical
finite-sample analysis may impose other requirements. Nevertheless, this asymptotic result provides a type of ideal in
designing adaptive SA algorithms.
Perhaps the first adaptive technique is that of Kesten (32),
where one looks at the signs of the difference k1 k in a
scalar process as a means of designing an adaptive gain
sequence ak (unlike the approaches described below, this approach does not explicitly use the connection with the Hessian
matrix as mentioned above). If there are frequent sign
changes, this is an indication that the iterate is near the solution, while if signs are not changing we have an indication
that the iterate is far from the solution. This forms the basis
for an adaptive choice of the gain ak, whereby a larger gain is
used if there are no sign changes and a smaller gain is used
if the signs change frequently. Kesten (32) established a.s.
convergence with such a scheme. A multivariate extension of
the Kesten idea is described in Delyon and Juditsky (33), including theoretical justification of the extension via a.s. convergence of the iterates.
Ljung and Soderstrom (34, Sec. 2.4, Sec. 3.4, and Chap. 4)
and Wei (35) discuss stochastic analogs of the Newton
Raphson search in the context of parameter estimation for
particular (possibly linear) models. [These represent extensions of the first paper on adaptive Hessian estimates for scalar problems (100).] In so doing, Refs. 34 and 35 demonstrate,
via the method of minimizing prediction error, that a batch
version of the problem of finding a * satisfying Eq. (2) (where
all the data are processed simultaneously) can be converted
to the recursive form (where the data are processed one at a
time) using only the current (instantaneous) input. The scalar
gain ak is then replaced in their formulation by a matrix that
approximates the (unknown) true inverse Hessian matrix corresponding to the current data points contribution to the
loss function.
Ruppert (36) describes an approach where the Hessian is
estimated by taking finite differences of a gradient measurement. The gradient used by Ruppert differs slightly from the
standard RM gradient in Eq. (3) in that he converts the basic
problem from one of minimizing L() to one of minimizing
L/2 (note that this yields the same * when there is a
unique minimum). Spall (102) presents a more efficient approach to general Hessian estimation based on the simultaneous perturbation idea discussed below.
Iterate Averaging. An important and relatively recent development in SA is the concept of iterate averaging. Like
many good ideas, this one is simple and, in some problems,
can be effective. The idea was jointly described by Polyak and
Juditsky (37) and Ruppert (Ref. 12, based on a 1988 internal
technical report). There are several variations, but the basic
idea is to replace k as our current best estimate of * after
k iterations with the average
k (k + 1)1
k
(5)
j=0
where each of the j summands is computed as in Eq. (3). The

singular achievement of the above references was to show
that k1/2(k *) is asymptotically normally distributed with
mean 0 and a covariance matrix that is as small as possible
in a matrix sense. These references establish this optimality
of Eq. (5) for gain sequences satisfying the standard conditions plus the conditions ak1 /ak 1 o(ak) [with o( ) implying a term that goes to 0 faster than the argument]. This
important additional condition implies that ak must decay at
a rate slower than the optimal rate of 1/k for the individual
iterates k (as discussed in the preceding subsection). The implications of this result are quite strong: namely, that one can
use the standard algorithm in Eq. (3) together with the simple averaging in Eq. (5) to achieve the same optimal rate of
533
convergence that previously was possible only with knowledge of the Hessian H(*), as discussed under Adaptive estimation and second-order algorithms above. This result is
available for any valid gain satisfying the above-mentioned
additional condition. Hence, in principle, iterate averaging
greatly reduces one of the traditional banes of SA implementationnamely, choosing the gain sequence in some optimal
way.
Variations on the above basic iterate averaging formulation are readily available. One obvious variation is to not average in the first few iterations, but rather start the average
at some N 0 or else use only a sliding window of the last
n N (say) measurements. In practical finite-sample implementations, such modifications are likely to help, since the
first few iterations tend to produce the poorest estimates. In
the sliding-window approach, formal asymptotically optimal
normality can be retained if the window length grows with
time (see, e.g., Ref. 38). A further modification to the basic
approach is to use the averaged value k (together with k) in
a modified form of the RMSA iteration [instead of k alone on
the right-hand side of the basic form (3)]. This is referred to
as the feedback approach in Ref. 39, and can be shown to
sometimes yield further improvement.
In practice, however, the results on iterate averaging are
more mixed than the above would suggest. Numerical studies
by the author have shown that a well-chosen ak sequence will
yield results superior to that possible by averaging (see also
Ref. 40, p. 57, and Ref. 41 for some cautionary notes). Once
again, this appears to be a consequence of the finite-sample
properties of practical problems. More study is required to
understand the full capabilities of iterate averaging in
practice.
Time-Varying Loss Functions. A final generalization of the
RobbinsMonro recursion that we discuss is one where the
loss function (and corresponding gradient with respect to )
varies with k. This problem is treated in Refs. 42 and 43. The
basic idea is that, while the loss function may change shape
with k, it is assumed that the underlying minimum * is either constant for all k or fixed in the limit as k (even
though the loss function may change shape indefinitely). The
two references mentioned above show a.s. convergence in
these cases for the scalar- setting; the proofs have to be
changed from those commonly seen in SA to accommodate the
time-varying loss.
Figure 1 depicts a time-varying loss function Lk() for a
scalar parameter. This figure shows a case where the loss
function and the minimizing parameter value both change
with time, but one where the optimal parameter value converges to a limiting point * . This is a situation for which the
above-mentioned theory would apply.
Applications of RobbinsMonro Stochastic Approximation
RMSA has been applied in a large number of engineering
(and other) problems, often under another name. This subsection provides a summary of several applications together with
some references for further study.
Neural network (NN) training via the well-known backpropagation algorithm has long been recognized as an application of the RMSA algorithm, two of the publications discussing this connection being Ref. 9 and Ref. 44 (pp.
534
L1( )
L2( )
L3( )
Lk+1( )
Lk( )
Time index, k
Figure 1. Example of time-varying loss functions with limiting minimum *.
190199). The essence of the backpropagation algorithm is to

systematically compute (via the chain rule applied to the neural network structure) the gradient g() [or its noisy measurement Y()] for use in an algorithm identical to RMSA (it appears that the authors of some of the earlier publications in
this areae.g., Ref. 45were unaware of the connection
with RMSA).
A popular area for application of RMSA is in recursive estimation for linear models, especially as applied in signal processing, fault detection, and adaptive control [e.g., Benveniste
et al. (16, Chap. I.1), Ljung et al. (21, Chap. III), and Solo
and King (101, Chap. 5)]. Several variations of the resulting
algorithms exist, most notably the least mean squares (LMS)
and recursive least squares (RLS) algorithms. The essential
idea in such algorithms is that the data are processed sequentially as they arrive according to a formula that weights each
new data point in light of the assumed linear relationship
between the input and output on the one hand and the data
that have already arrived on the other. This contrasts with
traditional linear regression methods where all the data are
assumed to be available at the outset and the processing is
done in batch form. Given the dynamic nature of problems in
signal processing, fault detection, adaptive control, and certain other areas, this recursive processing has obvious advantages. Further, it can be shown that the recursive algorithms
yield an estimate at time t that is close to that which would
result from batch processing if all the data up to time t were
available at the outset (when t is reasonably large).
Another popular area for application of RMSA is in simulation-based optimization, especially in the context of discreteevent dynamic systems such as queuing networks. Here the
goal may be to optimize some design aspect of a system (e.g.,
the position of certain machines on a factory floor) by running
experiments via computer simulations. A systematic approach to simulation-based optimization has been adapted
under the rubric of perturbation analysis, which was introduced in modern form by Y. C. Ho and his colleagues in the
late 1970s. In this method, one aims to get a gradient estimate Y() at any value based on only one or a small number
of simulation runs. Given the stochastic nature of the simulation, this gradient estimate will also only be a stochastic estimate of the true gradient L/. Some references on simulation-based optimization in the context of RMSA are Refs. 46,
47, 8, 48, and 49.
The final application we mention here is in image restoration. Here the task is to recover a true image of a scene from
a recorded image of the scene where the recorded image is
typically corrupted by noise and/or otherwise degraded from
the original image. The essential reason for using RMSA
rather than more conventional signal processing and deconvolution techniques is the ability to adapt to nonlinearities in
the process. Some recent references that discuss implementations of RMSA in the context of image restoration are Refs.
50 and 51. Abreu et al. (50) are concerned with removing impulse noise that may be due, for example, to noisy sensor or
transmission errors in collecting the image data. They note
that nonlinear methods have often proven superior to linear
methods for this problem. Cha and Kassam (51) describe an
approach to image restoration based on a type of NN called
the radial basis function network (e.g., Ref. 6, Sec. 9.5), where
RMSA is used to train the weights of this form of function
approximator. Their use of the radial basis function network
is as a spatially invariant filter taking measurement data as
input and producing estimates of the pixel-by-pixel gray levels in the image.
STOCHASTIC APPROXIMATION WITH GRADIENT
APPROXIMATIONS BASED ON FUNCTION MEASUREMENTS
Introduction and Contrast of Gradient-Based
and Nongradient Algorithms
There has been a growing interest in stochastic optimization
algorithms that do not depend on direct gradient information
or measurements. Rather, these algorithms are based on an
approximation to the gradient formed from (generally noisy)
measurements of the loss function. This interest has been motivated, for example, by problems in the adaptive control and
statistical identification of complex systems, the optimization
of processes by large Monte Carlo simulations, the training of
recurrent neural networks, the recovery of images from noisy
sensor data, and the design of complex queuing and discreteevent systems.
Overall, such algorithms exhibit certain convergence properties of the RobbinsMonro gradient-based algorithms considered above while requiring only loss-function measurements. A main advantage of such algorithms is that they do
not require the detailed knowledge of the functional relationship between the parameters being adjusted (optimized) and
the loss function being minimized that is required in gradient-based algorithms [in particular, they do not need the
Y() term in the RM recursion of Eq. (3)]. Such a relationship
can be notoriously difficult to develop in some areas (e.g., nonlinear feedback controller design or system optimization via
large-scale simulation), while in other areas (such as highdimensional statistical parameter estimation) there may be
large computational saving in calculating a loss function
rather than a gradient. In contrast, the approaches based on
gradient approximations require only conversion of the basic
output measurements to sample values of the loss function,
which does not require full knowledge of the system input
output relationships. Examples of approximation-based methods using loss-function measurements only are given below;
such methods include, as an early prototype, the Kiefer
Wolfowitz finite-difference SA algorithm (52).
Because of the fundamentally different information needed
in implementing these gradient-based (RM) and gradient-free
algorithms, it is difficult to construct meaningful methods of

comparison. As a general rule, however, the gradient-based
algorithms will converge faster than those using loss-function-based gradient approximations when speed is measured
in number of iterations. Intuitively, this is not surprising given
the additional information required for the gradient-based algorithms. In particular, based on asymptotic theory, the optimal rate of convergence measured in terms of the deviation
of the parameter estimate from the true optimal parameter
vector is of order k1/2 for the gradient-based algorithms (see
the subsection on asymptotic normality above) and of order
k1/3 for the algorithms based on gradient approximations,
where k represents the number of iterations (52). (Exceptions
to this maximum rate of convergence for the nongradient algorithms are discussed in Refs. 5355 and Ref. 56, where special cases are presented that achieve a rate arbitrarily close
to, or equal to, k1/2.)
In practice, of course, many other factors must be considered in determining which algorithm is most appropriate for
a given circumstance. Three examples of why this is true are:
(1) in cases where it is not possible to obtain reliable knowledge of the system inputoutput relationships, the gradientbased algorithms may be either infeasible (if no system model
is available) or undependable (if a poor system model is used);
(2) the total cost to achieve effective convergence depends not
only on the number of iterations required, but also on the cost
needed per iteration, which is typically greater in gradientbased algorithms (this cost may include greater computational burden, additional human effort required for determining and coding gradients, experimental costs for model building such as labor, materials, and fuel, etc.); and (3) the rates
of convergence are based on asymptotic theory, and may not
be representative of practical convergence rates in finite samples. For these reasons, one cannot say in general that a gradient-based search algorithm is superior to a gradient-approximation-based algorithm even though the gradient-based
algorithm has a faster asymptotic rate of convergence (and
with simulation-based optimization such as perturbation
analysis, as discussed above, requires only one system run
per iteration, while the approximation-based algorithm may
require multiple system runs per iteration). As a general rule,
however, if direct gradient information is conveniently and reliably available, it is generally to ones advantage to use this
information in the optimization process. The focus in this section is on the case where such information is not readily
available.
Spall (57) presents a summary of historical contributions
in the area of optimization with algorithms based on gradient
approximations using only loss-function measurements. The
summary below is a much condensed version of that review.
Background
As in the introduction consider the problem of minimizing a
(scalar) differentiable loss function L(), where Rp, p 1,
and where the optimization problem can be translated into
finding the minimizing * such that the gradient g() 0. It
is assumed that measurements of L() are available at various values of (actually, most of the algorithms have the
weaker requirement of only requiring measurements of the
difference of two values of the loss function, as opposed to
measuring the loss functions themselves). These measurements may or may not include added noise. No direct mea-
535
surements of g() are assumed available, in contrast to the

RM framework.
The recursive procedure we consider is in the general SA
form:
k+1 = k ak g k (k )
(6)
where gk( k) is the estimate of the gradient L/ at the iterate k based on the above-mentioned measurements of the loss
function. Under appropriate conditions, the iteration in Eq.
(6) will converge to * in some stochastic sense, usually a.s.
(see, e.g., Ref. 53 or 15). Typical convergence conditions are
similar to those mentioned above for the RMSA algorithm.
The essential part of Eq. (6) is the gradient approximation
gk( k). We discuss below two forms that have attracted attention. We let y( ) denote a measurement of L( ) at a design
level represented by the dot [i.e., y( ) L( ) noise], and ck
be some (usually small) positive number [if the noise has
mean 0, then y( ) Q( ) as defined in the section on RMSA].
One-sided gradient approximations involve measurements
y( k) and y( k perturbation) while two-sided gradient approximations involve measurements of the form y(k perturbation). The two general forms of gradient approximations
are:
Finite Difference (FD) (52,58). Each component of k is perturbed one at a time, and corresponding measurements
y( ) are obtained; each component of the gradient estimate is formed by differencing the corresponding y( )
values and then dividing by a difference interval. This
is the standard approach to approximating gradient vectors and is motivated directly from the definition of a
gradient as a vector of p partial derivatives, each constructed as the limit of the ratio of a change in the function value over a corresponding change in one component of the argument vector. Typically, the ith
component of gk( k) (i 1, 2, . . ., p) for a two-sided FD
approximation is given by
gki (k ) =
y(k + ck ei ) y(k ck ei )
2ck
where ei denotes a vector with a one in the ith place and

zeros elsewhere (an obvious analog holds for the onesided version; likewise for the SP form below).
Simultaneous Perturbation (SP) (59,60). All elements of
k are randomly perturbed together to obtain two measurements y( ), but each component of gk( k) is formed
from a ratio involving the individual components in the
perturbation vector and the difference in the two corresponding measurements. For two-sided SP, we have
gki (k ) =
y(k + ck k ) y(k ck k )
2ck ki
where the distribution of the user-specified random perturbations for SP, k (k1, k2, . . ., kp)T, satisfies conditions mentioned in the next section.
The algorithm [Eq. (6)] with one of the gradient approximations will be referred to as FDSA or SPSA, as appropriate.
[An approach in the same spirit as SPSA, called random di-
536
rections SA, is discussed in Refs. 61, 15, and 62, but it is

shown in Ref. 63 that SPSA will generally have a lower asymptotic mean squared error than RDSA for the same number of measurements y( ).] Note that the number of loss-function measurements y( ) needed in each iteration of FDSA
grows with p, while with SPSA only two measurements are
needed, independent of p. This, of course, provides the potential for SPSA to achieve a large saving (over FDSA) in the
total number of measurements required to estimate when p
is large. This potential is only realized if the number of iterations required for effective convergence to * does not increase
in such a way as to cancel the measurement savings per gradient approximation at each iteration. The next section will
discuss this efficiency issue further, demonstrating when this
potential can be realized.
SIMULTANEOUS PERTURBATION
STOCHASTIC APPROXIMATION
Introduction
The preceding section motivated the interest in techniques for
recursive optimization that rely on measurements of the loss
function only, not on measurements (or direct calculations) of
the gradient (or higher-order derivatives) of the loss function.
The focus was on two such techniques in the stochastic approximation setting: finite-difference (KieferWolfowitz) SA,
and simultaneous perturbation SA. This chapter will focus on
SPSA for reasons of its relative efficiency.
Recent applications of SPSA are described in Refs. 64 and
65 (queuing systems), 66 (industrial quality improvement), 67
(pattern recognition), 68 (neural network training), 69 and 70
(adaptive control of dynamic systems), 71 (statistical model
parameter estimation and fault detection), 72 (sensor placement and configuration), and 73 (vehicle traffic management).
As discussed in the preceding section, SPSA is based on a
highly efficient and easily implemented simultaneous perturbation approximation to the gradient: this gradient approximation uses only two loss-function measurements, independent of the number of parameters being optimized. This
contrasts, for example, with the standard (two-sided) finitedifference approximation associated with the well-known
KieferWolfowitz SA algorithm, which uses 2p function measurements to approximate the gradient. The fundamental
(and perhaps surprising) theoretical result in Ref. 60 (Sec. 4) is:
Under reasonably general conditions, SPSA and KieferWolfowitz
finite-difference-based SA (FDSA) achieve the same level of statistical accuracy for a given number of iterations even though SPSA
uses p times fewer function evaluations than FDSA (since each
gradient approximation uses only 1/p the number of function evaluations).
This theoretical result has been confirmed in many numerical

studies, even in cases where p is on the order of several hundred or thousand. The subsection below discusses this result
further. Of course, this result will not always hold in finitesample practice, because it is derived from asymptotic theory;
further, the asymptotic theory is based on conditions that
may be violated in some practical applications (the general
conditions are similar to those for the RMSA algorithm).
The next subsection summarizes the problem setting and
discusses some of the theory associated with the convergence
and efficiency of SPSA. The subsection after that summarizes

a reference dealing with guidelines for practical implementation of the algorithm. The subsection after that discusses
some extensions to the basic SPSA algorithm, including some
relatively recent results on a second-order version of SPSA
that emulates the NewtonRaphson algorithm of deterministic optimization while still only requiring loss-function measurements (i.e., no gradient or Hessian information). Numerical studies of SPSA are available in many of the references
(see, e.g., Ref. 60, 66, 64, 69, 70, or 63).
Basic Assumptions and Supporting Theory
Once again, the goal is to minimize a loss function L() over
C Rp. The SPSA algorithm works by iterating from an
initial guess of the optimal , where the iteration process depends on the above-mentioned simultaneous perturbation approximation to the gradient g(). The form of the SPSA gradient approximation was presented above.
Spall (74,60) presents sufficient conditions for convergence
of the SPSA iterate ( k * a.s.) using the differential equation approach discussed in the RMSA section in the context
of the RM algorithm. Because of the different form of the input, the conditions here are somewhat different from those of
the RM approach. In particular, we must impose conditions
on both gain sequences (ak and ck), the user-specified distribution of k, and the statistical relationship of k to the measurements y( ). We will not repeat the conditions here, since
they are available in Refs. 74 and 60 (with later extensions in
Refs. 7577). The main conditions are that ak and ck both go
to 0 at rates neither too fast nor too slow, that L() is sufficiently smooth (several times differentiable) near *, and that
the ki are independent and symmetrically distributed about
0 with finite inverse moments E(ki1) for all k, i. One particular distribution for ki that satisfies these latter conditions
is the symmetric Bernoulli 1 distribution; two common distributions that do not satisfy the conditions (in particular, the
critical finite-inverse-moment condition) are the uniform and
the normal.
Although the convergence result for SPSA is of some independent interest, the most interesting theoretical results in
Ref. 60, and those that best justify the use of SPSA, are the
asymptotic efficiency conclusions that follow from an asymptotic normality result. In particular, under some minor additional conditions in Ref. 60 (Proposition 2), it can be shown
that
dist
k /2 (k ) N(, )
as k
(7)
where 0 depends on the choice of gain sequences (ak and

ck), depends on both the Hessian and the third derivatives
of L() at *, and depends on the Hessian matrix at * (note
that in general 0, in contrast to many well-known asymptotic normality results in estimation). Given the restrictions
on the gain sequences to ensure convergence and asymptotic
normality, the fastest allowable value for the rate of convergence of k to * is k1/3. This contrasts with the fastest allowable rate of k1/2 for the RMSA algorithm. Hence, one measure
of the value of the gradient information in RM is the increase
in rate of convergence. There are exceptions to this result,
and cases arise where the SPSA rate can be made either arbitrarily close to the RM rate of k1/2 (following the logic of Ref.
53) or the same as the RM rate (e.g., the use in simulation-
537
based optimization with common random numberssee

Ref. 56).
Spall (60, Sec. 4) uses the asymptotic normality result in
Eq. (7) (together with a parallel result for FDSA) to establish
the relative efficiency of SPSA. This efficiency depends on the
shape of L(), the values for ak and ck, and the distributions
of the ki and measurement noise terms. There is no single
expression that can be used to characterize the relative efficiency; however, as discussed in Refs. 60 (Sec. 4) and 63, in
most practical problems SPSA will be asymptotically more efficient than FDSA. For example, if ak and ck are chosen as in
the guidelines mentioned in the subsection below, then by
equating the asymptotic mean squared errors E( k *2) in
SPSA and FDSA, we find
(the standard algorithm can be coded in 12 or fewer lines). In

addition, this reference provides some practical guidelines for
choosing the gain coefficients ak and ck. The values recommended in these guidelines differ from the asymptotically
theoretical optimal values in a reflection of the realities of
finite-samples analysis (e.g., it is recommended in practice
that the gains ak and ck decay at rates slower than the asymptotically optimal 1/k and 1/k1/6, respectively). Furthermore,
theoretical guidelines, such as discussed in Fabian (53) and
Chin (63), are not generally useful in practical applications,
since they require the very information on the loss function
and its gradients that is assumed unavailable.
no. of measurements of L( ) in SPSA

1
no. of measurements of L( ) in FDSA

p
Sadegh and Spall (79) consider the problem of choosing the

best distribution for the vector k. Based on asymptotic distribution results, it is shown that the optimal distribution for
the components of k is symmetric Bernoulli. This simple distribution has also proven effective in many finite-sample
practical and simulation examples.
Some extensions to the basic SPSA algorithm above are
reported in the literature. For example, its use in feedback
control problems, where the loss function changes with time,
is given in Refs. 69, 70. The former reference also reports on
a gradient smoothing idea (analogous to momentum in the
neural network literature) that may help reduce noise effects
and enhance convergence (and gives guidelines for how the
smoothing should be reduced over time to ensure convergence). Alternatively, it is possible to average several SP gradient approximations at each iteration to reduce noise effects
(at the cost of additional function measurements); this is discussed in Ref. 60. An implementation of SPSA for global minimization is discussed in Ref. 80 [for the case where there are
multiple minimums at which g() 0]; this approach is based
on a stepwise (slowly decaying) sequence ck (and possibly ak).
The problem of constrained (equality and inequality) optimization with SPSA is considered in Refs. 81 and 65 using a
projection approach. A one-measurement form of the SP gradient approximation is considered in Ref. 82; although it is
shown in that reference that the standard two-measurement
form will usually be more efficient (in terms of the total number of loss-function measurements to obtain a given level of
accuracy in the iterate), there are advantages to the onemeasurement form in real-time operations where the underlying system dynamics may change too rapidly to get a credible
gradient estimate with two successive measurements (and,
analogously to the perturbation analysis and related approaches in simulation-based optimization for discrete-event
systems, this would allow for a one-run gradient estimate, but
without requiring the gradient information needed in the
IPA-type approaches).
An accelerated form of SPSA is reported in Refs. 83 and
102. This approach extends the SPSA algorithm to include
second-order (Hessian) effects with the aim of accelerating
convergence in a stochastic analog to the deterministic NewtonRaphson algorithm. Like the standard (first-order) SPSA
algorithm, this second-order algorithm is simple to implement
and requires only a small numberindependent of pof lossfunction measurements per iteration (no gradient measurements, as in standard SPSA). In particular, only four measurements are required to estimate the loss-function gradient
and inverse Hessian at each iteration (though one additional
(8)
as the number of loss measurements in both procedures gets

large. This result implies that the p-fold saving per iteration
(gradient approximation) translates directly into a p-fold saving in the overall optimization process.
Figure 2 is an illustration of the relative efficiency of SPSA
and FDSA in an adaptive control problem related to wastewater treatment. The specific scenario is close to that described
in Ref. 70. The plot is showing the mean deviation (in a rootmean square, RMS sense) of the output of a treatment plant
from some specified target values for water cleanliness and
methane gas byproduct, the goal, of course, being to minimize
the RMS error. We see that on an iteration-by-iteration basis,
the SPSA and FDSA approaches yield similar RMS values.
However, the cost per iteration in FDSA [measured in number of L() measurements] is 412 ( p) times that of SPSA.
This leads to the very large overall cost savings shown in the
upper right box of the figure. In fact, one iteration of FDSA
takes over five times the number of loss measurements than
are taken in all 80 iterations of SPSA. The performance of the
two algorithms in Fig. 2 is very consistent with the theoretical
result in Eq. (8) above.
Implementation of Simultaneous
Perturbation Stochastic Approximation
Reference 78 includes a step-by-step summary of the implementation of SPSA together with a listing of MATLAB code
1.6
Total number of L( )measurements
SPSA
FDSA
Normalized L ( )
1.2
160
65,920
0.8
SPSA
0.4
FDSA
20
Minimum achievable long-run value
40
Iterations
60
80
Figure 2. Relative performance of SPSA and FDSA in wastewater

treatment system.
Further Results and Extensions to the Basic Algorithm
538
measurement is recommended as a check on algorithm behavior). The algorithm is implemented with two simple parallel
recursions: one for and one for the Hessian of L(). The recursion for is a stochastic analog of the well-known NewtonRaphson algorithm of deterministic optimization. The
recursion for the Hessian matrix is simply a recursive calculation of the sample mean of per-iteration Hessian estimates
that are formed using SP-type ideas.
earlier sections) is the willingness to give up the quick gain

of a rapid decrease in the loss function by allowing the possibility of temporarily increasing it. SAN derives this property
from the Boltzmann (or Gibbs) probability distribution of statistical mechanics, describing the probability of system having a particular energy state:

x
(9)
P(energy = x) = cT exp
cb T
SIMULATED ANNEALING
where cT 0 is a normalizing constant, cb 0 is known as

the Boltzmann constant, and T is the temperature of the system. Note that at high temperatures, the system is more
likely to be in a high-energy state than at low temperatures.
The optimization analogy derives from the fact that even at
a low temperature (equivalent to the optimization algorithm
having run for some time with a decreasing temperature),
there is some nonzero probability of reaching a higher energy
state (i.e., higher level of the loss function). So the SAN process sometimes goes uphill, but the probability of this decreases as the temperature is lowered. Hence, there is the
possibility of getting out of a local minimum in favor of finding a global minimum, and this possibility is especially prominent in the early iterations when the temperature is high.
It was Metropolis et al. (85) who first introduced the idea
of the BoltzmannGibbs distribution into numerical analysis
through constructing a means for simulation of a system at
some fixed temperature. In particular, if a system is in some
current energy state Ec, and some system aspects are changed
to make the system potentially achieve a new energy state
En, then the Metropolis simulation always has the system go
to the new state if Enew Ecurr. On the other hand, if Enew
Ecurr, then the probability of the system actually going to the
new state is

Enew Ecurr
(10)
exp
cb T
Introduction and Motivation from the Physics of Cooling

This section continues in the spirit of the preceding two in
considering algorithms that do not require direct gradient information. One of the algorithms that has attracted considerable attention is simulated annealing (SAN). SAN was originally developed for discrete optimization problems, but more
recently has found application in continuous optimization
problems of the type emphasized in the previous sections.
One of the main virtues of SAN is that, in principle, this
algorithm addresses the difficult global optimization problem
discussed in the introduction. The algorithm is designed to
traverse local minima en route to a global minimum to L().
Further, since the method can address both discrete and continuous optimization problems, there is no need to assume the
existence of a loss-function gradient (much less compute it).
The term annealing comes from analogies to the cooling
of a liquid or solid. A central issue in statistical mechanics is
analyzing the behavior of substances as they cool. At high
temperatures, molecules have much mobility, but as the temperature decreases this mobility is lost and the molecules may
tend to line themselves in a crystalline structure. This structure is the minimum-energy state for the system. Note the
qualifier may: temperature alone does not govern whether
the substance has reached a minimum-energy state. To
achieve this state, the cooling must occur at a sufficiently
slow rate. If the substance is cooled at too rapid a rate, an
amorphous (or polycrystalline) state may be reached that is
not a minimum-energy state of the substance. The principle
behind annealing in physical systems is the slow cooling of
substances to reach the minimum-energy state.
In optimization, the analogy to a minimum-energy state
for a system is a minimized value of the loss function. The
technique of SAN attempts to capture mathematically the
process of controlled cooling associated with physical processes, the aim being to reach the lowest value of the loss
function in the face of possible local minima. As with the
physical cooling process, whereby temporary higher-energy
states may be reached as the molecules go through their
alignment process, SAN also allows temporary increases in
the loss function as the learning process captures the information necessary to reach the global minimum. A more thorough explanation of the analogy between SAN and physical
cooling is given in, e.g., Ref. 84.
It is clear that the critical component of an SAN algorithm
is the mathematical analog of the rate of cooling in physical
processes. As with all other stochastic optimization algorithms, the choice of this algorithm- and problem-specific cooling schedule (analogous to the gains in stochastic approximation) has a strong effect on the success or failure of SAN.
A primary distinction between SAN and the majority of
other optimization approaches (including those discussed in
This expression is known as the Metropolis criterion. After

a large number of such decisions and outcomes, the system
eventually reaches an equilibrium where the system state is
governed by the BoltzmannGibbs distribution in Eq. (9).
This is predicated on the system being at the fixed temperature T. Kirkpatrick et al. (84) were apparently the first to use
this criterion for optimization together with the idea of a
changing temperature that decays according to an annealing
schedule (exponentially in their case). Geman and Geman
(86) go beyond the informal annealing guidelines of Kirkpatrick et al. (84) and establish conditions on the annealing
schedule for formal convergence of their form of the SAN algorithm iterate to the global minimum.
Algorithm Form
Using the principles discussed in the preceding subsection,
we now present the general SAN algorithm form. There are
variations depending on how one wants to implement the annealing schedule and how one performs the sampling required for generating a new candidate point. As before, we
consider the minimization of some loss function L(),
C Rp. Since the user has control over T, one can (without
loss of generality) take cb 1 in the Metropolis criterion (10).
Below are listed the general sequence of steps in SAN when
noise-free measurements of L are available:
Step 1. Choose an initial temperature T and set of current

parameter values curr; determine L(curr).
Step 2. Randomly determine a new value of , new, that is
close to the current value, and determine L(new).
Step 3. Compare the two L values via the Metropolis criterion (10): Let L(new) L(curr). Accept new if 0.
Alternatively, if 0, accept the new point new only if
a uniform (0, 1) random variable U (generated by Monte
Carlo) satisfies U exp( /T).
Step 4. Repeat steps 2 and 3 for some period until either
the budget of function evaluations allocated for that T
has been used or the system reaches some state of equilibrium.
Step 5. Lower T according to the annealing schedule, and
return to step 2. Continue the process until the total
budget for function evaluations has been used or some
indication of convergence is satisfied (analogous to the
system being frozen in its minimum-energy state).
The specifics of implementation for the five steps above can
vary greatly. In the annealing schedule, Kirkpatrick et al.
(84), Press et al. (87 p. 452), and Brooks and Morgan (88)
discuss the case where T decays geometrically in the number
of cooling phases (number of times T is lowered according to
step 5). Geman and Geman (86) present conditions such that
if T decreases at the rate 1/log k (where k is the number of
algorithm iterations) then the probability distribution for the
iteration converges to a uniform distribution over all possible
global minimum points. Szu and Hartley (89) present some
formal arguments justifying a faster rate of decay for T:
namely, having T decay at a rate 1/k. Another area for different implementations is in step 2, where a new value is generated randomly. Probably the most common form of perturbation for continuous optimization is to add a p-dimensional
Gaussian random variable to the current value curr (e.g., Ref.
6, p. 183). Alternative perturbations include the approach
based on changing only one component of at a time (88), the
approach using spherically uniform perturbations (90), and
the approach using a multivariate Cauchy distribution to
change all components (89,62).
Aside from the general variations in implementation
above, SAN is critically dependent on the specific values for
various algorithm parameters and decision criteria. In particular, these include the initial temperature T, the specific distribution parameters chosen for the perturbation distribution
(e.g., the covariance matrix for a Gaussian perturbation), the
specific parameter(s) associated with the decay of T (e.g., the
in Tnew Told, where Tnew and Told are the new and old temperatures if one is using a geometric decay), and the criterion
for determining when to lower the temperature (e.g., the maximum allowable number of function evaluations before a lowering of T).
There appears to be no fully satisfactory way to accommodate noisy function measurements in SAN, although some
work on this aspect of the discrete (combinatorial) optimization problem has been carried out in Ref. 91. (One of the particularly appealing features of the stochastic approximation
methods in the preceding three sections is that they handle
noisy function measurements in an essentially seamless manner. Namely, the forms of the algorithms do not have to be
altered, the convergence theory applies in the noisy case, and
there is a gradual degradation in performance as the noise
539
level increases from 0, in contrast with the sudden large degradation in approaches such as SAN that use explicit decision
criteria based on loss-function measurements). The primary difficulty arises in the critical decision step 3, where
curr is changed or not changed according to the value of
L(new) L(curr) together with the random sampling associated with the Metropolis criterion. With noisy function measurements, the value of , now equal to y(new) y(curr) instead of L(new) L(curr), will be altered from its underlying
true value according to noise-free function measurements. In
fact, even a modest level of noise will frequently alter the sign
of , which is likely to change the decision regarding the acceptance or rejection of the new point new. The obvious means
by which one can attempt to cope with noise is to average
several function measurements y( ) at each of the values
curr and new when performing the comparison at step 3. However, this may dramatically increase the cost of optimization
(specifically, the total number of function evaluations required), since a large amount of averaging will often be required to effectively remove the noise, especially in the region
around a local or global minimum where the function may be
relatively flat. An alternative way is to alter the acceptance
criterion to accept the inherent errors that will be made with
the noisy measurements. Hence we replace the criteria 0
or 0 with or , where is the function measurement noise standard deviation (or, more likely, an estimate of it) and the number of multiples of the standard
deviation that will be tolerated. The rationale for such a
change is that, while we are willing to accept a new that temporarily increases the loss function (a basic aspect of SAN),
we are less willing to forgo a new that decreases the loss function. Changing the unconditional acceptance criterion from
0 to will allow for a greater number of cases where
L(new) L(curr) even though y(new) y(curr) due to the noise.
With 2, one can be sure (through the Chebyshev inequality of probability) that most such cases will be caught, although at the expense of letting some additional new values
that increase the loss function be accepted.
Evaluation of Simulated Annealing
Although many positive studies regarding SAN have been reported in the literature (e.g., Refs. 9294), the author is unaware of any formal theoretical analysis comparing SAN with
the gradient-free SA algorithms discussed in the preceding
two sections. (Note that some connections of SAN with SA
have been explored in Refs. 95 and 96. The basic idea there
is, beginning with the basic SA recursion using either gradients or function measurements, to add a Monte Carlo-generated Gaussian noise term bkWk to the recursion, where bk
0 and Wk is the Gaussian random vector. This term is similar
to SAN in that it will force the iterate out of local minima
under some conditions that are established in the cited papers.) However, Fabian (97), Chin (103), Styblinski and Tang
(62), and the author (together with several students in a graduate class he taught) have conducted some numerical comparisons. Fabian (97), Chin (103), and Styblinski and Tang (62)
compare different forms of SAN against several random and
deterministic search algorithms for problems with noise-free
measurements of the loss function. The SAN algorithms generally compared poorly with the competing algorithms in
these two papers. The authors studies involved four different
loss functions: p 2 and p 20 fourth-order polynomials,
540
and the Example 1 (p 2, fourth-order polynomial) and Example 6 (p 10, trigonometric) functions in Ref. 62. The studies of the author considered both noise-free and noisy measurements of the four loss functions and compared SAN with
simple random search mentioned in Ref. 7 (pp. 186189) and
the global version of SPSA outlined in Ref. 80. Although SAN
was sometimes superior to the random search method, the
global SPSA method was significantly more efficient than
SAN in all cases considered, and, even more dramatically,
was sometimes convergent when SAN seemed unable to obtain a solution anywhere near the true optimal (this was most
apparent in the Example 6 problem of Ref. 62, which had a
very large number of local minima, one of which always
seemed to form a trap for SAN). In order to have a fair comparison in performing these studies, the algorithm coefficients
(e.g., the gains for the SA algorithms and the decay rate and
other coefficients mentioned above for SAN) were tuned to approximately optimize performance for the competing algorithms.
Since one must be careful in drawing conclusions beyond
the specific cases in any numerical analysis, the above should
not indicate that SAN is always a poor performer. Rather,
these results should be a cautionary note in view of some of
the positive results reported elsewhere. Certainly, SAN does
have a role in the area of global optimization, as evidenced by
its popularity and positive performance in some challenging
problems. Furthermore, the above studies were for problems
with smooth (differentiable) loss functions, and SAN (unlike
most SA algorithms) has the ability to deal with nonsmooth
loss functions.
CONCLUDING REMARKS
This article has surveyed several important algorithms with
stochastic optimization. The focus was on algorithms within
the stochastic approximation and simulated annealing
classes. Broadly speaking the gradient-based (Robbins
Monro) SA algorithms are most appropriate when enough
prior information is available about the system so that it is
possible to determine the gradient of the observed loss function. It is expected that when such information is available,
this type of algorithm may be the most efficient means for
optimization within the stochastic setting, especially when
coupled with one of the algorithm acceleration techniques
such as iterate averaging or second-order methods. However,
it was also pointed out that in many applications, it is difficult
or impossible to obtain the gradient information needed for
the application of the gradient-based methods. Then one is
faced with carrying out the optimization using only the (possibly noisy) measurements of the loss function itself. In that
context, we emphasized three different approaches: finite-difference and simultaneous perturbation SA (both of which are
based on approximating the unknown gradient vector) and
simulated annealing. The FDSA method is the oldest and
best-known of these approaches, and can work in a variety of
applications. The newer SPSA approach will, however, usually be more efficient than FDSA, with the gain in efficiency
being approximately equal to the dimension of the vector
being optimized (i.e., the SPSA algorithm will use only onepth the number of function evaluations to obtain the same
level of statistical accuracy as FDSA). Unlike the SA algorithms, simulated annealing is focused on global (as opposed
to local) optimization problems. Simulated annealing is based

on an intriguing analogy to the cooling of materials and the
achievement of an optimal state for the material by cooling
neither too fast nor too slow. While some positive experience
has been reported with optimization by simulated annealing,
it appears that there exist more efficient algorithms for
many problems.
Although stochastic optimization has the potential for
treating a broader class of problems than many traditional
deterministic techniques, their application may sometimes be
a challenge. A problem common to all stochastic optimization
techniques (including those not discussed in this article, such
as the genetic algorithm, sequential random search, and evolutionary programming) is that values must be specified for
the algorithms tunable coefficients. All stochastic optimization techniques have such coefficients (the gains in SA, terms
associated with the cooling schedule and probability of accepting a step in simulated annealing, etc.). These coefficient
values are typically problem-dependent and can have a profound effect on the performance of an algorithm.
Stochastic optimization is playing an ever larger role in
optimization, as it allows for the treatment of problems such
as global optimization and noisy loss-function evaluations
that arise frequently in areas such as network analysis, neural network training, image processing, and nonlinear control.
It is expected that the role of stochastic optimization will continue to grow as modern systems increase in complexity and
as population growth and dwindling natural resources force
tradeoffs that were previously unnecessary. Stochastic optimization allows for the treatment of a broader range of problems than possible with only standard deterministic methods.
The algorithms of this article provided a sampling of several
important stochastic optimization methods.
ACKNOWLEDGEMENTS
This work was partially funded by U.S. Navy Contract
N00024-97-C-8119 and the JHU/APL Independent Research
and Development Program. The author appreciates the comments of two anonymous reviewers and G. Cauwenberghs, V.
Fabian, M. Fu, J. L. Maryak, B. T. Polyak, M. A. Styblinski,
and S. Yakowitz on an earlier version of parts of this article.
BIBLIOGRAPHY
1. G. L. Nemhauser and L. A. Wolsey, Integer and Combinatorial
Optimization, New York: Wiley, 1988.
2. K. S. Tang et al., Genetic algorithms and their applications,
IEEE Sig. Process. Mag., November 1996, pp. 2237.
3. T. Back, Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, and Genetic Algorithms, Oxford, UK: Oxford University Press, 1996.
4. F. J. Solis and J. B. Wets, Minimization by random search techniques, Math. Oper. Res., 6: 1930.
5. A. A. Zhiglavsky, Theory of Global Random Search, Boston:
Kluwer Academic, 1991.
6. J.-S. R. Jang, C.-T. Sun, and E. Mizutani, Neuro-fuzzy and Soft
Computing, Upper Saddle River, NJ: Prentice-Hall, 1997.
7. W. H. Fleming, Functions of Several Variables, New York:
8. P. Glasserman, Gradient Estimation via Perturbation Analysis,
Boston: Kluwer, 1991.

9. H. White, Some asymptotic results for learning in single hidden
layer feedforward neural networks, J. Amer. Statist. Assoc., 84:
10031013, 1989.
10. H. Robbins and S. Monro, A stochastic approximation method,
Ann. Math. Statist., 22: 400407, 1951.
11. J. R. Blum, Approximation methods which converge with probability one, Ann. Math. Statist., 25: 382386, 1954.
12. D. Ruppert, Stochastic approximation, in B. K. Ghosh and P. K.
Sen (eds.), Handbook of Sequential Analysis, New York: Marcel
Dekker, 1991, pp. 503529.
13. J. S. Rustagi, Optimization Techniques in Statistics, New York:
14. L. Ljung, Analysis of recursive stochastic algorithms, IEEE
15. H. J. Kushner and D. S. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems, New York:
16. A. Benveniste, M. Metivier, and P. Priouret, Adaptive Algorithms and Stochastic Approximations, Berlin: Springer Verlag, 1990.
17. T. L. Lai, Stochastic approximation and sequential search for
optimum, in L. M. Le Cam and R. A. Olshen (eds.), Proc. Berkeley Conf. Honor of Jerzy Neyman and Jack Kiefer, Vol. II, Belmont, CA: Wadsworth, 1985.
18. V. Fabian, On asymptotic normality in stochastic approximation, Ann. Math. Statist., 39: 13271332, 1968.
19. J. Sacks, Asymptotic distribution of stochastic approximation
procedures, Ann. Math. Statist., 29: 373405, 1958.
20. R. G. Laha and V. K. Rohatgi, Probability Theory, New York:
Wiley, 1979.
21. L. Ljung, G. Pflug, and H. Walk, Stochastic Approximation and
Optimization of Random Systems, Basel: Birkhauser, 1992.
22. A. Okamura, T. Kirimot, and M. Kondo, A new normalized stochastic approximation algorithm using a time-shift parameter,
Electron. Commun. Jpn., Part 3, 78: 4151, 1995.
23. C. M. Kuan and K. Hornik, Convergence of learning algorithms
with constant learing rates, IEEE Trans. Neural Net., 2: 484
489, 1991.
24. O. Macchi and E. Ededa, Second-order convergence analysis of
stochastic adaptive linear filtering, IEEE Trans. Autom. Control,
AC-28: 7685, 1982.
25. H. J. Kushner and H. Huang, Asymptotic properties of stochastic approximations with constant coefficients, SIAM J. Control
Optim., 19: 87105, 1983.
26. G. C. Pflug, Stochastic minimization with constant step size: Asymptotic laws, SIAM J. Control Optim., 24: 655666, 1986.
27. G. G. Yin, Asymptotic optimal rate of convergence for an adaptive estimation procedure, in T. E. Duncan and B. Pasik-Duncan
(eds.), Stochastic Theory and Adaptive Control, New York:
Springer Verlag, 1992, pp. 480489.
28. G. G. Yin and K. Yin, Passive stochastic approximation with
constant step size and window width, IEEE Trans. Autom. Control, 41: 90106, 1996.
29. S. Mukherjee and T. L. Fine, Online steepest descent yields
wieghts with nonnormal limiting distributions, Neural Comp.,
8: 10751084, 1996.
30. M. Metivier and P. Priouret, Applications of a Kushner and
Clark lemma for general classes of stochastic algorithms, IEEE
Trans. Inf. Theory, IT-30: 14401451, 1984.
31. J. E. Dennis and R. B. Schnabel, A view of unconstrained optimization, in G. L. Nemhauser et al. (eds.), Optimization (Handbooks in OR and MS, Vol. 1) New York: Elsevier, 1989, pp. 172.
32. H. Kesten, Accelerated stochastic approximation, Ann. Math.
Statist., 29: 4159, 1958.
33. B. Delyon and A. Juditsky, Accelerated stochastic approximation, SIAM J. Control Optim., 3: 868881, 1993.
541

Identification, Cambridge, MA: MIT Press, 1983.
35. C. Z. Wei, Multivariate adaptive stochastic approximation, Ann.
Statist., 15: 11151130, 1987.
36. D. Ruppert, A NewtonRaphson version of the multivariate
RobbinsMonro procedure, Ann. Statist., 13: 236245, 1985.
37. B. T. Polyak and A. B. Juditsky, Acceleration of stochastic approximation by averaging, SIAM J. Control Optim., 30: 838
855, 1992.
38. H. J. Kushner and J. Yang, Stochastic approximation with averaging: Optimal rates of convergence for general processes, SIAM
J. Control Optim., 31: 10451062, 1993.
39. H. J. Kushner and J. Yang, Stochastic approximation with averaging and feedback: Rapidly convergent on-line algorithms,
40. I. J. Wang, Analysis of stochastic approximation and related algorithms, Ph.D. Thesis, Purdue University School of Electrical
Engineering, 1996.
41. J. L. Maryak, Some guidelines for using iterate averaging in
stochastic approximation, in Proc. IEEE Conf. Decision Control,
San Diego, CA, 22872290, 1997.
42. C. A. Goodsell and D. L. Hanson, Almost sure convergence for
the RobbinsMonro process, Ann. Probab., 4: 890901, 1976.
43. S. N. Evans and N. C. Weber, On the almost sure convergence
of a general stochastic approximation procedure, Bull. Austral.
Math. Soc., 34: 335342, 1986.
44. B. Kosko, Neural Networks and Fuzzy Systems, Englewood Cliffs,
45. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning
internal representations by error propagation, in Parallel Distributed Processing, Vol. 1, Cambridge, MA: MIT Press, 1986,
pp. 318362.
46. R. Suri, Perturbation analysis: The state of the art and research
issues explained via the Gl/G/1 queue, Proc. IEEE, 77: 114
137, 1989.
47. P. LEcuyer and P. W. Glynn, Stochastic optimization by simulation: Convergence proofs for the Gl/G/1 queue in steady state,
Management Sci., 40: 15621578, 1994.
48. G. C. Pflug, Optimization of Stochastic Models: The Interface
between Simulation and Optimization, Boston: Kluwer, 1996.
49. M. Fu and J.-Q. Hu, Conditional Monte Carlo: Gradient Estimation and Optimization Applications, Norwell, MA: Kluwer, 1997.
50. E. Abreu et al., A new efficient approach for the removal of impulse noise from highly corrupted images, IEEE Trans. Image
Proc., 5: 10121025, 1996.
51. I. Cha and S. A. Kassam, RBFN restoration of nonlinearity degraded images, IEEE Trans. Image Proc., 5, 964975, 1996.
52. J. Kiefer and J. Wolfowitz, Stochastic estimation of a regression
function, Ann. Math. Statist., 23: 462466, 1952.
53. V. Fabian, Stochastic approximation, in J. J. Rustigi (ed.), Optimizing Methods in Statistics, New York: Academic Press, 1971,
pp. 439470.
54. P. Glasserman and D. D. Yao, Some guidelnes and guarantees
for common random numbers, Management Sci., 38: 884908.
55. P. LEcuyer and G. Yin, Budget-dependent convergence rate of
stochastic approximation, SIAM J. Optim., 8: 217247, 1998.
56. N. L. Kleinman, J. C. Spall, and D. Q. Naiman, Simulationbased optimization with stochastic approximation using common random numbers, Management Sci., submitted.
57. J. C. Spall, Developments in stochastic optimization algorithms
with gradient approximations based on function measurements,
in J. D. Tew, M. S. Manivannan, D. A. Sadowski, and A. F. Seila
(eds.), Proc. Winter Simulation Conf., 1994, pp. 207214.
58. J. R. Blum, Multidimensional stochastic approximation methods, Ann. Math. Statist., 25: 737744, 1954.
542
59. J. C. Spall, A stochastic approximation technique for generating

maximum likelihood parameter estimates, in Proc. Amer. Control Conf., 1987, pp. 11611167.
60. J. C. Spall, Multivariate stochastic approximation using a simultaneous peturbation gradient approximation, IEEE Trans.
Automat. Control, 37: 332341, 1992.
61. B. T. Polyak and Y. Z. Tsypkin, Pseudogradient adaptation and
training algorithms, Automation Remote Control, 34: 377397,
1973.
62. M. A. Styblinski and T. S. Tang, Experiments in nonconvex optimization: Stochastic approximation with function smoothing
and simulated annealing, Neural Netw., 3: 467483, 1990.
63. D. C. Chin, Comparative study of stochastic algorithms for system optimization based on gradient approximations, IEEE
Trans. Syst. Man. Cybern. B, 27: 244249, 1997.
64. S. D. Hill and M. C. Fu, Transfer optimization via simultaneous
perturbation stochastic approximation, in C. Alexopoulos, K.
Kang, W. R. Lilegdon, and D. Goldsman (eds.), Proc. Winter Simulation Conf., 1995, pp. 242249.
65. M. C. Fu and S. D. Hill, Optimization of discrete event systems
via simultaneous perturbation stochastic approximation, Trans.
Inst. Ind. Eng., 29: 233243, 1997.
66. F. Rezayat, On the use of an SPSA-based model-free controller
in quality improvement, Automatica, 31: 913915, 1995.
67. Y. Maeda, H. Hirano, and Y. Kanata, A learning rule of neural
networks via simultaneous perturbation and its hardware implementation, Neural Netw., 8: 251259, 1995.
68. G. Cauwenberghs, Analog VLSI autonomous systems for learning and optimization, Ph.D. Dissertation, California Institute of
Technology, 1994.
69. J. C. Spall and J. A. Cristion, Nonlinear adaptive control using
neural networks: Estimation based on a smoothed form of
simultaneous perturbation gradient approximation, Statist.
Sinica, 4: 127, 1994.
70. J. C. Spall and J. A. Cristion, A neural network controller for
systems with unmodeled dynamics with applications to wastewater treatment, IEEE Trans. Syst. Man. Cybern. B, 27: 369
375, 1997.
71. A. Alessandri and T. Parisini, Nonlinear modeling and state estimation in a real power plant using neural networks and stochastic approximation, in Proc. Amer. Control Conf., 1995, pp.
15611567.
72. P. Sadegh and J. C. Spall, Optimal Sensor Configuration for
Complex Systems, in Proc. Test Technol. Symp. (sponsored by
U.S. Army TECOM), http://www.atc.army.mil/tecom/tts/
proceed/optsenr.html, 1996.
73. D. C. Chin and R. H. Smith, A traffic simulation for mid-Manhattan with model-free adaptive signal control, in Proc. Summer
Comput. Simulation Conf., (sponsored by the Society for Computer Simulation) 1994, pp. 296301.
74. J. C. Spall, A stochastic approximation algorithm for large-dimensional systems in the KieferWolfowitz setting, in Proc.
75. H. F. Chen, T. E. Duncan, and B. Pasik-Duncan, A stochastic
approximation algorithm with random differences, in Proc. 13th
IFAC World Congr., Vol. H, 1996, pp. 493496.
76. I.-J. Wang and E. K. P. Chong, A deterministic analysis of simultaneous perturbation stochastic approximation, in Proc. 30th
Conf. Inf. Sci. Syst., 1996, pp. 918922.
77. J. Dippon and J. Renz, Weighted means in stochastic approximation of minima, SIAM J. Control Optim., 35, 18111827, 1997.
78. J. C. Spall, Implementation of the simultaneous perturbation
method for stochastic optimization, IEEE Trans. Aerosp. Electron. Syst., 34: in press.
79. P. Sadegh and J. C. Spall, Optimal random perturbations for

multivariate stochastic approximation using a simultaneous
perturbation gradient approximation, IEEE Trans. Autom. Control, 43: in press.
80. D. C. Chin, A more efficient global optimization algorithm based
on Styblinski and Tang, Neural Netw., 7, 573574, 1994.
81. P. Sadegh, Constrained optimization via stochastic approximation with a simultaneous perturbation gradient approximation,
Automatica, 33: 889892, 1997.
82. J. C. Spall, A one-measurement form of simultaneous perturbation stochastic approximation, Automatica, 33: 109112, 1997.
83. J. C. Spall, Accelerated second-order stochastic optimization using only function measurements, Proc. IEEE Conf. Decision Control, 1997, pp. 14171424.
84. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, Optimization by
simulated annealing, Science, 220: 671680, 1983.
85. N. Metropolis et al., Equation of state calculations by fast computing machines, J. Chem. Phys., 21: 10871092, 1953.
86. S. Geman and D. Geman, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intell., PAMI-6: 721741, 1984.
87. W. H. Press et al., Numerical Recipes in C, 2nd ed., Cambridge:
Cambridge University Press, 1992.
88. S. P. Brooks and B. J. T. Morgan, Optimization using simulated
annealing, Statistician, 44: 241257, 1995.
89. H. Szu and R. Hartley, Fast simulated annealing, Phys. Lett. A,
122: 157162, 1987.
90. I. O. Bohachevsky, M. E. Johnson, and M. L. Stein, Generalized
simulated annealing for function optimization, Technometrics,
28: 209217, 1986.
91. S. Gelfand and S. K. Mitter, Simulated annealing with noisy
or imprecise energy measurements, J. Optim. Theor. Appl., 62:
4962, 1989.
92. P. J. M. Van Laarhoven, Theoretical and Computational Aspects
of Simulated Annealing, CWI Tract 51, Amsterdam: Center for
Mathematics and Computer Science, 1988.
93. R. Azencott (ed.), Simulated Annealing: Parallelization Techniques, New York: Wiley, 1992.
94. R. V. V. Vidal (ed.), Applied Simulated Annealing, Berlin:
95. S. Gelfand and S. K. Mitter, Simulated annealing type algorithms for multivariate optimization, Algorithmica, 6: 419
436, 1991.
96. S. Gelfand and S. K. Mitter, Metropolis-type annealing algorithms for global optimization in Rd, SIAM J. Control Optim.,
31: 111131, 1993.
97. V. Fabian, Simulated annealing simulated, Comput. Math.
Appl., 33: 8194, 1997.
98. H. J. Kushner and G. G. Yin, Stochastic Approximation Algorithms and Applications, New York: Springer-Verlag, 1997.
99. J. Tsitsiklis and B. Van Roy, An analysis of temporal-difference
learning with function approximation, IEEE Trans. Autom. Control, 42: 674690, 1997.
100. J. H. Venter, An extension of the RobbinsMonro procedure,
Ann. Math. Statist., 38: 181190, 1967.
101. V. Solo and X. Kong, Adaptive Signal Processing Algorithms,
102. J. C. Spall, Adaptive stochastic approximation by the simultaneous perturbation method, IEEE Trans. Autom. Control, submitted, 1998.
103. D. C. Chin, The simultaneous perturbation method for processing magnetospheric images, Opti. Eng., submitted, 1998.
JAMES C. SPALL
The Johns Hopkins University
STOCHASTIC SYSTEMS
STOCHASTIC SYSTEMS
Many control systems in practice are subject to imperfectly
known disturbances which may be taken as random. Such
disturbances have been ignored in the study of deterministic
control problems. In many control systems some unknown parameters occur. The system behavior depends on the parameters and the fact that the value of the parameters is unknown
makes the system unknown. Some crucial information concerning the system control is not available to the controller,
and this information should be learned during the systems
performance. The problem described is the problem of adaptive control. The adaptive control problem can be considered
the identification problem and the control problem. A solution
to an adaptive control problem will be understood as a solution to the identification problem and to the control problem.
We focus on control and adaptive control of continuous-time
linear stochastic systems. The theory of adaptive control of
continuous-time linear stochastic systems has been recently
developed as an important application of the stochastic control theory in engineering, biology, economics, finance, and
manufacturing systems. Continuous-time linear systems are
an important and commonly used class of systems (1,2). It is
assumed that the models evolve in continuous time rather
than discrete time because this assumption is natural for
many models and it is important for studying discrete time
models when the sampling rates are large and for analyzing
numerical round-off errors. Stochastic systems are described
by linear stochastic differential equations.
The general approach to adaptive control described here
exhibits a splitting or separation of the problems of identification of the unknown parameters and adaptive control. Maximum likelihood, least squares, or weighted least squares estimators are used to identify the unknown constant
parameters (3,4,5). These estimates are given recursively and
are strongly consistent, which means that the family of estimates converges to the true value of the parameter with the
probability one. It turns out that for some cases the weighted
least squares estimator is strongly consistent whereas the
least squares estimator is not. The adaptive control constructed by the so-called certainty equivalence principle, that
is, the optimal stationary control, is computed by replacing
the unknown true parameter values by the current estimates
of these values. Because the optimal stationary controls are
continuous functions of the unknown parameters, the selftuning property is verified, which means that asymptotically
adaptive control using the estimate of the unknown parameter is as good as optimal control if we know the system. It is
also shown that the family of average costs using the control
from the certainty equivalence principle converges to the optimal average cost. This verifies the self-optimizing property.
In describing a stochastic control model, the kind of information available to the controller at each instant plays an
important role. Several situations are possible:
(1) The controller has no information during the systems
operation. In this case a function of time is chosen as a control. These controls are often called open loop as distinct
from closed-loop or feedback controls, in which the actual
value of control input at time t is a function of the observation
at time t (6). The distinction between open-loop and feedback
control is fundamental. An open-loop control is deterministic
whereas a feedback law determines a random control process.
543
An open-loop control is formally a special case of a closed-loop

control. For a deterministic system the reverse also holds.
The important point is that this is not true for stochastic
systems.
(2) The controller knows the state of the system at each
time t. We call this the case of complete observations.
(3) The controller has partial knowledge of the system
states. This is called the case of partial observations.
For a deterministic control model, the state of the system
at any time t can be deduced from the initial data and the
control used up to time t by solving the corresponding differential equation. Thus observing the current state of the system at each time t does not really give more information than
knowing the initial data. For stochastic control systems, there
are many paths which the system states follow given the control and the initial data. In the stochastic case, the best system performance depends on the information available to the
controller at each time t.
When we consider continuous-time stochastic control problems, the system states are described by the stochastic process that changes according to a stochastic differential equation. We need to be aware that to deal with more realistic
mathematical models of the situation we allow for some randomness in some of the coefficients of an ordinary differential
equation (7). Randomness arises when we consider real life
situations with some disturbances or noise. These disturbances or noise are modeled by a stochastic process, often by
the so-called Brownian motion or Wiener process (8,9), which
we define later.
Stochastic control problems have been difficult to solve explicitly (10,11,12). One of the fundamental results of the stochastic control problem, where the optimal control is obtained
explicitly, is the linear quadratic Gaussian (LQG) problem
(5,6,13). It is called linear because the system is described by
a linear difference equation (in the discrete-time case) or by a
linear differential equation (in the continuous-time case). It
is called quadratic because the cost criterion to be minimized
or maximized is quadratic. It is quadratic in a state and in a
control. It is called Gaussian because the noise is modeled by
a Gaussian process.
Another stochastic control problem that has been solved
explicitly is the portfolio selection and consumption model
known as Mertons model (14) and described in Example 2
below.
A linear quadratic Gaussian problem and a portfolio selection and consumption model were the two of a very few stochastic control problems that have been solved explicitly. Recently Duncan and Upmeir (15) provided a pretty wide class
of stochastic control problems that are solved explicitly. The
stochastic control problems are the control of Brownian motion in noncompact symmetric spaces by a drift vector field.
In solving stochastic control problems the basic difficulty
lies in solving the so-called HamiltonJacobiBellman (HJB)
equation (11). To solve this equation it is assumed that the
solution, the value function, is smooth. Typically the solution
is not smooth enough to satisfy the HJB equation in a classical sense. The theory of viscosity solutions, first introduced by
M. G. Grandall and P. L. Lions, provides a convenient framework in which to study HJB equations. See Ref. 16 on the
theory of viscosity solutions.
Stochastic calculus provides powerful mathematical tools
for solving stochastic control problems and stochastic adap-
544
STOCHASTIC SYSTEMS
tive control problems. The mathematical tools of stochastic

calculus most commonly used are stochastic integral (17),
Itos differential rule, and martingales (7,8). These elements
of stochastic calculus together with the limit theorems of the
probability theory, the theory of stochastic processes
(18,19,20,21,22), and the theory of stochastic differential
equations are extremely useful in proving the results of stochastic control and stochastic adaptive control of continuous
time. This is another motivation for considering stochastic
control problems in continuous time rather than in discrete
time. There has been always a greater chance of stronger results because those mathematical techniques are more sophisticated and because of that, more powerful.
Stochastic adaptive control problems are applications of
stochastic control theory (23,24,25,26). Because we use the
certainty equivalence control as an adaptive control, it means
that we need the optimal control given explicitly or an almost
optimal control (27).
The stochastic adaptive control problems with discrete
time have been under investigation for a long time. Many references are in the books in Refs. 5, 6, 23, and 28, and in a few
others (17,24,29). Continuous-time stochastic adaptive control
problems have been investigated relatively recently. Many results have been obtained under conditions difficult to verify.
The goal of further investigation was to provide results which
could be obtained under the same assumptions as stochastic
control problems, that is, controllability and observability
which have been always considered the most natural conditions for optimal control (30).
The results presented here are obtained under these kinds
of conditions for linear systems. Corresponding results for
nonlinear systems do not yet exist. Stochastic adaptive control problems with continuous time are under intensive investigation.
It is important to mention that the problems of stochastic
adaptive control previously described are problems of stochastic adaptive control with complete observation of the state.
Let us describe the adaptive control scheme used here. The
control as feedback gains is obtained from solving an infinitetime, quadratic cost control problem by replacing the correct
values of parameters in this solution by the estimates of parameters at time t to obtain the feedback gain at time t. The
estimates of the parameters are the least squares estimates
or the weighted least squares estimates. The least squares
estimate based on the observations of the state until time t is
used as the correct value of the parameter to solve the infinite-time control problem by solving the algebraic Riccati
equation. This method gives a feedback gain at each time t
and therefore a control policy.
Problems of continuous-time stochastic adaptive control
with partial observations in a general setting remain still
open. Partial results have been provided by Bertrand (31).
For discrete time see Ref. 32 and references therein.
Many problems with partial observations of a state can be
solved by the filtering theory (33). To recognize the differences
among filtering problems, control problems, and adaptive control problems, let us consider some simple examples.
Example 1. Filtering problem (7,11,34). Suppose that we
would like to improve our knowledge about the solution of a
differential equation so we observe X(s) at times s t. Let us
imagine that the X(t) is the charge at time t at a fixed point
in an electric circuit. However, due to inaccuracies in our

measurements we do not really measure X(s) but a perturbed
version of it: Y(s) X(s) noise.
The filtering problem is, what is the best estimate of X(t)
satisfying Y(s) X(s) noise based on the observations Y(s)
where s t?
Initially the problem is to filter the noise away from the
observations optimally.
In 1961 Kalman and Bucy (33) proved what is now known
as the Kalman-Bucy filter. Basically the filter gives a procedure for estimating the state of a system which satisfies a
linear differential equation, based on a series of observations
with a noise.
Example 2. Stochastic control problem. Portfolio Selection
and Consumption Model. Let X(t), t 0 be the wealth of an
individual at time t who invests his wealth in two types of
assets: the safe asset has the return rate r and the risky asset
has the average return rate . The wealth X(t) at time t
changes according to the stochastic differential equation
dX (t) = r[1 U1 (t)]X (t) dt
+ U1 (t)X (t)[ dt + dW (t)] U2 (t) dt
where U1(t) is the fraction of the wealth invested in the risky
asset at time t and U2(t) is the consumption rate at time t; r,
, are constants with r and 0; and (W(t), t 0) is a
real-valued standard Wiener process.
The controls are naturally constrained as 0 U1(t) 1
and U2(t) 0. The stochastic control problem is to maximize
the expected discounted total utility
I (U ) = Ey
s
exp(t)F[U2 (t)] dt,
y = W (s)
where F(u) u with 0 1 is the utility function and

0 is the discount rate,
Example 3. Stochastic adaptive control problem. Portfolio
Selection and Consumption Model. Consider the situation described in Example 2. For the adaptive control problem, it is
assumed that is an unknown parameter such that [a1,
a2] with r a1. The adaptive control procedure in this adaptive setting is to define the control at a time t, that is, the
portfolio selection and the consumption rate, using the optimal infinite-time control, where the estimate of the unknown
parameter at time t is used for the unknown parameter.
Example 4. Stochastic adaptive control problem. Manufacturing Model (35). Consider a manufacturing system that produces n distinct part types using m identical machines. Let
U(t) n denote the vector of productions rates, X(t) n the
vector of total inventories/backlogs, and Z(t) n the vector
demands. These processes are related by the following stochastic differential equation:
dX (t) = U (t) dt dZ(t),
X (0) = x0 Rn
The demand Z( ) is given by the following stochastic differential equation:

dZ(t) = Z dt +
dW (t)
Z(0) = z0
STOCHASTIC SYSTEMS
where Z is a vector of unknown constants, 0 is a small

parameter, is a given n n matrix, and [W(t), t 0] is a
standard n-valued Wiener process defined on a complete
probability space (, F , P).
The manufacturing system under consideration consists of
machines subject to breakdown and repair. Let M 0, 1,
. . ., m denote the set of machine total capacity states, and
let the process [(t), t 0] where (t) M denotes the total
capacity process for the manufacturing system. Because only
a finite amount of production capacity is available at any
given time t, an upper bound is imposed on the production
rate U(t). For example, in the one-dimensional case (n 1),
the production constraint is 0 U(t) (t).
The cost function J is defined by
J[U ()] = E
Let us define now the unfair games in which we expect to

lose (supermartingale) and to win (submartingale)
Definition 2 A stochastic process (T(t), t 0) is a supermartingale with respect to the increasing family of -algebras
(F t, t 0) if it satisfies properties (1) and (2) of Definition
1 and
(3) if s t, then
E[X (t)|Fs ]
et G[X (t),U (t)] dt
E[X (t)|Fs ]
A noise that appeared in the above example is described

by a Wiener process. A Wiener process plays a very important
and exceptional role in stochastic control theory. It is useful
for modeling disturbances which corrupt transmitted information and also for modeling controlled system disturbances.
A Wiener process also occurs in the KalmanBucy filter (33).
It became as a model of Brownian motion, very popular in
physics since Einsteins and Smoluchowskis work, but it is
not well known that it appeared for the first time in the scientific literature as a model of a price process in Bacheliers
Ph.D. dissertation in 1900.
a.s.
(2)
> X (s)
a.s.
(3)
where a.s. stands for almost surely.

We are interested in the martingale theory because of its
strong connection with stochastic calculus and because of the
properties that the processes defined previously have. The
first property that we illustrate is the DoobKolmogorov inequality that tells how (and when) one can get information
about a family of ordered, random variables based on the
knowledge of the last one:
Theorem 3. Let (Y(t), F t, t [0, T]) be a submartingale (with
right continuous sample paths). Then

1
E[Y + (T )]
P sup Y (t) x
(4)
x
t[0,T ]
>
where x 0 and Y(T) max(Y(T), 0).
PRELIMINARY RESULTS
We shall assume that (, F , P) is a probability space and
that F t, t 0 is a nondecreasing, right-continuous family of
sub--algebras of F . We assume that the filtration (F t) is
complete with respect to P, that is, each F t contains all the
P-null sets of F .
One of the consequences of this result is the martingale

convergence theorem which is commonly used in proving convergence results for stochastic adaptive control problems.
Theorem 4. Let (Y(t), F t, t [0, T]) be a nonnegative submartingale. If supt0E[Y(t)] , then
lim Y (t) = Y
Martingales as Important Tools in Proofs (8,9)
A stochastic process is a parametrized collection of random

variables. We consider collections (and so processes) for which
the parameter t takes values in positive real numbers. It is
common to call this kind of process a continuous-time stochastic process. Let us start by introducing the concept of martingale. The intuitive idea of martingale is related to the description of a fair game: we are in fact in a situation in which the
fortune expected after a certain time is exactly equal to what
we have. Then we define this fair game.
Definition 1 A stochastic process (X(t), t 0) is a martingale with respect to the increasing family of -algebras (F t,
t 0) if
(1) X(t) is F t measurable;
(2) EX(t) for every t 0;
(3) if s t, then
E[X (t)|Fs ] = X (s) a.s.
6 X (s)
(X(t), t 0) is a submartingale with respect to the increasing

family of -algebras (F t, t 0) if it satisfies properties (1) and
(2) of Definition 1 and
(3) if s t, then
where G is the running cost of inventory/backlog and production and 0 is the discount rate. The problem is to find an
admissible control U( ) that minimizes J[U( )].
545
(1)
a.s.
(5)
and E[Y] .
The Wiener Process Used for Modeling the Noise
Definition 5. The stochastic process (W(t), t 0) is a
Brownian motion (or a Wiener process) if
(1) W(0) 0;
(2) for t 0, W(t s) W(t) is a random variable that is
normally distributed with mean zero and variance 2s where
2 is a constant; and
(3) the process has independent increments (i.e., if 0 t1
t2 tn, then W(t2) W(t1), W(t3) W(t2), . . .,
W(tn) W(tn1) are mutually independent random variables).
If 2 1, then the process is called standard Brownian
motion.
The following result provides important information about
the quadratic variation of the Wiener process:
546
STOCHASTIC SYSTEMS
Theorem 6. Consider a family of partitions Pnn1

of [0, T]
where Pn tin0 i n, 0 t0n t1n tnn T with
Pn Pn1 for all n and such that Pnn1

is dense in [0, T] (i.e.,
n
n
limn [supi(ti1 ti )] 0). Let W(t), t 0] be a standard
Brownian motion process. Then
n1
lim
n
[W (ti+1
) W (tin )]2 = T
a.s.
gration (in Itos sense) satisfactorily. We start by defining the

integral for simple processes:
Definition 10 Let F t be the -algebra generated by [W(s),
s t]. We call (t), t [0, T] a simple process, if there is a
partition 0 t0 t1 tn T such that
(6)
(t) = j
i=0
t [t j , t j+1 )
(11)
and j is measurable with respect to F tj.
and

lim E
n1
2
n
[W (ti+1
)
W (tin )]2
=0
(7)
Define the stochastic integral for simple processes when t

[tj, tj1) as
i=0
n
i1
n
i
(s) dW (s) =
that is
(W(t ) (W(t )) converges to t both almost
surely and in L2.
n1
i0
It follows from this result that almost every sample path

of a Brownian motion process has infinite arc length on every
nonempty closed interval. Moreover it follows that there is
not a closed interval in which Brownian motion is differentiable (otherwise by the mean value theorem we would have
convergence to zero). Much more than this is true:
j1
i [W (ti+1 ) W (t j )] + j [W (t) W (t j )]
i=0
(12)
Note that the integral is linear and continuous (almost
surely) in t. We want to preserve these properties and at the
same time we want to extend the class of functions that we
can integrate. To do this, we need the following result:
Proposition 11. Let (t), t [0, T] be a process adapted
to F t, t [0, T] such that
Theorem 7. Almost all sample paths of the Wiener process

are nowhere differentiable.
2 (s) ds < a.s.
(13)
Let us give another important result about the sample

path behavior of the Wiener process:
Theorem 8. The strong law of large numbers for Brownian
motion:
W (t)
= 0 a.s.
lim
t
t
|n (t) (t)|2 dt > = 0
lim P
Theorem 9. If W(t), F t, t [0, T] is a square integrable

martingale with continuous sample paths such that for t s
Now if we define Yn as the integral of the simple process

, that is,
(s) dW (s)
(15)
then using Proposition 11, the family Yn converges to a process Y uniformly in probability. Note that the uniform convergence and the continuity of sample paths for Yn imply the
continuity of the sample paths of the process Y. Now we can
define the stochastic integral as follows:
(9)
(s) dW (s) = Y (t)
(16)
The construction does not depend on the sequence of simple

processes used, and the integral is unique up to indistinguishability, that is, if
Stochastic Integrals
In this section we define the stochastic integral
(s) dW (s)
Yn (t) =
then W(t), t [0, T] is a Brownian motion process.
(14)
(8)
If (W(t), t 0) is a Brownian motion process with continuous sample paths and we let F t be the -algebras generated
by [W(s), s 0], it follows from the independence of increments property of Brownian motion that W(t), F tt [0, T]
is a square integrable martingale with continuous sample
paths. The converse of this statement is true:
E{[W (t) W (s)]2 | Fs } = t s
Then there is a sequence of simple processes n(t), t [0,

T] such that for every 0

(10)
(s) dW (s) = Y (t)
(17)
(s) dW (s) = Z(t)
(18)
The unbounded variation property of Brownian motion sample paths described in the previous section does not allow us
to use the definition of the RiemannStieljes integral. Nevertheless by proceeding similarly we can define stochastic inte-
and
t
0
STOCHASTIC SYSTEMS
547
then PY(t) Z(t) for all t [0, T] 1. The following result

is the strong law of large numbers for stochastic integrals
(22).
We mentioned previously that we shall present the differential rule that we must use in stochastic calculus. This rule
was introduced by Ito.
Theorem 12. Let (W(t), t 0) be a standard Wiener process,

and let F t be the -algebra generated by (W(s), s t). Let
( f(t), t 0) be a random process adapted to (F t, t 0) such
that for 0 T
Theorem 14. (Itos Lemma). For i 1, 2, . . ., k let ( fi(t), t

[0, T]) and (gi(t), t [0, T]) be two processes adapted to
(F t, t [0 t]) such that
f (t) dt < a.s.

2
(19)
f 2 (s) ds < a.s.
(26)
g2 (s) ds < a.s.
(27)
and
T
0
f 2 (t) dt = a.s.
(20)
and let F : k1 be twice continuously differentiable in the

first k variables and continuously differentiable in the last
variable. Let
Then
f (t) dW (t)
lim
= 0 a.s.
(21)
dXi (t) = g j (t) dt + f i (t) dW (t)
(28)
Y (t) = F[X1 (t), . . ., Xk (t), t]
(29)
and
f (t) dt
2
Then
An interesting feature of the stochastic integral is that it
does not satisfy the ordinary calculus rules of integration. A
standard example of this situation is the fact that
W (s) dW (s) =
0
t
W 2 (t)
2
2
(22)

k
1
F f (t) f j (t) + Fk+1 dt
dY (t) =
Fi dXi (t) +
2 i, j=1 ij i
i=1
dX (t) = a[t, X (t)] dt + [t, X (t)] dW (t),
X (0) = X0
Fij =
2F
[X (t), . . ., Xk (t), t]
Xi X j 1
[u, X (u)] dW (u)
(24)
We have the following result about the existence and uniqueness of the solution:
Theorem 13. If a(t, x) and (t, x) are continuous in t and satisfy the Lipschitz condition in x uniformly in t, that is, for
every t [0, T]
|a(t, x) a(t, y)| + | (t, x) (t, y)|
6 K|x y|,
Fk+1 =
F
[X (t), . . ., Xk (t), t]
t 1
i, j 1, . . ., k.
THE FILTERING PROBLEM (7)
a[u, X (u)] du +
0
F
[X (t), . . ., Xk (t), t]
Xi 1
(23)
if for every t [0, T] it satisfies the following equation

Fi =
and
In the previous section we introduced the concept of stochastic integration. Now we can use it to define stochastic differential equations. In fact we say that X(t), t [0, T] is a
solution for the stochastic differential equation
X (t) = X0 +
(30)
where
This is a consequence of the fact that Brownian motion has

nonzero quadratic variation (Theorem 6). We see in the next
section the differential rule that is to be used in stochastic
calculus.
Stochastic Differential Equations (7)
k
KR
(25)
then the solution to the stochastic differential Eq. (23) exists

and it is unique (up to indistinguishability).
Example 1 previously given is a special case of the following

general filtering problem:
Suppose that the state X(t) n of a system at time t is
given by a stochastic differential Eq. (23).
We assume that the observations H(t) m are performed
continuously and are of the form
(t)
H(t) = c[t, X (t)] + [t, X (t)]W
(31)
where c : n1 m, : n1 mr are functions satisfying

(t) denotes r-dimensional white noise. If we
Eq. (25) and W
introduce
t
Z(t) =
H(s) ds
(32)
0
548
STOCHASTIC SYSTEMS
then
where
dZ = c[t, X (t)] dt + [t, X (t)] dW (t)
S(t) = E{[X (t) X (t)]2 }
(33)
Z(0) = 0
where W(t) is an r-dimensional Wiener process.

Note that if H(s) is also known for 0 s t, then Z(s) is
also known for 0 s t and vice versa. So no information is
lost or gained by considering Z(t) as our observations instead
of H(t), but this allows us to obtain a well-defined mathematical model of the situation.
The filtering problem is the following:
Given the observations Z(s) satisfying Eq. (33) for 0 s
(t) of the state X(t) of the system
t, what is the best estimate X
Eq. (23) based on these observations?
(t) is based on the observaBy saying that the estimate X
tions Z(s) : s t, we mean that
X (t) is Ft -measurable,
where Ft is the -algebra generated by {Z(s), s 6 t}
G2 (t)
dS
= 2F (t)S(t) 2 S2 (t) + C2 (t),
dt
D (t)
S(0) = E{(X0 E[X0 ])2 }
(35)
(40)
Example 5. Estimation of a parameter. Suppose we want to

estimate the value of a (constant) parameter based on observations Z(t) satisfying the model
dZ(t) = X (t) dt + (t) dW (t)
where X(t), (t) are known functions. In this case the stochastic differential equation for is of course
d = 0
(34)
(t) is the best such estimate, we mean that

By saying that X

|X (t) X (t)|2 dP = E[|X (t) X (t)|2 ]
= inf{E[|X (t) = Y |2 ] : Y Y }
satisfies the (deterministic) Riccati equation
so the Riccati equation for S(t) E[ (t)]2 is

2
X (t)S(t)
dS
=
t
(t)
which gives
where

1
t
2
2
S(t) = S1
+
X
(s)
(s)
ds
0
Y := {Y :
Rn ; Y L2 (P) and Y is Ft -measurable}, (36)
where L2(P) L2(, P) and (, F , P) is a probability space.
One of the most important results is the following:
and the KalmanBucy filter is given by
X (t) = E[X (t)|Ft ]
d(t)
This is the basis for the general FujisakiKallianpurKunita equation of filtering theory. See, for example, Ref. 34.
Here we concentrate on the linear case, which allows an
explicit solution in terms of a stochastic differential equation
(t) (the KalmanBucy filter):
for X
In the linear filtering problem the system and observation
equations have the following form:
(linear system)
dX (t) = F (t)X (t) dt + C(t) dW (t);

F (t) Rnn , C(t) Rn p
(37)
This can be written as

t
S1
+
X (s)2 (t)2 d(t)
+ X (t)2 (t)2 dt
0
0
= X (t) (t)2 dZ(t)

We recognize the left-hand side as

t
1
2
2
d S0 +
X (s) (t) ds (t)
(linear observations) dX (t) = G(t)X (t) dt + D(t) dV (t);

G(t) Rmn , D(t) Rmr
(38)
Theorem 15. The One-Dimensional KalmanBucy Filter.

(t) E[X(t)F t] of the one-dimensional linear
The solution X
filtering problem
(linear system)
dX (t) = F(t)X (t) dt + C(t) dW (t);
F (t),C(t) R
(linear observations) dZ(t) = G(t)X (t) dt + D(t) dV (t);
G(t), D(t) R
satisfies the stochastic differential equation

G(t)S(t)
G2 (t)S(t)
X (t) dt +
dZ(t)
dX (t) = F (t)
D2 (t)
D2 (t)
X = E[X ]
0
X (t)S(t)
[dZ(t) = X (t)(t)]
(t)2
(39)
so
+
0 S1
0
S1
+
0
X (s) (s)2 dZ(s)
X (s)2 (t)2 ds
This estimate coincides with the maximum likelihood estimate in the classical estimation theory if S1
0 0. See (22).
For more information about estimates of drift parameters
in diffusion, see the next section.
Theorem 16. The Multidimensional KalmanBucy Filter.
The solution
X (t) = E[X (t)|Ft ]
STOCHASTIC SYSTEMS
of the multidimensional linear filtering problem

(linear system)
dX (t) = F (t)X (t) dt + C(t) dW (t);

F (t) Rnn , C(t) Rn p
(linear observations) dZ(t) = G(t)X (t) dt + D(t) dV (t);

G(t) Rmn , D(t) Rmr
satisfies the stochastic differential equation
dX (t) = [F SGT (DDT )1 G]X (t) dt + SGT (DDT )1 dZ(t) :

X (0) = E[X0 ]
tion of the following stochastic differential equation:

dX (t) = a t, X (t),U (t)] dt + t, X (t), U (t) dB(t)
X (0) = x
(45)
to minimize the objective function (also called the cost functional)

T

(46)
J(x,U ) = Ex
L s, X (s),U (s) ds + X (T )
0
where the admissible choices of the controls are all of the

smooth functions of the observations of X(t). We define the
HamiltonJacobiBellman equation of dynamic programming as
where

S(t) := E [X (t) X (t)][X (t) X (t)]T Rnn

W
+ min AsU W + L(s, X , U ) = 0
s
satisfies the matrix Riccati equation
(47)
where W(T, x) Es,x[X(T)] is the value function and (A t, t

0) is the infinitesimal generator of the semigroup associated
with the stochastic differential Eq. (45). The following result
is known as the verification theorem
dS
T
T
T
= FS + SF DG (DDT )1 GS + CC
dt

S(0) = E {X (0) E[X (0)]}{X (0) E[X (0)]}T
The condition on D(t) mr is now that D(t)D(t)T is invertible
for all t and that [D(t)D(t)T]1 is bounded on every bounded
t-interval.
A similar solution can be found for the more general situation
dX (t) = [F0 (t) + F1 (t)X (t) + F2 (t)Z(t)] dt

+ C(t) dW (t)
(observations) dZ(t) = [G0 (t) + G1 (t)X (t) + G2 (t)Z(t)] dt
+ D(t) dV (t)
(system)
where X(t) , Z(t) and B(t) [W(t), V(t)] is (n m)dimensional Brownian motion with appropriate dimensions
on the matrix coefficients. See (34), which also treats the nonlinear case.
For various applications of filtering theory see Ref. 33.
n
549
Theorem 17. Let W(s, y) be a solution of Eq. (47) such that

W C 2(Q) and it is continuous on the closure of Q, (we say
W is a smooth function). Then
(1) W(s, y) J(s, y, U) for any admissible control U and
any initial condition (s, y) Q; and
(2) if U* is an admissible control such that

AsU W + L(s, y, U ) = min AsU W + L(s, y, v)
(48)
vU
for all (s, y) Q, then

W (s, y) = J(s, y, U )
(49)
STOCHASTIC CONTROL (7,10,11,36)

Let (X(t), t 0) be the solution of the stochastic differential
Eq. (23) with X(0) x. Let f be a bounded, continuous function, and let T be the operator defined by
(Tt f )(x) = Ex { f [t, X (t)]}
(41)
This operator satisfies the semigroup property

Tt (Ts f )(x) = (Tt+s f )(x)
h0
Tt+h Tt
= At
h
(43)
where A t is an operator that acts on f in the following way:

At f (x) = a(t, x)
1
f
2 f
(x) + 2 (t, x) 2 (x)
x
2
x
The Linear Stochastic Gaussian Problem (7,11)

Suppose that the state X(t) of the system at time t is given by
the following linear stochastic differential equation:

dX (t) = H(t)X (t) + M(t)U (t) dt + (t) dB(t)
(50)
t s; X (s) = x
>
and the cost has the form
J u (s, x) = E s,x

t1

T
T
T
X (t) C(t)X (t) + U (t) D(t)U (t) dt + X (t1 ) RX (t1 )
s
(42)
If we differentiate this operator,

lim
Remark. U* is an optimal control.
(44)
(A t, t 0) is called the infinitesimal generator of the semigroup (Tt, t 0). Consider the problem of controlling the solu-
s 6 t1
(51)
where all the coefficients H(t) nn, M(t) nk, (t)

nm, C(t) nn, D(t) kk, and R nn, are t-continuous
and deterministic. We assume that C(t) and R are symmetric,
nonnegative-definite and D(t) is symmetric, positive-definite,
for all t. We also assume that t1 is a deterministic time.
[B(t), t 0] is a standard Brownian motion.
Then the problem is to choose the control u u[t, X(t)] so
that it minimizes Ju(s, x). We may interpret this as follows:
The aim is to find a control u which makes X(t) ( denotes
the norm) small quickly and so that the energy used (uT
Du) is small. The sizes of C(t) reflects the cost of large values
550
STOCHASTIC SYSTEMS
of X(t), whereas the size of D(t) reflects the cost (energy) of

applying large values of u(t).
In this case the HJB equation for W(s, x) becomes

0 = inf F v (s, x) + (LvW )(s, x)
v

n

W
W
+ inf xT C(t)x + vT D(t)v +
(H(t)x + M(t)v)i
=
v
s
xi
i=1

n
2W
1
for s < t1
+
( T )ij
(52)
2 i, j=1
xi x j
and
W (t1 , x) = xT Rx
(53)
We find a solution W of Eqs. (52)(53) of the form

W (t, x) = xT S(t)x + a(t)
t < t1
(55)
is an optimal control, and the minimum cost is given by

t1
W (s, x) = xT S(s)x +
tr( T S)t dt,
s < t1
(56)
where S(t) satisfies the corresponding Riccati type equation

from linear filtering theory. This formula shows that the extra cost due to the noise in the system is given by
t1
a(s) =

dX (t) = A()X (t) + BU(t) dt + dW (t)
(54)
where S(t) St nn is symmetric, nonnegative-definite,

a(t) are both a(t), and S(t) are continuously differentiable
w.r.t. t (and deterministic).
The following
u (t, x) = D(t)1 M(t)T S(t)x,
tion of the unknown parameters and adaptive control. Maximum likelihood (or equivalently least squares) estimates are
used to identify unknown constant parameters. These estimates are given recursively and are strongly consistent. The
adaptive control is usually constructed by the so-called certainty equivalence principle, that is, the optimal stationary
controls are computed by replacing the unknown true parameter values by the current estimates of these values. Because
the optimal stationary controls are continuous functions of
the unknown parameters, the self-tuning property is verified.
The family of average costs using the control from the certainty equivalence principle converges to the optimal average
cost. This verifies the self-optimizing property.
A model for the adaptive control of continuous-time linear
stochastic systems with complete observations of the state is
described by the following stochastic differential equation:
tr( T S)t dt
where X(t) n, U(t) m.

A() = A0 +
1
lim sup J(X0 , U, , t)
t
t
(57)
at our disposal, then the optimal control u*(t, ) (required to

be F t-adapted, where F t is the -algebra generated by
Z(r) : r t), is given by
u (t, ) = D(t)1 M(t)T S(t)X (t)()
i Ai
(58)
(t) is the filtering estimate of X(t) based on the obserwhere X

vations Z(r) : r t, given by the KalmanBucy filter. Comparing with Eq. (55), we see that the stochastic control problem in this case splits into a linear filtering problem and a
deterministic control problem.
LEAST SQUARES AND CONTINUOUS-TIME
STOCHASTIC ADAPTIVE CONTROL (2)
The general approach to adaptive control described here exhibits a splitting or separation of the problems of identifica-
(60)
Ai L (n) i 0, . . ., p, B L (m, n), (W(t), t ) is a

standard n-valued Wiener process and X0 a n. It is
assumed that
(5.A1) A p is compact and A .
(5.A2) (A(), B) is reachable for each A .
(5.A3) The family (Ai, i 1, . . ., p) is linearly independent.
Let (F t, t ) be a filtration such that Xt is measurable
with respect to F t for all t and (W(t), F t, t ) is a
Brownian martingale. The ergodic, quadratic control problem
for Eq. (59) is to minimize the ergodic cost functional
The Separation Principle
dZ(t) = g(t)X (t) dt + (t) dB(t)
p

i=1
The separation principle (5,18) states that if we only have

partial knowledge of the state X(t) of the system, that is, if
we have only noisy observations
(59)
(61)
where

J(X0 , U, , t) =

QX (s), X (s) + PU (s), U (s) ds
(62)
and t (0, ), X(0) X0, Q L (n); P L (m) are selfadjoint and P1 exists; [X(t), t ] satisfies Eq. (59); and
[U(t), t ] is adapted to (F t, t ). It is well known (33)
that, if is known, then there is an optimal linear feedback
control such that
U (t) = KX (t)
(63)
where K P1B*V and V is the unique, symmetric, nonnegative-definite solution of the algebraic Riccati equation
VA + AV VB P1 BV + Q = 0
(64)
For an unknown the admissible adaptive control policies

[U(t), t ] are linear feedback controls

U (t) = K(t)X (t) = K t, X (u), u
6 t X (t)
(65)
STOCHASTIC SYSTEMS
where [K(t), t 0] is an L (n, m)-valued process that is uniformly bounded and there is a fixed 0 such that (K(t), t
0) is measurable with respect to (Xu, u t ) for each t
and [K(t), t [0, )] is a deterministic function. For such
an adaptive control, it is elementary to verify that there is a
unique strong solution of Eq. (59). The delay 0 accounts
for some time required to compute the adaptive control law
from the observation of the solution of Eq. (59).
Let (U(t), t 0) be an admissible adaptive control and let
[X(t), t 0] be the associated solution of Eq. (59). Let A (t)
[aij(t)] and A (t) [aij(t)] be L (p)-valued processes such that
t

aij (t) =
Ai X (s), A j X (s) ds
where (X(t), t 0) is the solution of Eq. (59) with the admissible adaptive control [U(t), t 0] and 0 K and V is the
solution of the algebraic Riccati Eq. (64) with 0. Then
lim inf
T
1
J(X0 , U, 0 , T )
T
> tr V
a.s.
(71)
If U is an admissible adaptive control U(t) K(t)X(t) such

that
lim K(t) = k0
a.s.
(72)
where k0 P1B*V, then
a ij (t) =
551
aij (t)
lim
aii (t)
1
J(X0 , U, 0 , T ) = tr V
T
a.s.
(73)
To verify the strong consistency of a family of least squares

estimates, it is assumed that
Corollary. Under the assumptions of the Proposition, if Eq.

(72) is satisfied, then Eq. (69), and (70) are satisfied.
lim inf | det A (t)| > 0 a.s.
The previous results can be combined for a complete solution to the stochastic adaptive control problem Eqs. (59, 61).
The estimate of the unknown parameter vector at time t,

(t), for t 0 is the minimizer for the quadratic functional of
, L(t, ), given by
t

L(t, ) =
[A() + BK(s)]X (s), dX (s)
0
(66)

2
1 t
+
[A() + BK(s)]X (s) ds
2 0
where U(s) K(s)X(s) is an admissible adaptive control. The
following result (31) gives the strong consistency of these
least squares estimators.
Theorem 20. Assume that (5.A15.A4) are satisfied. Let

[ (t), t 0] be the family of least squares estimates where
(t) is the minimizer of Eq. (66). Let [K(t), t 0] be an admissible adaptive control law such that

K(t) = P1 BV (t
)
where V() is the solution of Eq. (64) for A . Then the
family of estimates ( (t), t 0) is strongly consistent.
lim K(t) = k0
a.s.
(74)
where k0 P1B*V(0), and
Theorem 18. Let (K(t), t 0) be an admissible adaptive feedback control law. If (5.A15.A4) are satisfied and 0 A ,
the interior of A , then the family of least squares estimates
[ (t), t 0] where (t) is the minimizer of Eq. (66), is strongly
consistent, that is,

(67)
P 0 lim (t)
= 0 = 1
WEIGHTED LEAST SQUARES AND CONTINUOUS-TIME

ADAPTIVE LQG CONTROL (30)
where 0 is the true parameter vector.
Introduction
The family of estimates [ (t), t 0] can be computed recursively because this process satisfies the following equation

d(t)
= A 1 (t) A (t)X (t), dX (t) A[(t)]X
(t) dt BU (t) dt

(68)
The linear, Gaussian control problem with an ergodic, quadratic cost functional is probably the most well-known ergodic
control problem. Because the optimal control is easily computed and the existence of an invariant measure follows directly from the stability of the optimal system, it is a basic
problem to solve for stochastic adaptive control. For discretetime linear systems it has been studied extensively, especially
for ARMAX models (see Refs. 6, 28, 3740 for many references). Although this adaptive control problem has been less
studied for continuous-time linear systems, it is nonetheless
an important problem as a model for physical systems that
naturally evolve in continuous time and as an approximation
for discrete-time sampled systems when the sampling rate is
high.
The adaptive control problem is solved using only the natural assumptions of controllability and observability. The
weighted least squares scheme is used to obtain the conver-
where (t)x, y (Aix, y) i 1, . . ., p.

Now the performance of some admissible adaptive controls
is described.
Proposition 19. Assume that (5.A15.A4) are satisfied and
that
1
V X (t), X (t) = 0 a.s.
t

1 t
lim sup
|X (t)|2 ds < a.s.
t 0
t
lim
(69)
(70)
lim
1
J(X0 , U, 0 , T ) = tr V
T
a.s.
(75)
552
STOCHASTIC SYSTEMS
gence of the family of estimates (self convergence), and the

scheme is modified by a random regularization to obtain the
uniform controllability and observability of the family of estimates. A diminishing excitative white noise is used to obtain
strong consistency. The excitation is sufficient to include the
identification of unknown deterministic linear systems.
This approach eliminates some other assumptions previously used that are unnecessary for the control problem for
a known system and are often difficult to verify. Furthermore,
this approach eliminates the need for random switchings or
resettings which often occur in previous work.
Weighted Least Squares Identification
Let [X(t), t 0] be the process satisfying the stochastic differential equation
dX (t) = AX(t) dt + BU (t) dt + D dW (t)
(76)
where X(0) X0, X(t) n, U(t) m, [W(t), t 0] is an pvalued standard Wiener process, and [U(t), t 0] is a control
from a family specified subsequently. The random variables
are defined on a fixed, complete probability space (, F , P),
and there is a filtration (F t, t 0) defined on this space that
is specified subsequently. It is assumed that A and B are unknown.
The following assumption is made:
(A1) (A, B) is controllable.
To describe the identification problem in a standard form,
let
= [A B]
(77)
A function f is slowly increasing if it is positive, increasing,

and f(x2) 0[f(x)] as x .
Applying the Ito formula to [XT(t)X(t), t 0], it is elementary to verify (31) that for D 0
lim r(t) = a.s.
If D 0, then the diminishing excitation used subsequently

implies this result.
The dependence of [(t), t 0] on [a(t), t 0] and the
dependence of [a(t), t 0] on f are suppressed for notational
convenience.
The following result describes some basic properties of the
WLS algorithm.
Proposition 21. Let [(t), P(t) t 0] satisfy Eqs. (80)(81).
The following properties are satisfied:
sup |P1/2 (t) (t)|2 < a.s.
(1)
>
(85)
t 0
(2)
a(t)| (t)(t)|2 dt < a.s.
(86)
lim (t) =
(3)
a.s.
(87)
where (t) (t) and is a matrix-valued random

variable.
Define (t, x) as
(t, x) = (t) P1/2 (t)x
and

X (t)
(t) =
U (t)
and
(78)
(t, x) = [A(t, x)
B(t, x)]
so that Eq. (76) is rewritten as
dX (t) = (t) dt + D dW (t)
(79)
A family of weighted least squares (WLS) estimates [(t),

t 0] is given by
d (t) = a(t)P(t)(t)[dX T(t) T(t) (t) dt]
dP(t) = a(t)P(t)(t) T(t)P(t) dt
A well-known test for controllability of (A, B) is the positivity

of F where

F (t, x) = det
n1
(80)
(81)
and similarly a test for observability of (A, C) is the positivity

of G where
G(t, x) = det
1
f [r(t)]
t
r(t) = e +
|(s)|2 ds
a(t) =
(82)
(83)
>0
i=0
where (0) is arbitrary, P(0) 0 is arbitrary
and f .

F = f | f : R+ R+ , f is slowly increasing

dx
< for some c
and
x f (x)
c
A (t, x)B(t, x)BT(t, x)A T(t, x)

i
(84)

n1

A (t, x)C CA (t, x)
iT
i=0
The linear transformation C is known. For the adaptive control problem, C Q1/2
where Q1 determines the quadratic
1
form of the state in the cost functional.
A random search method ensures uniform controllability
and observability of a family of estimates. Let (n, n ) be
a sequence of independent, identically distributed M (n
m,n)-valued random variables that is independent of [W(t),
t 0] so that each random variable n is uniformly distributed in the unit ball or the unit sphere for a norm of the
matrices. Define a sequence of M (n m,n)-valued random
STOCHASTIC SYSTEMS
If (t) is given by
variables by induction as follows:
T
(t) = A(t) B(t)Q1
2 B (t)R(t)
0 = 0

k = k
k1
if f (k, k )
otherwise
> (1 + ) f (k, k1 )
and
T
() = A() B()Q1
2 B ()R()
where (0, 2 1) is fixed and f(k, x) F(k, x)G(k, x). By

the compactness of the unit ball or the unit sphere and the
continuity of F(k, ) and G(k, ) for k , it follows that the
sequence (k, k ) terminates after some random integer so
there is a such that
= lim k
k
(t) = k
then
lim (t) = ()
and (t) and () are stable a.s.

The lagged certainty equivalence control is
a.s.
T
U (t) = Q1
2 B (t)R(t)X (t)
Define a family of estimates [ (t), t 0] as

t (k, k + 1]
(88)
[B(t), R(t)] = [B(0), R(0)]

k = (k) P1/2 (k)k
(89)
It is clear from Eqs. (81) and (87) that [ (t), t 0] converges

a.s. For notational simplicity the dependence on (k, k )
in Eq. (88) is suppressed so that
(t) = [A(t) B(t)]

Theorem 22. If ( (t), t 0) is the family of estimates given
by Eq. (88), (A, B) in Eq. (76) is controllable, and (A, Q1/2
1 ) is
observable, then for any admissible control [U(t), t 0],
[ (t), t 0] has the following properties a.s.:
(1) self-convergence,
(2) uniform controllability and observability, and
(3) semiconsistency.
Adaptive Control
Consider the following ergodic cost functional for the system
Eq. (76):
(91)
It is called lagged because B(t) and R(t) depend on X(s) and

j for s t and j t. This follows easily by induction recalling
the construction of ( (t), t 0), that is,
where k and
J(U ) = lim sup
553
1
T
T
0
[X T (t)Q1 X (t) + U T (t)Q2U (t)] dt
(90)
T
AT (t)R(t) + R(t)A(t) R(t)B(t)Q1
2 B (t)R(t) + Q1 = 0
has a unique, random, symmetric, positive solution R(t) a.s.

where [A(t) B(t)] (t). Because the solution of the algebraic
Riccati equation is a smooth function of the parameters of the
equation, there is a symmetric, positive L (n)-valued random
variable R() such that
lim R(t) = R()
[B(t), R(t)] = (Bk , Rk )

for t (k, k 1], and k [Ak, Bk] is given by Eq. (89).
To obtain strong consistency for the family of estimates
( (t), t 0), a diminishing excitation is added to the adaptive
control Eq. (91), that is,
dU (t) = Lk dX (t) + k dV (t)
a.s.
(92)
for t (k, k 1] and k where U(0) is arbitrary,

Lk = Q1
2 B(k)R(k)
and
log k
k2 =
k
for k 1. The process [V(t), t 0] is an m-valued standard
Wiener process that is independent of [W(t), t 0] and (k, k
). As noted before, it easily follows that there is a matrixvalued random variable L such that
lim Lk = L
where [U(t), t 0] is an admissible control, Q2 0,

and Q1 0.
The following assumption is made:
(A2) (A, Q1/2
1 ) is observable.
The stochastic algebraic Riccati equation
for t [0, 1],
a.s.
The sub--algebra F t is the P-completion of [X0, W(s), j,

V(s); s t, j t].
The state X(t) is augmented with the control U(t) as follows:

X (t)
(t) =
U (t)

A
B
Fk =
Lk A Lk B

D
0
Gk =
Lk D k I

W (t)
(t) =
V (t)
554
STOCHASTIC SYSTEMS
The stochastic differential equation for the augmented state

process ((t), t 0) is
d(t) = Fk (t) dt + Gk d (t)
for t (k, k 1].
The following result shows that the family of regularized,
weighted, least squares estimates is strongly consistent using
the lagged certainty equivalence control with diminishing excitation.
Proposition 23. Let [ (t), t 0] be the family of estimates
given by Eq. (88) using the control Eq. (92) in (Eq. 76). Then
lim (t) =
a.s.
where
= [A B]
Remark. It should be noted that the conditions of this proposition are satisfied if D 0 so that the identification of deterministic systems is included in this result.
Because the family of estimates [ (t), t 0] is strongly consistent, the self-optimality of the diminishingly excited lagged
certainty equivalence control Eq. (92) can be verified.
Theorem 24. Let (A1) and (A2) be satisfied for the stochastic
system
dX (t) = AX(t) dt + BU (t) dt + D dW (t)
with the cost functional Eq. (90) where A and B are unknown.
If the admissible adaptive control
dU (t) = Lk dX (t) + k dV (t)
Lk = Q1
2 B (k)R(k)
is used, then
J(U ) = inf J(U ) = tr(D RD)

U
a.s.
where R is the unique, positive, symmetric solution of the algebraic Riccati equation
A R + RA RBQ1
2 B R + Q1 = 0
and
J(U ) = lim sup
T
1
T
T
0
[X (t)Q1 X (t) + U (t)Q2U (t)] dt
a.s.
BIBLIOGRAPHY
1. V. Borkar, Optimal Control of Diffusion Processes, New York: Wiley, 1989.
2. T. E. Duncan and B. PasikDuncan, Adaptive control of continuous-time linear stochastic systems, Math. Control, Signals, Syst.,
3: 4560, 1990.
3. B. Bercu, Weighted estimation and tracking for ARMAX models,

4. V. Borkar and A. Bagchi, Parameter estimation in continuoustime stochastic processes, Stochastics, 8: 193212, 1982.
5. M. H. A. Davis and R. B. Vinter, Stochastic Modelling and Control, London: Chapman and Hall, 1985.
6. P. R. Kumar and P. Varaiya, Stochastic Systems: Estimation,
Identification and Adaptive Control, Englewood Cliffs, NJ: Prentice-Hall, 1986.
7. B. ksendal, Stochastic Differential Equations: An Introduction
with Applications, 4th ed., New York: Springer, 1995.
8. I. Karatzas and S. E. Shreve, Brownian Motion and Stochastic
Calculus, New York: Springer-Verlag, 1988.
9. S. Karlin and H. Taylor, A First Course in Stochastic Processes,
2nd ed., New York: Academic Press, 1975.
10. A. Bensoussan, Stochastic Control by Functional Analysis Method,
Amsterdam, The Netherlands: North Holland, 1982.
11. W. H. Fleming and R. W. Rishel, Deterministic and Stochastic
Optimal Control, New York: Springer Verlag, 1975.
12. J. Zabczyk, Lectures in Stochastic Control, Control Theory Centre Report No. 125, Univ. Warwick Control Theory Centre.
strom, Introduction to Stochastic Control, New York: Aca13. K. J. A
demic Press, 1970.
14. R. C. Merton, Continuous-Time Finance, Cambridge, MA: Blackwell, 1990.
15. T. E. Duncan and H. Upmeier, Stochastic control problems and
spherical functions on symmetric spaces, Trans. Amer. Math. Soc.,
347: 10831130, 1995.
16. W. H. Fleming and H. M. Soner, Controlled Markov Processes and
Viscosity Solutions, New York: Springer Verlag, 1993.
17. H. P. McKean, Stochastic Integrals, New York: Academic Press,
1969.
18. R. J. Elliott, Stochastic Calculus and Applications, New York:
19. I. I. Gihman and A. V. Skorohod, Controlled Stochastic Processes,
New York: Springer Verlag, 1979.
20. K. Ito and H. P. McKean, Diffusion Processes and Their Sample
Paths, New York: Springer Verlag, 1965.
21. R. S. Liptser and A. N. Shiryayev, Statistics of Random Processes,
New York: Springer Verlag, 1977, vol. I.
22. R. S. Liptser and A. N. Shiryayev, Statistics of Random Processes,
New York: Springer Verlag, 1978, vol. II.
strom and B. Wittenmark, Adaptive Control, 2nd ed.,
23. K. J. A
Reading, MA: AddisonWesley, 1995.
24. T. E. Duncan and B. PasikDuncan (eds.), Stochastic Theory and
Adaptive Control, Lecture Notes in Control and Information Sciences, Springer Verlag, 1992, vol. 184.
25. L. Gerencser and P. E. Caines (eds.), Topics in Stochastic Systems:
Modelling, Estimations, and Adaptive Control, Lecture Notes in
Control and Information Sciences, New York: Springer Verlag,
1991, vol. 161.
26. P. R. Kumar, A survey of some results in stochastic adaptive control, SIAM J. Control Optim., 23: 329380, 1985.
27. T. E. Duncan, B. PasikDuncan, and L. Stettner, Almost selfoptimizing strategies for the adaptive control diffusion processes,
J. Optim. Theor. Appl., 81: 479507, 1994.
28. H. F. Chen and L. Guo, Identification and Stochastic Adaptive
Control, Boston: Birkhauser, 1991.
29. W. Levine (ed.), The Control Handbook, Boca Raton, FL: CRC
Press, Piscataway, NJ: IEEE Press, 1996.
30. T. E. Duncan, L. Guo, and B. PasikDuncan, Adaptive continuous-time linear quadratic Gaussian control, IEEE Trans. Autom.
Control, submitted.
STOCK MARKETS
31. P. Bertrand, Adaptive control of partially observed linear stochastic systems, Stochastics, 54: 2151, 1995.
32. T. E. Duncan, B. PasikDuncan, and L. Stettner, Adaptive control of a partially observed discrete time Markov processes, Appl.
Math. Optim., 37: 269293, 1998.
33. R. E. Kalman and R. S. Bucy, New results in linear filtering and
prediction theory, Trans. ASME, Ser. D, J. Basic Eng., 83: 95
107, 1961.
34. G. Kallianpur, Stochastic Filtering Theory, New York: Springer
Verlag, 1980.
35. G. Yin and Q. Zhang (eds.), Mathematics in Stochastic Manufacturing Systems, Lecture Notes in Applied Mathematics, Providence:
Amer. Math. Soc., 1997, vol. 33.
36. H. J. Kushner, Stochastic Stability and Control, New York: Academic Press, 1967.
37. P. E. Caines, Linear Stochastic Systems, New York: Wiley, 1988.
38. G. Goodwin, P. Ramadge, and P. Caines, Discrete time stochastic
adoptive control, SIAM J. Control Optim., 19: 829853, 1981.
strom, and P. R. Kumar (eds.), Adaptive Con39. G. C. Goodwin, K. A
trol, Filtering and Signal Processing, IMA Volumes in Mathematics and Its Applications 74, New York: Springer, 1995.
40. G. Chen, G. Chen, and S. H. Hua, Stochastic Control Systems,
Boca Raton, FL: CRC Press, 1995.
BOZENNA PASIKDUNCAN
University of Kansas
555
SYSTEM IDENTIFICATION
The process of going from observed data to a mathemathical
model is fundamental in science and engineering. In the control area, this process has been termed system identification,
and the objective is then to find dynamical models (difference
or differential equations) from observed input and output signals. Its basic features are, however, common with general
model building processes in statistics and other sciences.
System identification covers the problem of building models of systems when insignificant prior information is available and when the systems properties are known, up to a few
parameters (physical constants). Accordingly, one talks about
black-box and gray-box models. Among black-box models,
there are familiar linear models such as ARX and ARMAX,
and among nonlinear black-box models we have, for example,
artificial neural networks (ANN).
THE PROBLEM
The area of system identification begins and ends with real
data. Data are required to build and to validate models. The
result of the modeling process can be no better than what
corresponds to the information contents in the data.
Look at two data sets:
Example 1 An Unstable Aircraft
Figure 1 shows some results from test flights of the new
Swedish aircraft, JAS-Gripen, developed by SAAB Military
Aircraft AB, Sweden. The problem is to use the information
in these data to determine the dynamical properties of the
aircraft for fine-tuning regulators, for simulations, and so on.
Of particular interest are the aerodynamical derivatives.
Example 2 Vessel Dynamics
Figure 2 shows data from a pulp factory. They are collected
from one of the buffer vessels. The problem is to determine
the residence time in the vessel. The pulp spends about 48 h
total in the process, and knowing the residence time in the
different vessels is important in order to associate various
portions of the pulp with the different chemical actions that
have taken place in the vessel at different times. (The -num-
263
20
40
60
80
100
120
140
160
180
20
40
60
80
100
120
140
160
180
20
40
60
80
100
120
140
160
180
20
40
60
80
100
120
140
160
180
Figure 1. Results from test flights of the new Swedish aircraft JASGripen, developed by SAAB Military Aircraft AB, Sweden. From
above (a) Pitch rate, (b) Elevator angle, (c) Canard angle, (d) Leading
edge flap.
ber is a quality property that in this context can be seen as a

marker allowing us to trace the pulp.)
So, the bottom line of these examples is that we have collected inputoutput data from a process or a plant, and we
need to extract information from these to find out (something
about) the process dynamical properties.
BACKGROUND AND LITERATURE
System identification has its roots in standard statistical
techniques, and many of the basic routines have direct interpretations as well-known statistical methods such as least
squares and maximum likelihood. The control community
took an active part in the development and application of
these basic techniques to dynamic systems right after the
birth of modern control theory in the early 1960s. Maximum
likelihood estimation was applied to different equations
(ARMAX models) by Ref. 1, and thereafter, a wide range of
estimation techniques and model parameterizations flourished. By now, the area is well matured with established and
well understood techniques. Industrial use and application of
the techniques has become standard. See Ref. 2 for a common
software package.
The literature on system identification is extensive. For a
practical user oriented introduction, we may mention (3).
Texts that go deeper into the theory and algorithms include
Refs. 4 and 5. A classical treatment is Ref. 6.
These books all deal with the mainstream approach to
system identification, as described in this article. In addition,
there is substantial literature on other approaches, such as
set membership (compute all those models that reproduce
the observed data within a certain given error bound), estimation of models from given frequency response measurement
(7), on-line model estimation (8), non-parametric frequency
domain methods (9), etc. To follow the development in the
264
The problem can be expressed as finding that model in the

candidate set, that best describes the data, according to the
criterion, and then evaluate and validate that models properties. To do this we need to penetrate a number of things:
Output #1
25
20
15
10
5
0
100
200
300
400
500
600
Input #1
20
1. First, we give a preview of the whole process, as applied

to the simplest set of candidate models.
2. Then, at some length, we display and discuss the most
common sets of candidate models used in system identification. In general terms, a model will be a predictor of
the next output y(t) from the process, given past observations Zt1, and parameterized in terms of a finite-dimensional parameter vector :
15
y(t|
) = g(, Zt1 )
(1)
10
0
100
200
300
400
500
600
3. We then discuss the criterion of fit for general model

sets. This will have the character
VN ( ) =
y(t) y(t|
(2)
80
60
40
20
0
0
100
200
300
400
500
600
200
150
100
We also discuss how to find the best model (minimize

the criterion) and how to assess its properties.
4. We shall describe special methods for linear black-box
models. This includes frequency analysis, spectral analysis and so called subspace methods for linear statespace models.
5. We then turn to the practical issues of system identification, to assure good quality of the data by proper experiment design, how to decide upon a good model
structure, and how to deal with the data.
DISPLAYING THE BASIC IDEAS: ARX MODELS

AND THE LINEAR LEAST SQUARES METHOD
50
0
0
100
200
300
400
500
600
Figure 2. From the pulp factory at Skutskar, Sweden. The pulp

flows continuously through the plant via several buffer tanks. From
above: (a) the -number of the pulp flowing into a buffer vessel, (b)
the -number of the pulp coming out from the buffer vessel, (c) flow
out from the buffer vessel, and (d) level in the buffer vessel.
The Model
We shall generally denote the systems input and output at
time t by u(t) and y(t), respectively. Perhaps the most basic
relationship between the input and output is the linear difference equation
y(t) + a1 y(t 1) + . . . + an y(t n)
= b1 u(t 1) + . . . + bm u(t m)
field, the IFAC series of Symposia on System Identification

[Budapest, Hungary (1991), Copenhagen, Denmark (1994),
Fukuoka, Japan (1997)] is also a good source.
OUTLINE
The system identification procedure is characterized by four
basic ingredients:
1. The observed data
2. A set of candidate models
3. A criterion of fit
4. Validation
(3)
We have chosen to represent the system in discrete time, primarily since observed data are always collected by sampling.
It is thus more straightforward to relate observed data to discrete time models. Nothing prevents us, however, from working with continuous time models: we shall return to that
later.
In Eq. (3), we assume the sampling interval to be one time
unit. This is not essential but makes notation easiser.
A pragmatic and useful way to see Eq. (3) is to view it as
a way of determining the next output value given previous
observations
y(t) = a1 y(t 1) . . . an y(t n)
+ b1 u(t 1) + . . . + bm u(t m)
(4)
For more compact notation, we introduce the vectors

= [a1 , . . ., an , b1 , . . ., bm ]T
(t) = [y(t 1) . . . y(t n)u(t 1) . . . u(t m)]
(5)
T
(6)
With these, Eq. (4) can be rewritten as
265
Once the vectors (t) are defined, the solution can easily be
found by modern numerical software, such as MATLAB, see
Ref. 2.
Example 3 First-Order Difference Equation
Consider the simple model
y(t) + ay(t 1) = bu(t 1)
y(t) = T(t)
This gives us the estimate according to Eqs. (5), (6) and (13)
To emphasize that the calculation of y(t) from past data [Eq.
(4)] indeed depends on the parameters in , we shall rather
call this calculated value y(t) and write
y(t|
) = T(t)
(7)

1

a N
y2 (t 1) y(t 1)u(t 1)

=
y(t 1)u(t 1) u2 (t 1)
b N

y(t)y(t 1)

y(t)u(t 1)
The Least Squares Method

Now suppose for a given system that we do not know the values of the parameters in , but that we have recorded inputs
and outputs over a time interval 1 t N:
All sums are from t 1 to t N. A typical convention is to

take values outside the measured range to be zero. In this
case, we would thus take y(0) 0.
ZN = {u(1), y(1), . . ., u(N), y(N)}
The simple model in Eq. (3) and the well-known least

squares method in Eq. (13) form the archetype of system identification. Not only that, they also give the most commonly
used parametric identification method and are much more
versatile then perhaps perceived at first sight. In particular,
one should realize that Eq. (3) can directly be extended to
several different inputs [this just calls for a redefinition of
(t) in Eq. (6)] and that the inputs and outputs do not have
to be the raw measurements. On the contrary, it is often most
important to think over the physics of the application and
come up with suitable inputs and outputs for Eq. (3), formed
from the actual measurements.
(8)
An obvious approach is then to select in Eqs. (3) through (7)

so as to fit the calculated values y(t) as well as possible to
the measured outputs by the least squares method:
min VN (, ZN )
(9)
where
V (, ZN ) =
N
1
(y(t) y(t|
))2
N t=1
N
1
=
(y(t) T (t) )2
N t=1
(10)
We shall denote the value of that minimizes Eq. (9) by N:

N = arg min VN (, ZN )
(11)
(arg min means the minimizing argument, i.e., that value of

which minimizes VN.)
Since VN is quadratic in , we can find the minimum value
easily by setting the derivative to zero:
0=
N
d
2
VN (, ZN ) =
(t)(y(t) T (t) )
d
N t=1
(t)y(t) =
t=1
N
(t) T (t)
v(t): the voltage applied to the heater

r(t): the temperature of the liquid
y(t): the temperature of the heater coil surface
Suppose we need a model for how y(t) depends on r(t) and
v(t). Some simple considerations based on common sense and
high school physics (semiphysical modeling) reveal the following:
The change in temperature of the heater coil over one
sample is proportional to the electrical power in it (the
inflow power) minus the heat loss to the liquid.
The electrical power is proportional to v2(t).
The heat loss is proportional to y(t) r(t).
which gives
N
Example 4 An Immersion Heater

Consider a process consisting of an immersion heater immersed in a cooling liquid. We measure
(12)
t=1
This suggests the model

y(t) = y(t 1) + v2 (t 1) (y(t 1) r(t 1))
or
N =

N

t=1
1
(t) (t)
T
N

t=1
which fits into the form
(t)y(t)
(13)
y(t) + 1 y(t 1) = 2 v2 (t 1) + 3 r(t 1))
266
This is a two input (v2 and r) and one output model and corresponds to choosing
(t) = [y(t 1) v2 (t 1) r(t 1)]T
in Eq. (7).
that E N 0, since e has zero mean. The estimate is consequently unbiased. Here, E denotes mathematical expectation.
We can also form the expectation of N TN, that is the covariance matrix of the parameter error. Denote the matrix within
brackets by RN. Take expectation with respect to the white
noise e. Then, RN is a deterministic matrix, and we have
2
y(t|
)
2
Some Statistical Remarks

Model structures, such as Eq. (7) that are linear in , are
known in statistics as linear regression, and the vector (t) is
called the regression vector (its components are the regressors). Regress here alludes to the fact that we try to
calculate (or describe) y(t) by going back to (t). Models such
as Eq. (3), where the regression vector (t) contains old values
of the variable to be explained y(t) are then partly auto-regressions. For that reason, the model structure in Eq. (3) has the
standard name ARX-model (auto-regression with extra
inputs).
There is rich statistical literature on the properties of the
estimate N under varying assumptions (10). So far, we have
just viewed Eqs. (9) and (10) as curve-fitting. Later, we shall
deal with a more comprehensive statistical discussion, which
includes the ARX model as a special case. Some direct calculations will be done in the following subsection.
since the double sum collapses to RN.

We have, thus, computed the covariance matrix of the estimate N. It is determined entirely by the input properties and
the noise level. Moreover, define
R = lim
y(t) = b1 u(t 1) + . . . bm u(t m)
(14)
Suppose that the observed data really have been generated

by a similar mechanism
y(t) = b01 u(t 1) + . . . b0m u(t m) + e(t)
(15)
where e(t) is a white noise sequence with variance , but otherwise unknown. (That is, e(t) can be described as a sequence
of independent random variables with zero mean values and
variances .) Analogous to Eq. (7), we can write this as
y(t) = T (t)0 + e(t)
(16)
We can now replace y(t) in Eq. (13) by the above expression

and obtain

N =

=
N
1
(t) T (t)
t=1
N
N
Ruu (i j) = Eu(t i)u(t j)

If the matrix R is non-singular, we find that the covariance
matrix of the parameter estimate is approximately (and the
approximation improves as N )
t=1
(t) T (t)
t=1
PN =
(t) T (t)0 +
t=1
N

(t)e(t)
t=1
N = N 0 =
N

t=1
1
(t) T (t)
(20)
A number of things follow from this. All of them are typical

of the general properties to be described later
The covariance decays like 1/N, so the parameters approach the limiting value at the rate 1/ N.
The covariance is proportional to the noise-to-signal ratio. That is, it is proportional to the noise variance and
inversely proportional to the input power.
The covariance does not depend on the inputs or noises
signal shapes, only on their variance/covariance properties.
Experiment design, that is, the selection of the input u,
aims at making the matrix R1 as small as possible.
Note that the same R can be obtained for many different
signals u.
MODEL STRUCTURES I: LINEAR MODELS
Starting from Eq. (3), there is actually another, quite different, way to approach the calculation of good values of ai and
bi from observed data in Eq. (8).
Equation (3) describes a linear, discrete-time system with
transfer function
or
1
R
N
Output Error Models
(t)y(t)
1
N
(19)
This will be the covariance matrix of the input, that is, the
i j-element of R is
Model Quality and Experiment Design

Let us consider the simplest special case, that of a finite impulse response (FIR) model. That is obtained from Eq. (3) by
taking n 0:
1
R
N N
N
G(z) =
(t)e(t)
(17)
t=1
Suppose that the input u is independent of the noise e. Then,

and e are independent in this expression, so it is easy to see
b1 zn1 + b2 zn2 + . . . + bm znm

zn + a1 zn1 + . . . + an
(21)
(assuming n m)
Here, z is the z-transform variable, and one may simply
think of the transfer function G as a shorthand notation for
the difference in Eq. (3).
We shall here use the shift operator q as an alternative

for the variable z in Eq. (21). The shift operator q has the
properties
qu(t) = u(t + 1)
(22)
(just as multiplying a z-transform by z corresponds to a time

shift).
Given only an input sequence
{u(t), t = 1, . . ., N}
y(t|
) = G(q)u(t)
Noise Models and Prediction Filters

A linear, finite-dimensional dynamical system can be described by the equation
y(t) =
B(q)
u(t)
A(q)
(23)
(26)
See Eqs. (21)(22). Based on Eq. (26) we can predict the next
output from previous measurements either as in Eq. (23)
y(t|
)=
we could then calculate the output for system in Eq. (21) by

running u as input to this system
267
B(q)
u(t)
A(q)
(27)
or as in Eqs. (4) and (7):

y(t|
) = (1 A(q))y(t) + B(q)u(t)
(28)
Which one shall we choose? We can make the discussion more

general by writing for Eq. (26)
Example 5 A First-Order System

Consider the system
y(t) = G(q, )u(t)
(29)
y(t + 1) + ay(t) = bu(t)

to indicate that the transfer function depends on the (numerator and denominator) parameters [as in Eq. (5)]. We can
multiply both sides of Eq. (29) by an arbitrary stable filter
W(q, ) giving
The output according to Eq. (23) is then obtained as

y(t|
)=

b
u(t) = b
(a)k1 u(t k)
q+a
k=1
W (q, )y(t) = W (q, )G(q, )u(t)
(30)
Then, we can add y(t) to both sides of the equation and rearrange to obtain
or
y(t
+ 1| ) + ay(t|
) = bu(t)
(24)
y(t) = (1 W (q, ))y(t) + W (q, )G(q, )u(t)
(31)
We assume that the filter W starts with a 1:

Notice the essential difference between Eqs. (23) and (7).
In Eq. (7), we calculated y(t) using both past measured inputs and also past measured outputs y(t k). In Eq. (23),
y(t) is calculated from past inputs only. As soon as we use
data from a real system [that does not exactly obey Eq. (13)],
there will always be a difference between these two ways of
obtaining the computed output.
Now, we could of course still say that a reasonable estimate of is obtained by minimizing the quadratic fit,
N = arg min
N
1
[ y(t) y(t|
)]2
N t=1
(25)
even when y(t) is computed according to Eq. (23). Such an

estimate is often called an output-error estimate since we
have formed the fit between a purely simulated output and
the measured output. Note that y(t) according to Eq. (23) is
not linear in , so the function to be minimized in Eq. (25) is
not quadratic in . Hence, some numerical search schemes
have to be applied in order to find N in Eq. (25). Most often
in practice, a GaussNewton iterative minimization procedure is used.
It follows from the discussion that the estimate obtained
by Eq. (25) will in general differ from the one from Eq. (9).
What is the essential difference? To answer that question, we
will have to discuss various ways of perceiving and describing
the disturbances that act on the system.
W (q, ) = 1 + w1 q1 + w2 q1 + w2 q2 + . . .
so that 1 W(q, ) actually contains a delay. We thus obtain
the predictor
y(t|
) = (1 W (q, ))y(t) + W (q, )G(q, )u(t)
(32)
Note that this formulation is now similar to that of Eq. (28).

We see that the method used in Eq. (27) corresponds to the
choice W(q, ) 1, while the procedure in Eq. (28) is obtained
for W(q, ) A(q).
Now, does the predictor in Eq. (32) depend on the filter
W(q, )? Well, if the inputoutput data are exactly described
by Eq. (29) and we know all relevant initial conditions, the
predictor in Eq. (32) produces identical predictions y(t), regardless of the choice of stable filters W(q, ).
To bring out the relevant differences, we must accept the
fact that there will always be disturbances and noise that affect the system, so instead of Eq. (29), we have a true system
that relates the inputs and outputs by
y(t) = G0 (q)u(t) + v(t)
for some disturbance sequence v(t). So Eq. (32) becomes
y(t|
) = {(1 W (q, ))G0 (q) + W (q, )G(q, )}u(t)

+ (1 W (q, ))v(t)
(33)
268
Now, assume that there exists a value 0, such that

G(q, 0) G0(q). Then the error of the above prediction becomes
(t, ) = y(t) y(t|
0 ) = W (q, )v(t)
(34)
To make this error as small as possible, we must thus match

the choice of the filter W(q, 0) to the properties of the noise
v(t). Suppose v(t) can be described as filtered white noise
v(t) = H0 (q)e(t)
1. In order to distinguish between different predictors, one

has to introduce descriptions of the disturbances that
act on the process.
2. If the inputoutput description is assumed to be
(36)
where e(t) is a white noise source, then the natural

predictor of y(t) given previous observations of inputs
and outputs will be
y(t|
) = [1 H 1 (q)] y(t) + H 1 (q)G(q)u(t)
(37)
This predictor gives the smallest possible error if

e(t) indeed is white noise.
3. Since the dynamics G(q) and the noise model H(q) are
typically unknown, we will have to work with a parameterized description
y(t) = G(q, )u(t) + H(q, )e(t)
(38)
The corresponding predictor is then obtained from Eq.

(37):
y(t|
) = [I H 1 (q, )] y(t) + H 1 (q, )G(q, )u(t)
(39)
We may now return to the question we posed earlier. What

is the practical difference between minimizing Eqs. (10) and
(25)? Comparing Eq. (23) with Eq. (29), we see that this predictor corresponds to the assumption that H 1, i.e., that
white measurement noise is added to the output. This also
means that minimizing the corresponding prediction error,
Eq. (25), will give a clearly better estimate, if this assumption
is more or less correct.
Linear Black-Box Model Parameterization
The model parameterization in Eq. (38) contains a large number of much-used special cases. We have already seen that
the ARX model in Eq. (3) corresponds to
G(q, ) =
1
B(q)
, H(q, ) =
A(q)
A(q)
G(q, ) =
C(q)
B(q)
, H(q, ) =
A(q)
A(q)
(41)
(35)
where e(t) is a sequence of independent random variables.

Here, we assume H0(q) to be normalized, so that H0(q) 1
h1q1 . . . . Then, it is easy to see from Eq. (34) that no filter W(q 0) can do better than 1/H0(q), since this makes the
prediction error (t, 0) equal to the white noise source e(t).
All this leads to the following summarizing conclusion
(which is the only thing one needs to understand from this
section).
y(t) = G(q)u(t) + H(q)e(t)
That is, we assume the system (plant) dynamics and the noise
model to have common poles and no numerator dynamics for
the noise. Its main feature is that the predictor y(t) will be
linear in the parameters according to Eq. (11) or Eq. (7).
We can make Eq. (40) more general by allowing also numerator dynamics. We then obtain the parameterization
(40)
The effect of the numerator C is that the current predicted

value of y will depend upon previous predicted values, not
just measured values. This is known as an ARMAX model,
since the C(q)-term makes the noise model a moving average
of a white noise source. Also, Eq. (41) assumes that the dynamics and the noise model have common poles, and is, therefore, particularly suited for the case where the disturbances
enter together with the input, early in the process, so to
speak.
The output error (OE) model we considered in Eq. (23) corresponds to the case
G(q, ) =
B(q)
, H(q, ) = 1
F (q)
(42)
[We use F in the denominator to distinguish the case from

Eq. (40).] Its unique feature is that the prediction is based on
past inputs only. It also concentrates on the model dynamics
and does not bother about describing the noise.
We can also generalize this model by allowing a general
noise model
G(q, ) =
C(q)
B(q)
, H(q, ) =
F(q)
D(q)
(43)
This particular model parameterization is known as the BoxJenkins (BJ) model, since it as suggested in the well-known
book by Box and Jenkins (6).
It differs from the ARMAX-model in Eq. (42) in that it assigns different dynamics (poles) to the noise characteristics
from the inputoutput properties. It is thus better suited for
cases where the noise enters late in the process, such as
measurement noise. See Fig. 3.
One might wonder why we need all these different model
parameterizations. As has been mentioned in the text, each
has its advantages which can be summarized as follows
ARX. Gives a linear regression, very simple to estimate
ARMAX. Gives reasonable flexibility to the noise description, assumes that noise enters like the inputs
OE. Concentrates on the inputoutput dynamics
BJ. Very flexible, assumes no common characteristics between noise and inputoutput behavior.
Physically Parameterized Linear Models
So far, we have treated the parameters only as vehicles to
give reasonable flexibility to the transfer functions in the general linear model in Eq. (38). This model can also be arrived
at from other considerations.
If we select the state variables
e
u
y(t)
y(t)
x(t) =
ARX
we obtain the state space form
e
u
B
F
0 1
x =
x+
0 a
y = (1 0) x + v
OE
269
0
u
b
(45)
where v denotes disturbances and noise. In this case, we thus

have
C
D
B
F
0
0
1
a

B( ) =
0
b
(46)
C = (1 0)
The parameterization reflects our insight that the system contains an integration, but is in this case not directly derived
from detailed physical modeling. Basic physical laws would,
in this case, have given us how depends on physical constants, such as resistance of the wiring, amount of inertia,
friction coefficients, and magnetic field constants.
A( ) =
ARMAX
a
b
Now, how do we fit a continuous-time model in Eq. (44a)

to sampled observed data? If the input u(t) has been piecewise
constant over the sampling interval
BJ
Figure 3. Linear black-box model structures.
u(t) = u(kT ) kT t < (k + 1)T

then the states, inputs, and outputs at the sampling instants
will be represented by the discrete time model
Consider a continuous time state space model

x(t)
= A( )x(t) + B( )u(t)
(44a)
y(t) = C( )x(t) + v(t)
(44b)
x((k + 1)T ) = A( )x(kT ) + B( )u(kT )

y(kT ) = C( )x(kT ) + v(kT )
(47)
where
Here, x(t) is the state vector and typically consists of physical
variables (such as positions and velocities, etc). The state
space matrices A, B, and C are parameterized by the parameter vector , reflecting the physical insight we have into the
process. The parameters could be physical constants (resistance, heat transfer coefficients, aerodynamical derivatives,
etc.) whose values are not known. They could also reflect
other types of insights into the systems properties.
Example 6 An Electric Motor
Consider an electric motor with the input u being the applied
voltage and the output y being the angular position of the
motor shaft.
A first but reasonable approximation of the motors dynamics is as a first-order system from voltage to angular velocity,
followed by an integrator:
G(s) =
b
s(s + a)
A( ) = e
A( )T

,
B( ) =
eA( ) B( ) d
(48)
If the input is not piecewise constant, other sampling rules

than Eqs. (47) and (48) will apply. This follows from solving
Eq. (44) over one sampling period. We could also further
model the added noise term v(kT) and represent the system
in the innovations form
x((k + 1)T ) = A( )x(kT ) + B( )u(kT ) + K( )e(kT )

y(kT ) = C( )x(kT ) + e(kT )
(49)
where e(kT) is white noise. The step from Eqs. (47) to (49)
is really a standard Kalman filter step: x will be the one-step
ahead predicted Kalman states. A pragmatic way to think
about it is as follows: In Eq. (47), the term v(kT) may not be
white noise. If it is colored, we may separate out that part of
v(kT) that cannot be predicted from past values. Denote this
part by e(kT): it will be the innovation. The other part of
270
v(kT), the one that can be predicted, can then be described as

a combination of earlier innovations, e(T), k. Its effect
on y(kT) can then be described via the states by changing
them from x to x, where x contains additional states associated with getting v(kT) from e(T), k .
Now, Eq. (49) can be written in inputoutput form as (let
T 1)
y(t) = G(q, )u(t) + H(q, )e(t)
(50)
where
(t) = (Z t1 )
Let the dimension of be d. As before, we shall call this vector the regression vector, and its components will be referred
to as the regressors. We also allow the more general case that
the formation of the regressors is itself parameterized:
(t) = (Z t1 , )
with
G(q, ) = C( )(qI A( ))1 B( )

H(q, ) = I + C( )(qI A( ))1 K( )
(51)
We are, thus, back at the basic linear model in Eq. (38). The
parameterization of G and H in terms of is, however, more
complicated than the ones we discussed earlier.
The general estimation techniques, model properties [including the characterization in Eq. (85)], algorithms, etc.
apply exactly as described in the section on general parameter estimation techniques.
From these examples, it is also quite clear that non-linear
models with unknown parameters can be approached in the
same way. We would then typically arrive at a structure
x(t)
= f (x(t), u(t), )
y(t) = h(x(t), u(t), ) + v(t)
(52)
In this model, all noise effects are collected as additive output

disturbances v(t) which is a restriction, but also a very helpful
simplification. If we define y(t) as the simulated output response to Eq. (52), for a given input, ignoring the noise v(t),
everything to be said about parameter estimation, model
properties, and so on is still applicable.
MODEL STRUCTURES II: NONLINEAR BLACK-BOX MODELS
In this section, we shall describe the basic ideas behind model
structures that have the capability to cover any nonlinear
mapping from past data to the predicted value of y(t). Recall
that we defined a general model structure as a parameterized
mapping in Eq. (1):
y(t|
) = g(, Z t1 )
(56)
which we for short write (t, ). For simplicity, the extra argument will, however, be used explicitly only when essential for the discussion.
The choice of the nonlinear mapping in Eq. (53) has thus
been reduced to two partial problems for dynamical systems:
1. how to choose the nonlinear mapping g() from the regressor space to the output space (i.e., from Rd to Rp)
2. how to choose the regressors (t) from past inputs and
outputs
The second problem is the same for all dynamical systems,
and it turns out that the most useful choices of regression
vectors are to let them contain past inputs and outputs, and
possibly also past predicted/simulated outputs. The regression vector will thus be of the character in Eq. (6). We now
turn to the first problem.
Nonlinear Mappings: Possibilities
Now, let us turn to the nonlinear mapping
g(, )
(57)
which for any given maps from Rd to Rp. For most of the
discussion, we will use p 1; that is, the output is scalarvalued. At this point, it does not matter how the regression
vector (1, . . ., d)T was constructed. It is just a vector
that lives in Rd.
It is natural to think of the parameterized function family
as function expansions:
g(, ) =
k gk ()
(58)
(53)
We shall consequently allow quite general nonlinear mappings g. This section will deal with some general principles
for how to construct such mappings, and will cover artificial
neural networks as a special case. See Refs. 11 and 12 for
recent and more comprehensive surveys.
Nonlinear Black-Box Structures
Now, the model structure family in Eq. (53) is really too general, and it turns out to be useful to write g as a concatenation
of two mappings: one that takes the increasing number of
past observations Zt1 and maps them into a finite dimensional vector (t) of fixed dimension and one that takes this
vector to the space of the outputs:
y(t|
) = g(, Z t1 ) = g((t), )
(55)
(54)
We refer to gk as basis functions, since the role they play in

Eq. (58) is similar to that of a functional space basis. In some
particular situations, they do constitute a functional basis.
Typical examples are wavelet bases (see below).
We are going to show that expansion in Eq. (58), with different basis functions, plays the role of a unified framework
for investigating most known nonlinear black-box model
structures.
Now, the key question is: How to choose the basis functions gk? The following facts are essential to understand the
connections between most known nonlinear black-box model
structures:
All the gk are formed from one mother basis function,
that we generically denote by (x).
This function (x) is a function of a scalar variable x.
Typically, gk are dilated (scaled) and translated versions

of . For the scalar case d 1, we may write
gk () = gk (, k , k ) = (k ( k ))
(59)
We, thus, use k to denote the dilation parameters and

k to denote translation parameters.
A Scalar Example: Fourier Series Take (x) cos(x).
Then Eqs. (58) and (59) will be the Fourier series expansion,
with k as the frequencies and k as the phases.
Another Scalar Example: Piecewise Constant Functions Take as the unit interval indicator function:

1 for 0 x < 1
(x) =
(60)
0 else
and take, for example, k k, k 1/, and k f(k). Then
Eqs. (58) and (59) give a piecewise constant approximation of
any function f. Clearly, we would have obtained a quite similar result by a smooth version of the indicator function, for
example the Gaussian bell:
2
1
(x) = ex /2
2
(61)
A Variant of the Piecewise Constant Case Take to be

the unit step function

0 for x < 0
(x) =
(62)
1 for x 0
We, then, just have a variant of Eq. (60), since the indicator
function can be obtained as the difference of two steps. A
smooth version of the step, like the sigmoid function
(x) = (x) =
1
1 + ex
(63)
will, of course, give quite similar results.

Classification of Single-Variable Basis Functions. Two classes
of single-variable basis functions can be distinguished depending on their nature:
Local basis functions are functions having their gradient
with bounded support, or at least vanishing rapidly at
infinity. Loosely speaking, their variations are concentrated to some interval.
Global basis functions are functions having a nonvanishing gradient for all arguments.
Clearly, the Fourier series is an example of a global basis
function, while Eqs. (60), (61), (62) and (63) are all local functions.
Construction of Multivariable Basis Functions. In multidimensional case (d 1), gk are multivariable functions. In practice,
they are often constructed from the single-variable function
in some simple manner. Let us recall the three most often
271
used methods for constructing multivariable basis functions

from single-variable basis functions.
1. Tensor Product. Given d single-variable functions of the
different components j of a d-dimensional vector ,
h1(1), . . ., hd(d) (identical or not), the tensor product
construction of the corresponding function from Rd is
then given by their product. In the present case, this
means that the basis functions are constructed from the
scalar function as
gk () =
d

(kj ( j kj ))
(64)
j=1
2. Radial Construction. For any single-variable function

, the radial construction of multivariable basis function of Rd has the form
gk () = gk f (, k , k ) = ( k
k )
(65)
where k denotes any chosen norm on the space of the

regression vector . The norm could typically be a quadratic norm
2
k
= T k
(66)
with k as a possibly k-dependent positive definite matrix of dilation (scale) parameters. In simple cases, k
may be just scaled versions of the identity matrix.
3. Ridge Construction. Let be any single-variable function. Then for all k Rd, k R, a ridge function is
given by
gk () = gk (, k , k ) = (kT + k ), Rd
(67)
The ridge function is thus constant for all in the subspace Rd : Tk constant. As a consequence, even
if the mother basis function has local support, the basis functions gk will have unbounded support in this
subspace. The resulting basis could be said to be semiglobal, but the term ridge function is more precise.
Approximation Issues. For any of the described choices, the
resulting model becomes
g(, ) =
en
k (k ( k ))
(68)
k=1
with the different exact interpretations of the argument

k( k) just discussed. The expansion is entirely determined by
the scalar valued function (x) of a scalar variable x
the way the basis functions are expanded to depend on a
vector
The parameterization in terms of can be characterized by
three types of parameters:
the coordinates
the scale or dilation parameters
the location parameters
272
A key issue is how well the function expansion is capable of

approximating any possible true system g0(). There is
rather extensive literature on this subject. For an identification oriented survey, see, for example, Ref. 12.
The bottom line is easy: For almost any choice of (x)except being a polynomialthe expansion in Eq. (68) can approximate any reasonable function g0() arbitrarily well for
sufficiently large n.
It is not difficult to understand this. It is sufficient to check
that the delta functionor the indicator function for arbitrarily small areascan be arbitrarily well approximated
within the expansion. Then, clearly, all reasonable functions
can also be approximated. For local with radial construction, this is immediate: By scaling and location, an arbitrarily
small indicator function can be placed anywhere. For the
ridge construction, one needs to show that a number of hyperplanes defined by and can be placed and intersect so that
any small area in Rd is cut out.
The question of how efficient the expansion is, that is, how
large n is required to achieve a certain degree of approximation is more difficult and has no general answer. We may
point to the following aspects:
If the scale and location parameters and are allowed
to depend on the function g0 to be approximated, then
the number of terms n required for a certain degree of
approximation is much less than if k, k; k 1, . . . is
an a priori fixed sequence.
For the local, radial approach, the number of terms required to achieve a certain degree of approximation of
a p times differentiable function is proportional to
n
1
(d/ p)
(69)
It thus increases exponentially with the number of regressors. This is often referred to as the curse of dimensionality.
Connection to Named Structures. Here we briefly review
some popular structures; other structures related to interpolation techniques are discussed in Refs. 11 and 12.
Wavelets. The local approach corresponding to Eqs. (58,
and 65) has direct connections to wavelet networks and
wavelet transforms. The exact relationships are discussed in Ref. 11. Loosely, we note that via the dilation
parameters in k, we can work with different scales simultaneously to pick up both local and not-so-local variations. With appropriate translations and dilations of a
single suitably chosen function (the mother wavelet), we can make expansion Eq. (58) orthonormal. This
is discussed extensively in Ref. 12.
Wavelet and Radial Basis Networks. The choice in Eq. (61)
without any orthogonalization is found in both wavelet
networks (13) and radial basis neural networks (14).
Neural Networks. The ridge choice in Eq. (67) with given
by Eq. (63) gives a much-used neural network structure
through the one hidden layer feedforward sigmoidal net.
Hinging Hyperplanes. If instead of using the sigmoid
function we choose V-shaped functions (in the form of
a higher-dimensional open book), Briemans hinging
hyperplane structure is obtained (15).

Nearest Neighbors or Interpolation. By selecting as in Eq.
(60) and the location and scale vector k, k in structure
Eq. (65), such that exactly one observation falls into
each cube, the nearest neighbor model is obtained:
just load the input-output record into a table, and for a
given , pick the pair (y, ) for closest to the given ;
y is the desired output estimate. If one replaces Eq. (60)
by a smoother function and allows some overlapping of
the basis functions, we get interpolation-type techniques such as kernel estimators.
Fuzzy Models. Fuzzy models based on fuzzy set membership belong to the model structures of class Eq. (58).
The basis functions gk then are constructed from the
fuzzy set membership functions and the inference rules
using the tensor approach Eq. (64). The exact relationship is described in Ref. 11.
Estimating Nonlinear Black-Box Models
The model structure is determined by the following choices:
the regression vector (typically built up from past inputs
and outputs)
the basic function
the number of elements (nodes) in the expansion Eq.
(58).
Once these choices have been made, y(t) g((t), ) is a well
defined function of past data and the parameters . The parameters are made up of coordinates in expansion Eq. (58)
and from location and scale parameters in the different basis
functions.
All the algorithms and analytical results of the next section can thus be applied. For neural network applications,
these are also the typical estimation algorithms used, often
complemented with regularization, which means that a term
is added to the criterion Eq. (74) that penalizes the norm of .
This will reduce the variance of the model, in that spurious
parameters are not allowed to take on large, and mostly random, values (11).
For wavelet applications, it is common to distinguish between those parameters that enter linearly in y(t) (i.e., the
coordinates in the function expansion) and those that enter
non-linearly (i.e., the location and scale parameters). Often,
the latter are seeded to fixed values, and the coordinates are
estimated by the linear least squares method. Basis functions
that give a small contribution to the fit (corresponding to nonuseful values of the scale and location parameters) can then
be trimmed away (pruning or shrinking).
GENERAL PARAMETER ESTIMATION TECHNIQUES
In this section, we shall deal with issues that are independent
of model structure. Principles and algorithms for fitting models to data, as well as the general properties of the estimated
models, are all model-structure independent and equally well
applicable to, say, ARMAX models and neural network
models.
The section is organized as follows. The general principles
for parameter estimation are outlined. Then, we deal with the
asymptotic (in the number of observed data) properties of the

models, while algorithms are described in the last section.
Fitting Models to Data
Earlier, we showed several ways to parameterize descriptions
of dynamical systems. This is actually the key problem in system identification. No matter how the problem is approached,
the bottom line is that such a model parameterization leads
to a predictor
y(t|
) = g(, Z t1 )
(70)
that depends on the unknown parameter vector and past

Zt1 [see Eq. (8)]. This predictor can be linear in y and u. This,
in turn, contains several special cases both in terms of blackbox models and physically parameterized ones, as was discussed earlier. The predictor could also be of general, nonlinear nature, which was also discussed.
In any case, we now need a method to determine a good
value of , based on the information in an observed, sampled
data set in Eq. (8). It suggests itself that the basic leastsquares like approach in Eqs. (9) through (11) still is a natural approach, even when the predictor y(t) is a more general
function of .
A procedure with some more degrees of freedom is the following one:
choose
L(q) = 1
t = 1, 2, . . ., N
(71)
(72)
so as to enhance or depress interesting or unimportant

frequency bands in the signals.
3. Choose a scalar valued, positive function ( ) so as to
measure the size or norm of the prediction error:
(F (t, ))
(73)
4. Minimize the sum of these norms:

N = arg min VN (, ZN )
(74)
An essential question is, of course, what properties will the

estimate resulting from Eq. (74) have. These will naturally
depend on the properties of the data record ZN defined by Eq.
(8). It is, in general, a difficult problem to characterize the
quality of N exactly. One normally has to be content with the
asymptotic properties of N as the number of data, N, tends
to infinity.
It is an important aspect of the general identification
method in Eq. (74) that the asymptotic properties of the resulting estimate can be expressed in general terms for arbitrary model parameterizations.
The first basic result is the following one:
N
as N
= arg min E(F (t, ))
This procedure is natural and pragmatic; we can still think of

it as curve-fitting between y(t) and y(t). It also has several
statistical and information theoretic interpretations. Most importantly, if the noise source in the system [like in Eq. (38)] is
supposed to be a sequence of independent random variables
e(t) each having a probability density function f e(x), then Eq.
(74) becomes the maximum likelihood estimate (MLE) if we
(78)
That is, as more and more data become available, the estimate converges to that value * that would minimize the expected value of the norm of the filtered prediction errors.
This is in a sense the best possible approximation of the true
system that is available within the model structure. The expectation E in Eq. (78) is taken with respect to all random
disturbances that affect the data, and it also includes averaging over the input properties. This means, in particular, that
* will make y(t*) a good approximation of y(t) with respect
to those aspects of the system that are enhanced by the input
signal used.
The second basic result is the following one. If (t, *) is
approximately white noise, then the covariance matrix of N
is approximately given by
E(N )(N )T
(75)
(77)
where
where
N
1
VN (, ZN ) =
(F (t, ))
N t=1
(76)
Model Quality
2. Possibly filter the prediction errors through a linear filter L(q),

F (t, ) = L(q)(t, )
and () = log f e ()
The MLE has several nice statistical features and, thus, gives
a strong moral support for using the outlined method. Another pleasing aspect is that the method is independent of the
particular model parameterization used (although this will affect the actual minimization procedure). For example, the
method of back propagation often used in connection with
neural network parameterizations amounts to computing N
in Eq. (74) by a recursive gradient method. We shall deal with
these aspects later.
1. From observed data and the predictor y(t), form the

sequence of prediction errors,
(t, ) = y(t) y(t|
),
273
[E (t) T (t)]1
N
(79)
where
= E 2 (t, )
(t) =
d
y(t|
)| =
d
(80)
(81)
Think of as the sensitivity derivative of the predictor with

respect to the parameters. Then Eq. (79) says that the covariance matrix for N is proportional to the inverse of the covari-
274
ance matrix of this sensitivity derivative. This is a quite natural result.

Note that for all these results, the expectation operator E
can, under most general conditions, be replaced by the limit
of the sample mean; that is,
E (t) T (t) lim
N
1
(t) T (t)
N t=1
Characterization of the Limiting Model in a General Class of

Linear Models.. Let us apply the general limit result in Eqs.
(77)(78) to the linear model structure in Eq. (38). If we
choose a quadratic criterion () 2 (in the scalar output
case), then this result tells us, in the time domain, that the
limiting parameter estimate is the one that minimizes the
filtered prediction error variance (for the input used during
the experiment.) Suppose that the data actually have been
generated by
(83)
Let u() be the input spectrum and v() be the spectrum

for the additive disturbance v. Then, the filtered prediction
error can be written
L(q)
[ y(t) G(q, )u(t)]
H(q, )
L(q)
=
[(G0 (q) G(q, ))u(t) + v(t)]
H(q, )

+
G0 (ei ) G(ei , ) 2
If the noise model H(q, ) H*(q) does not depend on [as

in the output error model Eq. (42)], the expression in Eq. (85)
thus shows that the resulting model G(ei, *) will give that
frequency function in the model set that is closest to the true
one, in a quadratic frequency norm with weighting function
Q() = u () L(ei ) 2 / H (ei )
(87)
FN = EV (N )
(88)
In general, the measure FN depends on a number of things:
The
The
The
The
model structure used

number of data points N
data properties for which the fit V is defined
properties of the data used to estimate N
FN V N ( ) 1 +
u () L(ei ) 2
d
H(ei , ) 2

(85)
v () L(ei ) 2 / H(ei , ) 2 d
= arg min
Here, expectation E is over the data properties [i.e., expectation over Z with the notation Eq. (8)]. Recall that expectation also can be interpreted as sample means as in Eq. (82).
Before we continue, let us note the very important aspect
that the fit V will depend not only on the model and the true
system but also on data properties, like input spectra, possible
feedback, etc. We shall say that the fit depends on the experimental conditions.
The estimated model parameter N is a random variable
because it is constructed from observed data that can be described as random variables. To evaluate the model fit, we
then take the expectation of V( N) with respect to the estimation data. That gives our measure
(84)
By Parsevals relation, the prediction error variance can also

be written as an integral over the spectrum of the prediction
error. This spectrum, in turn, is directly obtained from Eq.
(84), so the limit estimate * in Eq. (78) can also be defined
as

V ( ) = E y(t) y(t|
The rather remarkable fact is that if the two last data properties coincide, then, asymptotically in N (4, Chap. 16),
F (t, ) =
Some quite general expressions for the expected model fit,

that are independent of the model structure, can also be developed.
Let us measure the (average) fit between any model in Eq.
(70) and the true system as
(82)
The results in Eqs. (77) through (81) are general and hold for
all model structures, both linear and nonlinear ones, subject
only to some regularity and smoothness conditions. They are
also fairly natural and will give the guidelines for all user
choices involved in the process of identification. See Ref. 4 for
more details around this.
y(t) = G0 (q)u(t) + v(t)
Measures of Model Fit
(86)
This shows clearly that the fit can be affected by the choice of
prefilter L, the input spectrum u, and the noise model H .
*
dim
N

(89)
Here, * is the value that minimizes the expected criterion in

Eq. (78). The notation dim means the number of estimated
parameters. The result also assumes that the criterion function () 2, and that the model structure is successful in
the sense that F(t) is approximately white noise.
Despite the reservations about the formal validity of Eq.
(89), it carries a most important conceptual message: If a
model is evaluated on a data set with the same properties as
the estimation data, then the fit will not depend on the data
properties, and it will depend on the model structure only in
terms of the number of parameters used and of the best fit
offered within the structure.
The expression can be rewritten as follows: Let y0(tt 1)
denote the true one step ahead prediction of y(t), and let
W ( ) = E y0 (t|t 1) y(t|
(90)
and let
= E y(t) y0 (t|t 1)
(91)
Then is the innovations variance, that is, that part of y(t)

that cannot be predicted from the past. Moreover, W(*) is the
bias error, that is, the discrepancy between the true predictor
and the best one available in the model structure. Under the
same assumptions as above, Eq. (89) can be rewritten as
FN + W ( ) +
dim
N
(92)
is the unavoidable error, stemming from the fact that

the output cannot be exactly predicted, even with perfect
system knowledge.
W(*) is the bias error. It depends on the model structure
and on the experimental conditions. It will typically decrease as dim increases.
The last term is the variance error. It is proportional to
the number of estimated parameters and inversely proportional to the number of data points. It does not depend on the particular model structure or the experimental conditions.
Algorithmic Aspects
In this section, we shall discuss how to achieve the best fit
between observed data and the model, that is, how to carry
out the minimization of Eq. (74). For simplicity, we here assume a quadratic criterion and set the prefilter L to unity:
N
1
VN ( ) =
y(t) y(t
)
2N t=1
(93)
(i+1) = (i) i R1
i gi
(94)
Here, (i) is the parameter estimate after iteration number i.

The search scheme is thus made up of the three entities:
i step size
gi an estimate of the gradient VN( (i))
Ri a matrix that modifies the search direction
(95)
where
(t, ) =
y(t|
R( ) = VN ( ) =
N
d 2VN ( )
1
=
(t, ) T (t, )
d 2
N t=1
N
1
2
+
( y(t) y(t|
)) 2 y(t|
)
N t=1
(98)
The true Newton direction will, thus, require that the second
derivative
2
y(t|
)
2
be computed. Also, far from the minimum, R() need not be
positive semidefinite. Therefore, alternative search directions
are more common in practice:
Gradient Direction. Simply take
(99)
GaussNewton Direction. Use
Ri = Hi =
N
1
(t, (i) ) T (t, (i) )
N t=1
(100)
LevenbergMarquard Direction. Use

Ri = Hi + I
(101)
where Hi is defined by Eq. (100).

Conjugate Gradient Direction. Construct the Newton direction from a sequence of gradient estimates. Loosely,
think of V N as constructed by difference approximation
of d gradients. The direction in Eq. (97) is, however, constructed directly, without explicitly forming and inverting V .
It is generally considered (16) that the GaussNewton search
direction is to be preferred. For ill-conditioned problems, the
LevenbergMarquard modification is recommended.
Search Directions. The basis for the local search is the gradient
N
1
dVN ( )
=
( y(t) y(t|
)) (t, )
d
N t=1
(97)
where
Ri = I
2
No analytic solution to this problem is possible unless the

model y(t) is linear in , so the minimization has to be done
by some numerical search procedure. A classical treatment of
the problem of how to minimize the sum of squares is given
in Ref. 16.
Most efficient search routines are based on iterative local
search in a down-hill direction from the current point. We
then have an iterative scheme of the following kind:
VN ( ) =
The gradient is, in the general case, a matrix with dim

rows and dim y columns. It is well known that gradient
search for the minimum is inefficient, especially close to the
minimum. Then, it is optimal to use the Newton search direction
R1 ( )VN ( )
The three terms constituting the model error then have the
following interpretations:
275
(96)
Local Minima. All properties of the estimate that we have

discussed really refer to the global minimum of the criterion.
A fundamental problem with minimization tasks like Eq. (9)
is that VN() may have several or many local (non-global) minima, where local search algorithms may get caught. There is
no easy solution to this problem. It is usually well worth the
effort to find a good initial value (0) where to start the iterations. Other than that, only various global search strategies
are left, such as random search, random restarts, simulated
annealing, and the genetic algorithm.
276
SPECIAL ESTIMATION TECHNIQUES

FOR LINEAR BLACK-BOX MODELS
An important feature of a linear, time invariant system is
that it is entirely characterized by its impulse response. So if
we know the systems response to an impulse, we will also
know its response to any input. Equivalently, we could study
the frequency response, which is the Fourier transform of the
impulse response.
In this section, we shall consider estimation methods for
linear systems that do not use particular model parameterizations. First, we shall consider direct methods to determine
the impulse response and the frequency respone by simply
applying the definitions of these concepts.
Then, spectral analysis for frequency function estimation
will be discussed. Finally, a recent method to estimate general linear systems (of given order, by unspecified structure)
will be described.
1. The variables affected by the input in question. This

makes it easier to draw block diagrams for the system
and to decide which influences can be neglected.
2. The time constants of the system. This also allows us to
decide which relationships in the model can be described as static (that is, they have significantly faster
time constants than the time scale we are working with.
3. The characteristic (oscillatory, poorly damped, monotone, and the like) of the step responses, as well as the
levels of static gains. Such information is useful when
studying the behavior of the final model in simulation.
Good agreement with the measured step responses
should give a certain confidence in the model.
Frequency Analysis. If a linear system has the transfer
function G(q), and the input is
(102)
then the output after possible transients have faded away will
be
for t = T, 2T, 3T, . . .
Definitions. The cross spectrum between two (stationary

signals u(t) and y(t) is defined as the Fourier transform of
their cross covariance function, provided this exists:
Ryu ( )ei
(106)
Transient Analysis. The first step in modeling is to decide

which quantities and variables are important to describe
what happens in the system. A simple and common kind of
experiment that shows how and in what time span various
variables affect each other is called step-response analysis or
transient analysis. In such experiments, the inputs are varied
(typically one at a time) as a step: u(t) u0, t t0; u(t) u1,
t t0. The other measurable variables in the system are recorded during this time. We, thus, study the step response
of the system. An alternative would be to study the impulse
response of the system by letting the input be a pulse of short
duration. From such measurements, information of the following nature can be found:
y(t) = y0 cos(t + ),
Estimating the Frequency Response by Spectral Analysis
yu () =
Transient and Frequency Analysis
u(t) = u0 cos kT, (k 1)T t kT
If the system is driven by the input [see Eq. (102)] for a certain u0 and 1 and we measure y0 and from the output signal, it is possible to determine the complex number G(ei1T)
using Eqs. (104)(105). By repeating this procedure for a
number of different , we can get a good estimate of the frequency function G(eiT). This method is called frequency analysis. Sometimes, it is possible to see or measure u0, y0, and
directly from graphs of the input and output signals. Most of
the time, however, there will be noise and irregularities that
make it difficult to determine directly. A suitable procedure
is then to correlate the output with cos t and sin t.
where Ryu() is defined by

Ryu ( ) = Ey(t)u(t )
(107)
The (auto) spectrum u() of a signal u is defined as uu(),

that is, as its cross spectrum with itself.
The spectrum describes the frequency contents of the signal. The connection to more explicit Fourier techniques is evident by the following relationship
u () = lim
1
U ()
N N
(108)
where UN is the discrete time Fourier transform
UN () =
N
u(t)eit
(109)
t=1
The relationship in Eq. (108) is shown, for example, in Ref. 3.

Consider now the general linear model:
y(t) = G(q)u(t) + v(t)
(110)
It is straightforward to show that the relationships between

the spectra and cross spectra of y and u (provided u and v are
uncorrelated) is given by
yu () = G(ei )u ()
y () = G(ei )
u () + v ()
(111)
(112)
It is easy to see how the transfer function G(ei) and the noise
spectrum v() can be estimated using these expressions, if
only we have a method to estimate cross spectra.
(103)
Estimation of Spectra. The spectrum is defined as the Fourier transform of the correlation function. A natural idea
would then be to take the transform of the estimate
y0 = G(eiT ) u0
(104)
= arg G(eiT )
(105)
N
1
R N
y(t)u(t )
yu ( ) =
N t=1
where
(113)
That will not work in most cases, though. The reason could
N
yu
be described as follows: The estimate R
() is not reliable for
large since it is based on only a few observations. These
bad estimates are mixed with good ones in the Fourier
transform, thus creating an overall bad estimate. It is better
to introduce a weighting, so that correlation estimates for
large lags carry a smaller weight:
N

yu () =
i
R N
yu () w ()e
(114)
about
k <
(115)
From the spectral estimates u, y, and yu obtained in this

way, we can now use Eq. (111) to obtain a natural estimate
of the frequency function G(ei):
G N (ei ) =
N

yu ()
N

u ()
(116)
Furthermore, the disturbance spectrum can be estimated

from Eq. (112) as
N
N

v () = y ()
2
N

yu ()
N

u ()
(117)
To compute these estimates, the following steps are performed:

1. Collect data y(k), u(k), k 1, . . ., N.
2. Subtract the corresponding sample means from the
data. This will avoid bad estimates at very low frequencies.
3. Choose the width of the lag window w(k).
N
yN(k), R
uN(k), and R
yu
4. Compute R
(k) for k according to
Eq. (113).
N
yN(),
uN(), and
yu
5. Form the spectral estimates
() according to Eq. (114) and analogous expressions.
6. Form Eq. (116) and possibly also Eq. (117).
N and
wN are
Quality of the Estimates. The estimates G
formed entirely from estimates of spectra and cross spectra.
Their properties will, therefore, be inherited from the properties of the spectral estimates. For the Hamming window with
width , it can be shown that the frequency resolution will be
(118)
v ()
N u ()
(119)
N
2v ()
Var
v () 0.7
N
(120)
Var G N (i) 0.7
1
k
1 + cos
2
w (k) = 0
w (k) =
radians/time unit
This means that details in the true frequency function that

are finer than this expression will be smeared out in the estimate. It is also possible to show that the estimates variances
satisfy
=
This spectral estimation method is known as the Blackman

Tukey approach. Here, w() is a window function that decreases with . This function controls the trade-off between
frequency resolution and variance of the estimate. A function
that gives significant weights to the correlation at large lags
will be able to provide finer frequency details (a longer time
span is covered). At the same time, it will have to use bad
estimates, so the statistical quality (the variance) is poorer.
We shall return to this trade-off in a moment. How should we
choose the shape of the window function w()? There is no
optimal solution to this problem, but the most common window used in spectral analysis is the Hamming window:
277
and
[Variance here refers to taking expectation over the noise

sequence v(t).] Note that the relative variance in Eq. (119)
typically increases dramatically as tends to the Nyquist frequency. The reason is that G(i) typically decays rapidly,
while the noise-to-signal ratio v()/u() has a tendency to
increase as increases. In a Bode diagram, the estimates will
thus show considerable fluctuations at high frequencies.
Moreover, the constant frequency resolution in Eq. (118) will
look thinner and thinner at higher frequencies in a Bode diagram due to the logarithmic frequency scale.
See Ref. 3 for a more detailed discussion.
Choice of Window Size
The choice of is a pure trade-off between frequency resolution and variance (variability). For a spectrum with narrow
resonance peaks, it is thus necessary to choose a large value
of and accept a higher variance. For a more flat spectrum,
smaller values of will do well. In practice, a number of different values of are tried. Often, we start with a small value
of and increase it successively until an estimate is found
that balances the trade-off between frequency resolution (true
details) and variance (random fluctuations). A typical value
for spectra without narrow resonances is 20 30.
Subspace Estimation Techniques for State Space Models
A linear system can always be represented in state space
form:
x(t + 1) = Ax(t) + Bu(t) + w(t)
y(t) = Cx(t) + Du(t) + e(t)
(121)
We assume that we have no insight into the particular structure, and we would just estimate any matrices A, B, C, and
D, that give a good description of the input-output behavior
of the system. This is not without problems, among other
things because there are an infinite number of such matrices
that describe the same system (the similarity transforms).
The coordinate basis of the state-space realization, thus,
needs to be fixed.
Let us for a moment assume that not only are u and y
measured, but also the sequence of state vectors x. This
would, by the way, fix the state-space realization coordinate
basis. Now, with known u, y, and x, the model in Eq. (121)
becomes a linear regression: the unknown parameters, all of
278
the matrix entries in all the matrices, mix with measured signals in linear combinations. To see this clearly, let
Y (t) =
=
(t) =
x(t + 1)
y(t)
A
C
B
D
x(t)
u(t)
DATA QUALITY
w(t)
e(t)
E(t) =
Then, Eq. (121) can be rewritten as

Y (t) = (t) + E(t)
(122)
From this, all the matrix elements in can be estimated by

the simple least squares method, as described earlier. The covariance matrix for E(t) can also be estimated easily as the
sample sum of the model residuals. That will give the covariance matrices for w and e, as well as the cross covariance
matrix between w and e. These matrices will, among other
things, allow us to compute the Kalman filter for Eq. (121).
Note that all of the above holds without changes for multivariable systems, that is, when the output and input signals
are vectors.
The only remaining problem is where to get the state vector sequence x. It has long been known, for example (17,18),
that all state vectors x(t) that can be reconstructed from inputoutput data in fact are linear combinations of the components of the n k-step ahead output predictors
y(t
+ k|t),
k = {1, 2, . . ., n}
x(t) = L
y(t
+ 1|t)
..
.
y(t
+ n|t)
It is desirable to affect the conditions under which the data

are collected. The objective with such experiment design is to
make the collected data set ZN as informative as possible with
respect to the models to be built using the data. A considerable amount of theory around this topic can be developed and
we shall here just review some basic points.
The first and most important point is the following one
1. The input signal u must be such that it exposes all the
relevant properties of the system.
It must, thus, not be too simple. For example, a pure sinusoid
u(t) = A cos t
will only give information about the systems frequency response at frequency . This can also be seen from Eq. (85).
The rule is that
the input must contain at least as many different frequencies as the order of the linear model to be built. To
be on the safe side, a good choice is to let the input be
random (such as filtered white noise). It then contains
all frequencies.
(123)
where n is the model order (the dimension of x). See also Appendix 4.A in Ref. 4. We could then form these predictors and
select a basis among their components:
quoted earlier are not directly applicable. Experience has

shown, however, that confidence intervals computed according to the general asymptotic theory are good approximations. One may also use the estimates obtained by a subspace
method as initial conditions for minimizing the prediction error criterion [see Eq. (74)].
Another case where the input is too simple is when it is generated by feedback such as
u(t) = Ky(t)
(125)
If we would like to build a first-order ARX model
(124)
The choice of L will determine the basis for the state-space

realization and is done in such a way that it is well conditioned. The predictor y(t kt) is a linear function of u(s),
y(s), 1 s t and can efficiently be determined by linear
projections directly on the input output data. (There is one
complication in that u(t 1), . . ., u(t) k should not be
predicted, even if they affect y(t) k).)
What we have described now is the subspace projection approach to estimating the matrices of the state-space model in
Eq. (121), including the basis for the representation and the
noise covariance matrices. There are a number of variants of
this approach. See among several references, for example,
Refs. 19 and 20.
The approach gives very useful algorithms for model estimation and is particularly well suited for multivariable systems. The algorithms also allow numerically very reliable implementations. At present, the asymptotic properties of the
methods are not fully investigated, and the general results
y(t) + ay(t 1) = bu(t 1) + e(t)

we find that for any given , all models such that
a + bK =
will give identical inputoutput data. We can, thus, not distinguish between these models using an experiment with Eq.
(125). That is, we can not distinguish between any combinations of a and b if they satisfy the above condition for a
given . The rule is
If closed-loop experiments have to be performed, the
feedback law must not be too simple. It is to be preferred
that a set-point in the regulator is being changed in a
random fashion.
The second main point in experimental design is
2. Allocate the input power to those frequency bands
where a good model is particularly important.
This is also seen from the expression in Eq. (85).
If we let the input be filtered white noise, this gives information on how to choose the filter. In the time domain, it is often
useful to think like this:
Use binary (two-level) inputs if linear models are to be
built; this gives maximal variance for amplitude-constrained inputs.
Check that the changes between the levels are such that
the input occasionally stays on one level so long that a
step response from the system has time, more or less, to
settle. There is no need to let the input signal switch so
quickly back and forth that no response in the output is
clearly visible.
Note that the second point is really just a reformulation in the
time domain of the basic frequency domain advice: let the input
energy be concentrated in the important frequency bands.
A third basic piece of advice about experiment design concerns the choice of sampling interval.
Construct
experiment,
collect data
Data
Filter
data?
Polish and
present data
Data
Fit model
to data
Choose model
structure
MODEL VALIDATION AND MODEL SELECTION

A Pragmatic Viewpoint
The system identification process has, as we have seen, these
basic ingredients
The set of models
The data
The selection criterion
Once these have been decided upon, we have, at least implicitly, defined a model: The one in the set that best describes
the data according to the criterion. It is thus, in a sense, the
best available model in the chosen set. But is it good enough?
It is the objective of model validation to answer that question.
Often, the answer turns out to be no, and we then have to
go back and review the choice of model set, or perhaps modify
the data set. See Fig. 4.
How do we check the quality of a model? The prime
method is to investigate how well it is capable of reproducing
the behavior of a new set of data (the validation data) that
was not used to fit the model. That is, we simulate the obtained model with a new input and compare this simulated
output. One may then use ones eyes or numerical measurements of fit to decide if the fit in question is good enough.
Suppose we have obtained several different models in different model structures (say a fourth-order ARX model, a second-order BJ model, a physically parameterized one, and so
on) and would like to know which one is best. The simplest
and most pragmatic approach to this problem is then to simulate each one of them on validation data, evaluate their performance, and pick the one that shows the most favorable fit
to measured data. (This could indeed be a subjective criterion!)
Model
Validate
model
Data
not OK
Model structure
not OK
No
3. A typical good sampling frequency is 10 times the bandwidth of the system.

That corresponds roughly to 57 samples along the rise time
of a step response.
279
Accept
model?
Yes
Figure 4. The identification loop.
The Bias-Variance Trade-off

The heart of the model structure selection process is to handle
the trade-off between bias and variance, as formalized by Eq.
(92). The best model structure is the one that minimizes
FN, the fit between the model and the data for a fresh data
setone that was not used for estimating the model. Most
procedures for choosing the model structures are also aiming
at finding this best choice.
Cross Validation. A very natural and pragmatic approach
is cross validation. This means that the available data set is
N1
that is used to estisplit into two parts, estimation data, Zest
mate the models:
1)
N = arg min VN (, ZN
est
1
(126)
N2
for which the criterion is evaluated:
and validation data, Zval
2)
FN = VN (N , ZN
val
1
(127)
Here, VN is the criterion in Eq. (75). Then, FN will be an unbiased estimate of the measure FN, defined by Eq. (88), which
was discussed at length in the previous section. The procedure would the be to try out a number of model structures
and choose the one that minimizes FN1.
Such cross validation techniques to find a good model
structure have an immediate intuitive appeal. We simply
check if the candidate model is capable of reproducing data
it hasnt yet seen. If that works well, we have some confidence
in the model, regardless of any probabilistic framework that
might be imposed. Such techniques are also the most commonly used ones.
A few comments could be added. In the first place, one
could use different splits of the original data into estimation
280
and validation data. For example, in statistics, there is a common cross validation technique called leave one out. This
means that the validation data set consists of one data point
at a time but successively applied to the whole original set.
In the second place, the test of the model on the validation
data does not have to be in terms of the particular criterion
[see Eq. (127)]. In system identification, it is common practice
to simulate (or predict several steps ahead) the model using
the validation data and then visually inspect the agreement
between measured and simulated (predicted) output.
Estimating the Variance ContributionPenalizing the Model
Complexity. It is clear that the criterion in Eq. (127) has to
be evaluated on the validation data to be of any use; it would
be strictly decreasing as a function of model flexibility if evaluated on the estimation data. In other words, the adverse effect of the dimension of shown in Eq. (92) would be missed.
There are a number of criteria, often derived from entirely
different viewpoints, that try to capture the influence of this
variance error term. The two best known ones at Akaikes
Information Theoretic Criterion, AIC, which as the form (for
Gaussian disturbances)
2 dim
V N (, ZN ) = 1 +
N
N
1
2 (t, )
N t=1
(128)
and Rissanens Minimum Description Length Criterion, MDL,

in which dim in the expression above is replaced by log N
dim (21,22).
N is then to be minimized both with respect
The criterion V
to and to a family of model structures. The relation to the
expression in Eq. (89) for FN is obvious.
Residual Analysis
The second basic method for model validation is to examine
the residuals (the leftovers) from the identification process.
These are the prediction errors
(t) = (t, N ) = y(t) y(t|
N )
that is, what the model could not explain. Ideally, these
should be independent of information that was at hand at
time t 1. For example, if (t) and u(t ) turn out to be
correlated, then there are things in y(t) that originate from
u(t ) but have not been properly accounted for by y(t N).
The model has then not squeezed out all relevant information
about the system from the data.
It is good practice to always check the residuals for such
(and other) dependencies. This is known as residual analysis.
A basic reference for how to perform this is Ref. 10.
BACK TO DATA: THE PRACTICAL SIDE OF IDENTIFICATION
Software for System Identification
In practice, system identification is characterized by some
quite heavy numerical calculations to determine the best
model in each given class of models. This is mixed with several user choices, trying different model structures, filtering
data, and so on. In practical applications, we will thus need
good software support. There are now many different com-
mercial packages for identification available, such as Mathworks System Identification Toolbox (2), Matrixxs System
Identification Module (23) and PIM (24). They all have in
common that they offer the following routines:
Handling of data, plotting, and so on: Filtering of data,
removal of drift, choice of data segments, etc.
Non-parametric identification methods: Estimation of covariances, Fourier transforms, correlation- and spectralanalysis, etc.
Parametric estimation methods: Calculation of parametric estimates in different model structures.
Presentation of models: Simulation of models, estimation
and plotting of poles and zeros, computation of frequency
functions, and plotting Bode diagrams, etc.
Model validation: Computation and analysis of residuals
((t, N)). Comparison between different models properties, etc.
The existing program packages differ mainly in various user
interfaces and by different options regarding the choice of
model structure according to C above. For example, MATLABs Identification Toolbox (2) covers all linear model structures discussed here, including arbitrarily parameterized linear models in continuous time.
Regarding the user interface, there is now a clear trend to
make it graphically oriented. This avoids syntax problems
and relies more on click and move, at the same time as tedious menu-labyrinths are avoided. More aspects of CAD
tools for system identification are treated in Ref. 25.
How to Get to a Good Model?
It follows from our discussion that the most essential element
in the process of identificationonce the data have been recordedis to try out various model structures, compute the
best model in the structures using Eq. (38), and then validate
this model. Typically, this has to be repeated with quite a few
different structures before a satisfactory model can be found.
While one should not underestimate the difficulties of this
process, the following simple procedure to get started and
gain insight into the models could be suggested:
1. Find out a good value for the delay between input and
output, for example, by using correlation analysis.
2. Estimate a fourth order linear model with this delay
using part of the data, and simulate this model with the
input and compare the models simulated output with
the measured output over the whole data record. In
MATLAB language, this is simple
z = [y u];
compare(z,arx(z(1:200,:),[4 4 1]));
If the model/system is unstable or has integrators, use prediction over a reasonable large time horizon instead of simulation.
Now, either of two things happen:
The comparison looks good. Then, we can be confident
that with some extra worktrying out different orders
and various noise modelswe can fine-tune the model

and have an acceptable model quite soon.
The comparison does not look good. Then we must do
further work. There are three basic reasons for the failure.
1. A good description needs higher order linear dynamics. This is actually, in practice, the least likely reason, except for systems with mechanical resonances.
One then obviously has to try higher order models or
focus on certain frequency bands by band pass filtering.
2. There are more signals that significantly affect the
output. We must then look for what these signals
might be, check if they can be measured, and if so,
include them among the inputs. Signal sources that
cannot be traced or measured are called disturbances, and we simply have to live with the fact that
they will have an adverse effect on the comparisons.
3. Some important nonlinearities have been overlooked.
We must then resort to semiphysical modeling to find
out if some of the measured signals should be subjected to nonlinear transformations. If no such transformations suggest themselves, one might have to try
some nonlinear black-box model, like a neural network.
20
40
60
80
100
120
140
160
281
180
Figure 6. As Fig. 5 but using all three inputs.
inputs in Fig. 1. That is, the model is computed as

arx([y u1 u2 u3], [4 4 4 4 1 1 1])
Clearly, this advice does not cover all the art of identification,
but it is a reasonable first approximation.
on the same data set. The comparison is shown in Fig. 6. It

looks good. By further fine-tuning, as well as using model
structures from physical modeling, only slight improvements
are obtained.
Example 7 Aircraft Dynamics

Let us try the recipe on the aircraft data in Fig. 1. Picking
the canard angle only as the input, estimating a fourth order
model based on the data points 90 to 180, gives Fig. 5. (We
use 10-step ahead prediction in this example since the models
are unstable, as they should be; JAS has unstable dynamics
in this flight case.) It does not look good. Let us try alternative 2: More inputs. We repeat the procedure using all three
Example 8 Buffer Vessel Dynamics

Let us now consider the pulp process of Fig. 2. We use the number before the vessel as input and the -number after the
vessel as output. The delay is preliminarily estimated to 12
samples. Our recipe, where a fourth order linear model is estimated using the first 200 samples and then simulated over
the whole record gives Fig. 7. It does not look good.
6
4
2
0
2
4
6
8
0
20
40
60
80
100
120
140
160
180
Figure 5. Dashed line: actual pitch rate. Solid line: 10 step ahead
predicted pitch rate, based on the fourth order model from canard
angle only.
100
200
300
400
500
600
Figure 7. Dashed line: -number after the vessel, actual measurements. Solid line: simulated -number using the input only and a
fourth order linear model with delay 12, estimated using the first 200
data points.
282
SYSTEM INTERCONNECTS
9. D. Brillinger, Time Series: Data Analysis and Theory, San Francisco: Holden-Day, 1981.
10. N. Draper and H. Smith, Applied Regression Analysis, 2nd ed,
New York: Wiley, 1981, 2nd ed.
11. J. Sjoberg et al., Nonlinear black-box modeling in system identification: A unified overview, Automatica, 31 (12): 16911724,
1995.
6
4
2
12. A. Juditsky et al., Nonlinear black-box modeling in system identification: Mathematical foundations, Automatica, 31 (12): 1724
1750, 1995.
13. Q. Zhang and A. Benveniste, Wavelet networks, IEEE Trans.
Neural Networks, 3: 889898, 1992.
14. T. Poggio and F. Girosi, Networks for approximation and learning. Proc. IEEE, 78: 14811497, 1990.
6
0
4
15. L. Breiman, Hinging hyperplanes for regression, classification

and function approximation, IEEE Trans. Inf. Theory, 39: 999
1013, 1993.
6
8
50
100
150
200
250
300
350
400
Figure 8. Same as Fig. 7 but applied to resampled data.
Some reflection shows that this process indeed must be

non-linear (or time-varying): the flow and the vessel level
definitely affect the dynamics. For example, if the flow was a
plug flow (no mixing), the vessel would have a dynamics of a
pure delay equal to vessel volume divided by flow.
Let us thus resample the date accordingly, that is, so that
a new sample is taken (by interpolation from the original
measurement) equidistantly in terms of integrated flows divided by volume. In MATLAB terms, this will be
z = [y,u]; pf = flow./level;
t =1:length(z)
newt =
table1([cumsum(pf),t],[pf(1)sum:(pf)] );
newz = table1([t,z], newt);
We now apply the same procedure to the resampled data.
This gives Fig. 8. This looks good. Somewhat better numbers can then be obtained by fine-tuning the orders.
16. J. E. Dennis and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Englewood Cliffs,
17. J. Rissanen, Basis of invariants and canonical forms for linear
dynamic systems, Automatica, 10: 175182, 1974.
18. H. Akaike, Stochastic theory of minimal realization, IEEE Trans.
Autom. Control, AC-19: 667674, 1974.
19. P. V. Overschee and B. DeMoor, Subspace Identification of Linear
Systems: Theory, Implementation, Application, Boston, MA:
Kluwer, 1996.
20. W. E. Larimore, System identification, reduced order filtering
and modelling via canonical variate analysis, Proc. 1983 Amer.
Control Conf., San Francisco, 1983.
21. H. Akaike, A new look at the statistical model identification,
22. J. Rissanen, Modelling by shortest data description, Automatica,
14: 465471, 1978.
23. MATRIXx Users Guide, Santa Clara, CA: Integrated Systems
Inc., 1991.
24. I. D. Landau, System Identification and Control Design Using
P.I.M. Software, Englewood Cliffs, NJ: Prentice-Hall, 1990.
25. L. Ljung, Identification of linear systems, in E. D. Linkens (ed.),
CAD for Control Systems, New York: Marcel Dekker, 1993, Chap.
6, pp. 147165.
LENNART LJUNG
BIBLIOGRAPHY
strom and T. Bohlin, Numerical identification of linear
1. K. J. A
dynamic systems from normal operating records, IFAC Symp.
Self-Adapt. Syst., Teddington, England, 1965.
2. L. Ljung, The System Identification Toolbox: The Manual, Natick,
MA: MathWorks Inc., 1995, 4th ed.
3. L. Ljung and T. Glad, Modeling of Dynamic Systems, Englewood
4. L. Ljung, System IdentificationTheory for the User, Englewood
Cliffs, N.J: Prentice-Hall, 1987.
5. T. Soderstrom and P. Stoica, System Identification, London: Prentice-Hall Int., 1989.
6. G. E. P. Box and D. R. Jenkins, Time Series Analysis, Forecasting
and Control, San Francisco: Holden-Day, 1970.
7. J. Schoukens and R. Pintelon, Identification of Linear Systems: A
Practical Guideline to Accurate Modeling, London: Pergamon,
1991.
Identification, Cambridge, MA: MIT Press, 1983.
Linkoping University
SYSTEM IDENTIFICATION, MULTIDIMENSIONAL. See MULTIDIMENSIONAL SIGNAL PROCESSING.
592
ROBUST CONTROL
ROBUST CONTROL
The role of feedback in automatic control systems is to exploit
evolving real-time measurements to obtain more precise control over system behavior. A robust control system is one that
performs within specified tolerances despite uncertain variations in the controlled plant within given bounds. Robust control theory is that branch of mathematical system theory concerned with the design and analysis of robust feedback
control systems.
The block diagram in Fig. 1 represents a typical feedback
control system consisting of an uncertain plant and a controller. The plant might be an aircraft or missile, a robotic
lunar surface explorer, a chemical process, an automobile engine, or a nuclear power plant. It might be something as small
as the magnetic read-head positioning system in a computer
disk drive or as large as the global economy. It could be almost any complex dynamic system. The controller, on the
other hand, is typically a computer or a microprocessor,
though it could be simply the thermostat in a home heating
control system or the mechanical linkage in the eighteenthcentury flyball governor invented by James Watt for controlling steam engines. From a control theorists perspective the
uncertain plant and the controller are simply mathematical
relationships, for example, differential equations. Robust control theory is focused on the quantitative analysis of the consequences of plant uncertainty. The need for robust control
arises when a control system design must be based on an inexact mathematical model of the true physical plant.
ORIGINS OF ROBUST CONTROL THEORY

Since Watts invention of the flyball governor in 1788, designers of feedback control systems have implicitly sought robust
designs. Robustness considerations are implicit in the classical feedback design methods based on root locus and frequency responses. But prior to 1960, the issue of robustness
had not been generally recognized by mathematical control
theorists, and the term robustness itself did not appear in the
literature of mathematical control theory before 1975. Apparently the first clear mathematical formulation of a robust
feedback control problem was produced by I. Horowitz (6) in
the early 1960s. Horowitz studied simple linear time-invariant feedback control systems with a single feedback loop for
plant models with several uncertain parameters. He correctly
recognized that plant uncertainty is a dominant factor in determining what can and cannot be achieved with feedback
control, at least for simple single-loop feedback systems, and
d
Command
+
r
Error
e
K(s)
Control
u
Controller
G(s)
Output
y
Plant
Figure 1. Robust control concerns designing feedback controllers so

that the output y(t) will precisely track command signals r even when
the plant is uncertain.
he showed how to design uncertainty-tolerant controllers using classical root-locus and frequency-response methods.
Unfortunately, the significance of Horowitzs contribution
was not immediately noticed by control researchers. A gap
between mathematical control theory and control engineering
practice emerged in the 1960s and early 1970s, as the mathematical theory of feedback control increasingly decoupled
from engineering practice. Time-tested root-locus and frequency-response techniques were regarded by researchers as
ad hoc and simplistic. They sought to pose feedback design
as a mathematical optimization. But, explicit mathematical
representations of robustness issues were not incorporated in
their mathematical representations of the problems of feedback control. In those early days, mathematical control research generally presumed that available mathematical models were sufficiently accurate. The profound significance of
this omission was not immediately evident.
In the 1960s, mathematical proofs of optimality were
widely regarded as sufficient evidence that the emergent, but
as yet untried, modern mathematical theories would produce
superior results. But in the early 1970s, researchers began to
be jolted as early attempts to apply mathematical optimization theories to the design of complex multivariable feedback
controllers for military aircraft and submarines resulted in
some surprising failures. In one unpublished design study
carried out at Systems Control Inc. under the supervision of
D. Kleinman with MITs Michael Athans as consultant, a linear quadratic Gaussian (LQG) controller for a Trident submarine caused the vessel to unexpectedly surface in nonlinear
simulations involving moderately rough seas. In another example, a 1977 journal paper describing the disappointing results
of an LQG control design study for the F-8C Crusader aircraft
concluded euphemistically with the observation that The
study has pinpointed certain theoretical weaknesses . . . as
well as the need for using common sense pragmatic techniques
to modify the design based on pure theory. A lack of attention
to robustness was quickly identified as the dominant factor in
these failures.
In 1976, the modern field of robust control theory was
born. The term robustness was introduced into the control
theory lexicon in papers by E. J. Davison and by M. Safonov
and M. Athans. Mathematical control theorists in Michael
Athans MIT laboratory began to refocus their attention on
methods for analyzing and optimizing the robustness of feedback control systems. Researchers sought to salvage as much
as possible of the mathematical theory developed in the
1960s. They worked to link linear optimal control theory to
classical root-locus/pole-placement methods. In 1978 researcher Gunter Stein summarized the mood of the time, explaining that his goal was to make modern control theoretical methods work. Researchers were beginning to reexamine
classical frequency-response methods and to take a closer look
at Horowitzs work on uncertainty-tolerant single-loop feedback design. They were seeking multivariable generalizations
of Horowitzs ideas that would mesh well with the sophisticated mathematics of modern optimal control theory.
Though attempts at direct generalizations of Horowitzs
ideas to more complex multivariable feedback systems with
several feedback control loops proved unfruitful, mathematical control researchers quickly discovered that multivariable
robustness analysis methods could be developed based on the
so-called small-gain theorem of nonlinear stability theory.
ROBUST CONTROL
Along with the Lyapunov, ZamesSandberg and other nonlinear stability theories, this now became the core of the emergent mathematical theory of robust control. Prior to 1976, the
Lyapunov and ZamesSandberg theories were generally regarded as esoteric tools for nonlinear stability analysis. Despite some suggestive early remarks by stability theory pioneers G. Zames and V. Popov, by 1975, the robustness
implications of nonlinear stability theories had yet to be developed. Today, many still regard the mid 1976 change of focus of mathematical control theory from optimal control to
robustness as a revolutionary paradigm shift, though, unlike
many scientific revolutions, the shift occurred with remarkably little controversy.
In 1977 Safonov showed that the general problem of robust
stability analysis is equivalent to computing a topological separation of graphs of feedback operators. Lyapunov, conic sector, positivity small-gain and other nonlinear stability theories emerge as special cases. Other researchers, such as J. C.
Doyle, G. Stein and N. R. Sandell exploited H. H. Rosenbrocks multivariable version of the Nyquist stability criterion
to develop a simplified, and hence more widely accessible, linear explanation of multivariable robustness criteria, singular-value (-plots), and structured singular-value (-plots)
frequency-response plots.
By 1981, the concept of singular-value, loop shaping and
the closely related concept of mixed sensitivity became central
to understanding robust control. For a multivariable control
system with loop transfer function matrix L(s), design specifications were posed in terms of the sensitivity matrix
S(s) [I L(s)]1 and the complementary sensitivity matrix
T(s) L(s)[I L(s)]1. From these definitions, it follows that
there is the fundamental constraint
S(s) + T (s) = I
(1)
whence there is a fundamental tradeoff between sensitivity

S(s) and complementary sensitivity T(s).
In the mixed-sensitivity framework, the concept of multiplicative uncertainty M became central. Given a true plant
G(s) and a nominal plant model G0, a multiplicative uncertainty M is defined by
G(s) = [I + M (s)]G0(s)
When knowledge of plant uncertainty is limited to bounds on
the size of the multiplicative uncertainty matrix M, good control performance results when the size of S(s) is smaller than
unity over a specified control bandwidth, say B, with
s j. The singular values of a matrix are a particularly
useful measure of its size.
To this day in the majority practical robust control designs,
performance is specified in terms of S and T. Singular values
of S( j) are constrained to be small compared to unity over
that desired control bandwidth ( B). And, singular values
of T( j) are constrained to be smaller than unity at higher
frequencies (beyond B) at which multiplicative uncertainties
are expected to be large because of the effects of small unmodeled time delays, phase lags, and other sorts of parasitic dynamics.
Optimal H2, H, and -synthesis are very flexible robust
control theories introduced in the 1980s. They are currently
the principal methods employed for designing robust, multi-
593
variable, feedback control systems. Though each of these

methods has the flexibility to optimize robustness in a very
general setting, their most common use is in conjunction with
weighted, mixed-sensitivity, performance criteria where they
are used to find a controller K(s) for a plant G(s) so as to
minimize the size of the following transfer function matrix:
W1 (s)S(s)
W2 (s)K(s)S(s)
W3 (s)T (s)
Here W1, W2, W3 are frequency-dependent weights specified by
the designer, L(s) G(s)K(s) is the loop transfer function,
K(s) is the controller, and G(s) is the plant to be controlled.
SOURCES OF UNCERTAINTY
Uncertainty is always present to some degree in mathematical models of physical systems. It arises from ignorance of
physical details, from approximations, or from unpredictable
external effects, such as noise or weather changes.
Models derived from idealized physical laws and theoretical principles are uncertain. This includes, for instance,
Newtons force law, the second law of thermodynamics,
Kirchhoff s current law, Boyles ideal gas law, Maxwells
electromagnetic equations, and the law of supply and demand
from economic theory. Though some laws are very good predictors of behavior, all such laws are idealizations with fundamental limits of accuracy. Moreover, applications of such laws
and principles usually entail assumptions that further degrade the models accuracy of prediction.
Approximations produce modeling uncertainty. Linearized
models are uncertain, and neglected nonlinear distortion becomes uncertainty. Engineers designing stability augmentation systems for aircraft, robot manipulator arms, or similar
mechanical systems employ a rigid-body approximation neglecting flexible bending modes, thereby introducing uncertainty into the mathematical models of these systems. Deliberate truncation of modes to simplify a model is called model
reduction. Uncertainty arises even when all flexible modes
are included in models, because the numerical values in the
equations for all modes are seldom known with precision.
Mathematical models derived from experimental data are
also uncertain. Knowledge of the past is uncertain and predictions of the future are always uncertain. The approximate
curve-fitting processes implicit in most algorithms for system
identification mean that experimentally identified models
usually can not exactly reproduce even the past data. And
even when they do, forecasts of future behavior derived from
such models are still never guaranteed to be accurate.
STABILITY AND ROBUSTNESS
It turns out that the effects of even very small model uncertainties are amplified many times by a feedback system.
Thus, it is vital that control engineers quantify the tolerance
of their designs to such uncertainties. Because modeling uncertainty is usually a dominant factor in determining how
precisely it is possible to control a plants behavior, an engineer wishing to optimize control performance must quantify
ROBUST CONTROL
the intrinsic tradeoff between uncertainty tolerance and control precision, that is the intrinsic tradeoff between robustness and performance.
Examples of extreme sensitivity to small effects abound in
nature. For example, a ball balanced on the top of a hill is
unstable. Left undisturbed, it would, in principle, remain forever at rest on the mountain peak. Yet, a disturbance smaller
than the flapping of a butterfly wing causes the ball to plummet. Such extreme sensitivity to small disturbances is called
instability. Unstable systems are never robust.
On the other hand, a ball at rest in the cusp of a valley is
stable and is not dislodged even by a major earthquake. Further, so long as gravity remains an attractive force, then balls
at rest in valleys remain stably at rest. Not only is the ball
stable, but it remains stable for any large nonnegative variation in the gravitational constant. A ball in a valley is said to
possess the property of stability robustness.
But even the property of stability itself can be extremely
sensitive to small effects. If one had a knob which could reduce the Earths gravitational attraction until gravitational
attraction became negative, then balls in valleys would no
longer have stability robustness. As the pointer on the knob
approached zero, the property of stability itself would be extremely sensitive to the position of the knob. With the knob
set a little bit above zero, a ball in a valley would remain
stably at rest. But with the pointer on the knob ever so
slightly below zero, attraction would become repulsion, and
stability robustness would no longer hold. The ball would be
launched into space in response to the slightest change in the
knobs setting. The property of stability for a ball in a valley
is thus not robust to negative variations in the gravitational
constant.
Of course, although there is little prospect that gravity will
reverse itself, there are many situations encountered by engineers in which stability is not robust to variations in physical parameters.
Robustness in Classical Single-Loop Control
Although the term robustness was not generally used in control theory until the 1970s, robustness has always concerned
feedback control system designers. For example, consider the
flyball governor system invented by James Watt in 1788 to
regulate the rotational velocity of a steam engine. A sketch of
Watts invention is shown in Fig. 2. The flyball governor device is coupled to the main shaft of a steam engine via a pulley. It maintains a set value for the rotational velocity of the
shaft by opening the steam valve when velocity decreases and
closing it when it increases.
A simplified linear model for a steam engine feedback system incorporating a flyball governor is given by the following
differential equations:
d2
d
y(t) + 2 y(t) + y(t) = Ks u(t)
dt 2
dt
d
Flyball governor:
u(t) + u(t) = Kg (r(t) y(t))
dt
Steam engine:
Here y(t) denotes the angular velocity of the steam engines

output shaft at time t, u(t) indicates that the steam valve position at time t, and r(t) denotes an externally specified desired shaft velocity. The constants Kg and Ks are constants
Steam
594
Figure 2. An early example of a feedback control system is the Watt

flyball governor. The device enhances the robustness of steam plants
against engine load variation and steam pressure changes.
corresponding, respectively, to the position of the fulcrum in

the mechanical linkage between the flyball governor and the
steam valve and the physical properties of the steam valve.
The equilibrium solution of these equations (obtained by setting all the d/dt and d2 /dt2 terms to zero) is y(t) K/(1
K)r where K KgKs. Hence, if K is much larger than one,
then y(t) is very nearly equal to its desired value r, assuming
that equilibrium is achieved.
A more detailed analysis of the differential equations that
model the flyball governor shows that they have a time-varying solution of the form
y(t) =
K
r + c1 et + c2 e t sin(t + )
1+K
where 1 K1/3, 1 0.5K1/3, 0.5 3 K1/3, and

the three numbers c1, c2, are determined by the initial conditions (d/dt)y(0), y(0), and u(0). When 1 K 8, the timevarying terms c1et and c2et sin(t ) both decay in magnitude to zero as time t increases toward infinity, so that equilibrium is approached asymptotically. However, if K 8, then
0, and the et term becomes increasingly large, predicting
that the steam engines velocity y(t) alternately increases and
decreases with ever growing amplitude. For example, if the
steam valve has gain constant Ks 1, then the mathematics
predicts that the flyball governor system is unstable for governor gain Kg 8. A governor gain of Kg 7 would be fine. But,
if the true value of the engine gain Ks were uncertain and
known to within only 33%, then one might have Ks 1.33
in which case the model predicts that the flyball governor system is unstable for governor gain Kg 8/1.33 6. Whence, a
value of governor gain Kg offers insufficient robustness to accommodate a 33% variation in steam engine gain about a
nominal value of Ks 1.
ROBUST CONTROL
595
MULTIVARIABLE ROBUSTNESS
In the 1960s and 1970s as increasingly complex control systems with multiple feedback loops were deployed, engineers
typically evaluated the robustness of these systems against
variations one-loop-at-at-a-time. That is, they examined the
effects of varying each feedback while holding all the other
gains fixed at their nominal values. This proved unwise, because a system may tolerate large, even infinite, variations in
individual feedback gains, yet may be destabilized by even
very small simultaneous variations in several feedback gains.
The term multivariable robustness refers to the tolerance of
feedback systems to such simultaneous variations.
As early as 1971, H. H. Rosenbrock and M. Athans urged
greater attention to the issue of simultaneous variations in
several feedback gains. And Rosenbrock even developed crude
methods for designing robust controllers for such systems,
provided that the loops were approximately decoupled, a condition that Rosenbrock called diagonal dominance. But, for
many, the telling example that made the case for multivariable robustness theory was the following very simple example
produced by J. C. Doyle in 1978.
1 + 1
u
s 100
1
s2 + 100
1 + 2
Multiplicative
uncertainties
Plant G0 (s)
Unstable
1
Stable
Unstable
1
Unstable
Figure 3. Doyles 1978 two-loop feedback example shows that oneloop-at-a-time stability robustness analysis is unwise. In this case,
the feedback system remains stable for large variations in either of
the two gain perturbations 1 and 2 when the other is fixed to be
zero, but is unstable for small simultaneous variations.
If one closes either one of the two feedback loops around this
system, the open-loop transfer function in the other loop is
simply given by
g(s) =
1
s
whence the system remains stable if the gain in the other loop
assumes any nonnegative value. Moreover, this implies that
gain margin is infinite and the phase margin is 90 in each
loop, when evaluated one-loop-at-a-time. The system seems to
be extraordinarily robust when examined one-loop-at-a-time.
However, this is not the case when gains in both loops are
varied simultaneously, as the stability diagram in Fig. 3 indicates.
Stable
1
Multivariable Robustness
Consider a two-loop multivariable feedback system with nominal loop transfer function

s 100
10(s + 1)
1
G0 (s) = 2
s + 100 10(s + 1)
s 100
10(s + 1)
10(s + 1) s 100
G(s) = G0 (s) + A (s)

A multiplicative uncertainty is a matrix M(s) such that the
true and nominal are obtained by multiplying the model
G0(s) by [I M(s)]. Because matrix multiplication is not generally commutative, the right multiplicative uncertainty
[say Mr(s)] and left multiplicative uncertainty [say Ml(s)]
are generally different, that is,
G(s) = G0 (s)[I + Mr (s)] = [I + Ml (s)]G(s)
does not imply that Mr(s) equals Ml(s). Feedback control systems with additive and multiplicative uncertainties are depicted in Fig. 4.
Uncertainty
An important aspect of robust control theory is quantifying
the difference between two transfer function matrices, say
G(s) and G0(s). Typically, G0(s) is a nominal mathematical
model, and G(s) is the corresponding true system and the
difference is called uncertainty. The uncertainty may be
viewed as additive or multiplicative, or it may be regarded in
terms of more exotic linear fractional transformation (LFT) or
normalized coprime matrix fraction description (MFD) transformations. Useful quantitative measures of size are singular
values, H norms, and the gap metric.
Additive and Multiplicative Uncertainty. An additive uncertainty is a matrix, say A(s), which produces the true system
G(s) when added the nominal model G0, that is
LFT Uncertainty. LFT is an acronym standing for linear

fractional transformation. In the case of LFT uncertainty, the
true plant is presumed to be related to a perturbational matrix via a relationship of the form
G(s) = P22 + P21(I P22)1 P12.
(2)
where

P
P(s) = 11
P21
P12
P22

(3)
is a given transfer function matrix. The LFT transformation

Eq. (2) is also sometimes called the Redheffer star product
596
ROBUST CONTROL
where [N0(s), D0(s)] and [N(s), D(s)] are stable normalized coprime left MFDs of G0(s) and G(s), respectively. The gap metric has attracted much interest from theorists because it has
a rich mathematical structure with many subtle symmetries
and geometric interpretations.
True plant G(s)

A
+
G0(s)
K(s)
Singular Values and H Norms. Given a true plant G(s) and

a nominal plant G0(s), it is useful to have a quantitative measure of the size of the difference between the two. In robust
control theory, singular values and H norms provide the desired measures of size.
An n m matrix, say A, is a mapping of m-vectors x into
n-vectors Ax. Likewise, a transfer function matrix, say A(s),
is mapping of signal vectors x(s) into signal vectors A(s)x(s).
Singular values and H-norms indicate the size or, more
precisely, the gain of such mappings.
Every complex n m matrix A with rank r has a singularvalue decomposition (SVD)
True plant G(s)

M
+
K(s)
G0(s)
Figure 4. Inaccuracies in a plant model G(s) can be represented as

additive uncertainty A or multiplicative uncertainty M.
where U and V are square unitary matrices

0

=
0 0
and denoted
G(s)
,P
MFD Uncertainty and the Gap Metric. Every real transfer

function matrix G(s) can be decomposed into a ratio
G(s) = N(s)D1 (s)
where N(s) and D(s) are both stable transfer function matrices. Such a decomposition is called a stable right matrix fraction description (MFD) of G(s).
Stable MFDs are not unique. For example, another stable
MFD is given by
A = UV

N(s)
N(s)
=
Q(s)
D(s)
D(s)
(4)
is an r r diagonal matrix
and
0
1 0
0
0
2
=.

..
.
.
.

N(s)
U (s) =
D(s)

N(s)
N0 (s)

(G(s), G0(s)) = min

Q(s)

Q(s) stable D(s)
D0 (s)
i ui vi
i=1
The largest singular value of A is also a measure of the

vector gain of the matrix A, that is
1 (A) = max
x
has the all-pass property U(s)U(s) I.

MFDs play a role in some more exotic quantitative measures of uncertainty used in robust control theory. An example is the gap metric (G, G0) which, roughly speaking, is the
sine of the angle between the graphs of G(s) and G0(s). Formally, the gap is computed via solution of a certain H optimization problem, namely
with 1 2 . . . r 0. The positive numbers 1, 2,

. . ., r are called the singular values of A. The orthogonal
columns u1, . . ., un. v1, . . ., vm of U and V are called the
singular vectors. The SVD of A is sometimes also written in
terms of the singular vectors as
A=
where Q(s) is any stable invertible transfer function matrix.

A stable right MFD N(s), D(s) of G(s) is said to be coprime if
every other stable MFD of G(s) can be written in the form of
Eq. (4) for some stable Q(s). A right MFD is said to be normalized if the matrix
0
0
..
Ax
x
where x denotes the Euclidean norm

x = x21 + x22 + + x2n
The H norm of a stable transfer function matrix, say
P(s), is defined in terms of the largest singular value of its
frequency response
P
, sup
1 [P( j)]
For an unstable transfer function P(s), the H-norm by definition is infinite, that is, whenever P(s) has unstable poles
P
.
ROBUST CONTROL
The following are other useful properties of singular

values:
1.
2.
3.
4.
5.
6.
1(A B) 1(A) 1(B).

1(AB) 1(A)1(B).
1(A) 1/ n(A1) for A an n n invertible matrix.
r
Trace(A*A) i1 i2(A).
1(Aij) 1(A) for any submatrix Aij of A.

A
2 max{1 (A), 1 (B)}
max{1 (A), 1 (B)} 1
B
(5)
According to an interpretation of the small-gain theorem

known as the performance robustness theorem, robustness of
stability for all n uncertainties i 1 (i 1, . . ., n) is
equivalent to performance robustness in the absence of the
fictitious uncertainty n.
The canonical robust control problem is described mathematically as follows. Given a multi-input, multi-output
(MIMO) plant transfer function matrix P(s), find a stabilizing
controller F(s) so that the closed-loop transfer function matrix
remains stable for all diagonal uncertainty matrices
0
0
0
1
0
0
0
2
= .
..
..
.
.
.
.
Canonical Robust Control Problem
The canonical robust control problem concerns systems of the

general form depicted in Fig. 5. The nth fictitious uncertainty
is not an actual uncertain gain. This fictitious uncertainty of
size n 1 is wrapped as a purely imaginary loop from a
disturbance input d to regulated error output signal e to represent the performance specification that the gain of the
transfer function matrix from d to e should be less than one,
that is,
1 [Ted ( j)] < 1 for all
Fictitious
uncertainty
n
1 1
, D
n - 1
Sigma Plots and Mu Plots
Plant P(s)
P11
P12
P21
P22
n
( j)]

inf 1 () | det[I Ty 1 u 1 ( j)] = 0
u1
, Km [Ty u
y1
with 1 and, perhaps, subject to an additional constraint D where D is a specified subset of the set of diagonal matrices with H-norm less than one.
Closely related to the canonical robust control problem are
the concept of the multivariable stability margin and the
structured singular value. Consider the system in Fig. 5.
Denote by Ty1u1(s) the closed-loop transfer function from u1 to
y1 with the uncertainty removed. Loosely, Ty1u1(s) is the
closed-loop transfer function that the diagonal uncertainty
sees as it looks back into the rest of the system. Given a stable transfer function matrix Ty1u1(s), the multivariable stability margin Km(Ty1u1) associated with Ty1u1(s), by definition, is
the size of the smallest destabilizing D. The structured
singular value [Ty1u1(s)], by definition, is the reciprocal of
the multivariable stability margin. More formally, for any
stable transfer function Ty1u1( j),
1
[Ty 1 u 1 ( j)]
Uncertainty
597
A well-known stability result called the small-gain theorem

says that, if the loop gain of a feedback system is less than
one, then it is stable. For linear time-invariant systems, the
H norm of the system transfer function is the relevant measure of gain. Hence, for the canonical robust control problem,
a sufficient condition for closed-loop stability is that
Ty1u1 1. Now, the uncertainty has, by hypothesis,
1. Hence, by the small-gain theorem, a sufficient condition closed-loop stability is that Ty1u1 1 or, equivalently,
1 [Ty 1 u 1 ( j)] < 1 for all
y2
u2
Controller
K(s)
Figure 5. The canonical robust control problem represents a wide

spectrum of uncertain control design problems. It accommodates robust performance requirements via a fictitious uncertainty n.
The problem of choosing the feedback K(s) so that the foregoing H inequality holds is known as the H control problem.
To gain a sense of the tolerance of uncertainty at each frequency, control engineers plot singular value Bode plots, such
as in Fig. 6. The number 1/ 1[Ty1u1( j)] is interpreted as a
lower bound on the size Km( j) of the smallest destabilizing
uncertainty ( j). Hence, singular values provide a convenient upper bound on the structured singular value, namely,
[Ty 1 u 1 ( j)] 1 [Ty 1 u 1 ( j)]
(6)
598
ROBUST CONTROL
matrix
10
103
Peak value:
976.93 =
18.89 =
1.08 =
Matlab function:
sigma.m (complex full-matrix )
ssv.m with perron option (complex )
ssv.m with muopt (real )
102
0
d2
0
0
..
.
0
0
..
.
dn
that (T) 1(DTD1), whence,

(T )
101
, D diagonal
inf 1 (DTD1 )
Usually, the upper bound is a fairly tight bound on the true

value of .
100
10-1
d1
0
D=
..
.
100
101
Hz
Computation of via LMIs. In practice, the foregoing optimization over D is usually reduced to an equivalent optimization
problem involving a linear matrix inequality (LMI), namely
max
Figure 6. Singular value Bode plots are widely used to evaluate robustness, but other less-conservative upper-bounds on (Ty1u1) are better when the uncertainty is structured. Shown here are Bode plots
of several upper-bounds computed using the Matlab Robust Control
Toolbox functions for a particular Ty1u1.
The bound is conservative in the sense that generally the

inequality is strict. In the special case in which the uncertainty is unstructured, however, then [Ty1u1( j)]
1]Ty1u1( j)]. An uncertain matrix is said to be unstructured
if it is a full (not a diagonal) matrix about which nothing else
is known (so the set D is the set of all complex matrices).
From the definition of , it follows that a necessary and
sufficient condition for the linear time-invariant in Fig. 5 to
be stable is given by
1 [( j)][Ty 1 u 1( j)] < 1 for all
If one defines 1 and as the gains of and Ty1u1, respectively, then this constitutes a sort of nonconservative version
of the loop-gain-less-than-one small gain. It is known informally as the small theorem. This interpretation of as a
nonconservative replacement for the singular value in applications of the small-gain theorem involving diagonally structured s is in fact the reason why is called the structured
singular value.
Conservativeness, Scaling, and Multipliers. As noted previously, the singular value 1(T) of a matrix T is generally
only a conservative upper bound on the desired quantity
(T). And in general this bound can be arbitrarily poor. For
example, if

0 1000
T=
0
0
subject to
X T X T 0
The greatest value of for which this optimization is feasible
yields (T) , and the optimal diagonal scaling D X1/2.
Solutions to LMI optimization problems are computed via
standard algorithms. An LMI-based D, G-scaling technique
extends the LMI concept to further tighten the upper bound
on when some of the uncertainties i are known to be real
constants. For example, the Matlab Robust Control Toolbox
function muopt.m computes this way.
Kharitonov Theorem
A system is stable if and only if all its poles have negative
real parts. The characteristic polynomial associated with a
transfer function Ty1u1(s) is any polynomial whose roots are its
poles. One very simple but useful test for robustness allows
evaluating the stability of a system whose characteristic polynomial has several uncertain real coefficients. Consider the
polynomial p(s) ansn an1sn1 a1 a0. Further,
suppose that each of the coefficients ai (i 1, . . ., n) is a real
number which varies in the range ai ai ai. The Kharitonov theorem (6) gives necessary and sufficient conditions for
all of the roots of the polynomial p(s) to be robustly stable
(i.e., have negative real parts) for all values of the coefficients
in the specified ranges. The Kharitonov theorem says that robust stability is assured if, and only if, four specific polynomials are stable. The four Kharitonov polynomials are obtained
by setting all of the even and all of the odd coefficients to
their minimal values and maximal values, respectively. Specifically, the four Kharitonov polynomials are
p1 (s) =
a i si +
then one may readily compute from the definitions that

1(T) 1000 and (T) 0. So, the ratio of 1 / can be very
large. It can even be infinite.
Diagonal scaling is one technique for reducing the conservativeness of the bound on provided by singular values.
In particular, it can be shown that, for any diagonal scaling
a i si +
ai si +
i odd
a i si
a i si
i even
i odd
p4 (s) =
i even
i odd
p3 (s) =
a i si
i even
i odd
p2 (s) =
ai si +
i even
a i si
ROBUST CONTROL
For example, if p(s) is a degree three polynomial with each

coefficient ai ranging between 3 and 5, then
Further suppose, for simplicity, that the following conditions

hold:
p2 (s) = 3s + 5s + 3s + 5
2
p3 (s) = 5s3 + 3s2 + 5s + 3

p4 (s) = 5s3 + 5s2 + 5s + 5
Robust Controller Synthesis. From the small theorem, it
follows that the canonical robust control problems are solved
if the controller K(s) stabilizes P(s) and if the resultant closedloop system Ty1u1(s) satisfies the condition [Ty1u1( j)] 1 for
all . Formally, this leads to the -synthesis optimization
opt
, K (s)min
sup [Ty u
stabilizing
1 1
( j)]
(7)
The previous -synthesis problem Eq. (7) constitutes a formal

mathematical formulation of the canonical robust control
problem. The canonical robust problem is solved if and only
if opt 1.
H Synthesis
Computing , a nonconvex optimization, is generally difficult.
The solution to the -synthesis problem Eq. (7) is generally
even harder. Therefore, in practice, engineers often choose to
minimize a conservative singular-value upper bound on ; see
Eq. (6). This leads to the problem
opt
, K (s)min
sup 1 [Ty u
stabilizing
1 1
( j)]
D12 D12 = I
D21 D21 = I
D11 = 0
D22 = 0
p1 (s) = 3s3 + 3s2 + 3s + 3

3
(8)
599
D12C1 = 0
B1 D21 = 0
(11)
The foregoing conditions are not very restrictive. Indeed, by a

suitable choice variables, these simplifying conditions can be
made to hold in nearly all cases [see (21)].
The formula for an H controller K(s) that solves the standard H control problem is most commonly expressed in
terms of stabilizing solutions P and Q of the following two
algebraic Riccati equations:
0 = PA + A P P(B2 B2 B1 B1 )P + C1 C1

and
0 = AQ + QA Q(C2 C2 C1 C1 )P + B1 B1
or, equivalently, in terms of the stable eigenspaces of two
Hamiltonian matrices

A
B2 B2 B1 B1
(12)
HP
C1C1
A

A
C2 C2 C1 C1
HQ
(13)
B1 B1
A
Provided that there exists a controller K(s) that solves the

standard H problem, one such K(s) is given in terms of solutions to P, Q of these two Riccati equations as
u2 = F (I QP)1 x
x = (A + QC1T C1 )x + B2 u2 + H(y2 C2 x)
or, equivalently,
opt
, K (s)min
Ty u
stabilizing
1 1
.
(9)
This is the H optimal control problem. It is closely related to

the standard H control problem which is to find, if it exists,
a controller K(s) for the system in Fig. 5 such that
Ty 1 u 1 < 1
(10)
Optimal H controllers possess the property that their closedloop singular-value frequency-response is completely flat, that
is
opt = Ty 1 u 1 = 1 [Ty 1 u 1 ( j)] for all
This flatness of optimal H controllers is called the all-pass
property, analogous to the flat response of all-pass filters that
arise in circuit theory and signal processing.
H Control Riccati Equations. Suppose that the plant P(s) in
Fig. 5 is given in state-space form as

x
A

y1 = C1
y2
C2
B1
D11
D21

B2
x

D12 u1
D22
u2
where F B2P and H QC2.

H Existence Conditions. The foregoing formula gives a solution to the standard H control problem Eq. (10), provided
that a solution exists. In general, no solution may exist, so it
is important to have precise and numerically testable mathematical existence conditions. The following are the necessary
and sufficient conditions for the existence of a solution:
1. Rank(D12) dim(u2) and rank(D21) dim(y2).
2. 1((D12
)D11) 1 and 1(D11(D21)) 1 where (D) denotes an orthogonal matrix whose column span is
null(D).
3. Neither of the Hamiltonians Eqs. (1213) has any
purely imaginary eigenvalues; that is, Imag[i(HP)] 0
i and Imag[i(HQ)] 0 i. (This is necessary and sufficient for the two Riccati equations to have stabilizing
solutions that is, solutions such that A (B1B1
B2B2)P and A Q(C1C1 C2C2) have only eigenvalues
with negative real parts.)
4. Both A B2F and A HC2 are stable (i.e., both have
all eigenvalues with strictly negative real parts). (Note:
In theory, this condition holds if and only if the Riccati
solutions P and Q are both positive-semidefinite matrices, but in practice semidefiniteness cannot be checked
reliably with a finite precision computation.)
600
ROBUST CONTROL
5. All of the eigenvalues of PQ have moduli less than one:

|i (PQ)| < 1
for all i.
Implicit in condition 4 above is the requirement that the plant

be both stabilizable and detectable, that is, the pair (A, B2)
must be stabilizable and the pair (C2, A) must be detectable.
If this is not so, then condition 4 cannot hold.
-Iteration. The solution to the H optimal control problem

Eq. (9) is computed via successive solutions to the standard
H problem Eq. (10) via a technique called -iteration. The
basic idea is to scale the plant matrix P(s) in Eq. (3) by multiplying its first row by a nonnegative scalar and then to test
the H existence conditions for various values of . The greatest value of , for which all five of the H existence conditions
hold, yields the minimal value of the H cost opt as opt 1/.
The H optimal controller is the solution to the corresponding
standard H problem.
-Synthesis
The problem of -synthesis involves computing a controller
K(s) that solves optimization Eq. (7). No such algorithm exists, but the combination of H optimal control together with
diagonal scaling is employed to produce good suboptimal solutions. The algorithm is called the DK-iteration, and it proceeds as follows:
Input: The plant P(s) shown in Fig. 5.
Initialize: Set D(s) I, K(s) 0, Ty1u1(s) P11(s).
Step 1. Replace the plant by the diagonally scaled plant

D
P(s)
0
0
I

P11
P21
P12
P22

D
0
1
0
I
(14)
Step 2. Compute the solution K(s) to the H optimization problem Eq. (9) for the diagonally scaled
plant Eq. (14) using the -iterative technique,
and set
Ty 1 u 1 = P11 + P12 K(I P11 K)2 P21
Step 3. At each frequency , compute the diagonal scaling D( j) that minimizes 1(DTy1u1D1).
Step 4. Using a curve-fitting method, find a polynomial
transfer function D(s) that approximates the
frequency response D( j) computed in Step 3.
Step 5. Go to Step 1.
There are no theoretical guarantees that this algorithm
converges to a joint minimum in K(s) and D(s). In fact, it may
not even find a local minimum. But, at least it improves the
controller, which is enough for it to be useful in some engineering applications. Refinements on this algorithm have recently appeared in which D, G-iteration handles real uncertainties with less conservatism. Also, a technique for
bypassing the curve fitting in Step 4 is available. The optimal
D(s) is a prespecified order computed via an LMI-related technique [see (22)].
Mixed Sensitivity and Loop Shaping

There is one H robust control problem of considerable practical significance, the weighted mixed-sensitivity problem in
which
Ty 1 u 1
W1 (s)S(s)
= W2 (s)K(s)S(s)
W3 (s)T (s)
Figure 7 shows how a conventional control system is augmented with weights W1, W2, W3 to produce an augmented
plant P(s) such that the closed-loop Ty1u1 has the indicated
form, namely,
W1 W1 G
0
W2
P(s) =
0 W3 G
G
I
Let us suppose (as is often the case in applications of the theory) that the control-signal weight W2 is absent, so that Ty1u1
reduces to
Ty 1 u 1

W1 (s)S(s)
=
W3 (s)T (s)
The all-pass property of H optimal controllers ensures that

opt = Ty 1 u 1 = 1 [Ty1 u1 ( j)] for all .
From this, combined with the singular-value property Eq. (5)
and the fundamental constraint S(s) T(s) I, it follows
that, to within a factor of 2,
opt max{1 [W1 ( j)S( j)], 1 [W3 ( j)T ( j)]}.
Whence, it follows from the properties of singular values that,
at frequencies where 1[S( j)] 1,
i [L( j)] n [L( j)]
W1 ( j)
1
1 [S( j)]
opt
and, at other frequencies where 1(T( j)) 1,

i [L( j)] 1 [L( j)] 1 [T ( j)]
opt
W3 ( j)
where L(s) G(s)K(s) is the loop transfer function matrix of

the control system in Fig. 7. The situation is depicted in Fig.
8. The key feature to notice is that, inside the control loop
bandwidth, the singular values i[L( j)] of the loop transfer
function are bounded below by W1( j)/ opt 1 and, outside
the control bandwidth, the singular values are bounded above
by opt /W3( j) 1. All of the loop transfer function singular
values cross over in the intermediate frequency range, as depicted in Fig. 7.
The implication is that the shapes of the Bode magnitude
plots of W1( j) and 1/W3( j) specify with considerable precision the actual shapes of the optimal singular-value plots of
ROBUST CONTROL
601
Augmented plant P (s)
G(s)
y1a
W2(s)
y1b
W3(s)
y1c
u2
y2
Controller
K(s)
the optimal loop-transfer function L( j) G( j)K( j). Thus,

the weights W1(s) and W3(s) are high-precision knobs for
shaping the loop-transfer functions singular values. The H
theory automatically ensures that the controller K(s) is stabilizing. Because of this, the weighted mixed-sensitivity H loop
shaping is a highly popular and useful method for multiloop
control design.
FURTHER READING
History. For pre-1900 control history, including an account
of the invention of the Watt flyball governor, see Ref. 1. For
an accurate account of the early developments in robust control theory, see Ref. 3 and the monograph Ref. 2. Reprints of
key journal papers are provided in the anthology (5). Reference 4 is an extensive bibliography on robust control.
dB
1(L)
1(T)
Performance
bound W1
0
1
1(S)
n(L)
y1
+
u1
W1(s)
1
W3
Robustness
bound
Figure 8. The mixed-sensitivity weights W1(s) and W3(s) provide engineers with the means to directly specify the desired shape of the
compensated loop transfer function L(s). The Bode plot of L(s) is sandwiched between the plot of W1(s) and the plot of 1/W3(s). Loop-shaping
is achieved by manipulating the weights W1(s) and W3(s).
Figure 7. In mixed-sensitivity H robust

control design, the plant model is augmented with weighting filters W1(s), W2(s)
and W3(s) that determine the shape of the
resulting closed-loop sensitivity and complementary sensitivity Bode plots.
Textbooks. Reference 6 describes the early 1960s work of

Horowitz on uncertainty tolerant design for single-loop feedback control systems. References 7 and 8 provide a good discussion of the most useful techniques of robust control theory
along with numerous design examples. More theoretical
treatments are in the texts (9), (11), and (12). Adaptive robust
control theory is treated in Ref. 10. A more specialized focus
on robustness analysis methods related to the Kharitonov
theory is in (13).
Software. Software for robustness analysis, H control design, -synthesis, and related topics is provided by the Matlab
toolboxes (14), (15), and (16). The users guides accompanying
these software products all contain extensive tutorial texts
covering the theory, its use, and numerous design examples.
Advanced Topics. The robustness implications of the gap
metric is examined in Ref. 19. Recently LMI-based methods
and have become increasingly important in robust control
theory, leading to significant reductions in conservatism and
extensions to difficult multi-objective problems, simultaneous
stabilization, and gain scheduling for slowly varying plants;
see Refs. 17 and 18. A key issue in applying robust control
theory is the question of how evolving experimental data are
incorporated to identify uncertainty sizes and to adaptively
enhance robustness. The latter is the focus of unfalsified control theory; see Ref. 20.
BIBLIOGRAPHY
1. O. Mayr, The Origins of Feedback Control, Cambridge, MA: MIT
Press, 1970.
2. M. G. Safonov, Stability and Robustness of Multivariable Feedback
Systems, Cambridge, MA: MIT Press, 1980. Based on the authors
PhD Thesis, Robustness and stability aspects of stochastic multivariable feedback system design, MIT, 1977.
602
3. M. G. Safonov and M. K. H. Fan, Editorial, Special issue on

multivariable stability margin, Int. J. Robust Nonlinear Control,
7: 97103, 1997.
4. P. Dorato, R. Tempo, and G. Muscato, Bibliography on robust
control, Automatica, 29: 201213, 1993.
5. P. Dorato, ed., Robust Control, New York: IEEE Press, 1987.
6. I. Horowitz, Synthesis of Feedback Systems, New York: Academic
Press, 1963.
7. J. M. Maciejowski, Multivariable Feedback Design, Reading, MA:
AddisonWesley, 1989.
8. S. Skogestad and I. Postlethwaite, Multivariable Feedback Control, New York: Wiley 1996.
10. P. Ioannou and J. Sun, Stable and Robust Adaptive Control, Englewood Cliffs, NJ: Prentice-Hall, 1994.
11. M. Green and D. J. N. Limebeer, Linear Robust Control, Englewood Cliffs, NJ: Prentice-Hall, 1995.
12. T. Basar and P. Bernhard, H-Optimal Control and Related Minimax Design Problems A Dynamical Game Approach, Boston, MA:
Birkhauser, 1995.
13. B. R. Barmish, New Tools for Robustness in Linear Systems, New
14. R. Y. Chiang and M. G. Safonov, Robust Control Toolbox, Mathworks, Natick, MA, 1988 (Ver. 2.0, 1992).
15. G. J. Balas et al., -Analysis and Synthesis Toolbox (-Tools),
Mathworks, Natick, MA, 1991.
16. P. Gahinet et al., LMI Control Toolbox, Mathworks, Natick, MA,
1995.
17. K. C. Goh, M. G. Safonov, and J. H. Ly, Robust synthesis via
bilinear matrix inequalities, Int. J. Robust Nonlinear Control, 6:
10791095, 1996.
18. S. Boyd et al., Linear Matrix Inequalities in Systems and Control
Theory, Philadelphia, PA: SIAM, 1994.
19. T. T. Georgiou and M. C. Smith, Optimal robustness in the gap
metric, IEEE Trans. Autom. Control, AC-35: 673686, 1990.
20. M. G. Safonov and T. C. Tsao, The unfalsified control concept
and learning, IEEE Trans. Autom. Control, 42: 843847, 1997.
21. M. G. Safonov, D. J. N. Limebeer, and R. Y. Chiang, Simplifying
the H theory via loop-shifting, matrix-pencil and descriptor concepts, Int. J. Control, 50 (6): 24672488, 1989.
22. M. G. Safonov and R. Y. Chiang, Real/complex Km-synthesis
without curve fitting, in C. Leondes, (ed.), Control and Dynamic
Systems, New York: Academic Press, 1993. Vol. 56 (Part 2), pp.
303324.
MICHAEL G. SAFONOV
University of Southern California

Robustness is a property inherently sought after in engineering systems. The concept is directly linked to such issues
as design viability and system reliability. In broad terms, robustness can be regarded as the capability to withstand unknown, unexpected, and often hostile conditions that can adversely affect a systems behavior. A system must be
sufficiently robust in order to function properly under undesirable circumstances, conducting its task as designed. As engineering systems are becoming more and more complex and
are required to operate in increasingly more uncertain environments, robustness has become increasingly more crucial.
Robust control can be generally defined as the control, by

means of fixed compensators, of uncertain plants (i.e., of systems with uncertain dynamics and unknown disturbance signals). Robustness in a control system refers to its ability to
cope with uncertainties in a satisfactory manner, maintaining
its stability and performance as desired. Uncertainty in signals and systems is inevitable, reflecting both the complexity
of the physical world and the limitation in human understanding. Uncertain signals typically arise as a result of the
randomness and unpredictability of environmental effects
and are of an unmeasurable and unpredictable nature. Uncertain system dynamics, on the other hand, can be attributed
to changes in the actual system and to modeling errors, be
they accidental or deliberate. Generally, uncertain dynamics
may come from the following sources:
1. Imperfect or incomplete knowledge of physical processes. This represents the information unattainable
because of ones limited knowledge or inadequate measurements. It can be particularly acute for complex systems and processes (e.g., those found in biomedical engineering).
2. Parameter variations. Every physical system will undergo a change in parameter values under different operating conditions. Aging itself can be a factor.
3. Neglected high-frequency dynamics, time delays, nonlinearities, and the like. It may occur as a result of a
sheer lack of knowledge, or the difficulty to model these
characteristics. It may also occur because of the desire
for models of low complexity.
While robustness is a concept of universal significance, robustness analysis for control systems is the study of whether
a system, however designed, can meet specified stability and
performance goals in the presence of uncertainty within a
prescribed range.
Uncertain plants can be modeled in various ways. In particular, models can be stochastic or purely deterministic. In
robust control, uncertain systems are typically modeled deterministically, as bounded sets of system models. A property is
then said to hold robustly if it holds for every model in the
set. The simplest case is that of unstructured uncertainty: the
model set consists of all systems in a certain neighborhood
(e.g., all transfer functions lying within a certain distance
of a distinguished nominal system). Such a description is
particularly appropriate to account for unmodeled dynamics.
One rather typical example is the modeling of flexible structures. It is well known that, in general, a flexible structure
cannot be accurately represented by a finite-dimensional system. For control design purposes, however, we desire, and
most often are compelled to find, an approximate finite-dimensional model with a relatively low order. In doing so, a
common practice is to include in the nominal model a small
number of dominant modes in the low-frequency range and
to treat the high-frequency modes as modeling uncertainty.
Evidently, this description is also appropriate for modeling
errors resulting from model reduction, or from any frequency
response truncation. Moreover, it can be used to cover, albeit
in a conservative way, parameter variations. The latter
amounts to drawing up a frequency response envelope to describe the range of parameter variation in frequency domain.
Finally, in robustness analysis, it is common to introduce a

fictitious uncertainty to represent a performance objective.
A more accurate account of parametric uncertainty calls
for model sets within which individual models are uniquely
characterized by the value(s) of one or more parameter(s)
(e.g., transfer function coefficients). Typically each such parameter takes values in a known range. Such models account
for the fact that parameters in physical devices are bound to
vary with time, environment, and operating conditions. Ackermanns car steering problem (1) offers a good illustrative
example. In his study of a four-wheel steering vehicle, he
found that the vehicle mass and the adhesion between tires
and road surface are significant uncertain parameters. This
is easily understandable. The vehicle mass certainly varies
with load, and in a more subtle sense, it varies with fuel consumption. The adhesion changes with road condition, wearing
of tires, and weather condition. We can think of other uncertain parameters by considering the human-vehicle system as
a whole or by considering a whole batch of vehicles as a family
of systems. In these scenarios, differences among individual
drivers and vehicles can all constitute significant uncertain
factors. Yet a more striking example is an aircraft, whose
aerodynamic coefficients vary in large magnitudes due to
changes in altitude, maneuvering, and weather.
There is an inherent trade-off between fidelity and simplicity in modeling uncertain systems. In a sense, unstructured
and parametric uncertainties may be considered the two extremes. While an unstructured perturbation furnishes a simple characterization and is useful for simplifying robustness
analysis, it contains little information and may often be too
conservative. On the other hand, uncertain parameters often
yield a more natural and accurate model, but such elaborate
descriptions tend to complicate analysis. The process of robustness analysis, therefore, calls for a judicious balance between uncertainty description and complexity of analysis. In
its full generality, however, the description of an uncertain
system should take into account both parametric variations
and unmodeled dynamics. Uncertainty descriptions of this
kind are called structured.
To be sure, robustness is not entirely a new concept in control system analysis and design. In retrospect, the need for a
control system to tolerate unmodeled dynamics and parameter variations is precisely what motivated feedback control,
and it has been a primary goal in control system design since
its birth. This is well recognized in classical control design
and, at least implicitly, is embedded in classical loop shaping
methods. Concepts such as gain and phase margins may well
be regarded as elementary robustness measures. However, it
was not until the late 1970s that the term began to appear
routinely in the control literature, when the need for robustness was reexamined and was gaining increasing recognition. Robust control as a research direction soon thrived and
became a defining theme. After two decades of intense activity, it has evolved into a broad research area rich in theory
and potential applications. The progress has been rapid and
vast, leading to the development of a variety of key concepts
and techniques, among which notably are the H / theory,
the Kharitonov/polynomial approach, and analyses based on
state-space formulations and the Lyapunov theory.
The structured singular value (2), also known as , was
introduced in the early 1980s as a very general framework
for studying structured uncertainty in linear time-invariant
603
models. It defines an exact measure of robust stability and

performance in the frequency domain. The method is a natural progression of earlier work on using singular values to
extend the concept of stability margin to multivariable systems, an idea that was heavily influenced by operator theoretic results such as the Small Gain Theorem. The main impetus for the development of the structured singular value
theory, evidently, has been the recognition that unstructured
uncertainty is too crude a model, often leading to excessive
conservatism.
From a computational point of view, the success of
hinges critically upon whether and how it can be computed
both accurately and efficiently. Unfortunately, this is known
to be difficult. Recent studies have shown that computation of
generally amounts to solving a so-called NP-hard decision
problem, which, in the language of computing theory, is one
that suffers from an exponential growth in its computational
complexity. Although this by no means implies that every
problem is computationally difficult, it nevertheless points to
the unfortunate conclusion that the computation of in general poses an insurmountable difficulty in the worst case. In
retrospect, it thus comes as no surprise that the major progress in computing has been made by some forms of approximation, specifically, readily computable bounds. While no definitive conclusion has been drawn concerning the gap
between and such bounds, it is reassuring that the bounds
are often reasonably tight and that they have other interpretations of engineering significance.
The Kharitonov theory is another robustness analysis
approach, developed in parallel with , which deals almost
entirely with robust stability issues under parametric uncertainty; only in rare cases is unstructured uncertainty also
taken into consideration. The research in this area has a natural heritage from such a classical stability test as the RouthHurwitz criterion and was mainly triggered by a landmark
result published by V. L. Kharitonov (3) in 1978, later referred to as Kharitonovs theorem. Kharitonov considered the
question of stability of parameterized families of polynomials. Here polynomials are thought of as characteristic polynomials of systems described by rational transfer functions and
thus are stable if their zeros all lie in the open left-half of
the complex plane (continuous time) or in the open unit disk
(discrete time). In the continuous-time case, Kharitonov
showed that, for an uncertain interval polynomial whose coefficients each vary independently in a given interval, stability of the entire family can be assessed by testing merely four
simply constructed extreme polynomials. From an aesthetic
point of view, Kharitonovs theorem possesses remarkable elegance, reducing an otherwise seemingly impossible task to a
simple problem. From an engineering perspective, however,
the theorem is likely to find only limited utility because very
rarely would an uncertain system yield a family of characteristic polynomials of the form required in the problem description. Thus, Kharitonovs work triggered a flurry of activities
in the search for more realistic results, and soon came various
generalizations. Two notable features stand out from the robust stability conditions available in this category. First, they
are stated either in terms of a finite number of polynomials
or as graphical tests requiring a frequency sweep. Second, the
main success to date pertains to uncertainty descriptions in
which polynomial coefficients depend linearly on uncertain
parameters.
604
Robust stability analysis in state-space formulation often

comes under the names of stability radius, interval matrix,
and stability bound problems. The key issue here is to determine the largest uncertainty size under which stability is preserved. Unlike in the aforementioned two approaches, parametric uncertainty in state-space representation is defined in
terms of perturbations to system matrices, which can be either unstructured or structured, but most often are only allowed to be real. In a natural way, the robust stability problem translates to one of how the perturbations may affect the
eigenvalues of the system matrix, the solution of which can
draw upon rich theories from linear algebra and Lyapunov
analysis. Thus, unsurprisingly, most of the robustness conditions have this flavor. Alternatively, the problem can also be
recast as one of analysis or one of polynomial stability. In
the former case, we need to compute with respect to solely
real uncertainties, for which the computation schemes are
known to be conservative. In the latter case, the coefficients
in the resultant polynomial will depend on the unknown parameters in a multilinear or multinomial fashion. The Kharitonov approach cannot provide a satisfactory answer for such
polynomial families either. The problem thus appears to be a
very difficult one and is known to have been solved only for a
number of isolated cases. Most notably, recent progress has
made it possible to compute the stability radius in an efficient
manner when the matrix perturbation is unstructured. For
structured uncertainty, unfortunately, the problem is completely open; recent studies showed too that it is in general
NP-hard. Accordingly, the majority of results in the latter
class are sufficient conditions for robust stability.
In conclusion, robustness is a key concept, vital for engineering design in general and for control system design in
particular. Robust control has matured into a field rich in theory and potential applications. By focusing on these three selected areas, this chapter is limited only to robustness analysis of linear time-invariant systems, that is, control of linear
time-invariant plants by linear time-invariant controllers,
which itself is condensed from a vast collection of results and
techniques. Nevertheless, we should note that the concept
and theory of robust control goes far beyond the boundary of
linear time-invariant systems and, in fact, has been quickly
branching to the domains of nonlinear control, adaptive control, and the like. As a whole, it has been, and will continue
to be, a driving force behind the evolution of control theory.
THE STRUCTURED SINGULAR VALUE
Thus each plant in the model set corresponds to the selection of one element in each of the uncertain blocks. The nominal plant corresponds to the choice 0 for all elementary uncertain blocks. Note that the assumption that all the
uncertainty balls have unity radius is made at no cost: any
size information can be embedded into known blocks (e.g.,
connected in cascade with the uncertain blocks). It turns out
that, for uncertain block diagrams of this type, the transfer
function (or transfer function matrix) between any two nodes
(or tuples of nodes) in the block diagram is given by a linear
fractional transformation (LFT). Similarly, when such an uncertain plant is connected with a feedback compensator, the
transfer function between any two nodes will also be an LFT.
LFTs have attractive mathematical properties that can be
used to advantage at the modeling, analysis, and synthesis
stages. More on LFTs can be found, e.g., in Ref. 7.
Robust Stability
It should be intuitively clear that block diagrams of the type
just described can always be redrawn in the form of an
M loop as depicted in Fig. 1 (external input and outputs
have been left out). Here M corresponds to the nominal system, which is comprised of closed-loop transfer functions as
elements and has an input channel and an output channel for
each elementary uncertain block, and is a block-diagonal
matrix whose diagonal blocks are the elementary uncertain
blocks. For its generality, the M loop paradigm has found
wide acceptance in robust control (see, for example, Refs. 2
and 48).
Throughout most of this article, we will assume that the
nominal system, or equivalently M, is linear and time-invariant, as are all instances of the uncertainty blocks, equivalently, of . We will also assume that M and all instances of
are in H . In this case, an immediate payoff of the LFT
uncertainty description and the ensuing representation of the
system via the M loop is the following strong form of the
Small Gain Theorem, a necessary and sufficient condition for
well-posedness and stability of the M loop, in the case
where consists of a single uncertainty block, ranging over
the unit ball in H .
Small Gain Theorem. Let M H . Then the M loop is
well-posed and BIBO stable for all H with 1 if
and only if M 1.
As alluded to earlier, the H norm of a causal, stable, continuous-time transfer function matrix M is defined as
M = sup (M( j))
Uncertain Systems
Throughout this article, we consider model sets P with the
following property: P can be represented by a block diagram
with some of the blocks being fully known systems, and others
being elementary uncertain blocks. The latter are elementary sets, namely, unit balls in simple vector spaces. For
example, some uncertainty blocks might be the real interval
[1, 1] and others might be the unit ball of H , the set of
transfer function matrices (linear time-invariant systems)
that are causal and bounded-input bounded-output (BIBO)
stable; the size or H -norm of H is defined to be the
supremum of its largest singular value over the imaginary
axis (continuous time) or unit disk (discrete time).
where denotes the largest singular value.
Figure 1. M loop.
As an example consider a model set of the output multiplicative uncertainty type. Specifically, let
namics, consider now the model set (1 w1c)Pa with Pa explicitly given as
P = (I + w)P0
where P0 is the transfer matrix of a linear, time-invariant
nominal model, w is a scalar weighting transfer function,
and ranges over the unit ball in H . The weight w is introduced to account for the fact that the amount of uncertainty
is usually frequency-dependent; in particular, system dynamics are often poorly known at high frequencies. Suppose that
a feedback controller K has been tentatively selected to stabilize the nominal system P0. (We use the negative feedback
convention, i.e., the loop transfer function is KP0.) Isolating
from the nominal closed-loop system, we obtain an M
loop with
M = wKP0 (I + KP0 )1
Since K stabilizes P0, M is stable.
As a note of interest, we expect a keen connection between
the Small Gain Theorem and the classical Nyquist criterion.
Indeed, this can be best observed by examining single-input/
single-output systems. In such case P and K are scalar, and
thus so is M. Since both M and are stable, Nyquists criterion implies that the M loop is stable whenever the Nyquist plot of M does not encircle the critical point 1 j0.
Clearly, this will be the case for every satisfying ( jw) 1
for all w if and only if M( jw) 1 holds at all frequencies
(including ).
The Small Gain Theorem suggests that one way to obtain
a robustly stable system, or more generally to obtain a robust
design, is to make sure that the H norm of a certain system
transfer function is small enough. This has triggered an entire field of research known as H design, which is discussed
elsewhere in this encyclopedia. The focus of the present article is the case when is block-diagonal (i.e., when the uncertainty model consists of several blocks or, in other words,
when the uncertainty is structured). Typically, two types of
uncertainty blocks are considered in the literature: (i) the set
of real, constant scalar multiples of the identity, with the scalar having magnitude no larger than one, and (ii) the set of
causal and BIBO stable (H ) transfer function matrices, with
H -norm no larger than one. The latter corresponds to unmodeled dynamics. The former, on the other hand, is used to
represent parametric uncertainty, particularly when a same
uncertain parameter affects more than one coefficients in a
transfer function. For example, concurrent variation as a
function of temperature (e.g., dilation) of multiple quantities
in a mechanical system can result in such a block. This description is more general than the simpler scalar nonrepeated blocks.
Examples with structured uncertainty arise with plants
modeled as being affected by uncertainty at more than one
physical location, e.g.,
P = (I + w1 1 )P0 (I + w2 2 )
where both input and output uncertainty are accounted for.
Another instance arises in the context of the robust performance problem, discussed in a later section. For an example
including both parametric uncertainty and unmodeled dy-
605
Pa (s) =
1
sa
where a can take any value in the interval [0.9,1.1]. We may

write a 1 w2r with w2 0.1 and r 1, and Pa can be
represented as a loop with P0(s) 1/(s 1) in the forward
path and w2r in the feedback path (again using the negative feedback convention). Let K be a feedback controller (still
with the negative feedback convention) that stabilizes P0. By
extracting the uncertainty blocks c and r, we obtain an
M loop with
w2 P0 (I + P0 K)1
M=
w1 P0 (I + P0 K)1

r 0
=
0 c
w2 KP0 (I + P0 K)1
w1 KP0 (I + P0 K)1
where r is a real number ranging over [1,1] and c is a

scalar transfer function ranging over the unit ball in H .
Clearly, the condition M 1 in the Small Gain Theorem
remains sufficient for robust stability when is restricted to
be block-diagonal. However, it is in general no longer necessary. A refinement of the Small Gain Theorem for the structured uncertainty case was proposed by Doyle and Safonov in
the early 1980s (2,4). We adopt here the framework introduced by Doyle, that of the structured singular value also
known as . The Small Theorem states that, if the uncertainty is restricted to be block-diagonal, then the correct refinement is essentially (see Ref. 9 for a precise statement) to
replace M with M, where for a continuous-time transfer
function matrix M,
M = sup (M( j))
and ( ) denotes the structured singular value of its matrix

argument with respect to the block-structure under consideration. The set e is the extended real line; if no
parametric uncertainty is present, however, ( ) is continuous and e can be replaced by . Similarly, for a discrete-time
system,
M = sup (M(e j ))
[0,2 )
But what specifically is ? This is discussed next.

The Structured Singular Value
Let us denote by the set of values taken by ( j) [or (ej)]
as ranges over the set of block diagonal transfer function
matrices of interest, with the unit ball restriction lifted,
namely let
= {diag( r , c , C ) : r r , c c , C C }
606
with
tightened to the following (12):
max R (MQ) (M)
r := {diag(1r Ik , . . ., mr r Ik m ) : ir R}
1
c :=
C :=
C
{diag(1C , . . ., m
)
C
ic
ic
C}
{ 0 : MDM H + GM H MG 2 D < 0}
k m +m +i k m +m +i
r
c
r
c
}
The first and third block types, often referred to as repeated

real and full complex blocks, correspond to values of parametric and dynamic uncertainty, respectively. The second block
type, known as repeated complex, often arises in analyzing
multidimensional (10) and time-delay systems (11), and is
also used sometimes when an LFT state-space representation
of transfer functions is sought (5). It is worth noting that
while as just defined is usually adequate for representing
uncertainties frequently encountered, it can be extended further to accommodate more general situations. For example,
full real blocks (i.e., unknown real matrices) may be added
whenever desired.
The structured singular value (M) of a matrix M with respect to the block structure is defined to be 0 if there is no
such that det(I M) 0, and
;
inf
DD + ,GG
QQ
{diag(1c Ik
, . . ., mc c Ik m +m c )
m r +1
r
(M) = min{ () : det(I M) = 0}
1
otherwise. It can be checked that for structures simply consisting of one full complex block as in the Small Gain Theorem, (M(j) becomes the largest singular value of M(j),
and M is thus equal to M.
Given a matrix M and a block structure , computing
(M) is generally not an easy task. Indeed, this computation
is known to be NP-hard, even when is simplified to a structure containing only full complex blocks. Thus estimates of
(M), e.g., upper and lower bounds on (M), are often used
instead. These, as well as other properties of , are discussed next.
Let U be the set of unitary matrices in and D be the set
of nonsingular matrices that commute with every . The
latter consist of block-diagonal matrices with scalar multiples
of the identity in correspondence with full complex blocks
(C), and with arbitrary blocks in correspondence with those
constrained to be scalar multiples of the identity (r, c). Then
the following result holds:
Here the superscript H indicates complex conjugate transpose, Q is the subset of consisting of matrices whose complex blocks are unitary, D is the subset of D consisting of
Hermitian positive definite matrices, G is the subset of D
consisting of skew-Hermitian matrices (i.e., GH G) with
zero blocks in correspondence with repeated real blocks in ,
and the sign indicates that the matrix expression is constrained to be negative definite. The lower bound in condition
(2) is always equal to (M). The upper bound is never greater
than that in condition (1) (it reduces to it when G 0 is imposed) and, as was the case for condition (1), can be computed
by solving an LMI problem. See, for example, Section 8.12 in
Ref. 8.
For the class of problems where the matrix M has rank
one, Young (13) showed that the right-hand side in inequalities [Eq. (2)] is equal to (M). In that case, Chen et al. (14)
obtained an explicit formula for (M). Let M baH, where a
and b are column vectors. Let also a and b be partitioned into
subvectors ai and bi compatibly with . For i 1, . . ., mr
mc, let i aiHbi. Moreover, define
m
r +m c
|i | +
i=m r +1
Then,
(M) = inf
x

m
ai 2 bi 2
|Re(i ) + x Im(i )| +
1+x
!
2
(3)
Furthermore, if we assume with no loss of generality that for

some l mr, Im(i) 0 for 1 i l, and Im(i) 0 for i
l, and that
Re(l )
Re(1 )

Im(1 )
Im(l )
then the infimum is achieved at one of the following points:
Re(k )
,
Im(k )
l
k = 1, . . ., l
Here R is the largest absolute value of a real eigenvalue of

its matrix argument. Inequalities [see Eq. (1)] are of special
interest in the case of purely complex uncertainty structures.
In that case, (i) the lower bound is equal to (M) and R can
be replaced by the spectral radius , and (ii) the upper bound
is equal to (M) whenever mC 2mc is no greater than 3, and
extensive numerical experimentation suggests that it is never
(or at least seldom) much larger. Moreover, the upper bound
can be computed efficiently by solving a convex optimization
problem, in fact, a linear matrix inequality (LMI) problem.
LMIs define a special class of convex optimization problems
and are discussed elsewhere in this encyclopedia.
For uncertainty structures where real (scalar multiple of
the identity) blocks are present, inequalities [Eq. (1)] can be
x 0 =
;
x k =
DD
i=1
xk =
U U
m r +m c +m C
i=m r +m c +1
(1)
max R (MU ) (M) inf (DMD1 )
(2)
i=1
|Im(i )|
l
i=1
|Im(i )|
k
l
i=1 |Im(i )|
i=k+1 |Im(i )|
l

k
2
i=1 |Im(i )|
i=k+1 |Im(i )|
Finally, the infimum cannot be achieved at x0 unless x0

(, x1], and for k 1, . . ., l, it cannot be achieved at xk
unless xk [xk, xk1].
The rank-one case just alluded to is one of the rare instances for which one can obtain an explicit expression for
(M). This expression not only simplifies the computation of
the upper bound in condition (2) but also was found useful in
studying robust stability of uncertain polynomials. Indeed, as
blocks p(s) such that p 1. It thus follows that robust

performance holds if and only if
1
M
607
sup (M( j)) < 1
Figure 2. Robust performance setup.
will be discovered shortly, an important class of stability

problems for uncertain polynomials can be formulated in
terms of a rank-one problem. Consequently, the result furnishes a unifying tool for the stability problems and a link
between analysis and the Kharitonov approach to robustness analysis.
Robust Performance
A key reason for the popularity of the framework is that it
encompasses not only the robust stability problem but also
the following robust performance problem: determine
whether, for all plants in the given model set, the energy (integral of the square of the magnitude) in a specified error output signal remains below a specified threshold whenever the
disturbance inputs energy is less than a specified value.
Consider the block diagram of Fig. 2 where, as compared
to Fig. 1, external (disturbance) input and (error) output are
made explicit. Given a block diagram such as the one of Fig.
2, the input-output transfer function in the continuous-time
case is given by the linear-fractional transformation
F (M(s), (s)) = M11 (s) + M12 (s)(s)(I M22 (s)(s))1M21 (s)
where Mij(s) is the transfer function from input j to output i
of M(s), i,j 1,2, when the feedback connection through (s)
is removed. [Thus M22(s) is the transfer function matrix formerly denoted M(s).]
The issue at hand is to determine, under the assumption
that M(s) H , whether robust performance holds, that is,
whether it is the case that, for all (s) in our unit uncertainty
ball,
F (M(s), (s)) < 1
(4)
This is readily handled by noting that, in view of the Small

Gain Theorem, for any fixed (s) such that the system is stable, condition (4) is equivalent to the stability of the augmented system depicted in Fig. 3 for all fictitious uncertainty
1
M
Figure 3. Fictitious uncertainty for robust performance.
where now denotes the structured singular value corresponding to the augmented block structure diag(kpkp, )
(i.e., the block-structure corresponding to the actual uncertainty, augmented with a full complex block).
For an example of a typical robust performance problem,
consider an uncertain plant described by the multiplicative
uncertainty model set
P = (I + w1 )P0
with a fixed feedback controller K. It is desired to determine
whether w2S 1 for all possible in a possibly structured
unit uncertainty ball, where S is the sensitivity function (i.e.,
using the negative feedback convention, S (I PK)1).
Here w1 and w2 are stable transfer functions introduced for
frequency-weighting purposes. For simplicity, w1 and w2 are
assumed to be scalars. Using the transformation just outlined, we obtain
w2 (I + P0 K)1
M=
w1 (I + KP0 )1 K
w2 (I + P0 K)1 P0
w1 (I + KP0 )1 KP0

(5)
In the single-input/single-output case, M has rank one. In the

present case mr mc 0, so the right-hand side of Eq. (3) is
simply and the right-hand side of the expression defining
reduces to its second term. Thus
(M( j)) = |w2 ( j)(1 + P0 ( j)K( j))1 |

+ |w1 ( j)(1 + K( j)P0 ( j))1 K( j)P0 ( j)|
and the condition for robust performance can be expressed as
|w2 ( j)| + |w1 ( j)K( j)P0 ( j)| < |1 + P0 ( j)K( j)| Re
Extensions
The structured singular value may be further generalized in
many directions, depending on uncertainty descriptions and
characterizations. Some of these generalizations are summarized next.
Nondiagonal uncertainty structure. The uncertainty
structure need not be diagonal. It can contain unknown,
independently bounded blocks in every entry. Doyle (2)
and Kouvaritakis and Latchman (15) showed that the
analysis may be converted into one based on the standard , but this may lead to a substantial increase in
computational effort. Chen et al. (16,17) proposed a computational scheme that renders the computation growth
insignificant.
Uncertainty with phase information. Tits et al. (18)
adopted a notion of with phase, in which not only are
uncertainties known to be bounded by given quantities,
but also their phases are known to vary in given ranges.
The formulation gives a more detailed uncertainty de-
608
scription, and it requires extensions of the concept of

phase and of .
1-norm bounded uncertainty. Khammash and Pearson
(19,20) studied structured uncertainties bounded in 1
norm, which is another active research area in robust
control, concerning peak-to-peak system response. They
showed that robust stability can be assessed by computing the spectral radius of a positive matrix constructed
from the impulse response of the nominal system.
Time-varying uncertainty. Shamma (21) and Megretsky
(22) examined the robust stability problem with respect
to structured time-varying uncertainties. They showed
that if the uncertainty is allowed to vary arbitrarily over
time, robust stability holds if and only if for some D
D , DM(s)D 1. It is readily checked that the lefthand side (known as scaled H -norm), is similar to the
right-hand side in condition (1), except that here the
same D must be used at all frequencies. Subsequently,
Poolla and Tikku (23) showed that, if the time variation
of the uncertainty is arbitrarily slow, then robust stability holds if and only if the right-hand side in condition
(1) is less than 1 at all frequencies.
Finally, while may be custom made and seems to be an
all-encompassing paradigm when extended appropriately, it
cannot be applied to models in which the uncertainty block
is allowed to be unstable. An effective robustness measure for
the latter situation is furnished by the gap metric, a concept
discussed elsewhere in this encyclopedia.
where q is an unknown vector that may or may not represent

physical parameters. When q varies over a bounded set Q
m, a family of polynomials are generated:
P = { p(s, q) : q Q}
(7)
The problem of concern is to determine if the polynomial family P is robustly Hurwitz stable, by which we mean that every member in P is Hurwitz stable. We shall assume that the
coefficients ak(q) are continuous functions of q. Furthemore,
we assume that an(q) 0 for all q Q (i.e., all polynomials
in P have the same degree). For control system analysis, it is
typical to restrict the polynomial family P to the following
classes, arranged by order of increased complexity.
1. P a: the coefficients ak(q) are affine functions of q. For
example,
p(s, q) = s2 + (q1 + 2q2 + 3)s + (4q1 + 5q2 + 6)
2. P m: the coefficients ak(q) are multiaffine functions of q.
For example,
p(s, q) = s3 + (2q1q2 + 2q1 q3 + q3 + 1)s2
+ (4q2q3 + 5)s + (q1 q2 q3 + 1)
3. P p: the coefficients ak(q) are multivariate polynomials
in q. For example,
p(s, q) = s3 + (2q21 q2 + 2q1 q23 + q1 q3 + 1)s2

+ (4q2 q3 + 5)s + (q21 q22 q23 + 1)
THE KHARITONOV APPROACH

The Kharitonov approach, named after Russian mathematician V. L. Kharitonov whose celebrated 1978 theorem is often
considered to be the cornerstone of the field, is largely concerned with the issue of determining zero locations for a family of polynomials whose coefficients vary in a bounded set.
Hence, by nature, it can be best presented in a framework
different from that of the M loop, or , namely, directly as
a polynomial stability problem. This issue, however, is connected to the M loop paradigm in an intimate fashion. To
see this, simply consider a model set comprising proper real
rational functions whose coefficients take values in certain
bounded intervals. To determine robust stability of such a
plant together with any compensator will then amount to
checking whether the set of all resultant closed-loop characteristic polynomials have zeros in the stability region. For
continuous-time systems, our main focus, the stability region
of interest, is the open left half of the complex plane. Other
regions of interest include the open unit disk, a shifted left
half plane, and a sector; these regions can be imposed to
study stability of discrete-time systems or to enforce pole
placement constraints. A polynomial is generally said to be
Hurwitz stable, or is referred to as a Hurwitz polynomial, if
its zeros lie in the open left half plane.
A general description for a set of polynomials of interest is
p(s, q) =
n

k=0
ak (q)sk ,
(6)
It should be rather evident that P a P m P p and hence

that the complexity in analysis increases in that same order.
At present, the only available methods for tackling P m and
P p are largely ad hoc, via either local optimization or graphical approaches, and they are either conservative or computationally formidable. In particular, when Q is an ball [i.e., a
hyperrectangle (box) parallel to the coordinate axes], the
problem of testing the stability of P m is known to be NP-hard.
The class P a, as the sole tractable case, merits a particularly thorough study. A polynomial family P in this class consists of all polynomials of the form
p(s, q) = p(s, q0 ) +
(qk q0k )pk (s)
(8)
k=0
Here q0 belongs to Q and may be regarded as the nominal

value of uncertain parameter vector q, and the pk(s)s are fixed
polynomials. Evidently, one can assume with no loss of generality that p(s, q0) is Hurwitz stable, which is necessary for P
to be robustly Hurwitz stable as q varies over Q.
Let p [1, ], and let p be the standard p Holder norm
defined on the Euclidean space m. That is,

!1/ p
m

|qi |
, 1 p<
q p =
i=1
max |qi |,
p=
1im
Then a common description for Q adopts the notion of unit

p balls centered at q0, defined as
Q = {q : q q0 p 1}
(9)
When the coefficient ak(q) depends on qk alone [i.e., when

pk(s) is a constant multiple of sk], the p ball description gives
rise to a class of most studied polynomial families. Such families can be expressed as

n

k
0
P =
k qk s : q q p 1
(10)
k=0
for some (possibly different) q0 and some fixed scalars k. In

particular, for p , 1, 2, such P is referred to as an interval
polynomial, a diamond polynomial, and a spherical polynomial, respectively.
Arguably, the entire success of the Kharitonov approach
may be best summarized as a triumph over the polynomial
family in Eq. (8) with the p norm characterization in Eq. (9),
but not beyond, of which the most shining example is Kharitonovs celebrated theorem.
Interval Polynomials
An interval polynomial can be equivalently expressed as

n

k
(11)
P =
q k s : q k q k qk
k=0
where the qks and the qks are fixed. Kharitonovs original
treatment of the interval polynomial problem is of an algebraic nature. He constructed four extremal members of P ,
K1 (s) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 +
K2 (s) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 +
K3 (s) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 +
K4 (s) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 +
later dubbed Kharitonov polynomials. In a remarkable fashion, Kharitonov showed that the stability of the entire interval polynomial family can be ascertained by testing merely
these four.
Kharitonovs Theorem. The interval polynomial is Hurwitz
stable if and only if K1(s), K2(s), K3(s), and K4(s) are all Hurwitz stable.
Subsequent development showed that for polynomials of degree 5, 4, and 3, the test can be further simplified, requiring
checking only 3, 2, and 1 of the four Kharitonov polynomials,
respectively.
Dasgupta (24) gave a geometrical interpretation to Kharitonovs Theorem in frequency domain, which sheds light on
why such a puzzling result would hold. The interpretation
makes use of the concept of value set. The idea is to examine
the image of the polynomial family when s lies on the boundary of the stability region, namely
p( j, Q) = {p( j, q) : q Q}
609
Because p(s, q0) is stable and by assumption an(q) never vanishes on Q, we can conclude from the continuity properties of
the zeros of polynomials that if some member in P is not
Hurwitz stable, then some polynomial in P must have a zero
on the imaginary axis. Thus the entire family P is Hurwitz
stable if and only if
0
/ p( j, Q),
that is, the value set never contains the origin. This is a
rather straightforward fact known as early as in 1929 and is
generally referred to in the literature as the zero exclusion
principle. Note, in particular, that for a polynomial p(s, q) in
the family described by Eq. (11), for any q Q, the real and
imaginary parts of p( j, q) are respectively given by
Re p( j, q) = q0 q2 2 + q4 4
and
Im p( j, q) = j(q1 q3 2 + q5 4 )
Thus the real part (resp. imaginary part) of p( j, q) depends
only on the parameters with even (resp. odd) subscript. For
an interval polynomial, therefore, it becomes clear that
Re K1 ( j) = Re K2 ( j) Re p( j, q) Re K3 ( j) = Re K4 ( j)
Im K1 ( j) = Im K4 ( j) Im p( j, q) Im K3 ( j) = Im K2 ( j)
Because the four Kharitonov polynomials are themselves in
P , it follows that, at each , the value set is a rectangle in
the complex plane with vertices Ki( j), as depicted in Fig. 4.
As increases, the rectangle evolves in a continuous fashion. Its dimensions vary, but its edges remain parallel to the
axes, and the relative positions of the Ki( j)s do not change.
Now, a polynomial p(s) of degree n with positive coefficients
is Hurwitz stable if and only if the phase of p( j) monotonically increases from 0 to n/2 as goes from 0 to . This is
best seen by noting that p(s) can always be factorized as
p(s) an(s si) in terms of its zeros si, and by considering
the phase of each factor separately. When this is understood,
Kharitonovs Theorem becomes self-evident. Indeed, if the
Kis are Hurwitz stable, then the phase of each Ki( j) increases monotonically from 0 to n/2 and the entire Kharitonov rectangle rotates around the origin by a total angle of
n/2 without ever touching it. Stability of P then follows from
the zero exclusion principle.
Kharitonovs Theorem opened an era epitomized by the
search for so-called vertex results in robust stability analysis,
as manifested by the construction of the four Kharitonov polynomials at the vertices of the hyperrectangle Q. This theme
Im
K2(j)
K3(j)
K1(j)
K4(j)
Re
Figure 4. Kharitonov rectangle.
610
persisted in much of the subsequent work. Although Kharitonovs Theorem is without question a milestone in control theory, and perhaps a measured intellectual triumph in general,
we should note that the interval polynomial family is nevertheless a very special instance of an affine uncertainty class,
and hence the theorem has a rather limited scope in application. The quest thus continues.
Edge Theorem
In light of Kharitonovs Theorem, it is tempting to contemplate the possibility that a similar vertex result would hold
for the general affine polynomial family with Q a hyperrectangle. This, unfortunately, turns out to be false; a counterexample can be readily constructed to demonstrate the opposite.
What can be said about a polynomial family P in class P a?
Bartlett et al. (25) provided an answer. In what is now known
as the Edge Theorem, they took the bounding set Q to be a
convex polytope in m. Let qi, i 1, . . ., l, be the vertices of
Q. Then, it is well known that Q can be represented as a
convex hull of the vertices. That is,

Q = conv{q1 , . . ., ql } =
l

i=1
i q i :

i = 1, i 0
i=0
Because ak(q) is an affine function of q, it follows that

P = conv{ p(s, q1 ), . . ., p(s, q l )}
This implies that the value set p( j, Q) is a polygon, generated by the vertex polynomials p(s, qi). It should be rather
clear that the interval polynomial family is generated by a
polytopea hyperrectangleand so is the diamond polynomial family.
Edge Theorem. The affine polynomial family (polytope of
polynomials) P a is Hurwitz stable if and only if for each edge
point q of Q, p(s, q) is Hurwitz stable.
Let qi and qj be two vertices of Q connected by an edge. When
varies from 0 to 1, the polynomial
pij (s, ) = p(s, q i ) + (1 )p(s, q j )
defines a line segment (of polynomials) connecting p(s, qi) and
p(s, qj). The theorem shows thatquoting directly from Ref.
25it suffices to check the edges. Because an edge polynomial
involves only one parameter, its stability can be readily
tested, by resorting to either a graphical test based upon a
root locus, or the numerical solution of a generalized eigenvalue problem (see, for example, Chapter 4 in Ref. 26).
The heuristics behind the Edge Theorem are simple. Because for large values of , the value set does not contain the
origin (this is easy to see), the polynomial family will be Hurwitz stable if and only if there is no frequency at which the
origin belongs to the boundary of the (polygonal) value set. It
should be intuitively clear that every point on the boundary
of the value set must be the image of a point on an edge of
the polytope of polynomials. Thus, if for some the origin
does belong to this boundary, the corresponding edge polynomial must be unstable. Note that it is wasteful to check the
entire set of all polynomial segments joining two vertices of
the polytope because some of these segments are not edges.

In some cases, however, it is not an easy task to check which
such segments are edges and which are not.
The Edge Theorem generalizes Kharitonovs Theorem in
two important aspects. First, it is applicable to general affine
polynomial families, with Q an arbitrary polytope. Secondly,
the stability region may be any open, simply connected set
(27,28), although only stated here in terms of the open left
half plane. In contrast, Kharitonovs Theorem addresses only
Hurwitz stability. On the other hand, as the number of parameters increases, the computation effort required in the
Edge Theorem can be enormous. For example, when Q is a
box in m, with 2m vertices, the number of edges is m2m1, a
very large number even for moderate values of m. Finally, to
a lesser extent, the Edge Theorem is not applicable to situations where the boundary of Q is curved, of which the spherical polynomial family is an example.
Graphical Tests
The complexity of the Edge Theorem provides a direct motivation to search for computationally more tractable stability criteria, and graphical conditions become a natural avenue to
explore. Not only are graphical criteria time-honored tools in
classical stability analysis, but the zero exclusion principle,
with all its simplicity and transparency, also prompts a
deeper investigation of such tools. We can generally feel that
an important asset of the zero exclusion principle is its generality, both in terms of uncertain polynomial families and in
terms of stability regions.
Barmish (29) studied the issue systematically; earlier conditions had appeared sporadically on stability of perturbed
polynomials in isolated cases. Barmishs approach stems from
a geometrical argument: a convex polygon in the complex
plane does not intersect the origin as long as it can be separated from it by a straight line or, equivalently, as long as the
vertices can be separated, as a whole, from the origin by such
a line. This observation led to his construction of a testing
function, which is to be evaluated along the boundary of the
stability region. After this is accomplished, we can determine
stability by plotting the testing function. Barmishs test is certainly one step forward compared to a pure brute-force computation; however, it remains somewhat ad hoc and is computationally overwhelming. Because it requires evaluations at
all vertices, it does not clear the hurdle of exponential growth
in complexity.
On a lesser scale, Tsypkin and Polyak (30) obtained a
graphical test for a simpler problem. They examined the polynomial family in Eq. (10). Let p [1, ] be given. Furthermore, for r [1, ] such that (1/p) (1/r) 1, define
Xr () = (0r + (2 2 )r + (4 4 )r + )1/r
Yr () = (1r + (3 2 )r + (5 4 )r + )1/r
R() = 0 q00 2 q02 2 + 4 q04 4 +
I() = 1 q01 3 q03 2 + 5 q05 4 +
Note that R() Re(p( j, q0)) and I() Im(p( j, q0)).
Tsypkin-Polyak Criterion. The polynomial family P of Eq.
(10) is Hurwitz stable if and only if p(s, q0) is Hurwitz stable,
qn0 1, q00 1, and
|R()| p |I()| p
Xr ()
Yr ()
> 1,
(0, )
(12)
This condition was independently obtained by Hinrichsen and

Pritchard (31).
Up to this point, we could assert that the stability problem
for affine polynomial families remains largely unresolved.
However, as yet another observation, we find that at each ,
the zero exclusion condition defines two linear constraints in
terms of perturbed coefficients, imposed on the real and imaginary parts of p( j, q), respectively. These constraints, together with a convex bounding set Q, define in turn a convex
feasibility condition; when the parameters vary independently, it reduces further to a linear program. This is a simple
but conceptually appealing observation. It led to a reformulation via linear programming, due to Saridereli and Kern (32)
and to Tesi and Vicino (33), which can be solved readily for
each and then plotted graphically. Qiu and Davison (34)
went further to demonstrate that for very general bounding
sets it suffices to solve an optimization problem with one variable only, and the problem can be solved explicitly in special
cases. Finally, Chen et al. (14,35) recognized that the problem
can be reformulated as a special rank-one problem for each
, and showed that stability can be ascertained by evaluating
an explicit formula. These results led to the final resolution
of the affine polynomial family.
and the Kharitonov Approach

Indeed, there is an inherent linkage between analysis and
Kharitonov approach, whenever the latter applies, in that
both approaches yield necessary and sufficient conditions for
problems of the same nature. However, for a rather long time
a clear link seemed elusive. The main cause, it seems, lay in
how to reconcile an optimization-based formulation such as ,
and explicit results from the Kharitonov approach. Can one,
for example, derive Kharitonovs theorem from , or vice
versa?
The explicit formula of rank-one given earlier lends an
answer. Specifically, for a general affine polynomial family
P a [(Eq. (8)] with Q the unit ball, robust stability can be
checked by computing , with a rank-one matrix M(s) constructed as

1

pm (s)
.. p1 (s)
M(s) = .
p(s, q0 )
p(s, q0 )
1
To see this, observe that p(s, q) in Eq. (8) is the characteristic
polynomial of the M loop of Fig. 1 with diag(q1
q10, . . ., qm qm0 ). Indeed,
m

pk (s)
p(s, q)
(qk q0k ) =
det(I M(s)) = 1 +
0
p(s, q )
p(s, q0 )
k=0
Thus, stability of p(s, q) for all q Q is equivalent to stability

of the M loop for all diagonal matrices with real (parametric) entries lying in [1, 1]. The condition for this is that
M 1 where the structured singular value is computed
611
with respect to the block structure diag(1, . . ., m): i

. In light of the formula for rank-one , an explicit condition can then be stated. Such a result clearly applies to general stability regions, and it furnishes a frequency sweeping
condition for robust stability. Note that we may interpret this
result alternatively based upon the zero exclusion principle.
Indeed, under the condition that Q is the unit ball centered
at q0, all the polynomials in P will have zeros in a specified
region if and only if the zeros of p(s, q0) are in that region
and (M(s)) 1 for all s on the boundary of the region. This
follows because, according to the zero exclusion principle, it is
both necessary and sufficient that

m

min q q0 : p(s, q0 ) +
(qk q0k )pk (s) = 0 > 1
k=0
in order for the polynomial family p(s, q) in Eq. (8) to have no

zero on or exterior to the boundary of the region, for all possible q Q.
More generally, it is possible to extend the definition of
by means of more general norms and to use this extended
to study the robust stability of an affine family P with a more
general bounding set Q. Such a generalization also leads to a
similar expression when the M matrix in question has rank
one (14,35). In particular, when the stability region is restricted to the open left half plane, and Q is the unit p ball
centered at q0 with ak(q) kqk, the expression for the generalized rank-one , denoted as p( ) for purpose of distinction,
is found to be
Yr ()/|I()|
if R() = 0
if I() = 0
Xr ()/|R()|
p (M( j)) =
()Y
()
X
r
r
p
p
p
p 1/ p
(Xr ()|I()| + Yr ()|R()| )
otherwise
which leads to a similar condition for robust Hurwitz stability. This condition is slightly more general than, but essentially replicates, the graphical criterion by Tsypkin and Polyak. Note that for p , the polynomial family becomes an
interval polynomial, and the stability condition reduces to
checking whether p(s, q0) is Hurwitz stable, q0 1, and

X1 () Y1 ()
,
< 1,
(0, )
min
|R()| |I()|
A little thought reveals that the latter is equivalent to determining whether one of the four conditions Re(K1( j)) 0,
Re(K3( j)) 0, Im(K1( j)) 0, and Im(K3( j)) 0 holds.
Clearly, this is further equivalent to the requirement that the
rectangular value set in Fig. 4 never contains the origin.
Extensions
There is an immense body of literature devoted to polynomial
stability problems. Various extensions to Kharitonovs Theorem have been obtained. They generally fall into the categories of vertex results and frequency-sweeping conditions, consisting of delicate studies and intricate technical details. We
summarize some of the highlights next. A recent and comprehensive account can be found in the books by Barmish (26)
and by Bhattacharyya et al. (36).
612
Vertex and Edge Results. Much of the work in this direction

continues the thread in Kharitonovs Theorem, focusing on
simple uncertainty descriptions and leading to stability tests
based on vertex and/or edge polynomials. Some notable examples follow.
Complex Interval Polynomials. The polynomial family
has complex coefficients whose real and imaginary parts
are allowed to vary independently in given intervals.
Eight vertex polynomials need to be tested to ascertain
stability.
Diamond Polynomials. This polynomial family is described in Eq. (10), with p 1. Eight vertex polynomials
are required as well.
Stabilization of Interval Plants via First-Order Compensators. The numerator and denominator of the plant transfer function are interval polynomials, and it is to be stabilized by a first-order compensator in closed loop. It
suffices to stabilize 16 vertex plants, constructed based
upon the vertex numerator and denominator polynomials.
Generalized Kharitonov Theorem. It concerns linear combination of interval polynomials and requires checking
certain polynomial segments in addition to vertices.
Other stability conditions based on vertex polynomials are
also available. As the complexity of uncertainty structure increases slightly, they usually require testing more (e.g., 32 or
64) vertex polynomials. A clear insight concerning uncertainty structure and the required number of vertices, however, remains unavailable.
Performance Issues. The entire Kharitonov theory is largely
successful for determining stability of uncertain polynomials.
However, a number of results are also available regarding
properties of transfer functions, which have implications toward performance issues. Some examples follow.
H Norm of Interval Transfer Functions. When the numerator and denominator of a transfer function are both
interval polynomials, the H norm of the transfer function can be computed over 16 vertex transfer functions,
provided that the four Kharitonov polynomials associated with the denominator are stable.
Peak Magnitudes of Closed-Loop Transfer Functions. The
peak H norm of closed-loop transfer functions can be
computed over the edges of the plant family, when it is
an interval plant.
Nyquist and Bode Envelopes. The Nyquist and Bode plots
of open or closed-loop transfer functions associated with
an interval plant lie in envelopes determined by plots
generated by vertex and edge plants.
Other Extensions. Additional extensions may be found in
the following categories.
Schur Stability. The Kharitonov theory has been extended with varying degrees of success to other stability
regions, such as the unit circle. These results are useful
for studying stability of discrete-time systems and for addressing other performance issues.
Unstructured Uncertainty. Unmodeled dynamics may be

included along with parametric uncertainties. They may
be accommodated either in the rank-one formula or by
small gain-type conditions involving vertex and edge
plants.
Nonlinear Systems. In a system consisting of an interval
plant and a static, sector bounded nonlinear component,
stability conditions similar to the Popov and circle criteria have been obtained, which also require the testing of
vertex and edge plants.
Multilinear Uncertainty Structure. The entire success in
the Kharitonov approach relies on the key assumption
that polynomial coefficients depend linearly on uncertain
parameters, and the utility of all the results in this area
is thus measured by how much the uncertainty can deviate from this description. Little success has been
achieved in this endeavor. A fundamental barrier, as implicated by the zero exclusion principle, is that the stability problem is one of optimization subject to nonlinear,
nonconvex constraints.
STATE-SPACE APPROACH
Dynamical systems are often described by state-space equations. Accordingly, it is common to model system uncertainty
as perturbations to system matrices. An uncertain continuous-time system in this spirit may be described by
x(t)
= (A + BC)x(t)
(13)
Here A, B, and C are known matrices of appropriate dimensions. The matrix A is assumed to be stable. The system uncertainty is represented by a set of allowed values for the
real matrix , which may be unstructured or structured. Typical perturbation classes considered in the literature are as
follows, arranged in increasing order of generality. In all
cases, 0 is given.
Unstructured Perturbation. The set consist of all real
matrices with spectral norm less than a given number:
U = { real : () }
Element-by-Element Perturbation. Each element in
varies in a given interval. Let rij 0 be given. The set
is defined as
r11 11 r1m 1m
.
..
..
E = real : =
..
.
. ,
rn1 n1 rnm nm
= max{|ij | : rij > 0}
i, j
Linear Combination. The allowable set of perturbations

is described by

k

L = real: =
i Ei , = max |i |
i=1
where the Eis are given.
613
Evidently, an uncertain discrete-time system can be described

in exactly the same manner.
The problem of interest is to determine the size of the perturbation matrix, measured by a norm of choice, so that the
system remains stable. This issue naturally translates into
one concerning how the eigenvalues of a stable matrix A
would vary when it is perturbed by . More specifically, would
the eigenvalues cross the stability boundary? And if they do,
what is the minimal such that at least one of the eigenvalues leaves the stability region? These questions may be addressed by examining the characteristic polynomial
There essentially exists no result for the structured stability radius other than those already known for . For the unstructured stability radius, much of the early work was devoted to derivation of bounds. One representative example is
(s, ) = det(sI A BC)
Real Stability Radius. Let G(s) C(sI A)1B, and 2( )

be the second largest singular value. Then,
Re[G(s)]
Im[G(s)]
r(A, B, C)1 = sup inf 2 1

Im[G(s)]
Re[G(s)]
s D (0,1]
or equivalently, the characteristic equation

det(I C(sI A)1 B) = 0
Thus, it becomes clear at the outset that the problem may be
tackled in principle by using a polynomial approach. Similarly, it can also be analyzed as a problem. The latter can
be easily seen with respect to E and L, by rearranging the
elements of these sets into diagonal matrices and by defining
the M matrix appropriately. For U, we may simply adopt a
full real block structure and define accordingly. It should
be pointed out, nevertheless, that both and the polynomial
approach will lead to complications in the present context. On
the one hand, the computation of with respect to a real is
generally very difficult, and approximation by its upper bound
can be very conservative. On the other hand, the characteristic polynomial (s, ) will generally exhibit a multilinear or
multinomial dependence of its coefficients on , for which the
Kharitonov theory is ill-equipped; indeed, it is not difficult to
see that the coefficients of (s, ) are multilinear in ij if
E, and are multinomial functions of k if L. In summary,
both approaches are ineffective and conservative.
By far this uncertainty description poses the most difficult
challenge in robust stability analysis, and the state-space approach is the least developed. Results are scarce, and only in
rare cases are they nonconservative.
Stability Radius
A notion frequently encountered in studying the state-space
uncertainty description is that of stability radius. This notion
is closely related to , but it is less developed. Let be a
stability region of concern, and be its boundary. Furthermore, denote by (A) the spectrum of A. Then for any
norm of interest, the stability radius associated with the
triple (A, B, C) is defined by
r(A, B, C) = inf{ : , (A + BC) D = }
In other words, it defines the minimal perturbation size leading to instability, or the distance of A to the set of unstable
matrices. By definition, it thus follows directly that the matrix family A BC: , has all eigenvalues in
whenever r(A, B, C) . Moreover, in view of the preceding
discussion, we may regard the stability radius as the reciprocal of the maximum of a certain , with respect to an appropriate block structure and a matrix M. For further distinction,
the stability radius is said to be unstructured if is unstructured and structured otherwise.
r(A, B, C)
1
C(sI A)1 B
(14)
This, of course, is a rather straightforward consequence of the

Small Gain Theorem. Recently, however, Qiu et al. (37) obtained the following exact, readily computable formula.
The significance of this result lies in that for any s , the

function 2( ) is unimodal in over (0, 1), and hence its infimum can be computed effectively. Furthermore, when is
the open left half plane or the open unit disk, that is, when
Hurwitz or Schur stability is of concern, Sreedhar et al. (38)
developed a fast-converging algorithm for the maximization
with respect to s. Consequently, from a computational standpoint, the unstructured stability radius problem can be considered largely resolved.
Interval Matrices
An interval matrix is a family of real matrices in which all
elements are known only within certain closed intervals. In
precise terms, the interval matrix AI [A, A] is the set of
matrices defined by
AI = {A : a ij aij aij }
that is, each aij of A is confined elementwise to lie within an
interval determined by aij and aij, the corresponding elements
of A and A, respectively. An interval matrix AI is said to be
stable if every A AI is stable. Evidently, interval matrix and
set E share the same uncertainty description.
Interval matrices are direct matrix analogues of interval
polynomials, and hence there has been a lingering temptation
for extension of Kharitonovs Theorem to the former. Unfortunately, neither vertex nor edge results exist for interval matrices. In fact, more recent studies showed that in order to
determine stability of an interval matrix, we must solve an
NP-hard decision problem. This in a way explains why only
sufficient stability conditions are available.
One approach of attack is to analyze eigenvalue distribution. Heinen (39) and Argoun (40) examined the problem on
the basis of Gershgorins Theorem, and their developments
culminated in a subsequent work of Chen (41), leading to a
number of simple, albeit conservative, stability conditions. As
a representative example of these results, consider an interval matrix AI such that aii 0. Let W be constructed as
0
i= j
max{|a ij |, |aij |}
W = [wij ],
wij =
i = j
|aii |
614
Then a sufficient condition for Hurwitz stability of AI is found

by Chen (41) to be
(W ) < 1
A useful feature of this result, and more generally of conditions obtained by using Gershgorins Theorem, is that it lends
a ready characterization via the so-called M-matrices. The
latter aspect makes it possible to unify a number of sufficient
stability conditions in different forms.
Alternatively, Yedavalli and others studied interval matrices from a Lyapunov analysis standpoint. This is collectively
inspected next.
Lyapunov Analysis
The Lyapunov theory, as anticipated, is widely employed in
robust stability analysis pertaining to state-space formulation, yielding various results concerning stability radius and
interval matrices. One common thread in this approach is to
find a single quadratic Lyapunov function applicable to the
entire family of the perturbed matrices; the technique is often
referred to in the literature as quadratic stability. Another
lies in the simplicity of the stability conditions.
Let us begin with the unstructured uncertainty set U. By
constructing the usual Lyapunov function
V (x) = 12 x T Px
C(sI A)1 B < 1/

The latter condition clearly coincides with condition (14).
More results are available for the special case when B
C I. For the structured uncertainty set E, Yedavalli (43)
gave the sufficient condition
|P|R + RT |P|
2
< 1/
(16)
for the Hurwitz stability of the matrix family A :

E. Here P 0 is the unique solution to the Lyapunov equation
PA + AT P = 2I
(17)
P denotes the modulus matrix of P (i.e., each entry of P is

the absolute value of the corresponding entry of P) and R is
given by
r11
.
R = ..
rn1
..
.
r1n
..
.
rnn
k
1
(|PEi + EiT P|)
2 i=1
< 1/
(18)
where, again, P 0 is the unique solution to Eq. (17). Subsequent developments led to further extensions for problems
with even more detailed uncertainty descriptions. For example, the ks may be allowed to vary in asymmetric intervals.
Moreover, because rather obviously any interval matrix can
be represented alternatively in the form of A : E,
these conditions can be applied to determine the Hurwitz stability of an interval matrix as well.
Yet another issue clearly of interest is whether it is possible to derive vertex versions of these sufficient conditions.
Boyd and Yang (44) examined stability problems for matrix
polytopes. Specifically, they postulated the uncertainty description
A = conv{A1 , . . ., Ak }
A sufficient condition for A to be Hurwitz stable can be easily
found to be the existence of a P 0 such that
i = 1, . . ., k
(19)
Similarly, for the uncertainty set L, a vertex condition can

be obtained as
(15)
This, of course, does not come as a surprise. According to the

well-known Bounded Real Lemma (42), it is equivalent to
PAi + ATi P < 0,
we find that the entire family of matrices A BC: U

is Hurwitz stable if there exists a positive definite matrix
P 0 such that
PA + AT P + 2 BBT + CTC < 0
Furthermore, Zhou and Khargonekar (42) observed that the

uncertainty description E can be regarded as a special case
of L, for which they provided the stronger Hurwitz stability
condition
k
1
(PEi + EiT P)
2 i=1 i
< 1/
(20)
for all combinations of the i in 1, 1. It should be rather

evident that this condition improves upon inequality (18).
Both conditions (19) and (20) may be regarded as vertex results in the matrix perturbation case, and both can be posed
and solved as LMI problems.
CONCLUSION
Summary
For the past two decades, modeling uncertainty and robustness has resurfaced as a dominating theme in control theory and application and is now held unanimously by theoreticians and practitioners as the most important concern in
control system design. For both its intrinsic appeal and practical significance, robust control as a whole attracted considerable interest and underwent a period of immense development, bringing control theory to a new height. Many
important issues have been addressed. Many remain unresolved. The ultimate puzzle, it now appears, lies in the fundamental conflict between problem complexity and computational tractability.
Of the three main research areas surveyed in this article,
the structured singular value provides the most general formulation for uncertainty modeling and is the most systematically developed tool in robustness analysis. The major issues
in analysis are clearly generality of uncertainty description,

conservatism of analysis, and ease of computation. The main
success achieved with this approach, unquestionably, lies in
the progress in computing . While it cannot be computed
exactly in general, various computational schemes have been
developed for computing it approximately, and commercial
software programs are now available. This paves the way for
its application to a series of engineering design problems,
ranging from disk drive control to flight control. Successful
applications to other potential areas, including robot manipulators, flexible structures, magnetic bearings, and chemical
processes, have also been reported in laboratory experiments.
The Kharitonov approach, unlike analysis, was more restrictive in scope in its early phase of development. However,
it has undergone a bottom-up growth pattern as the uncertainty descriptions become progressively more general and sophisticated. Overall, the Kharitonov and state-space methods
may be broadly classified as a parametric approach toward
robustness analysis, originating from interval polynomials
and culminating at state-space uncertainty descriptions. The
main appeal of this approach, it appears, lies in its quest for
analytical solutions, more appealing than mere computationbased tools. The main success in the entire parametric approach, which remains the state-of-the-art today, is the resolution of the affine uncertain polynomial family case, for
which necessary and sufficient stability conditions are available, in terms of both edge tests and graphical conditions. On
the other hand, the multilinear/multinomial polynomial family and the state-space uncertainty description are the weakest link, for which only sufficient stability conditions are
available with unknown conservatism, and more systematic,
efficient, computation-based approximate tests are called for.
At present, only a few applications of the Kharitonov theory
are reported in the literature, including Ackermanns car
steering problem and an automotive engine control problem
investigated by Abate et al. (45) (see also Chapter 3 in Ref.
26). It should be rather evident that the fundamental bottleneck in all robustness analysis methods, be it analysis or
Kharitonov approach, lies in computational complexity, and
the ultimate challenge is in the conquest over the curse of
dimensionality. No matter whether this can be achieved or
not, we should be consciously aware that the dilemma is the
natural cause of problem generality and hence complexity and
results from the search of optimal solutions. In engineering
system design, we should therefore reconcile and seek a judicious trade-off between these conflicting requirements.
To Probe Further
In light of the difficulties encountered in robustness analysis
with respect to structured and/or parametric uncertainties, a
number of researchers recently examined complexity issues
from a computational standpoint, drawing upon concepts and
techniques from computing science and operation research.
The main discoveries are in the following areas.
with Real Uncertainties. Braatz et al. (46) showed that
the computation of real is NP-hard.
with Complex Uncertainties. Toker and Ozbay (47)
proved that the computation of complex is also NPhard.
615
with Real and Complex Uncertainties. Braatz et al. (46)

and Toker and Ozbay (47) both showed that the computation of is NP-hard.
The Bounds. Toker (48) and Fu (49) showed that the
problem of finding an accurate bound for is NP-hard.
Interval Matrix. Coxson and DeMarco (50) showed that
stability of interval matrices amounts to an NP-hard
problem.
These results indicate that a worst-case instance exists in
each class of the problems, for which it is rather unlikely that
computational complexity can be bounded via a polynomial
function of the problem dimension. It thus comes as no surprise that the problems are difficult, and indeed are intractable in general.
From a technical standpoint, the computational difficulty
in question may be best seen as an outcome of nonlinear, nonconvex optimization problems. Although only explored systematically in recent years, complexity issues have been under contemplation for a long time and have led to alternative,
computationally tractable approximations and formulations.
One notable remedy is to resort to formulations based upon
LMIs, and problems in this class include those that can be
described via integral quadratic constraints (IQC). Both LMIs
and IQCs offer in essence an energy-based perspective toward
system analysis, and they draw heavily upon concepts in classical passivity and dissipativity theory, leading to readily
computable, albeit only sufficient, robust stability conditions.
For a comprehensive treatment of control-relevant LMI and
convex programming problems, see Ref. 51, or the relevant
chapter in this encyclopedia. Megretsky and Rantzer (52) provided a detailed account of the IQC technique.
The computational complexity results just discussed are
strongly linked to the worst-case nature of the robustness
problems; that is, the requirement of robustness must be met
for all possible instances. Is so stringent a requirement truly
necessary? This question prompted a reexamination of robustness issues, and it led to a recent venture departing almost entirely from the worst-case formulation. A number of
researchers argued that worst-case scenarios hardly occur in
practice, that a worst-case analysis is not only overly demanding but also too pessimistic, and that, after all, worstcase analysis problems are often intractable. The argument
thus motivated the description of uncertainty via probabilistic
measures, and accordingly probabilistic approaches to robustness analysis. In this new thinking, the deterministic uncertainty description is discarded altogether and is replaced
by a probability description characterizing the likelihood that
the uncertainty may lie in a bounded set. The robustness condition then amounts to determining the probability under
which the system may become unstable. Recent studies (53
56) show that a variety of problems, which are NP-hard in
the deterministic setting, become readily solvable computationally when formulated probabilistically. The area, however, is entirely open and is not without obstacles of its own.
BIBLIOGRAPHY
1. J. Ackermann, Robust Control Systems with Uncertain Physical
Parameters, London: Springer-Verlag, 1996.
616
2. J. Doyle, Analysis of feedback systems with structured uncertainties, IEE Proc., 129-D (6): 242250, 1982.
3. V. L. Kharitonov, Asymptotic stability of an equilibrium position
of a family of systems of linear differential equations, Differentialnye Uraveniya, 14 (11): 14831485, 1978.
4. M. G. Safonov, Stability margins of diagonally perturbed multivariable feedback systems, IEE Proc., 129-D: 251256, 1982.
5. A. Packard and J. C. Doyle, The complex structured singular
value, Automatica, 29 (1): 71109, 1993.
6. W. S. Levine (ed.), The Control Handbook, Boca Raton, FL: CRC
Press, 1996.
9. A. L. Tits and V. Balakrishnan, Small- theorems with frequency-dependent uncertainty bounds, Math. Control. Signals,
Syst., in press, 1998.
10. J. M. Krause, Structured singular value analysis of multidimensional system stability, IEEE Trans. Autom. Control, 34 (6): 638
639, 1989.
11. J. Chen and H. A. Latchman, Frequency sweeping tests for stability independent of delay, IEEE Trans. Autom. Control, 40 (9):
16401645, 1995.
12. M. K. H. Fan, A. L. Tits, and J. C. Doyle, Robustness in the presence of mixed parametric uncertainty and unmodelled dynamics,
13. P. M. Young, The rank one mixed problem and Kharitonovtype analysis, Automatica, 30: 18991911, 1994.
and stability of uncertain polynomials, Part 1: The generalized ,
15. B. Kouvaritakis and H. Latchman, Necessary and sufficient stability criterion for systems with structured uncertainties: The
major principle direction alignment principle, Int. J. Control, 42
(3): 575598, 1985.
with non-diagonal structures, Part 1: Characterizations, IEEE
with non-diagonal structures, Part 2: Computations, IEEE Trans.
Autom. Control, 41 (10): 15111516, 1996.
18. A. L. Tits, V. Balakrishnan, and L. Lee, Robustness under
bounded uncertainty with phase information, IEEE TAC, in press.
19. M. Khammash and J. B. Pearson, Robust disturbance rejection in
l1-optimal control systems, Syst. Control Lett., 14: 93101, 1990.
20. M. Khammash and J. B. Pearson, Performance robustness of discrete-time systems with structured uncertainty, IEEE Trans. Autom. Control, 36 (4): 398412, 1991.
21. J. Shamma, Robust stability with time-varying structured uncertainty. IEEE Trans. Autom. Control, 39 (4): 714724, 1994.
22. A. Megretski, Necessary and sufficient conditions of stability: A
multiloop generalization of the circle criterion, IEEE Trans. Autom. Control, 38 (5): 753756, 1993.
23. K. Poolla and A. Tikku, Robust performance against time-varying
structured perturbations, IEEE Trans. Autom. Control, 40 (9):
15891602, 1995.
24. S. Dasgupta, Kharitonovs theorem revisited, Syst. Control Lett.,
11: 381384, 1988.
25. A. C. Bartlett, C. V. Hollot, and L. Huang, Root locations of an
entire polytope of polynomials: It suffices to check the edges,
Math. Control, Signals, Syst., 1: 6171, 1988.
26. B. R. Barmish, New Tools for Robustness of Linear Systems, New
27. M. Fu and B. R. Barmish, Polytopes of polynomials with zeros in

a prescribed set, IEEE Trans. Autom. Control, 34 (5): 544546,
1989.
28. A. L. Tits, Comments on polytopes of polynomials with zeros in
a prescribed set, IEEE Trans. Autom. Control, 35 (11): 1276
1277, 1990.
29. B. R. Barmish, A generalization of Kharitonovs four polynomial
concept for robust stability problems with linearly dependent coefficient perturbations, IEEE Trans. Autom. Control, 34 (2): 157
165, 1989.
30. Ya. Z. Tsypkin and B. T. Polyak, Frequency domain criteria for
lp-robust stability of continuous linear systems, IEEE Trans. Autom. Control, 36 (12): 14641469, 1991.
31. D. Hinrichsen and A. J. Pritchard, An application of state space
methods to obtain explicit formulae for robustness measures of
polynomials, in M. Milanese et al. (ed.), Robustness in Identification and Control, Boston, MA: Birkhauser, 1989.
32. M. K. Saridereli and F. J. Kern, The stability of polynomials under correlated coefficient perturbations, Proc. 26th IEEE Conf.
Decision Control, 1987, pp. 16181621.
33. A. Tesi and A. Vicino, Robustness analysis for linear dynamical
systems with linearly correlated parametric uncertainties, IEEE
34. L. Qiu and E. J. Davison, A simple procedure for the exact stability robustness computation of polynomials with affine coefficient
perturbations, Syst. Control Lett., 13: 413420, 1989.
and stability of uncertain polynomials, Part 2: A missing link,
36. S. P. Bhattacharyya, H. Chapellat, and L. H. Keel, Robust Control: The Parametric Approach, Upper Saddle River, NJ: PrenticeHall, 1995.
37. L. Qiu et al., A formula for computation of the real stability radius, Automatica, 31 (6): 879890, 1995.
38. J. Sreedhar, P. Van Dooren, and A. L. Tits, A fast algorithm to
compute the real structured stability radius, in R. Jeltsch and M.
Mansour (eds.), Stability Theory: Proc. Centenary Conf. Int. Series
Numerical Math. (ISNM), Basel: Birkhauser, 1996, vol. 232, pp.
219230.
39. J. A. Heinen, Sufficient conditions for stability of interval matrices, Int. J. Control, 39 (6): 13231328, 1984.
40. M. B. Argoun, On sufficient conditions for the stability of interval
matrices, Int. J. Control, 44 (5): 12451250, 1986.
41. J. Chen, Sufficient conditions on stability of interval matrices:
Connections and new results, IEEE Trans. Autom. Control, 37
(4): 541544, 1992.
42. K. Zhou and P. P. Khargonekar, Stability robustness bounds for
linear state-space models with structured uncertainty, IEEE
43. R. K. Yedavalli, Improved measures of stability robustness for
linear state space models, IEEE Trans. Autom. Control, 30 (6):
577579, 1985.
44. S. Boyd and Q. Yang, Structured and simultaneously Lyapunov
functions for system stability problems, Int. J. Control, 49 (6):
22152240, 1989.
45. M. Abate et al., Application of some new tools to robust stability
analysis of spark ignition engines: A case study, Proc. Amer. Control Conf., 1992, pp. 932936.
46. R. P. Braatz et al., Computational complexity of calculation,
47. O. Toker and H. Ozbay, On the complexity of purely complex
computation and related problems in multidimensional systems,
IEEE Trans. Autom. Control., 43 (3): 409414, 1998.
ROUNDOFF ERRORS
48. O. Toker, On the conservatism of upper bound tests for structured singular value analysis, Proc. 35th Conf. Decision Control,
Kobe, Japan: 1996, pp. 12951300.
49. M. Fu, The real is hardly approximable, IEEE Trans. Autom.
Control, 42 (9): 12861288, 1997.
50. G. E. Coxson and C. L. DeMarco, The computation complexity of
approximating the minimal perturbation scaling to achieve instability in an interval matrix, Math. Control, Signals Syst., 7 (4):
279292, 1994.
Theory, Studies in Applied Mathematics, Philadelphia: SIAM,
1994, vol. 15.
52. A. Megretski and A. Rantzer, System analysis via integral quadratic constraints, IEEE Trans. Autom. Control, 42 (6): 819
830, 1997.
53. R. F. Stengel and L. R. Ray, Stochastic robustness of linear timeinvariant systems, IEEE Trans. Autom. Control, 36 (1): 8287,
1992.
54. B. R. Barmish and C. M. Lagoa, The uniform distribution: A rigorous justification for its use in robustness analysis, Proc. 35th
55. R. Tempo, E. W. Bai, and F. Dabbene, Probabilistic robustness
analysis: Explicit bounds for the minimum number of samples,
Proc. 35th IEEE Conf. Decision Control, 1996, pp. 34243428.
56. X. Chen and K. Zhou, On the probabilistic characterization of
model uncertainty and robustness, Proc. 36th IEEE Conf. Decision
Control, 1997, pp. 38163821.
JIE CHEN
ANDRE L. TITS
ROBUSTNESS ANALYSIS. See ROBUST CONTROL

ANALYSIS.
ROBUST SIGNAL PROCESSING. See NONLINEAR

SYSTEMS.
ROLLBACK RECOVERY. See PROGRAM DIAGNOSTICS.

ROLLING METALS. See METALS INDUSTRY.
ROTATION SPEED. See TACHOMETERS.
617
LINEAR DYNAMICAL SYSTEMS, APPROXIMATION
403
Two basic and desirable features of an approximation theory are the existence of:
a constructive methodology for obtaining reduced-complexity models
an appropriate quantization of the approximation error,
in other words, the estimation of some measure of the
error between the given high-complexity model and the
derived reduced-complexity model(s)
The importance of the second item cannot be overemphasized:
In approximating a system, one wishes to have some idea of
what has been eliminated.
Often an additional desirable feature consists in
looking for reduced-complexity models within a specified
class, e.g. the class of stable systems

Approximation is an important methodology in science and
engineering. In this essay we will review a theory of approximation of linear dynamical systems that has a number of desirable features. Main ingredients of this theory are: the 2norms used to measure the quantities involved, in particular
the Hankel norm, the infinity norm, and a set of invariants
called the Hankel singular values. The main tool for the construction of approximants is the all-pass dilation (unitary extension) of the original dynamical system.
For instance, in case the nominal model is stable, if the intended use of a reduced-complexity model is in open-loop, it
is imperative that the latter also be stable.
We will deal with linear, time-invariant, discrete- and continuous-time, finite-dimensional systems described by convolution sums or integrals. In addition, we will only consider the
approximation of stable systems. Most of the approximation
methods availablefor example, Pade approximationfail to
satisfy the requirements listed above. We will discuss the theory of Hankel-norm approximation and the related approximation by balanced truncation. The size of systemsincluding
error systemsis measured in terms of appropriately defined
2-norms, and complexity is measured in terms of the (least)
number of state variables (i.e., first-order differential or difference equations) needed to describe the system.
This approach to model reduction has an interesting history. For operators in finite-dimensional spaces (matrices),
the problem of optimal approximation by operators of lower
rank is solved by the SchmidtMirsky theorem (see Ref. 1).
Although this is not a convex problem and consequently conventional optimization methods do not apply, it can be solved
explicitly by using an ad hoc tool: the singular value decomposition (SVD) of the operator. The solution involves the truncation of small singular values.
In the same vein, a linear dynamical system can be represented by means of a structured (Hankel) linear operator in
appropriate infinite dimensional spaces. The AdamjanArov
Krein (AAK) theory (see Refs. 2 and 3) generalizes the
SchmidtMirsky result to dynamical systems, that is to structured operators in infinite-dimensional spaces. This is also
known as optimal approximation in the Hankel norm. The original setting of the AAK theory was functional analytic. Subsequent developments due to Glover (4) resulted in a simplified
linear algebraic framework. This setting made the theory
quite transparent by providing explicit formulae for the quantities involved.
The Hankel norm approximation problem can be addressed and solved both for discrete- and continuous-time systems. However, the following is a fact: The discrete-time case
is closer to that of finite-dimensional operators and to the
SchmidtMirsky result. Therefore, the intuitive understanding of the results in this case is more straightforward. In the
continuous-time case, on the other hand, while the interpreta-
404
tion of the results is less intuitive than for the discrete-time

case, the corresponding formulas turn out to be simpler (see
Remark 3). Because of this dichotomy we will first state the
results in the discrete-time case, trying to make connections
and draw parallels with the SchmidtMirsky theorem (see
section entitled The SchmidtMirsky Theorem and the AAK
Generalization). The numerous formulas for constructing approximants, however, will be given for the continuous-time
case (see section entitled Construction of Approximants).
The Hankel-norm approximation, as well as model reduction by balanced truncation, inherits an important property
from the SchmidtMirsky result: Unitary operators form the
building blocks of this method and cannot be approximated; a
multiple singular value indicates that the original operator
contains a unitary part. This unitary part has to either be
included in the approximant as a whole or discarded as a
whole; it must not be truncated. The same holds for the extension to Hankel-norm approximation. The fundamental building blocks are all-pass systems, and a multiple Hankel singular value indicates the existence of an all-pass subsystem. In
the approximation, this subsystem will be either eliminated
or included as a whole; it must not be truncated. Consequently, it turns out that the basic operation involved in constructing approximants is all-pass dilation (unitary extension)that is, the extension of the number of states of the
original system so that the aggregate becomes all-pass (unitary) [see Main Theorem (2.3)].
The article contains three main sections. The first, entitled
The SchmidtMirsky Theorem and the AAK Generalization, reviews 2-norms and induced 2-norms in finite dimensions and states the SchmidtMirsky result. The second half
of this section presents the appropriate generalization of
these concepts for linear discrete-time systems. An operator
which is intrinsically attached to a linear system is the convolution operator S . Therefore, the problem of optimal approximation in the 2-induced norm of this operator arises
naturally. For reasons explained in the section entitled Approximation of in the 2-Induced Norm of the Convolution
Operator, however, it is currently not possible to solve this
problem, except in some special cases [see Remark 4(b)]. A
second operator, the Hankel operator H , is obtained by restricting the domain and the range of the convolution operator. The 2-induced norm of H is the Hankel norm of . It
turns out that the optimal approximation problem is solvable
in the Hankel norm of . This is the AAK result, stated in
the section entitled The AAK Theorem. The main result of
optimal and suboptimal approximation in the Hankel norm is
presented next. The fundamental construction involved is the
all-pass dilation (unitary extension) of .
The most natural norm for the approximation problem is
the 2-induced norm of the convolution operator, which is
equal to the infinity norm of the associated rational transfer
function (i.e., the maximum of the amplitude Bode plot). However, only a different problem can be solved, namely the approximation in the 2-induced norm of the Hankel operator
(which is different from the convolution operator). Although
the problem solved is not the same as the original one, the
singular values of the Hankel operator turn out to be important invariants. Among other things, they provide a priori
computable boundsboth upper and lowerfor the infinity
norm of the error systems.
The first part of the section entitled Construction of

Approximants presents the fundamentals of continuous-time
linear systems. The convolution and the Hankel operators are
introduced together with the grammians and the computation
of the singular values of the Hankel operator H . These are
followed by Lyapunov equations, inertia results, and statespace characterizations of all-pass systemsall important ingredients of the theory. Subsequently, various formulas for
optimal and suboptimal approximants are given. These formulae are presented for continuous-time systems since (as
mentioned earlier) they are simpler than their discrete-time
counterparts. First comes an inputoutput approach applicable to single-input single-output systems (see section entitled
InputOutput Construction Method for Scalar Systems); it
involves the solution of a polynomial equation which is
straightforward to set up and solve. The remaining formulas
are all state-space-based. The following cases are treated (in
increasing complexity): square systems, suboptimal case;
square systems, optimal case; general systems, suboptimal
case. These are followed by the important section entitled
Error Bounds of Optimal and Suboptimal Approximants;
these bounds concern the 2-induced norms of both the Hankel
and the convolution operators of error systems. The section
entitled Balanced Realizations and Balanced Model Reduction, presents the closely related approximation method by
balanced truncation. Again error bounds are available, although balanced approximants satisfy no optimality properties.
This article concludes with three examples and a brief
overview of some selected recent developments, including an
overview of the approximation problem for unstable systems.
Besides the original sources, namely Refs. 24, parts of the
material presented below can also be found in books (5,6) and
in lecture notes (7). For the mathematical background needed
we refer to Ref. 8.
THE SCHMIDTMIRSKY THEOREM
AND THE AAK GENERALIZATION
In this section we will first state the SchmidtMirsky theorem concerned with the approximation of finite-dimensional
unstructured operators in the 2-norm, which is the operator
norm induced by the vector Euclidean norm. Then, discretetime linear dynamical systems are introduced together with
the associated convolution operator; this is a structured operator defined on the space of square summable sequences. It is
argued in the section entitled Approximation of in the 2Induced Norm of the Convolution Operator that due to the
fact that the convolution operator has a continuous spectrum,
it is not clear how the SchmidtMirsky result might be generalized in this case. However, roughly speaking by restricting
the domain and the range of the convolution operator, the
Hankel operator is obtained. It turns out that the Schmidt
Mirsky result can be generalized in this case; this is the famous AAK theorem followed by the section entitled The
Main Result.
2-Norms and Induced 2-Norms in Finite Dimensions
The Euclidean or 2-norm of x n is defined as
x
:=
x21 + + x2n
Given the linear map A : n m, the norm induced by the

Euclidean norm in the domain and range of A is the 2-induced
norm:
x
:= sup
2-ind
x = 0
Ax 2
x 2
(1)
Ax 22
x A Ax
max (A A)
=
x 22
x x
where ( )* denotes complex conjugation and transposition. By
choosing x to be the eigenvector corresponding to the largest
eigenvalue of A*A, the above upper bound is attained, and
hence the 2-induced norm of A is equal to the square root of
the largest eigenvalue of A*A:
2-ind
max (A A)
:=
trace(A A) =

A2i, j
Consider a rectangular matrix A nm; let the eigenvalue

decomposition of the symmetric matrices A*A and AA* be
AX
2-ind
k+1 (A)
(5)
The lower bound k1(A) of the error is attained by X*, which

is obtained by truncating the dyadic decomposition of A, to
the leading k terms:
X :=
k

i ui vi
(6)
Remark 1. (a) The importance of this theorem lies in the fact

that it establishes a relationship between the rank k of the
approximant, and the (k 1)st largest singular value of A.
(b) The minimizer X* given above is not unique, since each
member of the family of approximants
X (1 , . . ., k ) :=
A A = V SV V , AA = USU U
k

(i i )ui vi ,
0 i k+1, i = 1, . . ., k
i=1
(7)
where U, V are (square) orthogonal matrices of size n, m, respectively (i.e., UU* In, VV* Im). Furthermore SV, SU are
diagonal, and assuming that n m we have
SV = diag(12 , . . ., n2 , 0, . . ., 0),
i i+1 0
Given the orthogonal matrices U, V and the nonnegative real

numbers 1, . . ., n, we have
A = USV
where S is a matrix of size n m with i on the diagonal and
zeros elsewhere. This is the singular value decomposition
(SVD) of A; i, i 1, . . ., n, are the singular values of A,
while the columns ui, vi of U, V are the left, right singular
vectors of A, respectively. As a consequence, the dyadic decomposition of A follows:
i ui vi
12 + + n2
i=1
The SVD and the SchmidtMirsky Theorem
n

Theorem 1. SchmidtMirsky. Given is the matrix A of

rank n. For all matrices X of the same size and rank at most
k n, there holds:
(3)
The Frobenius norm is not an induced norm. It satisfies

A2-ind AF.
A=
The solution of this problem is provided by the Schmidt

Mirsky theorem (see, e.g., Ref. 1, page 208):
i, j
SU = diag(12 , . . ., n2 ),
The following central problem can now be addressed.
(2)
The m n matrix A can also be considered as an element of

the (mn)-dimensional space mn. The Euclidean norm of A
in this space is called the Frobenius norm:
A
values the Frobenius norm of A defined by Eq. (3), is
PROBLEM 1. OPTIMAL LOW-RANK APPROXIMATION.

Given the finite matrix A, find a matrix X of the same size
but lower rank such tht the 2-induced norm of the error E :
A X is minimized.
It readily follows that
405
(4)
i=1
This is a decomposition in terms of rank one matrices; the

rank of the sum of any k terms is k. In terms of the singular
attains the lower bound, namely k1(A).

(c) The problem of minimizing the 2-induced norm of A
X over all matrices X of rank at most k, is a nonconvex optimization problem, since the rank of the sum (or of a linear combination) of two rank k matrices is, in general, not k. Therefore, there is little hope of solving it using conventional
optimization methods.
(d) If the error in the above approximation is measured in
terms of the Frobenius norm, the lower bound in Eq. (5) is
replaced by k21 n2. In this case, X defined by
*
Eq. (6) is the unique optimal approximant of A, which has
rank k.
2-Norms and Induced 2-Norms in Infinite Dimensions
The 2-norm of the infinite sequence x : is
x
:=
+ x(1)2 + x(0)2 + x(1)2 +
The space of all sequences over which have finite 2-norm is

denoted by 2(); 2 is known as the Lebesgue space of squaresummable sequences. Similarly, for x defined over the negative or positive integers , , the corresponding spaces of
sequences having finite 2-norm are denoted by 2(), 2().
406
The 2-norm of the matrix sequence X : pm is defined as

X
:=
+ X (1)
2
F
+ X (0)
2
F
+ X (1)
2
F
The space of all p m sequences having finite 2-norm is denoted by 2pm(). Given the linear map A : X Y, where X, Y
are subspaces of 2(), the norm induced by the 2-norm in the
domain and range is the 2-induced norm, defined by Eq. (1);
again Eq. (2) holds.
Linear, Discrete-Time Systems
A linear, time-invariant, finite-dimensional, discrete-time system is defined as follows:
:
x(t + 1) = Ax(t) + Bu(t)

,
tZ
x(t) n is the value of the state at time t, u(t) m and y(t)

p are the values of the input and output at time t, respectively; and A, B, C, and D are constant maps. For simplicity,
we will use the notation
:=
A
C
B
D
R(n+ p)(n+m)
(8)
h (t) = CAt1 BI(t) + D(t)
(9)
To we associate the convolution operator S which maps

inputs u into outputs y:
where y(t) = (h u)(t)

:=
t

=
..
.
h(0)
h(1)
h(2)
h(3)
..
.
h(0)
h(1)
h(2)
..
.

S
is called the transfer function of . The system is stable if the

eigenvalues of A are inside the unit disk in the complex plane,
that is, i(A) 1; this condition is equivalent to the impulse
response being an 2 matrix sequence: h 2pm(). If is not
stable, its impulse response does not have a finite 2-norm.
However, if A has no eigenvalues on the unit circle, the impulse response can be reinterpreted so that it does have a
finite 2-norm. This is done next. Let T be a basis change in
the state space such that the matrices A, B, C can be partitioned as
A=
A+
0

B=
B+
B
C (C+ C )
where the eigenvalues of A and A are inside and outside of

the unit disk, respectively. In contrast to Eq. (9), the 2-impulse response denoted by h2 of is defined as follows:
(12)
where
h2+ (t) := C+ At+ B+ I(t) + (t)D

h2 (t) := C At B I(t)
where as before is the unit step function and is the Kronecker symbol. Accordingly, we will write 2 . Notice that the algebraic expression for he transfer function remains the same in both cases:
+ D + C (zI A )1 B = H (z)
h (t )u( ), t Z (10)
h(0)
h(1)
..
.
0
A
H2 (z) = H2+ (z) + H2 (z) = C+ (zI A+ )1 B+
What is modified is the region of convergence of H from
This convolution sum can also be written in matrix notation:
0 . 1 0.
BB .. CC BB . .
BBy(2)CC BB
BBy(1)CC = BB
BB y(0) CC BB
BB y(1) CC BB
@
A @
1 i p, 1 j m
h2 := h2+ + h2
The dimension (or order) of the system is n: dim n. We

will denote the unit step function by I ( I(t) 1 for t 0, and
zero otherwise) and the Kronecker delta by ((0) 1, and
zero otherwise). The impulse response h of is
S : u y,
S has (block) Toeplitz and lower-triangular structure. The

rational p m matrix function

pi j (z)
t
1
,
H (z) :=
h (t)z = C(zI A) B + D =
qi j (z)
t=0
h(0)
..
.
..
1
CC
CC
CC
CC
CC
A
This is equivalent to trading the lack of stability (poles outside the unit disk) for the lack of causality (h2 is nonzero for
negative time). Thus a system with poles both inside and outside of the unit disk (but not on the unit circle) will be interpreted as a possibly antistable 2 system 2, by defining the
impulse response to be nonzero for negative time. Consequently h2 2(). The matrix representation of the corresponding convolution operator is
0 . 1
BB .. CC
BBu(2)CC
BBu(1)CC
BB u(0) CC
BB u(1) CC
@
A
..
.
|max (A )| < |z| to |max (A+ )| < |z| < |min (A )|
(11)
0.
BB . .
BB
B
S2 = B
BB
BB
@
..
.
h2+ (0)
h2+ (1)
h2+ (2)
h2+ (3)
..
.
..
.
h2 (1)
h2+ (0)
h2+ (1)
h2+ (2)
..
.
..
.
h2 (2)
h2 (1)
h2+ (0)
h2+ (1)
..
.
..
.
h2 (3)
h2 (2)
h2 (1)
h2+ (0)
..
.
1
CC
C
C
C
C
C
CC
C
A
..
.
(13)
Notice that S 2 has block Toeplitz structure, but is no longer

lower triangular. All discrete-time systems with no poles on
the unit circle will be interpreted as 2 systems 2. For simplicity of notation, however, they will still be denoted by .
They are composed of two subsystems: (a) the stable and
causal part and (b) the stable but anti-causal part :
.
For square 2 systemsthat is, 2 systems having the same
number of inputs and outputs p mthe class of all-pass
2 systems is defined as follows: For all pairs u, y satisfying
Eq. (10), there holds
y
= u
(14)
for some fixed positive constant . This is equivalent to

S S = 2 I H (z1 )H (z) = 2 Im ,
|z| = 1
This last condition says that the transfer function (scaled by

) is a unitary matrix on the unit circle.
Approximation of in the 2-Induced
Norm of the Convolution Operator
Let be an 2 system. The convolution operator S can be
considered as a map:
p
S : m
2 (Z) 2 (Z)
The 2-induced norm of is defined as the 2-induced norm of

S :

2-ind
:= S
2-ind
= sup
u = 0
Due to the equivalence between the time and frequency domains (the Fourier transform is an isometric isomorphism),
this norm can also be defined in the frequency domain. In
particular,

2-ind
= sup maxH (z) =: H

|z|=1
This latter quantity is known as the norm of H. If the

system is single-input single-output, it is the supremum of
the amplitude Bode plot. In the case where is stable, H is
analytic outside the unit disk, and the following holds:

2-ind
= sup maxH (z) =: H

|z|1
This is known as the h norm of H. For simplicity, we will

use the notation

2-ind
:= S
2-ind
= H
to explore the possibility of optimal approximation of in the

2-induced norm of the convolution operator S . In this regard
the following result holds.
Proposition 1. Let : infz1 min(H(z)), while * :
*
supz1 max(H(z)). Every in the interval [*, *] is a singular
value of the operator S .
We conclude that since the singular values of S form a
continuum, it is not a priori clear how the SchmidtMirsky
result might be generalized to the convolution operator of .
Remark 2. Despite the above conclusion, the approach discussed below yields as byproduct the solution of this problem
in the special case of the one-step model reduction; see Remark 4(b).
Approximation of in the 2-Induced
Norm of the Hankel Operator
The next attempt to address this approximation problem is
by defining a different operator attached to the system . Recall that m2 (I ) denotes the space of square-summable sequences of vectors, defined on the interval I , with entries in
m. Given the stable and causal system , the following operator is defined by restricting the domain and the range of the
convolution operator, Eq. (10):
H : m (Z) p (Z+)
u y+ ,
where y+ (t) =
1

=
h (t )u ( ),
t0
S u 2
u 2
(15)
It will be clear from the context whether the subscript

stands for the norm of the h norm. If is all-passthat
is, Eq. (14) is satisfiedthen 2-ind H .
Our aim is the generalization of the SchmidtMirsky result for an appropriately defined operator. It is most natural
407
(16)
H is called the Hankel operator of . Its matrix representation in the canonical bases is

y(0)
y(1)
y(2)
..
.
=

h(1)
h(2)
h(3)
..
.
h(2) h(3)
h(3) h(4)
h(4) h(5)
..
..
.
.

..
.

u(1)
u(2)
u(3)
..
.
H
(17)
Thus the Hankel operator of maps past inputs into future
outputs. It has a number of properties given next. The first
one for single-input single-output systems is due to Kronecker.
Proposition 2. Given the system defined by Eq. (8), the
rank of H is at most n. The rank is exactly n if, and only if,
the system is reachable and observable. Furthermore, if
is stable, H has a finite set of nonzero singular values.
In order to compute the singular values of the Hankel operator, we define the reachability matrix R and the observability matrix O :
R(A, B) = [B AB A2 B
],
O (C, A) = [R(A , C )]
408
both of these matrices have rank at most n. The reachability

and observability grammians are
P := RR =
At BB (A )t ,
Q := O O =
t0
(A )t CCAt
An important property of the Hankel singular values of is

that twice their sum provides an upper bound for the 2-induced norm of (see also the section entitled Error Bounds
of Optimal and Suboptimal Approximants).
t0
(18)
The quantities P and Q are n n symmetric, positive semidefinite matrices. They are positive definite if and only if is
reachable and observable. The grammians are (the unique)
solutions of the linear matrix equations:
AP A + BB = P ,
A QA + CC = Q
(19)
which are called discrete-time Lyapunov or Stein equations.

The (nonzero) singular values i of H are the square roots
of the (nonzero) eigenvalues i of H * H . The key to this computation is the fact that the Hankel operator can be factored
in the product of the observability and the reachability matrices:
H = O (C, A)R(A, B)
For ui m2 (), ui 0, there holds

2
= i2 ui RR
OO Rui = i Rui
Q
Thus, ui is an eigenfunction of H * H corresponding to the

nonzero eigenvalue i2 iff R ui 0 is an eigenfunction of the
product of the grammians P Q :
i2 (H ) = i (H H ) = i (PQ)
(20)
Proposition 3. The nonzero singular values of the Hankel

operator H associated with the stable system are the
square roots of the eigenvalues of the product of the grammians P Q .
Definition 1. The Hankel singular values of the stable system as in Eq. (10), denoted by
1 () > > q () with multiplicity

ri , i = 1, . . ., q,
q

ri = n
(21)
i=1
are the singular values of H defined by Eq. (16). The Hankel

norm of is the largest Hankel singular value:

The AAK Theorem. Consider the stable systems and of

dimension n and k, respectively. By Proposition 2, H has
rank n and H has rank k. Therefore, the SchmidtMirsky
theorem implies that
H H
k+1 (H )
2-ind
The question which arises is to find the infimum of the above

norm, given the fact that the approximant is structured (block
Hankel matrix): inf H H 2-ind. A remarkable result due
to Adamjan, Arov, and Krein, code-named AAK result, asserts
that this lower bound is indeed attained for some or dimension k. The original sources for this result are Refs. 2 and 3.
H H
2-ind
= k+1 (H )
(24)
If p m 1, the optimal approximant is unique.

The result says that every stable and causal system can
be optimally approximated by a stable and causal system *
of lower dimensions; the optimality is with respect to the 2induced norm of the associated Hankel operator (see Fig. 1).
The Main Result. In this section we will present the main
result. As it turns out, one can consider both suboptimal and
optimal approximants within the same framework. Actually,
as shown in the sections entitled State-Space Construction
for Square Systems: Suboptimal Case and State-Space Construction for Square Systems: Optimal Case, the formulas
for suboptimal approximants are simpler than their optimal
counterparts.
PROBLEM 2. Given a stable system , we seek approximants * satisfying
k+1 ()
< k ()
:= 1 ()
The Hankel operator of a not necessarily stable 2 system

is defined as the Hankel operator of its stable and causal part
: H : H .
( )
Thus, the Hankel norm of a system having poles both inside

and outside the unit circle is defined to be the Hankel norm
of its causal and stable part. In general,
(23)
Theorem 2. AAK Theorem. Given the 2pm() sequence of

matrices h (h(t))t 0, such that the associated Hankel matrix
H has finite rank n, there exists an 2pm() sequence of matrices h (h (t))t 0, such that the associated Hankel matrix
*
*
H has rank k and in addition
*
H H ui = i2 ui R O ORui
P
Lemma 1. Given the stable system with Hankel singular

values i, i 1, . . ., n (multiplicities included), the following
holds true: 2-ind 2(1 n).
2-ind
(22)
e
Figure 1. Construction of approximants.
This is a generalization of Problem 1, as well as the problem

solved by the AAK theorem. The concept introduced in the
next definition is the key to its solution.
Definition 2. Let e be the parallel connection of and :
e : . If e is an all-pass system with norm , is
called an -all-pass dilation of .
As a consequence of the inertia result of the section entitled
The Grammians, Lyapunov Equations, and an Inertia Result, the all-pass dilation system has the following crucial
property.
Main Lemma 1. Let be an -all-pass dilation of , where
satisfies Eq. (25). It follows that has exactly k poles inside
the unit disk, that is, dim k.
We also restate the analog of the SchmidtMirsky result [Eq.
(23)], applied to dynamical systems:
Proposition 4. Given the stable system , let have at
most k poles inside the unit disk. Then

k+1 ()
This means that the 2-induced norm of the Hankel operator

of the difference between and is no less than the (k
1)st singular value of the Hankel operator of . Finally, recall
that if a system has both stable and unstable poles, its Hankel norm is that of its stable part. We are now ready for the
main result which is valid for both discrete- and continuoustime systems.
Theorem 3. Let be an -all-pass dilation of the linear, stable, discrete- or continuous-time system , where
k+1 () < k ()
k+1 ()
stable approximant of in the 2-induced norm of the convolution operator (i.e., the norm).
(b) We are given a stable system and seek to compute an
approximant in the same class (i.e., stable). In order to
achieve this, the construction given above takes us outside
this class of systems, since the all-pass dilation system has
poles both inside and outside the unit circle. In terms of matrices, we start with a system whose convolution operator S
is a (block) lower triangular Toeplitz matrix. We then compute a (block) Toeplitz matrix S , which is no longer lower
triangular, such that the difference S S is unitary. It
then follows that the lower left-hand portion of S , which is
the Hankel matrix H , has rank r and approximates the Hankel matrix H , so that the 2-norm of the error satisfies
Eq. (25).
(c) The suboptimal and optimal approximants can be constructed using explicit formulae. For continuous-time systems, see the section entitled Construction Formulas for
Hankel-Norm Approximants.
CONSTRUCTION OF APPROXIMANTS
The purpose of this section is to present, and to a certain extent derive, formulas for suboptimal and optimal approximants in the Hankel norm. Because of Theorem 3, all we need
is the ability to construct all-pass dilations of a given system.
To this goal, the first subsection is dedicated to the presentation of important aspects of the theory of linear, continuoustime systems; these facts are used in the second subsection.
The closely related approach to system approximation by balanced truncation is briefly discussed in the section entitled
Balanced Realizations and Balanced Model Reduction.
Linear, Continuous-Time Systems
L 2 Linear Systems. For continuous-time functions, let
(25)
It follows that has exactly k stable poles and consequently

<
(26)
L n (I ) := { f : I Rn , I R}
Frequent choices of I : I , I or I . The 2-norm
of a function f is
In case k1() ,
k+1 () =
409
:=
tI
f (t) 22 dt
1/2
f L n (I )
The corresponding L 2 space of square-integrable functions is

Proof. The result is a consequence of the following sequence
of equalities and inequalities:
+
k+1 ()
=

L2n (I ) := { f L n (I ), f
Remark 3. (a) For 1(), the above theorem yields the

solution of the Nehari problem, namely to find the best anti-
< }
The 2-norm of the matrix function F : I pm, is defined as
=
The first inequality on the left side is a consequence of Main

Lemma 1, the equality follows by definition, the second inequality follows from Eq. (22), and the last equality holds by
construction, since is -all-pass.

F
:=
t I
F (t)
2
F
1/2
dt
where the subscript F denotes the Frobenius norm. The

space of all p m matrix functions having finite 2-norm is
denoted by L 2pm(I ). Let A : X Y, where X and Y are subspaces of L 2q(), for some q. The 2-induced norm of A is defined as in Eq. (1), and Eq. (2) holds true.
410
We will consider linear, finite-dimensional, time-invariant,

continuous-time systems described by the following set of differential and algebraic equations:
:
dx(t)
= Ax(t) + Bu(t)
,
dt
tR
where u(t) m, x(t) n, and y(t) p are the values of the

input, state, and output at time t, respectively. The system
will be abbreviated as
A
=
C
B
D
(n+ p)(n+m)
(27)
In analogy to the discrete-time case, we define the continuous-time unit step function I(t) ( I(t) 1 for t 0, and zero
otherwise) and the delta distribution . The impulse response
of this system is
h (t) = CeAt BI(t) + (t)D
(28)
H (s) = C(sI A)1 B + D

Unless A has all its eigenvalues in the left half-plane (LHP),
the impulse response h is not square-integrablethat is,
does not belong to the space L 2pm(). In this case we shall say
that the system is not an L 2 system. If, however, A has no
eigenvalues on the j axis, it can be interpreted as an L 2
system by appropriate redefinition of the impulse response.
As in the discrete-time case, let there be a state transformation such that
A=
A+
0
0
A

B=
B+
B
The Convolution Operator. The convolution operator associated to is defined as follows:
S : u y,
C = (C+
C )
h2 := h2+ + h2
2-ind
:= S
h2 (t) := C e
tR
(30)
(29)
B+ I(t) + (t)D
B I(t)
As in the discrete-time case,
H2 (s) = H2+ (s) + H2 (s) = C+ (sI A+ )1 B+

+ D + C (sI A )1 B = H (s)
This is equivalent to trading the lack of stability (poles in the
RHP) to the lack of causality (h2 is nonzero for negative
time). What is modified is the region of convergence from
Re(max (A )) < Re(s) to
Re(max (A+ )) < Re(s) < R(min (A ))
2-ind
= H
(31)
The Grammians, Lyapunov Equations, and an Inertia Result.

For stable systems (i.e., R e(i(A)) 0), the following quantities are defined:
eAt BB eA t dt,
Q :=
eA t CCeAt dt
(32)
They are called the reachability and observability grammians

of , respectively. By definition, P and Q are positive semidefinite. It can be shown that reachability and observability
of are equivalent to the positive definiteness of each one of
these grammians. As in the discrete-time case, the grammians are the (unique) solutions of the linear matrix equations
AP + P A + BB = 0,
A t
h (t )u( ) d ,
If is all-pass [Eq. (14)], then 2-ind H .
where
h2+ (t) := C+ e
Proposition 5. Let * : inf min(H( j)), while * :

sup max(H( j)). Every in the interval [ , *] is a singu*
lar value of the operator S .
P :=
where the eigenvalues of A and A are in the LHP and right

half-plane (RHP), respectively. The L 2-impulse response is
defined as follows:
A+ t
Corresponding to Proposition 1 we have the following:

,
where y(t) = (h u)(t) :=
Since the singular values of S form a continuum, the same

conclusion as in the discrete-time case follows (see also Remark 2). Again, the 2-induced norm of turns out to be the
infinity norm of the transfer function H:
while the transfer function is
From now on, all continuous-time systems with no poles on

the imaginary axis will be interpreted as L 2 systems; to keep
the notation simple, they will be denoted by instead of 2.
The stable and causal subsystem will be denoted by , and
the stable but anti-causal one will be denoted by :
.
A Q + QA + CC = 0
(33)
which are known as the continuous-time Lyapunov equations

[cf. Eq. (19)]. Such equations have a remarkable property
known as inertia result: There is a relationship between the
number of eigenvalues of A and (say) P in the LHP, RHP,
and the imaginary axis. More precisely, the inertia of A
nn is
in (A) := {(A), (A), (A)}
(34)
where (A), (A), and (A) are the number of eigenvalues of A

in the LHP, on the imaginary axis and in the RHP, respectively.
Proposition 6. Let A and X X* satisfy the Lyapunov equation: AX XA* R, where R 0. If the pair (A, R) is reach-
able, then the inertia of A is equal to the inertia of X:

in(A) in(X).
The Hankel singular values of H and their multiplicities will

be denoted as in Eq. (21). Finally,

Main Lemma 1 is based on this inertia result.

Remark 3. One of the reasons why continuous-time formulas
are simpler than their discrete-time counterparts is that the
Lyapunov equations [Eqs. (34)] are linear in A, while the
discrete-time Lyapunov equations [Eqs. (19)] are quadratic
in A.
All-Pass L 2 Systems. All-pass systems are a subclass of L 2
systems. As indicated in Theorem 3, all-pass systems play an
important role in Hankel norm approximation. The following
characterization is central for the construction of suboptimal
and optimal approximants (see sections entitled State-Space
Construction for Square Systems: Suboptimal Case and
State-Space Construction for Square Systems: Optimal
Case). The proof is by direct computation; for details see
Ref. 7.
Proposition 7. Given is as in Eq. (27) with p m. The
following statements are equivalent.
is -all-pass.
For all inputoutput pairs (u, y) satisfying y h u,
there holds: y2 u2.
H* (j)H( j) 2Im
There exists Q Q * nn, such that the following
equations are satisfied:
A Q + QA + CC = 0
QB + C D = 0
(35)
AP + P A + BB = 0,
2-ind
A Transformation Between Continuous- and Discrete-Time Systems. As mentioned in the introductory paragraphs of this article, the formulas for the construction of approximants will
be given for the continuous-time case because they are generally simpler than their discrete-time counterparts. Thus, in
order to apply the formulas to discrete-time systems, the system will have to be transformed to continuous-time first, and
at the end the continuous-time optimal or suboptimal approximants obtained will have to be transformed back to discretetime approximants.
One transformation between continuous- and discrete-time
systems is given by the bilinear transformation z (1
s)/(1 s) of the complex plane onto itself. The resulting relationship between the transfer function Hc(s) of a continuoustime system and that of the corresponding discrete-time system Hd(z) is
Hc (s) = Hd
1 + s
1s
The state space maps
A
C
B
,
D
d :=
F
H
G
J
are related as given in Table 1. Furthermore, the proposition

that follows states that the Hankel and infinity norms remain
unchanged by this transformation.
A Q + QA + CC = 0
The Continuous-Time Hankel Operator. In analogy to the

discrete-time case, we define the operator H of the stable
system which maps past inputs into future outputs:
u y+ , where y+ (t) =
= +
Lemma 2. Let the distinct Hankel singular values of the stable system be i, i 1, . . ., q; following holds: 2-ind
2(1 q).
c :=
satisfy P Q 2In, and in addition we have D*D 2Im.
H : L2m (R ) L2p (R+ )
Lemma 1 can be strengthened in the continuous-time case

(see also the section entitled Error Bounds of Optimal and
Suboptimal Approximants).
D D = 2 Im
The solutions P and Q of the Lyapunov equations
411
Proposition 8. Given the stable continuous-time system c

with grammians P c and Q c, let d with grammians P d and
Q d, be the discrete-time system obtained by means of the
transformation given above. It follows that P c P d and
Q c Q d. Furthermore, this bilinear transformation also preserves the infinity norms (i.e., the 2-induced norms of the
associated convolution operators): c d.
h (t )u ( ) d ,
t0
(36)
H is called the Hankel operator of . Unlike in the discretetime case, however (see Eq. (17)), H has no matrix representation.
It turns out that just as in Eq. (20), it can be shown that
the nonzero singular values of the continuous-time Hankel
operator are the eigenvalues of the product of the two grammians:
i2 (H ) = i (H H ) = i (PQ)
(37)
Table 1. Transformation Formulas

Continuous-Time
Discrete-Time
A, B, C, D
z
1s
1s
A (F I)1(F I)
B 2(F I)1G
C 2H(F I)1
D J H(F I)1G
s
z1
z1
F (I A)(I A)1
G 2(I A)1B
H 2C(I A)1
J D C(I A)1B
F, G, H, J
412
Construction Formulas for Hankel-Norm Approximants

We are ready to give some of the formulas for the construction
of suboptimal and optimal Hankel-norm approximants. As
mentioned earlier, all formulas describe the construction of
all-pass dilation systems [see Eq. (3)]. We will concentrate on
the following cases (in increasing degree of complexity):
An inputoutput construction applicable to scalar systems. Both optimal and suboptimal approximants are
treated. The advantage of this approach is that the equations can be set up in a straightforward manner using
the numerator and denominator polynomials of the
transfer function of the given system (see the section
entitled InputOutput Construction Method for Scalar
Systems).
A state-space-based construction method for suboptimal
approximants (see the section entitled State-Space Construction for Square Systems: Suboptimal Case) and for
optimal approximants (see the section entitled StateSpace Construction for Square Systems: Optimal Case)
of square systems.
A state-space-based parameterization of all suboptimal
approximants for general (i.e., not necessarily square)
systems (see the section entitled General Case: Parameterization of All Suboptimal Approximants).
The optimality of the approximants is with respect to the
Hankel norm (2-induced norm of the Hankel operator).
The section entitled Error Bounds of Optimal and Suboptimal Approximants gives an account of error bounds
for the infinity norm of approximants (2-induced norm of
the convolution operator).
The section entitled Balanced Realizations and Balanced Model Reduction discusses model reduction by
balanced truncation which uses the same ingredients as
the Hankel norm model reduction theory. The approximants have a number of interesting properties, including
the existence of error bounds for the infinity norm of the
error. However, no optimality holds.
InputOutput Construction Method for Scalar Systems. Given
the polynomials a i0 aisi, b i0 bisi, and c i0 cisi

satisfying c(s) a(s)b(s), the coefficients of the product c are
a linear combination of those of b:
b
c = T(a)b
where c : (c c1 . . . c1 c0)* 1, b : (b . . . b0)*
1, and (a) is a Toeplitz matrix with first column (a . . .
a0 0 . . . 0)* 1, and first row (a 0 . . . 0) 1(1).
We will also define the sign matrix
K = diag(
...
, 1, 1, 1)
of appropriate size. Given a polynomial a with real coefficients, the polynomial c* is defined as c(s)* : c(s). This
means that
c = K c
The basic construction given in Theorem 3 hinges on the construction of an -all-pass dilation of . Let
H (s) =
p(s)
,
q(s)
H (s) =
p(s)
q(s)
We require that the difference H H He be -all-pass.

Therefore, the problem is as follows: Given and the polynomials p and q such that deg(p) deg(q) : n, find polynomials p and q of degree at most n such that
p
q q
p
=
pq q p = q q
q
q
qq
(38)
This polynomial equation can be rewritten as a matrix equation involving the quantities defined above:
T(p)q T(q) p = T(q )q = T(q )K q

Collecting terms we have
(T(p) T(q )K ,
T(q))

q
p
=0
(39)
The solution of this set of linear equations provides the coefficients of the -all pass dilation system . Furthermore, this
system can be solved for both the suboptimal i and the
optimal i cases. We will illustrate the features of this
approach by means of a simple example. For an alternative
approach along similar lines, see Ref. 9.
Example. Let be a second-order system, that is, n 2. If
we normalize the coefficient of the highest power of q, that
is, q2 1, we obtain the following system of equations:

0
p2 q2
p1 + q1
p0 q0
0
0
0
p2 + q2
p1 q1
p0 + q0

W ()
q2
q1
q0
0
0
0
q2
q1
q0
0
0
0
q2
q1
q0

q 1
q 0
p 2
p 1
p 0
p2 + q2
p1 q1
p0 + q0
0
0
This can be solved for all which are not roots of the equation
det W() 0. The latter is a polynomial equation of second
degree; there are thus two values of , 1, and 2, for which
the determinant of W is zero. It can be shown that the roots of
this determinant are the eigenvalues of the Hankel operator
H ; since in the single-input single-output case H is selfadjoint (symmetric), the absolute values of 1 and 2, are the
singular values of H . Thus both suboptimal and optimal approximants can be computed this way. (See also the first example of the section entitled Examples.)
State-Space Construction for Square Systems: Suboptimal
Case. In this section we will discuss the construction of ap-
proximants in the case where m p. Actually this is no loss

of generality because if one has to apply the algorithm to a
nonsquare system, additional rows or columns of zeros can be
added so as to make the system square. For details we refer
to Ref. 4. Consider the system as in Eq. (27), with
R e(i(A)) 0 and m p. For simplicity it is assumed that
the D-matrix of is zero. Compute the grammians P and Q
by solving the Lyapunov equations [Eqs. (34)]. We are looking
for
B
D
A
C
=

such that the parallel connection of and denoted by e, is

-all-pass. First, note that e has the following state-space
representation:
A
B
B
A
C
e =
C
(40)
According to the last characterization of Proposition 7, e is all-pass iff the corresponding grammians and D matrices satisfy
= 2 Im
D
D
Pe Qe = 2 I2n ,
(41)
/ is a unitary matrix of size m. The key

This implies that D
to the construction of P e and Q e, is the unitary dilation of P
and Q . Consider the simple case where the grammians are
scalar P p, Q q, and 1. In this case we are looking
for filling in the ? so that
p
?
?
?
q
?

?
?
1
0
0
1
assuming that pq 1. It readily follows that one solution is
p
1 pq
q
1 pq
q(1 pq)
1
p
1 pq

=
1
0
0
1

(42)
In the general (nonscalar) case, this suggests defining the

quantity:
:= 2 In PQ
(43)
Assuming that is not equal to any of the eigenvalues of the

product P Q (i.e., to any of the singular values of the Hankel
operator H ), is invertible. Keeping in mind that P and Q
do not commute, the choice of P e and Q e corresponding to Eq.
(42) is
Pe =
P

Q
Qe =
Q
In
In
1 P
(44)
Once the matrices P e and Q e for the dilated system e have

been constructed, the next step is to construct the matrices
, B
of the dilation system . According to Eq. (35) of
, and C
A
Proposition 6, the Lyapunov equation A*e Q e Q eAe C*e Ce
0 and the equation Q eBe C*e De 0 have to be satisfied.
413
Solving these two equations for the unknown quantities, we

obtain
)
A = A + C (CP DB
B = QB + C D
(45)
)
C = (CP DB
has exwhere * : (*)1. There remains to show is that A
actly k stable eigenvalues. From the Lyapunov equation for
1P . By construction [see Eq.
Q e it also follows that Q
(43)], has k positive and n k negative eigenvalues; the
(i.e., in (Q
) k, 0, n k). Furthersame holds true for Q
, A
is observable, because otherwise A has
more, the pair C
eigenvalues on the j axis, which is a contradiction to the
original assumption that A is stable. Then by Proposition 6,
, which completes the
is equal to that of A
the inertia of Q
proof.
Corollary 1. The system has dimension n and the dilated
system e has dimension 2n. The stable subsystem has
dimension k.
State-Space Construction for Square Systems: Optimal Case.
The construction of the previous section can be extended to
include the optimal case. This section presents the formulae
which first appeared in Section 6 of Ref. 4.
In this case we need to construct the all-pass dilation e
for equal to the lower bound in Eq. (25), namely k1().
Thus, k1 is an eigenvalue of the product of the grammians
P Q of multiplicity, say r. There exists a basis change in the
state space such that
P =
Ir k+1
0
0
P2
Q=
Ir k+1
0
0
Q2
(46)
The balancing transformation Eq. (60) discussed in the section entitled Balanced Realizations and Balanced Model Reduction accomplishes this goal.
Clearly, only the pair P 2 and Q 2 needs to be dilated. As in
the suboptimal case, explicit formulae for can be obtained
by using Eq. (35) of Proposition 7. Partition A, B, and C conformally with P and Q :
A=
A11
A21
A12
A22

B=
B1
B2
C = (C1
C2 )
where A11 rr, B1, C*1 rm, and Ir denotes the r r

identity matrix. The (1, 1) block of the Lyapunov equations
yields B1B*1 C*1 C1; this implies the existence of a unitary
matrix U of size m, such that
B1U = C1 ,
UU = Im
Using Eqs. (45) we construct an all-pass dilation of the subsystem (A22, B21, C12); solving the corresponding equations we
obtain
= U
D
k+1
B = Q2 B2 + C2 D
)( )1
C = (C2 P2 DB
2
2
A = A22 + C2C
(47)
414
2
where 2 : k1
P 2Q 2 (nr)(nr). Hence unlike the subop is not arbitrary. In the single-input single-outtimal case, D
is completely determined by the above relationput case, D
ship (is either 1 or 1) and hence there is a unique optimal
approximant. This is not true for systems with more than one
input and/or more than one output. Furthermore, in the onestep reduction case (i.e., k1 n) the all-pass dilation system
has no antistable part; that is, it is the optimal Hankelnorm approximant.
Corollary 2. (a) In contrast to the suboptimal case, in the

optimal case, has dimension r, and the dilated system e
has dimension 2n r. Furthermore, the stable subsystem
has dimension n r. (b) In the case where k1 n, since
, is stable and all-pass with norm n.
General Case: Parameterization of All Suboptimal Approximants. Given the system as in Eq. (27), let satisfy Eq.
(25). The following rational matrix (s) has size (p m)
(p m):
1
(s) := I p+m C (sI A )1 Q
C J
where

C :=
A :=
Q :=
C
0
0
B
A
0
0
A

In
P
Ip
0
0
Im
R2n2n
R2n2n ,
(s) =
:=
11
21
I
12
22
1 ( )
2 ( )
(s)
:= (s)
Im

=
C(sI A)1 1 B
Im B (sI + A )1 Q 1 B
(49)
11 (s) (s) + 12 (s)

21 (s) (s) + 22 (s)
(50)
The proof of the following result can be found in Chap. 24 of

Ref. 10. Recall Theorem 3.
Theorem 4. is an -all-pass dilation if and only if
H H = 1 ( )2 ( )1
(52)
We now approximate q by q1 through a one-step optimal

Hankel-norm reduction. By successive application of the same
procedure, we thus obtain a sequence of all-pass systems i,
i 1, . . ., q, and a system 0 consisting of the matrix D0,
such that the transfer function of is decomposed as follows:
H (s) = D0 + H1 (s) + + Hq (s)
D0
i = 1, . . ., n rq
(53)
H1 (s) + + Hq (s)

q
Evaluating the above expression at infinity yields
1 1
P C
p + C(sI A)
B (sI + A )1 C
i (q ) = i (),
By construction, is J-unitary on the j axis; that is,

*(j)J( j) J. Define
Thus q is all-pass of magnitude q, and consequently we

have
H (s) D0
R( p+m)( p+m)
Putting these expressions together we obtain
q)
q := (
where Hk(s) is the transfer function of the stable all-pass sysq

tem k having dimension 2n ik ri, k 1, . . ., q; the dik
k
mension of the partial sums i1 Hi(s) is equal to i1 ri, k
1, 2, . . ., q. Thus Eq. (53) is the equivalent of the dyadic
decomposition Eq. (4), for .
From the above decomposition we can derive the following
upper bound for the H norm of . Assuming that H()
D 0, we obtain
R( p+m)2n
Q
In
J :=
(48)
Error Bounds of Optimal and Suboptimal Approximants. Given

is the stable m m system , having Hankel singular values
i and multiplicity ri, i 1, . . ., q [see Eq. (21)]. Let be the
-all-pass dilation of , where q (the smallest singular
value). Following Corollary 2, the dimension of is n q,
which implies that ; that is, it is stable. Thus by the
same corollary, for a one-step reduction, the all-pass dilation
system has no unstable poles; for simplicity we will use the
notation
(51)
where (s) is a p m anti-stable contraction (all poles in the

right half-plane and (s) 1).
1 + + q
Thus, combining this inequality with H(s) D02 1

q, yields the bound given in Lemma 2:
H (s)
2(1 + + q )
(54)
This bound can be sharpened by computing an appropriate

D0 mm as in Eq. (53):
H (s) D0
1 + + q
(55)
Finally, we state a result of Ref. 4, on the Hankel singular

values of the stable part e of the all-pass dilation e. It will
be assumed for simplicity that each Hankel singular value
has multiplicity one: ri 1. Assume that e , where
is an optimal Hankel approximant of of dimension k; the
following holds: 1(e) 2k1(e) k1(),
2k2(e) 1( ) k2(), . . ., nk(e) nk1( )
n(). Using these inequalities we obtain an error bound for
the H norm of the error of with a degree k optimal approximant . First, note that
:= D0
) + + nk1 (
)
1 (
This implies D0 k1() . Combining all

previous inequalities we obtain the bounds:
+ D0
k+1
)
k+1 () + 1 (
) k+1 () + + n ()

+ + nk1 (
(56)
The left-hand-side inequality in Eq. (25), and the above finally

yield upper and lower bounds on the H norm of the error:
+
k+1
2(k+1 + + n )
(57)
The first upper bound in Eq. (56) above is the tightest; but
both a D term and the singular values of the antistable
part of the all-pass dilation are needed. The second upper
bound in Eq. (56) is the second-tightest; it only requires
knowledge of the optimal D term. Finally the upper bound in
Eq. (57) is the least tight among these three. It is, however,
the most useful, since it can be determined a priori, that is,
before the computation of the approximants, once the singular values of the original system are known.
Remark 4. (a) In a similar way a bound for the infinity norm
of suboptimal approximants can be obtained. Given , let
be an all-pass dilation satisfying Eq. (25). Then, similarly to
Eq. (56), the following holds:
+
r+1
2( + r+1 + + n )
(b) For a one-step Hankel-norm reduction, is allpass; that is,

=
= q
which shows that in this case is an optimal approximant

not only in the Hankel norm but in the infinity norm as well.
Thus for this special case, and despite Propositions 1 and 5,
the Hankel-norm approximation theory yields a solution for
the optimal approximation problem in the 2-norm of the convolution operator S ; recall that this is the problem we initially wished to solve.
Balanced Realizations and Balanced Model Reduction
the input is zero (u 0), is equal to

xo Qxo
(59)
We conclude that the states which are difficult to reachthat

is, those which require a large amount of energy to reach
are in the span of the eigenvectors of the reachability grammian P corresponding to small eigenvalues. Moreover, the
states which are difficult to observethat is, those which
yield small amounts of observation energyare those which
lie in the span of the eigenvectors of the observability grammian Q corresponding to small eigenvalues as well.
This observation suggests that one way to obtain reduced
order models is by eliminating those states which are difficult
to reach and/or difficult to observe. However, states which are
difficult to reach may not be difficult to observe, and vice
versa; as simple examples show, these properties are basisdependent. This suggests the need for a basis in which states
which are difficult to reach are simultaneously difficult to observe, and vice versa. From these considerations, the question
arises: Given a continuous- or discrete-time stable system ,
does there exist a basis of the state space in which states
which are difficult to reach are also difficult to observe?
The answer to this question is affirmative. The transformation which achieves this goal is called a balancing transformation. Under an equivalence transformation T (basis change
TAT1,
in the state space: x : Tx, det T 0) we have A
CT1, while the grammians are transformed
TB, and C
B
as follows:
P = TP T ,
= T QT 1 P Q
= T (PQ)T 1
Q
The problem is to find T, det T 0, such that the transformed

and Q
are equal. This will ensure that the
grammians P
states which are difficult to reach are precisely those which
are difficult to observe.
Definition 3. The stable system is balanced iff P Q .
is principal-axis balanced iff
P = Q = S := diag(1 , . . ., n )
The existence of a balancing transformation is guaranteed.
A model reduction method which is closely related to the Hankel-norm approximation method is approximation by balanced truncation. This involves a particular realization of a
linear system given by Eq. (27), called balanced realization
(the D matrix is irrelevant in this case). We start by explaining the rational behind this particular state-space realization.
The Concept of Balancing. Consider a stable system with
positive definite reachability and observability grammians P
and Q . It can be shown that the minimal energy required for
the transfer of the state of the system from 0 to some final
state xr is
xr P 1 xr
415
(58)
Similarly, the largest observation energy produced by observing the output, when the initial state of the system is xo and
Lemma 3. Balancing Transformation. Given the stable

system and the corresponding grammians P and Q , let the
matrices R, U, and S be defined by P : R*R and RQ R* :
US2U*. A (principal axis) balancing transformation is given
by
T := S1/2U R
(60)
where (as before) R* : (R*).1

To verify that T is a balancing transformation, it follows
by direct calculation that TP T* S and T*Q T1 S. We
also note that if the Hankel singular values are distinct (i.e.,
are deterhave multiplicity one), balancing transformations T
mined from T given above, up to multiplication by a sign matrix Lthat is, a diagonal matrix with 1 on the diagonal:
LT.
T
416
Model Reduction. Let be balanced with grammians equal

to S, and partition:
A=
B=
A11
A21
A12
A22

B1
B2
The systems
i :=
S=
S1
0
0
S2
,
(61)
C = (C1 C2 )
A
ii
Bi
Ci
i = 1, 2
are called reduced-order systems obtained form by balanced

truncation. These have certain guaranteed properties. However, these properties are different from discrete- and continuous-time systems. Hence we state two theorems. For a proof
see, for example, Ref. 6.
Theorem 5. Balanced Truncation: Continuous-Time Systems. Given the stable (no poles in the closed right halfplane) continuous-time system , the reduced-order systems
i, i 1, 2, obtained by balanced truncation have the following properties:
1. i, i 1. 2, satisfy the Lyapunov equations AiiSi
SiA*ii BiB*i 0, and A*ii Si SiAii C*i Ci 0. Furthermore, Aii, i 1, 2, have no eigenvalues in the open right
half-plane.
2. If S1 and S2 have no eigenvalues in common, both 1
and 2 have no poles on the imaginary axis and are, in
addition, reachable, observable, and balanced.
3. Let the distinct singular values of be i, with multiplicities mi, i 1, . . ., q. Let 1 have singular values
i, i 1, . . ., k, with the corresponding multiplicities
mi, i 1, . . ., k, k q. The H norm of the difference
between the full-order system and the reduced-order
system 1 is upper-bounded by twice the sum of the neglected Hankel singular values:
1
2(k+1 + + q )
(62)
If the smallest singular value is truncatedthat is,

S2 qIrq equality holds.
Theorem 6. Balanced Truncation: Discrete-Time Systems. Given the stable, discrete-time system (no poles in the
complement of the open unit disk) , the reduced-order systems i obtained by balanced truncation have the following
properties:
1. i, i 1, 2, have no poles in the closed unit disk; these
systems are, in general, not balanced.
2. If min(S1) max(S2), 1 is, in addition, reachable and
observable.
3. The h norm of the difference between full- and reduced-order models is upper-bounded by twice the sum
of the neglected Hankel singular values, multiplicities
included:
1
2 trace(S2 )
Remark 5. (a) The last part of the above theorems says that
if the neglected singular values are small, then the Bode plots
of and 1 are guaranteed to be close in the H norm. The
difference between part 3 for continuous- and discrete-time
systems above is that the multiplicities of the neglected singular values do not enter in the upper bound for continuoustime systems.
(b) Proposition 8 implies that the bilinear transformation
between discrete- and continuous-time systems preserves balancing (see also A Simple Discrete-Time Example below).
(c) Let hank and bal be the reduced-order systems obtained
by one step Hankel-norm approximation, and one step balanced truncation, respectively. It can be shown that hank
bal is all-pass with norm q; it readily follows that
bal
2q
and
bal
2q
A consequence of the above inequalities is that the error for

reduction by balanced truncation can be upper-bounded by
means of the singular values of as given in Theorem 5. Furthermore, this bound is valid both for the H norm and for
the Hankel norm.
(d) Every linear, time-invariant, continuous-time and stable system , can be expressed in balanced canonical form. In
the generic case (distinct singular values) this canonical form
is given in terms of 2n positive numbers, namely the singular
values i 0, and some bi 0, as well as n signs si 1,
i 1, . . ., n. The quantities i : sii are called signed singular values of ; they satisfy H(0) 2(1 n). For
details on balanced canonical forms we refer to the work of
Oberfor example, Ref. 11.
EXAMPLES
In this section we will illustrate the results presented above
by means of three examples. The first deals with a simple
second-order continuous-time system. The purpose is to compute the limit of suboptimal Hankel-norm approximants as
tends to one of the singular values. The second example discusses the approximation of a discrete-time system [third-order finite impulse response (FIR) system] by balanced truncation and Hankel-norm approximation. The section concludes
with the approximation of the four classic analog filters (Butterworth, Chebyshev 1, Chebyshev 2, and Elliptic) by balanced truncation and Hankel-norm approximation.
A Simple Continuous-Time Example
Consider the system given by Eq. (27), where n 2, m
p 1, and
1
1
21
1 + 2
A=
1
1
1 + 2
22
C = (1 1),
D=0
B=
1
,
1
where 1 2. This system is in balanced canonical form; this

means that the grammians are P Q diag(1, 2) : S;
this canonical form is a special case of the forms discussed in
Ref. 11.
We wish to compute the suboptimal Hankel-norm approximants for 1 2. Then we will compute the limit of this
family for 2 and 1, and we will show that the system obtained is indeed the optimal approximant. From Eq.
(43), 2I2 S2 diag(2 12, 2 22); the inertia of is
1, 0, 1; furthermore, from Eq. (45) we obtain
A =
B =
C =

1
21 ( + 1 )
2
(1 + 2 )( + 1 )
1
2
1
+ 1
,
1
+ 2
1
(1 + 2 )( + 2 )
2
22 ( + 2 )
Approximation by Balanced Truncation. A balanced realization of this system is given by
C =
2 1
22 (1 + 2 )
2 1
21 (1 + 2 )
0
1
1 + 2
1
22
B =
2 1
,
21 (1 + 2 )
d,2 =
,
d,1 =
1 2
,
21 (1 + 2 )

=
D
2
B = 2 1 ,
C=
1
,
1 + 2
=
D
2
. If we choose it to
Equations (45) depend on the choice of D
be , the limit still exists and gives a realization of the optimal system which is equivalent to A, B, C, D given above.
Finally, if 1, after a pole-zero cancellation, we obtain
the following reachable and observable approximant:
A =
3 55
10
B = 1 2 ,
C =
1
,
1 + 2
=
D
1
c =
A
C
In this section we will consider a third-order discrete-time

FIR system described by the transfer function:
We will first consider approximation by balanced truncation.

The issue here is to examine balanced truncation first in discrete-time and then in continuous-time, using the bilinear
transformation of the section entitled A Transformation Between Continuous- and Discrete-Time Systems, and compare
the results. Subsequently, Hankel-norm approximation will
be investigated.
G2
J2
F1
H1
G1
J1
0
=

=
0
0
0
0
1 2 2
2 2
2
2 2
+
1
2
2
1 + 2 2
where 2( ). Notice that c is balanced. We now

compute first and second reduced-order systems c,1 and c,2
by truncating c:
A Simple Discrete-Time Example
(63)
F2
H2
B
=
D
c,2 =
z2 + 1
z3
5+1
2
Notice that d,2 is balanced, but has singular values which are
different from 1 and 2. d,1 is also balanced since G1 H2,
but its grammians are not equal to 1.
Let c denote the continuous-time system obtained from
d by means of the bilinear transformation described in the
section entitled A Transformation Between Continuous- and
Discrete-Time Systems
This is the best antistable approximant of that is, the

Nehari solution [see Remark 3(a)].
H(z) =
1 =
The second- and first-order balanced truncated systems are
This system is not reachable but observable (i.e., there is a

pole-zero cancellation in the transfer function). A state-space
representation of the reachable and observable subsystem is
A =
P = Q = S = diag(1 , 2 , 3 ),
51
3 =
2 = 1,
2
0
,
where the reachability and observability grammians are

equal and diagonal:
is equal to the inertia of , A

has one
Since the inertia of A
stable and one unstable poles (this can be checked directly by
is negative). As 2 we
noticing that the determinant of A
obtain
0
F G
d :=
=
H J
0
0
0

3 5+5
1/4
,
=
=5
,
10
=
D
A =
417
c,1 =
A2
C2
B2
D2
A1
C1
B1
D1
1 2 2
2
+

=
1 2 2
+
+
2
2
2
Let d,2 and d,1 be the discrete-time systems obtaining by

transforming c,2 and c,1 back to discrete-time:
d,2 =

d,1 =

F2
H 2
G 2
J
F1
H 1
G 1
J1
0
=

=
3 /2
( + )3 /2 2
( + )3 /2 2
2 ( 2 + )2 /(1 + 2 )
418
The conclusion is that d,2 and d,1 are balanced and different
from d,2 and d,1. It is interesting to notice that the singular
value of d,1 is 2 (1 1)/ 5, while that of d,1 is 1; 2
satisfies 2 2 1. Furthermore, the singular values of
d,2 are 52 /4, 52 /4 which satisfy the following interlacing
inequalities:
3 <
Hd,2 (z) = Hc,2
5 2 /4 < 2 < 5 2 /4 < 1
The numerical values of the quantities above are 1

1.618034, 3 0.618034, 0.66874, 1.08204,
0.413304, 2.47598, and .361241.
Hankel-Norm Approximation. The Hankel operator of the
system described by Eq. (63) is
Again using the transformation of the section entitled A

Transformation Between Continuous- and Discrete-Time Systems we obtain the following discrete-time optimal approximant:
01
BB0
BB
= B1
BB0
@
0
1
0
0
..
.
1
0
0
0
1
C
CC
CC
CC
A
.
0
0
0
0

Hd (z) Hd,2 (z) = 3
1
0
1
0
1
0
3
5
1
0
0
0
2
0
0
1
0
01
BB 0
BB
B3
=B0
BB 2
B@3
0
1
0
r 1
3 C
5 C
C
0 C
r CCA
1
1
(64)
where i, i 1, 2, 3, are as given earlier. It is tempting to

conjecture that the optimal second-order approximant is obtained by setting 3 0 in Eq. (64). The problem with this
procedure is that the resulting approximant does not have
Hankel structure.
To compute the optimal approximant, the system is first
transformed to a continuous-time system using the transformation of the section entitled A Transformation Between
Continuous- and Discrete-Time Systems; we obtain the
transfer function
Hc (s) = Hd
1 + s
1s
2(s3 s2 + s 1)
(s + 1)3
where Hd is the transfer function defined in Eq. (63). Applying

the theory discussed in the section entitled State-Space Construction for Square Systems: Optimal Case, we obtain the
following second-order continuous-time optimal approximant:
Hc,2 (s) =
z
z 2 3
1 3 z 2
z3 (z2 3 )
0
3
0
32
0
..
.
3
0
32
0
33
32
0
33
0
34
0
32
0
33
0
1
C
CC
CC
CC
CC
A
.
..
is
In this particular case the 3 3 principal submatrix of H
also an optimal approximant of the corresponding submatrix
of H .
0r
BB 15
0
BB 0
0
B r
3 B
@ 3
is all-pass with magnitude equal to 3 on the unit circle [as

predicted by Corollary 2(b)]. The corresponding optimal Hankel matrix of rank 2 is
..
r 1
3 C
5 C
C
0 C
r CCA
1
z+1
Notice that the optimal approximant is not an FIR system. It

has poles at 3. Furthermore, the error
The SVD of the 3 3 principal submatrix of H is
0r

BB 15
1
B 0
0 =B
BBr
0
@
z 1
(s2 1)
2
(1 3 )s + 21 s + (1 3 )
0
3
0
3
0
0r

BB 15
3
B 0
0
=B
BBr
32
@ 3
0
1
0
1 + 32
0
0
r 1
3 C
5 C
C
0 C
r CCA
1
0r
BB 15
0
B 0
0 B
B r
0 B
@ 3
0
3
0
0
1
0
r 1
3 C
5C
C
0 C
r CCA
1
(65)
Notice that the above decomposition can be obtained from Eq.

(64) by making use of the freedom mentioned in Eq. (7). Finally it is readily checked that the Hankel matrix consisting
of 1 as (1, 1) entry and 0 everywhere else is the optimal approximant of H of rank one. The dyadic decomposition [Eq.
(53)] of H is
H(z) = H1 (z) + H2 (z) + H3 (z) = 1

+ 3
1 + 3 z2
z3 (z2 3 )

1
1 3 z 2
+ 2
z
z(z2 3 )
Notice that each Hi is i-all-pass, and the degree of H1 is one,

that of H1 H2 is two, and finally that of all three summands
is three.
A Higher-Order Example
In our last example, we will approximate four well-known
types of analog filters by means of balanced truncation and
Hankel norm approximation. These are:
1.
2.
3.
4.
B Butterworth
C1 Chebyshev 1:
C2 Chebyshev 2:
E Elliptic:
1% ripple in the pass band (PB)

1% ripple in the stop band (SB)
0.1% ripple both in the PB and
in the SB
In each case we will consider 20th-order low-pass filters, with

pass-band gain equal to 1, and cut-off frequency normalized
to 1. Figure 2 shows the Hankel singular values of the fullorder models. It follows from these plots that in order to obtain roughly comparable approximation errors, B and C2 will
HSV Butterworth N = 20
HSV Chebyshev 1 N = 20
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
10
15
20
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
10
Butterworth
9.9998e-01
9.9952e-01
9.9376e-01
9.5558e-01
8.2114e-01
5.6772e-01
2.9670e-01
1.1874e-01
3.8393e-02
1.0402e-02
2.3907e-03
4.6611e-04
7.6629e-05
1.0507e-05
1.1813e-06
1.0622e-07
7.3520e-09
3.6804e-10
1.1868e-11
1.8521e-13
15
20
Chebyshev 1
9.5000e-01
9.5000e-01
9.5000e-01
9.4996e-01
9.4962e-01
9.4710e-01
9.3362e-01
8.8268e-01
7.5538e-01
5.5133e-01
3.3740e-01
1.8209e-01
9.8132e-02
6.3256e-02
5.2605e-02
5.0362e-02
5.0036e-02
5.0003e-02
5.0000e-02
5.0000e-02
10
15
) 9 ()
+ + 11 (
20
HSV Elliptic N = 20
1
have to be approximated by systems of lower order than C1

and E; we thus choose to approximate B and C2 by 8thorder models, and we choose to approximate C1 and E by
10th-order models. It is interesting to observe that for the
Chebyshev-2 filter the difference of the singular values, 13
20, is of the order 107. Thus C2 has an (approximate) 8thorder all-pass subsystem of magnitude 0.05. Consequently,
C2 cannot be approximated with systems of order 13 through
19. Similarly, the Chebyshev-1 filter has an all-pass subsystem of order 3; consequently approximations of order 1, 2, and
3 are not possible.
The subscript bal stands for approximation by balanced
truncation, the subscript hank stands for optimal Hankelnorm approximation, FOM stands for full-order model,
and ROM stands for reduced-order model.
Figure 3 gives the amplitude Bode plots of the error systems and tabulates their H norms and upper bounds. We
observe that the 10th-order Hankel-norm approximants of
C1 and E are not very good in the SB; one way for improving
them is to increase the approximation order; another is to
compute weighted approximants (see, e.g., Ref. 12). In Table
2, more detailed bounds for the approximants will be given in
the case where the approximant contains an optimal D-term
Bound 1 is the first expression on the right-hand side of Eq.
(56), and Bound 2 is the second expression on the righthand side of the same expression:
)
Bound 11 := 9 () + 1 (
HSV Chebyshev 2 N = 20
1
419
+ + 20 () =: Bound 12
)
Bound 21 := 11 () + 1 (
) 11 ()
+ + 9 (
+ + 20 () =: Bound 22
Finally, Figure 4 shows the amplitude Bode plots of the
ROMs obtained by Hankel and balanced reductions.
Chebyshev 2
8.9759e-01
7.3975e-01
5.1523e-01
3.0731e-01
1.6837e-01
9.5205e-02
6.3885e-02
5.3342e-02
5.0643e-02
5.0103e-02
5.0014e-02
5.0002e-02
5.0000e-02
5.0000e-02
5.0000e-02
5.0000e-02
5.0000e-02
5.0000e-02
5.0000e-02
5.0000e-02
10
15
20
Elliptic
9.8803e-01
9.8105e-01
9.6566e-01
9.3602e-01
8.8469e-01
8.0582e-01
6.9997e-01
5.7677e-01
4.5166e-01
3.3873e-01
2.4576e-01
1.7413e-01
1.2135e-01
8.3613e-02
5.7195e-02
3.9014e-02
2.6729e-02
1.8650e-02
1.3613e-02
1.0869e-02
Figure 2. Analog filter approximation: Hankel singular values

(curves and numerical values).
CONCLUSION
The computational algorithms that emerged from balancing
and Hankel-norm model reduction have found their way to
software packages. The Matlab toolboxes, robust control toolbox (13) and -toolbox (14), contain m-files which address the
approximation problems discussed above. Two such m-files
are sysbal and hankmr; the first is used for balancing and
balanced truncation, while the second is used for optimal
Hankel-norm approximation (including the computation of
the anti-stable part of the all-pass dilation).
We will conclude with a brief discussion of some articles
which were written since Glovers seminal paper (4), namely,
Refs. 9, 12, 1520.
Hankel-norm approximation or balanced truncation can be
applied to stable systems. The approximants are guaranteed
to be close to the original system, within the bounds given in
the section entitled Error Bounds of Optimal and Suboptimal
Approximants; the important aspect of this theory is that
these bounds can be computed a priorithat is, before computing the approximant. The bounds are in tems of the H
norm which is a well-defined measure of distance between
420

Butterworth: FOMROM
Chebyshev 1:: FOMROM
10
10
10
10
20
20
30
30
40
40
50
101
100
101
50
101
Chebyshev 2: FOMROM
10
10
10
20
20
30
30
40
40
101
100
101
Elliptic: FOMROM
10
50
100
101
50
101
100
Norm of the Error and Bounds
hank
bal
2( 9 + . . . + 20)
Butterworth
Chebyshev 2
0.0383
0.0506
0.0388
0.1008
0.0779
0.0999
0.1035
1.2015
11
hank
Chebyshev 1
Elliptic
0.3374
0.2457
0.4113
0.2700
bal 2( 11 + . . . + 20)
0.6508
0.3595
stable systems and has a direct relevance in feedback control theory.

When the system is unstable, however, Hankel-norm approximation may not be the right thing to do, since it would
not predict closeness in any meaningful way. To bypass this
difficulty Ref. 15 proposes approximating unstable systems
(in the balanced and Hankel norm sense) using the (normalized) coprime factors, derived from the transfer function. Another way to go about reducing unstable systems is by adopting the gap metric for measuring the distance between two
systems; the gap is a natural measure of distance in the context of feedback control. For details on this approach we refer
to the work by Georgiou and Smith (16), and references
therein.
1.9768
1.5818
101
Figure 3. Analog filter approximation:

Bode plots of the error systems for model
reduction by optimal Hankel-norm approximation (continuous curves), balanced truncation (dashdot curves), and
the upper bound [Eq. (57)] (dashdash
curves). The table compares the peak values of these Bode plots with the lower and
upper bounds predicted by the theory.
The second paper (9) authored by Fuhrmann uses the authors polynomial models for a detailed analysis of the Hankel
operator with given (scalar) rational symbol (i.e., single-input
single-output systems). Furhmanns computations follow a
different approach than the one used in Ref. 4. The Schmidt
pairs (singular vectors) are explicitly computed and hence the
optimal Hankel approximants are obtained. This analysis
yields new insights into the problem; we refer to Theorem 8.1
on page 184 of Ref. 9, which shows that the decomposition of
the transfer function in the basis provided by the Schmidt
pairs of the Hankel operator provides the balanced realization
of the associated state space representation. Another advantage is that it suggests the correct treatment of the polynomial equation [Eq. (38)], which yields a set of n linear equa-
Table 2. Various Upper Bounds for the Norm of the Error

Analog Filter
Optimal D
9
hank
Bound 11
Bound 12
Butterworth
Chebyshev 2
0.0384
0.1010
0.0384
0.0506
0.0389
0.1008
0.0390
0.5987
0.0517
0.6008
Optimal D
11
hank
Bound 21
Bound 22
0.2988
0.2441
0.3374
0.2458
0.4113
0.2700
0.5030
0.3492
0.9839
0.7909
Chebyshev 1
Elliptic

Butterworth: ROM
(Hankel and Balanced)
0
10
10
20
20
30
30
40
40
102
101
100
problem of approximating a finite-dimensional Hankel matrix

remains largely unknown. Some partial results are provided
in Ref. 20 which relate to results in Ref. 9.
Chebyshev 1: ROM
101
BIBLIOGRAPHY
1. G. W. Stewart and J. Sun, Matrix Perturbation Theory, New York:
102
Chebyshev 2: ROM
101
100
101
Elliptic: ROM
10
10
20
20
30
30
40
421
2. V. M. Adamjan, D. Z. Arov, and M. G. Krein, Analytic properties

of Schmidt pairs for a Hankel operator and the generalized
SchurTakagi problem, Math. USSR Sbornik, 15: 3173, 1971.
3. V. M. Adamjan, D. Z. Arov, and M. G. Krein, Infinite block Hankel matrices and related extension problems, Amer. Math. Soc.
Trans., 111: 133156, 1978.
4. K. Glover, All optimal Hankel-norm approximations of linear
multivariable systems and their L -error bounds, Int. J. Control,
39: 11151193, 1984.
5. M. Green and D. J. N. Limebeer, Linear Robust Control, Upper
Saddle River, NJ: Prentice-Hall, 1995.
40
101

Figure 4. Analog filter approximation: Bode plots of the reducedorder models obtained by balanced truncation (dashdot curves) and
Hankel-norm approximation (continuous curves).
7. A. C. Antoulas, Lectures on optimal approximation of linear systems, Draft, Dept. Elec. Comp. Eng., Rice Univ., Houston, TX,
1998.
102
101
100
101
102
101
100
8. P. A. Fuhrmann, Linear Systems and Operators in Hilbert Space,

tions, instead of 2n 1 as in Eq. (39). Further developments

along the same lines are given in Ref. 21.
Weighted Hankel-norm approximation was introduced in
Ref. 17. Reference 12 presents a new frequency-weighted balanced truncation method. It comes with explicit L norm error bounds. Furthermore, the weighted Hankel-norm problem
with anti-stable weighting is solved. These results are applied
to L -norm model reduction by means of Hankel-norm approximation.
Reference 18 discusses a rational interpolation approach to
Hankel-norm approximation.
The next article (19) addresses the issue of balanced truncation for second-order form linear systems. These are systems which arise in applications and which are modeled by
position and velocity vectors. Thus in simplifying, if one decides to eliminate the variable qi, the derivative qi should be
eliminated as well, and vice versa. Toward this goal, new sets
of invariants are introduced and new grammians as well.
However, no bounds on the H norm of the error are provided.
The question whether in bounds in Eqs. (57) and (62) are
tight arises. It can be shown that this property holds for systems having zeros which interlace the poles; an example of
such systems are RC circuits [this result follows from Eq. (57)
and Remark 5(d), after noticing that the Hankel operator in
this case is positive definite]. For an elementary exposition of
this result we refer to Ref. 22; for a more abstract account,
see Ref. 23.
The problems of approximating linear operators in the 2induced norm, which are (a) finite-dimensional, unstructured
and (b) infinite-dimensional structured (Hankel), have been
solved. The solutions of these two problems exhibit striking
similarities. These similarities suggest the search of a unifying framework for the approximation of linear operators in
the 2-induced norm. For details see Ref. 24. In particular the
9. P. A. Fuhrmann, A polynomial approach to Hankel norm and

balanced approximations, Linear Alg. Appl., 146: 133220, 1991.
10. J. A. Ball, I. Gohberg, and L. Rodman, Interpolation of Rational
Matrix Functions, Basel: Birkhauser Verlag, 1990.
11. R. Ober, Balanced parameterization of classes of linear systems,
12. K. Zhou, Frequency-weighted L norm and optimal Hankel norm
model reduction, IEEE Trans. Autom. Control, 40: 16871699,
1995.
13. R. Y. Chiang and M. G. Safonov, Robust Control Toolbox, Natick,
MA: The MathWorks, 1992.
14. G. J. Balas et al., -analysis and Synthesis Toolbox, Natick, MA:
The MathWorks, 1993.
15. D. G. Meyer, A fractional approach to model reduction, Proc.
Amer. Control Conf., Atlanta, 1988, pp. 10411047.
16. T. T. Georgiou and M. C. Smith, Approximation in the gap metric: Upper and lower bounds, IEEE Trans. Autom. Control, 38:
946951, 1993.
17. G. A. Latham and B. D. O. Anderson, Frequency-weighted optimal Hankel normal approximation of stable transfer functions,
18. T. Auba and Y. Funabashi, Interpolation approach to Hankel
norm model reduction for rational multi-input multi-output systems, IEEE Trans. Circuits Syst. I; Fundam. Theory Appl., 43:
987995, 1996.
19. D. G. Meyer and S. Srinivasan, Balancing and model reduction
for second-order form linear systems, IEEE Trans. Autom. Control, 41: 16321644, 1996.
20. A. C. Antoulas, On the approximation of Hankel Matrices, in U.
Helmke, D. Pratzel-Wolters, and E. Zerz, (eds.), Operators, Systems and Linear Algebra, Stuttgart: Teubner Verlag, 1997, pp.
1723.
21. P. A. Fuhrmann and R. Ober, A functional approcah to LQG balancing, Int. J. Control, 57: 627741, 1993.
22. B. Srinivasan and P. Myszkorowski, Model reduction of systems
zeros interlacing the poles, Syst. Control Lett., 30: 1924, 1997.
422
LINEAR NETWORK ELEMENTS
23. R. Ober, On Stieltjes functions and Hankel operators, Syst. Control Lett., 27: 275278, 1996.
24. A. C. Antoulas, Approximation of linear operators in the 2-norm,
Linear Algebra Appl., special issue on Challenges in Matrix Theory, 1998.
A. C. ANTOULAS
Rice University
LINEAR ELECTRIC COMPONENTS. See LINEAR NETWORK ELEMENTS.
LINEAR FREQUENCY MODULATION. See CHIRP

MODULATION.
LINEARLY SEPARABLE LOGIC. See THRESHOLD LOGIC.
VOLTAGE-TO-FREQUENCY CONVERTERS
INTRODUCTION
The Voltage-to-Frequency Converters (VFCs) belong to the
category of the oscillator circuits. The function of an oscillator is to produce periodic time-varying signals with
only dc excitations 1 . These signals may have quite diverse waveform shapes such as triangular, square, impulsive, exponential, sinusoidal, etc. For some oscillator applications, the shape of the generated signal is an important
design criteria. For instance, in the production of periodic
sweep voltages for display systems; or in the generation of
quasi-sinusoidal signals for testing and instrumentation
purposes. In other applications, the important design criteria is the signal frequency, while the spectral purity of
the signal remains secondary. This is for instance the case
of the oscillators used to generate clock signals for timing
purposes. And this is also the case of VFCs.
The function of VFCs is to convert the value of an analog input voltage or current into an output frequency; i.e.
they must generate signals of any shape whose frequency
is a faithful representation of the analog input data. Thus,
they also belong to the category of the data converter circuits, whose function is to convert the information among
different domains with minimum degradation of the information itself. From this point of view, the VFCs have some
similarities with the Analog-to-Digital Converters (ADCs),
whose function is to convert an analog data into a digital word with the minimum error possible. In the case of
VFCs the domain of the transformed information is the frequency of a signal, instead of a digital word, and the aim
is to obtain the smallest possible error in the voltage-tofrequency mapping. The design criteria for VFCs include
the following (14):
Dynamic Range, i.e. the range of input voltage values

where the transformation takes place. A related magnitude is the frequency range the range of output
frequencies which corresponds to the input voltage
range. 60dB and larger dynamic ranges 2 are commonplace.
Linearity of the voltage-to-frequency map, f =
0
F (vin ). This map is required to be highly linear,
F (vin ) = k vin + E(vin )
(1)
with error E(vin ) much smaller than 1% of full scale.

Scale Factor Accuracy and Stability. The parameter k
in Eq. (1) must be externally set for given voltage-tofrequency gain. After its setting, the gain must remain
stable with temperature 100ppm/ C , aging, and
changes of the power supply 0.1%/V.
Such requirements are related to the main application
area of the VFCs, namely the encoding of information for
telemetry or remote control. In these applications, the analog data is converted into frequency at the emitter frontend by using a VFC. The generated signal is then transmitted through an optical or radio link a process in which the
signal frequency remains essentially unchanged. Then, at

the receiver front-end, the analog data is recovered from
the signal frequency by using a Frequency-to-Voltage Converter (FVC).
From the construction point of view, the VFCs are in
the same category as the so-called Volt-age-ControlledOscillators (VCOs), in the sense that the circuit structures
employed to realize VFCs and VCOs are actually quite similar. However, the typical applications of VCOs usually demand low precision in the voltage-to-frequency map and,
in some cases, high spectral purity. In this chapter, we will
start by presenting a broad overview of the different circuit
structures that qualify to generate periodic signals with
voltage-controlled frequency. Then, in the remaining of the
chapter we will focus on those which are better suited to
achieve accuracy in the volt-age-to-frequency mapping.
BASIC CONCEPTS AND MODELS FOR OSCILLATOR
DESIGN
Circuit synthesis is the process of interconnecting circuit
components so as to realize some targeted behavior. The
rst step for systematic circuit synthesis is to identify
mathematical descriptions for the targeted behavior. In
the case of oscillators, the target is to obtain a steadystate cyclic solution with only dc excitations. For VCOs and
VFCs, the timing of these cyclic solutions must also be electronically controllable.
The simpler the better; this is a basic motto for electronic circuit design. In the case of oscillators, two ineludible ingredients are dynamics and nonlinearity. Thus, oscillator circuits must contain capacitors and/or inductors
together with nonlinear elements. Strictly speaking, the
dynamics must be at least of second-order typemeaning
that the corresponding circuit requires at least two energystorage elements. With regards to the nonlinearity, the simplest one is the N-like shape of Figure 1(a), which can be
approximated by a series expansion,
(x) = NI x + NE x3
(2)
where NI and NE are real parameters which can be used

to t the required function shape. Alternatively, Figure 1(a)
can be approximated by the piecewise-linear curve depicted
in Figure 1(b), where the tting is realized through the real
parameters NI and NE .
Basic Oscillator Model
The basic oscillator model is built by combining the ingredients above into
dx
x
= y (x)
dt
(3)
dy
=x
y
dt
Figure 2(a) shows a conceptual realization of this equation using two integrators, one adder and one block with
nonlinear transfer function (x) all realizable in electronic form. Figure 2(b) and (c) show realizations using capacitors, inductors and nonlinear resistors; it is
easy to conrm that Figure 2(b) maps onto Eq. (3)
for x = v, y = RiL , x = RC, y = L/R and assuming
Voltage-to-Frequency Converters
Figure 1. N-like shaped characteristics necessary to implement oscillator circuits and associated piecewise-linear approximation.
Figure 2. Conceptual realization of the basic oscillator model in Eq. (3) (a) and LC realizations for x = v, y = RiL , y = L/R (b) and for x =
RiL , y = vC , x = L/R, y = CR (c).
that the resistor has a voltage-controlled characteristic

iR = (v); similarly, Figure 2(c) maps onto Eq. (3) for
x = RiL , y = vc , x = L/R, y = CR and assuming that
the resistor has a current-controlled characteristic v =
(iR).
The model of Eq. (3) generates stable oscillationscalled
limit cyclesdue to the dynamic equilibrium between two
opposite forces, one expansive, the other contractive. The
expansive force happens for |x|< due to the negative slope
of the nonlinearity in this regionsee Figure 1(b) 3 . This
negative slope can only be realized through an active element which injects energy into the system thereby making
|x| increase with time. On the other hand, the contractive
force occurs for |x|> due to the positive slope of the nonlinearity in this region. Because of this positive slope, the
system dissipates energy and, consequently, |x| decreases
with time.
The actual trajectories in the transient evolution towards the limit cycle depend on the values of the parameters x , y , NI , NE and . The shape of the limit cycle
orbit and the frequency of the motion are also dependent
on these values. Finding closed relationships among the
shape and frequency, on the one side, and the parameter
values, on the other, is not even possible in the more general case. However, for our purposes here it sufces to study
the approximated solutions in two particular cases. First
of all, we nd convenient rewriting Eq. (3) in terms of a
normalized time variable = t/o with o = x y . It yields
dx
= [y (x)]
d
dy
1
= x
d
(4)
where = y /x .The two particular cases of interest occur for extreme values of the parameter ; they are respectively caalled the harmonic oscillator casewhich cor-
responds to 1, and the relaxation oscillator casewhich

corresponds to 1.
Harmonic Oscillator Case
This corresponds to 1, i.e. to y x . For a better understanding of what happens in this case, it is convenient
to recast Eq. (4) into the following scalar form,
d2x
ddx
+
+x=0
(5)
d 2
dxd
obtained after differentiating the top expression in Eq. (4)
and using the bottom expression to substitute dy/d.
Assume rst that the nonlinearity is null. Then d/dx =
0 and the solution of Eq. (5) for initial conditions x(0) = x0
and dx/d|=0 = 0 is x() = x0 cos(), and hence,
x(t) = x0 cos(0 t)
(6)
where o 1/0 . Obviously, such a solution could not

be kept in practice due to the unavoidable energy
dissipationwhich is why nonlinearity is needed for practical oscillator design. However, because 1, the second
term in Eq. (5) is negligible and at the limit cycle x(t) is
quasi-sinusoidal; x(t) cos(0 t) with amplitude for xed
and 0 depending on NI , NE and .
The set of drawings in Figure 3 illustrate the harmonic
oscillation case. On the one hand, they are intended to highlight the dynamic equilibrium process underlying the onset of oscillations. On the other, they give insight on the
inuence of the model parameters. All the drawings are
for x = 10, y = 0.1 and = 1. The three drawings at
the top are for NE = 1 and NI = 1. The one on the left in
this subset shows the solution trajectories in the x-y plane
for different initial conditions. The opposed expansive and
contractive forces are highlighted by this gure. Their equilibrium results in the system evolving towards a limit cycle
from any initial condition located either inside or outside it.
Figure 3. Trajectories, signals and spectrum for the oscillator model in the harmonic oscillation
case for x = 10, y = 0.1 and = 1. The three drawings at the top are for NE = 1 and NI = 1, those
in the center row are for NE = 1 and NI = 0.2, the drawings at the bottom were obtained for NE
= 1 and NI = 3. We see that the oscillation frequency is practically the same in the three cases
(basically set by 0 = x y ). We also observe that the larger NI , the larger the signal amplitude
and distortion, while the dynamic evolution toward the limit cycle is faster.
The waveform at the center in the top set of gures shows

x(t) at the steady-state. Because thisxt() steady-state is obtained as the result of a nonlinear equilibrium process, this
waveform is not a pure sinusoidal, but contains harmonics. The spectrum on the right shows these harmonics. The
three drawings in the center row in Figure 3 are for NE = 1
and NI = 0.2 . Here the frequenc is practically the same as
in the previous case, although the amplitude and distortion
of are x(t) now smaller. On the other hand, the evolution
from inside the limit cycle towards the limit cycle itself is
much slower. However, the last set of drawings, for NE = 1
and NI = 3, disply, again, practically the same frequency
but much larger amplitude and distortion, and a very fast
evolution to the limit cycle.
The following observations on the inuence of parameters can be made regarding the harmonic oscillator case:
The oscillation frequency is basically set by 0 =
x y . The voltage control of this frequency is hence
achieved through electronic tuning of the time constants x and y For the LC circuits of Figure 2(b) and
(c), the frequency can be made voltage-controlled by
resorting to the use of tunable capacitors or inductors
(1524).
On the one hand, for given values of x , y , NE and
, the larger NI , the larger the signal amplitude and
the signal distortion. Also, the faster the dynamic evolution towards the limit cycle from the inside. On
the other hand, for given values of x , y , NI and ,
the smaller NE , the smaller the signal distortion, although the speed of the dynamic evolution towards
the limit cycle from the inside does not change with
NE . Finally, the value of inuence either the signal
amplitude, but has no inuence on either the signal
distortion or the speed of the evolution towards the
limit cycle.
Figure 4. (a) Representation of the basic oscillator model in Figure 2(a) as the interconnection of
a linear dynamic block and a nonlinear block and (b) general harmonic oscillator model with these
blocks. The phase shift of the linear block is null at the oscillation frequency, and the oscillation
occurs at the amplitude for which the loop gain is just unity.
General Harmonic Oscillator Model
racy of the oscillation frequency setting.
Figure 2(a) can be redrawn as depicted in Figure 4(a), consisting of the interconnection of a linear dynamic block
with transfer function in the s-domain given by
1
H(s) =
s x
X(s)
= 2
Z(s)
s + s NE
+
x
1
x y
(7)
and a nonlinear block with transfer characteristics (x).

Note that the linear block acts as a bandpass lter
with resonant frequency 0 1/0 and quality factor Q =
1/(NE ).Note also that the gain of this lter at the resonant frequency is 1/NE , and that the nonlinear block gains
NE + NI inside its inner linear region.
Assume x(t) cos(t) . The system can oscillate, provided that the amplitude and the phase of this signal is
maintained while the signal is being transmitted across
the loop. Consider rst the phase. On the one hand, the
nonlinearity is static, and does not produce any phase shift.
On the other hand, the phase of the linear block is null only
for = 0 which is why the oscillation actually happens
at this frequency. Consider now the amplitude. On the one
hand, for > , the loop has a gain of 1 + NI /NE , larger
than unity, so that the amplitude grows while the signal
transverses the loop. On the other hand, for > , the gain
is compressed due to the saturation nonlinearity. The compression increases with until a critical amplitude value
0 is reached where the effective gain is unity.
Figure 4(b) shows the block diagram of a generic harmonic oscillator model. Based on the explanations made
for Figure 4(a) the following statements are made for this
generic model:
The phase shift of the linear block must be frequencydependent and null at a single frequency = 0 ,
which is the osocillation frequency. The sharper the
phase-vs-frequency characteristics of the linear block
around this zero-phase frequency, the larger the accu-
Consider now the oscillation amplitude. On the one

hand, the loop gain at the zero-phase frequency must
be larger than unity for low amplitudes. This ensures
that the oscillations will start. On the other hand, the
compression exercised by the nonlinear block must
render the loop gain smaller than unity for large amplitude values. The oscillation occurs at the amplitude
for which the compressed loop gain at the zero-phase
frequency is just unity.
This can be applied to the ring oscillator structure of Figure 5(a)commonly employed in 10 VCOs. It is composed of
N identical stages, Neach with linearized transfer function
(22) ,
F (s) =
Gm R
1 + sRC
(8)
connected into a feedback loop. On the one hand, note that

each stage produces a phase shift of atan(RC); the
zero-phase frequency is hence readily calculated as,
f0 =
1
tan
{3 + (1)N }
2RC
2N
(9)
On the other hand, the nonlinearity, and hence the control

of the oscillation amplitude, is inherent to the active devices used to realize the transconductors. For the simple
circuits typically employed in these transconductors, such
as for instance those in Figure 5(b), the transconduc-tance
decreases with the signal amplitude. Hence, the condition
for the generation of oscillations is Gm R > 1.
Relaxation Oscillator Case
It corresponds to in Eq. (4) atbeing much larger than
unity, i.e. to x y . In the limit, as , its factor in the
top equation of Eq. (4) becomes negligible and hence,
y = (x)
(10)
Figure 5. (a) Ring oscillator structure and its linearized model as a specic example of Figure 4.
(b) Simple typical circuit realizations for the tunable transconductors.
Figure 6. The limit cycle in the relaxation case ts with the nonlinear function (x) in (a) as
depicted in (b), where we assume that the horizontal transitions occur instantaneously. The bottom
drawing (c) shows the limit cycle, the x(t) waveform and its spectrum for x = 0.1, y = 10, = 1
and NE = NI = 1.
meaning that the shape of the limit cycle ts with the nonlinear function (x), drawn with a thick solid trace in Figure 6(a).
This piece of information has to be complemented with
the analysis of the solution trajectories of Eq. (4) the different linear pieces of (x) : Let us consider rst the inner
piece, where y = NI x. Analysis obtains,

1
[y(0) NI x(0)]eNI
NI
y() [y(0)]e NI
x()
(11)
valid for |x|<. Note that in the limit, fo , y() remains

practically constant while the system evolves in this inner region. Quite on the contrary, x(t) changes at very high
speed; either decreasing, for y(0) > NI x(0), or increasing,
for y(0) < NI x(0). The arrows drawn with thin solid lines
in Figure 6(a) indicate the direction and sense of the trajectories in this inner region. From this picture, it is readily
inferred that the nonlinearity inner piece is not part of the
limit cycle.
Let us now consider the outer pieces, where y =
NE x , where the minus is for the left-hand piece, the
plus for the right-hand one, and is the absolute value
of the ordinate at the origin. Analysis obtains a fast transient across a quasi-horizontal trajectory followed by a slow
asymptotic behavior whose trajectory is described by
1
(
)
1
NE
x()
[y(0) ( )]e
NE
(12)
1
(
)
x() [y(0) ( )]e NE
Note that this slow asymptotic behavior results in trajectories which t exactly with the corresponding outer piece
of the nonlinear function.
As a summary of these considerations, the limit cycle
can be built as shown in Figure 6(b), where we assume that
the horizontal transitions occur instantaneously. Thus, the
oscillation frequency is basically determined by the time
invested in making the asymptotic evolutions on the outer
pieces of the nonlinearity, which can be calculated from Eq.
(12) as
Tc = 2y NE ln
+ NI
NI
(13)
These summary considerations are conrmed by the

simulations depicted in Figure 6(c), which shows the
limit cycle, the x(t) waveform and its spectrum for
x = 0.1, y = 10, = 1 and NE = NI = 1.
Let us at this point explore the implications of the conditions leading to the relaxation oscillator case on the LC
circuits of Figure 2(b) and (c). These conditions are met by
making C 0 in the former case; and by making L 0 in
the latter. For circuit implementation purposes, these negligible elements can be considered parasitics and, hence,
the relaxation oscillator can be built by simply connecting
either an inductor or a capacitor to a nonlinear resistor.
Figure 7 shows these realizations, where the nonlinear resistor is N-shaped for Figure 7(a), and S-shaped for Figure
7(b).
CONVERTERS BASED ON MULTIVIBRATORS
Since the frequency in oscillators can be controlled by
means of tunable capacitors, resistors or inductors, any
oscillator can be used to translate a voltage into a frequency, provided that the voltage to be converted is that
used to tune the devices (1524). However, in the case of
harmonic oscillators, these mechanisms do not provide a
good enough linear dependence of the output frequency
on the input voltage nor temperature stability for most
voltage-to-frequency applications. They are used instead in
PLL (phase locked loops) applications, many allowing nonlinearities up to several tens of percent and temperature
induced variations as long as the tuning range accommodates them (23). The jitter in these circuits is also lower
than in relaxation oscillators, which make them more appropriate for timing applications (11). Multivibrators have
usually poor timing accuracy for frequencies above a few
hundred kHzs because of the random uctuations of the
threshold levels and charging and discharging currents
and their increasing importance with respect to shorter
clock periods (1), but they provide the best linearity and
temperature and supply stability at medium-low frequencies, as well as wide control and signal ranges (8). Since
the latter are features required for voltage-to-frequency
conversion, most VFCs on the market are based on multivibrators, thus we will focus on this approach in the following.
Implementations of VFCs are usually based on capacitors as energy storage elements, and the nonlinear resistor in Figure 7(b) is commonly implemented as depicted
in Figure 8(a). The current switches in Figure 8(a) are set
to the Ch or Dh position by the reversal block, corresponding to the charging phase and the discharging phase respectively. During the charging phase, the current iC enters the capacitor input node (+) and the voltage vC grows,
while it decreases during the discharging phase, wherein
the current iD leaves the capacitor input node. The circuit
operation consists in the succession of these two phases
in a cyclic behavior. The period of the cycle is determined
on the one hand by the value of iC and iD and the capacitance C associated to the timing capacitor, which set the
rate for the voltage vC to change inside each phase, and on
the other hand by the reversal block, which senses vC and
turns the switches to the Ch or Dh positions. Assuming
constant values for the currents iC and iD , the time intervals corresponding to the charging and discharging phases
are (see Figure 8(c)) ,
tC =
C(vCH vCL )
iC
tD =
C(vCH vCL )
iD
(14)
where vCH and vCL are the top and bottom limits respectively of the voltage swing in the capacitor terminals. In
the specic case of having i = ic = iD , the time invested in a
cycle is
Tc = tC + tD =
2C(vCH vCL )
i
(15)
where we have supposed that the delay associated with the

reversal block and the current switches is negligible, which
corresponds to the assumption x y in the previous section. Actually, the expression for Tc above is also derived
from Eq. (13) by making NE ,which provides
Tc = 4y NI
(16)
for the time invested in completing the cyclic diagram in

Figure 8(b) (see also Figure 6). Note that Eq. (16) yields
Eq. (15) if NI is obtained from Figure 8(a) as
NI =
vCH vCL
2iR
(17)
and taking into account that y = RC.

The VFCs currently on the market can be classied into
two major categories that follow the previous approach, the
Schmitt trigger type converters and the charge balancing
type converters. The main difference between both types
Figure 7. Simplied rst-order abstractions for a relaxation oscillator. The conditions leading to the relaxation oscillator case on the LC
circuits of Figure 2(b) and Figure 2(c) are met by making C -> 0 in the former case (a) and L 0 in the latter (b).
Figure 8. Realization of relaxation oscillators based on Figure 7(b). The reversal block controls the switches to alternatively charge and
discharge the capacitor (a). The associated cyclic diagram corresponding to the general case in Figure 6 is shown in (b), while (c) depicts
the resultant waveform.
lies in the role that the reversal block plays. In the former, the reversal block works like a Schmitt trigger, which
senses vC (t) and turns the current switch to the position Ch
or Dh any time it reaches the low vCL and high vCH threshold values respectively. Thus, the reversal block imposes
the thresholds vCH and vCL , and sets directly the voltage
swing (vCH vCL ) in the CLacitor during the circuit operation. Eq. (14) applies here and the frequency of the waveform associated to vC is controlled by means of voltage controlled current sources, thus making iC = KC vin and iD =
KD vin obtain
f =
1
KC KD vin
=
t C + tD
(KC + KD)C(vCH vCL )
(18)
where vin is the input voltage that is converted into the

frequency f. In the particular case of having KC = KD , we get
a symmetrical triaKC=KDngular shaped periodic signal.
On the other hand, in the charge-balancing type converters, the reversal block does not impose lower and upper limits or thresholds to the voltage drop in the capacitor, but a xed duration of the charging phase T0 . These
circuits also are built to fulll iC = i0 iD , where i0 is a
constant current. The basic working principles are similar
to those in the Schmitt trigger converters, and the circuit
evolves in a cycle with successive charging and discharging
phases. Let us begin in a certain point inside the discharging phase. The voltage vC ramps down until it equals a
lower threshold vCL . At this time, the switch is turned to

Ch by the reversal block and the capacitor is charged but
now for a xed period T0 . Thus, the voltage vC changes an
amount given by
vC =
(i0 iD )T0
iC T0
=
C
C
(19)
Once this charging phase is concluded, the switch is set to

Dh by the reversal block and the capacitor is discharged
by iD until the lower threshold is reached again, hence the
voltage vC falls an amount of vC and the duration of this
phase is
tD =
CvC
i0
= ( 1)T0
iD
iD
(20)
since T0 and i0 are constant values, the frequency is

changed by tuning iD in Eq. (20). If iD = KD vin , we obtain
the following expression for the frequency of vC (t),
f =
iD
KD vin
1
=
=
tC + tD
i0 T0
i0 T0
(21)
In the following we will show implementations of these two

classes of converters and we will discuss design issues and
non idealities.
Figure 9. (a) Direct implementation of a Schmitt trigger type VFC converter, where the reversal
block in Figure 8 is implemented by a Schmitt trigger circuit.(b) Two realizations of the Schmitt
trigger block.
SCHMITT TRIGGER TYPE CONVERTERS
with supply voltage is 2%/V.
Direct Implementation
Emitter-coupled multivibrator
Figure 9(a) shows a straightforward implementation of a

Schmitt trigger type converter with a grounded capacitor.
Two possible realizations of the Schmitt trigger block are
also depicted in Figure 9(b) (14). In the single-comparatortype Schmitt trigger on the left of Figure 9(b), for very low
values of vi the comparator output is low and the current
switch is open, thus the voltage reference at the inverting
input is
Instead of a grounded capacitor, the circuit in Figure 10

uses a oating one which is charged and discharged by
means of the bias current sources iC and iD . Basically, the
circuit xes the voltage at one terminal of the capacitor,
for instance, the emitter of Q3 , to a nearly constant voltage
VM , while the other terminal is connected to the current
source iC that charges it. Eventually, the voltage in the
latter drops enough to allow the base-to-emitter voltage
of Q2 to reach the transistor threshold, which causes the
voltage at the base of Q3 to fall and the transistor is turned
off. The voltage at the emitter of Q2 is now xed to two baseto-emitter drops below the supply voltage and the capacitor
is being discharged by iD . Since the circuit is symmetrical,
the process is repeated periodically. Let us split the cycle
in periods related to important events in order to derive
the expression of the output wave frequency:
viH = VCC
R2 + R3
R1 + R2 + R3
(22)
For increasing values of the input voltage, eventually the

reference in Eq. (22) is crossed and the comparator output
changes and turns the switch on, thus the resistor R3 is
short circuited. Hence, if the input voltage decreases at
this point, the new reference voltage at the inverting input
of the comparator is
viL = VCC
R2
R1 + R2
(23)
Note that Eq. (22) and Eq. (23) constitute the high and low
threshold voltages, respectively, of the single-comparator
Schmitt trigger. The main drawback of this circuit is that
the parasitic capacitors at the nodes between the resistors have to be charged and discharged every time the
switch is closed or opened, which is a time-consuming task
and causes errors that limit the high frequency capability of the circuit. A better behavior is obtained by using
two comparators and xed voltage references to dene the
threshold values. The right part of Figure 9(b) shows the
dual-comparator Schmitt trigger which is based on this approach and whose threshold voltages are determined by the
resistor ladder as
R3
R2 + R3
viL = VCC
viH = VCC
(24)
R1 + R2 + R3
R1 + R2 + R3
Besides other general limitations that will be seen later,
the basic disadvantage of this strategy is the poor supply voltage stability, due to the dependence of the threshold values on VCC . Table 1 shows the typical performance
parameters associated to the NE5664 , which follows the
direct approach in Figure 9(a) and whose maximum drift
Figure 10(b) shows the circuit working somewhere

in the cycle with the transistor Q2 and the diode D1
in the cut-off region, which is indicated by using a
white trace. Consider the clamping block in Figure
10(a) formed by one diode and one resistor in parallel. The voltage drop across this block is depicted in
Figure 10(a), as well as the response to a step in the
current ID . As long as Q2 is off, the voltage across the
block equals zero if we neglect the base and leakage
currents. Since Q1 and Q3 are on, the voltage at node
B is clamped to VM = VCC 2 VB Eon approximately. On
the other hand, the current source iC is charging the
capacitor with a constant current, thus the voltage at
the emitter of Q2 falls to a lower threshold VL beyond
which Q2 is not longer cut-off. Since D2 and Q4 are on,
such a limit is given by the equation,
VL = VCC (VD2 + VBEQ4 + VBEQ2 ) VCC 3 VB Eon (25)
Once the voltage at node A reaches VL , the transistor

Q2 turns on and its collector current makes the voltage
drop across the left clamping block rise to VBEcon , thus
the base of Q1 drops to VCC VB Eon and the emitter of
Q1 follows this voltage, hence the voltage at the base
of Q3 is now VCC 2VB Eon . Since its emitter was xed
Table 1. Typical performance parameters of the NE566

Maximum operating frequency
Sweep range
Frequency drift with supply voltage
Frequency drift with temperature
FM distortion (10%deviation)
1MHz
10:1
0.2 %/V
600 ppm/ C
0.4%
Table 2. Typical performance parameters of the AD537 JD

Frequency range
Nonlinearity (fmax = 100kHz)
Error drift with temperature (0 to 70 C)
Error drift with supply voltage (fmax < 100kHz)
0kHz to 150kHz
0.15%
50 ppm/ C
0.01%/V
Table 3. Typical performance parameters of the AD VFC32KN

Frequency range
Nonlinearity (fmax = 500kHz)
Error drift with temperature (0 to 70 C)
Error drift with supply voltage (max)
0kHz to 500kHz
0.05%
75 ppm/ C
0.015% of FSR%
Table 4. Typical performance parameters of the AD652 JP

Gain Temperature Coefcient
fCLOCK = 200kHz
fCLOCK = 4MHz
Linearity Error
fCLOCK = 200kHz
fCLOCK = 4MHz
Power Supply Rejection Ratio
25ppm/ C
25ppm/ C
0.002%
0.02%
0.001 %/V
Figure 10. Emitter-coupled multivibrator. The clamping block in (a) causes a voltage drop of value VBEon for a current ID VBEon /R, that
makes the basis voltage of Q2 fall, thus it is cut-off and the capacitor is charged through . The situation reiCmains until the voltage drop
at the emitter of Q2 is low enough to turn on Q2 , thus the clamping block at the top-left corner cuts-off Q3 and the situation is reversed.
The process is repeated cyclically.
to the same voltage, the transistor Q3 turns off. Note

that at this point, the voltage drop across the capacitor
is VC = VB VA = VBEon
Since Q3 is cut off, its collector current vanishes and
the voltage drop across the right clamping block goes
down to zero. As a consequence, the voltage at node A
is pulled up to the value VCC 2VB Eon . On the other
hand, since the switching time is very short, the capacitor remains charged and the voltage increment is
translated into an increment in the voltage at node B,
which reaches the value,
VH = VCC VBEon
(26)
Note that the situation is symmetrical to that in the

rst point, but now the voltage at nodeA remains constant while the current source iD discharges the capacitor, which is equivalent to the initial point above.
From the previous discussion we conclude that in each
charging or discharging phase, the voltage at one capacitor
terminal A or B remains constant, while the other changes
from the voltage VH to VL . It is then easy to derive the
voltage increment in the capacitor,
vC = vCH vCL = (VH VM ) (VL VM ) = 2VBEon
(27)
Since the problem is equivalent to that treated in the previous section, equation Eq. (18) is also valid here and the
10
frequency of oscillation is given by the expression,

f =
Kvin
4CVBEon
(28)
where KC =KD =K.

Improved emitter-coupled multivibrator
Since the base-to-emitter voltage has a strong temperature
coefcient, the circuit in Figure 10 has as a main drawback
the dependence of Eq. (28) on the temperature. One strategy to compensate this dependence consists of making that
the current that charges and discharges the capacitor is
dependent on the temperature for cancelling the global dependence (14). Such an approach achieves circuits whose
output frequency does not change with the temperature for
vin =0, while there is a dependence for values of vin different
from zero. Another strategy tries to make vC = vCH vCL
equal to a quantity very stable under temperature variations. That is the strategy followed in the circuit of Figure
11, which basically works in the same way as the previous one in Figure 10. The main difference consists in the
change of the voltage values used to clamp the collectors of
Q1 and Q2 . For this purpose, the simple diodes in Figure 10
are substituted by a more complex circuitry depicted at the
top of Figure 11. In addition to the current sources, the circuit uses two reference voltages to implement the strategy:
a very stable voltage reference VR , and the reference voltage VCC 2VBEon generated by D5 and D6 . In addition, one
diode bridge derives the voltages at the collectors of Q1 and
Q2 from these references. Figure 11 illustrates the same
situation as in Figure 10, where Q1 was cut-off, the voltage
at node B was xed, and the capacitor was being charged by
iC . The same process is going on now in Figure 11 where the
devices in white are off, but the voltages at the collectors
of Q1 and Q2 are set to VCC VBEon and VCC VR VBEon , respectively, by the above referred top circuitry. Taking into
account the voltage drops across the emitter followers as
well as that in Q2 , the voltage at B is
VM = VCC 3 VBEon
(29)
and the voltage atA beyond which Q1 is not off anymore is

VL = VCC VR 3 VBEon
(30)
When this limit is reached, the situation is reversed, Q1 is

turned on and Q2 is turned off. At this point, the voltage
across the capacitor is vC = VB VA = VM VL = VR , and
the nal voltage at B after the switching will be
VH = VM + VR = VCC 3VBEon + VR
(31)
Now from Eqs. (29), (30) and (31) vC = vCH vCL = (VH
VM ) (VL VM ) = 2VR and Eq. (18) reduces to
f =
K Vin
4CVR
(32)
The frequency of oscillation can now be stable if VR is

designed carefully to have a low temperature coefcient,
which is described later. Table 2 shows some typical parameters of the AD537JD (2). The data refer to the current-tofrequency converter block without taking into account the
deviations introduced by the voltage-to-current converter

at input.
The other major class of commercial voltage-tofrequency converters are based on a charge balancing system. As said in Section 3, these circuits are characterized
by having a xed length in time in the charging phase. We
can classify them into two major categories, those based
on one-shot timers and those which are synchronized to a
clock signal.
Charge balancing converters based on one-shot timers
Figure 12 illustrates a typical implementation of these converters, whose working principle we now describe. Suppose
that vC vin , then the comparator provides a high output.
As a consequence, the one-shot timer output rises and the
current switch is set to the Ch position. This behavior of the
one-shot continues under the start-up conditions or in the
case of a change in the input voltage, and the frequency of
the timer output signal is zero. Once the condition vC > vin
is reached, the one-shot is reset and the current switch
is set to the Dh position, thus the capacitor is discharged
through R. However, as soon as the condition vC vin is fullled again, the timer is triggered and the switch is turned
to Ch again to allow the current ow into the integrator.
This time the one-shot timer operates as usual and generates a pulse of xed duration tos , thus a packet of charge is
injected in the RC integrator and vC ramps upward by an
amount
dvc
(i0 iD )
vC = tos
= tos
(33)
dt
C
which is enough to make vC > vin , thus the one-shot is reset
and the capacitor is discharged again until vC reaches vin .
If vC vC , we can approximate iD vin /R, which is a constant current that discharges C, thus the time tD is given
by
tD
|vC |
| dvdtC |
tos ((i0 iD )/C)

iD
C
= tos (
i0 R
1)
vin
(34)
The cycle is repeated again as soon as vC equals vin , thus

the output of the one-shot timer is a periodic signal whose
frequency is
f =
1
vin
=
tos + tD
tos i0 R
(35)
Note that this coincides with Eq. (21) for KD = 1/R and T0 =
tos . The circuit in Figure 12 has two main drawbacks. First,
the change of vC causes an error in the charging current
due to the nite output impedance of the current source
that generates it, as well as in the discharging current iD ,
which is not constant as supposed above. Second, the passive RC integrator is very slow, especially for large input
voltage changes, which require a large amount of charge
to be injected in or extracted from the capacitor. Both aspects can be improved by using an active integrator, as
Figure 13 depicts (327). The negative feedback provides
low impedance at the inverting input terminal of the integrator, whose voltage remains nearly constant, thus the
error in the charging and discharging currents is reduced.
Equations Eq. (33)-Eq. (35) are valid in Figure 13(b), but
now iD = vin /R exactly. With respect to Figure 13(a), vC
11
Figure 11. Improved emitter-coupled multivibrator. The clamping circuitry is redesigned to make the voltage variations at the capacitor
depend on a voltage (VR ) very stable against temperature changes, thus the frequency drift with temperature is improved.
Figure 12. Basic diagram of a charge-balancing converter based on a one-shot timer. The charging phase has xed length in time, which
is the width of the pulse generated by the one-shot timer (tos ) when it is triggered by the comparator. For vC vC , the average voltage at
the comparator inverting input is vin , thus iD vin /R. The average current injected in the bottom integrator must equal iD and depends
on the rate the one-shot is triggered with, hence this rate depends on vin .
Figure 13. Improved converters based on one-shot timers. The use of active integrators instead
of the passive one in Figure 12 achieves smaller errors and improves the dynamic response.
diminishes in the reset mode, thus the derivative in Eq.

(33) is negative, as is the voltage increment, but Eq. (34)
and Eq. (35) are valid. In both cases, the output current of
the op amp in the integrator is the same for the charging
and the discharging modes, which improves the dynamic
response because this minimizes the transients. In addition, Figure 13(a) provides high impedance for the input
terminal, which can be useful for applications with low
load requirements. Finally, note that the capacitance of the
integrating capacitor does not appear in Eq. (35), thus it
12
does not inuence directly the output frequency. However,

it certainly inuences the shape of vC (t), because if C is too
small, vC in Eq. (33) could become too large and exceed
the linear range of the integrator. In (16), a nonlinearity
error of 0.024% FS (f= 10Hz to 11 kHz) is reported for
the approach in Figure 12, while it is reduced to 0.006% FS
for the improved converter in Figure 13(b).
The input voltage offset Voff of the comparator in Figure
12 and the integrators in Figure 13 introduces an error
Vo f f
f = |
| in the expected off frequency - an effect that
tos i0 R
can be signicant, especially for low frequencies, in terms
of relative error. On the other hand, precision and low temperature coefcient current sources to derive io are generated on-chip by means of a voltage band-gap reference and
a precision voltage-to-current conversion circuit (see Section 7), while further reections and replications should
also be carried out by cascode current mirrors. Nevertheless, accuracy as well as high frequency capability of these
converters is primarily determined by the precision of the
one-shot section.
Figure 14 shows the simplest form of an exponentialramp-type one-shot timer (14). The principle of operation is as follows. As long as the ip-op is reset (low input), the voltage at the output Q is high and the current
switch (which can be implemented in the simplest way by
a grounded transistor) is open, thus the voltage drop in the
capacitor is zero. If the circuit is triggered by a high input,
the outputs Q and Q of the ip-op are set to a high and a
low voltage respectively, thus the switch is turned off and
the capacitor is charged through R, and vCos grows exponentially with a time constant of RCos . However, as soon
as the reference voltage at the non-inverting input (which
is determined by the voltage divider formed by Ra and Rb )
is reached, the comparator output changes to a low voltage
value and the ip-op is reset, hence the switch is closed
and the capacitor is discharged quickly. Simple calculations on the circuit formed by the supply voltage source,
R and Cos provides
tos = RCos ln(1 +
Rb
)
Ra
(36)
The voltage divider formed by Ra and Rb is usually implemented on-chip and the accuracy obtained for the ratio
Rb /Ra is within 1%, and does not depe1nd on the temperature. Thus, the precision, as well as the temperature
stability in the expected tos are mainly determined by the
resistor R and the capacitor. However, for small values of
tos , the time delays in the input comparator, the ip-op and
the switch limit the accuracy. In addition, for low frequencies the leakage currents in the capacitor and the switch
introduce an error in the timing prediction.
Table 3 shows some typical performance parameters of
the monolithic converter ADVFC32KN (4).
Synchronized charge balancing type converters
Further improvements can be achieved by replacing the
one-shot timer by circuitry clocked by a precise oscillator,
usually a crystal one, which determines the charging phase
duration (5). Figure 15 shows the circuit which results from
substituting the one-shot timer with a block composed of
one bistable and one AND logic gate, which works as follows. If the integrator output crosses the threshold vref , the
comparator places a high value at the bistable input (suppose that Q is high). However, the output does not follow
it immediately as in Figure 13(b), but it is synchronized
with the clock by the ip-op and changes at the next clock
rising edge. At the same time, the output complement (low
value) is put at the ip-op input by the feed-back loop,
thus the output will go back down at the next rising edge
of the clock and the duration of the output pulse is just one
clock cycle. Note that the circuit operates in the charging
phase as long as the output is high, thus TCLK replaces tos in
the equations Eq. (33), Eq. (34) and Eq. (35), which remain
valid. However, since the output is now synchronized, its
frequency must be a multiple of the clock frequency ,
f =
1
1
=
tD
TCLK + TCLK iS ( TCLK
)
TCLK [1 + iS ( iv0inR 1)]
(37)
where is in Eq. (37) means the next superior integer of the

argument. Table 4 shows some typical performance parameters of the synchronized converter AD652JP(5).
In addition to other design issues that will be discussed
in the next section, Figure 15 has a specic problem to solve
due to the lack of synchronism between the input voltage
and the clock. The latter could cause that the comparator
output changes at the same time as the clock arrives at the
D ip-op. Since the latter are usually designed to work
synchronously, a setup time must be respected for the input signal to change before the clock edge, otherwise there
could be metastability problems that do not guarantee a
correct value of the op output, thus tC can be shorter or
larger and the circuit does not work properly. Is it possible
to avoid this problem by building the D ip-op with two
ip-ops in cascade or with a ip-op and a latch in cascade
(5), like the right part of Figure 15 shows, where the clock
edge for both ops is delayed. The simplest way to have delayed edges for both ops consists in using the clock falling
edge to trigger the rst ip-op, while the second op is
triggered by the rising edge, thus they are delayed half
a clock cycle. With this strategy, the metastability of the
rst ip-op has ended by the time the second ip-op is
clocked, and the pulse width of the output is a clock cycle,
thus there is not error in tC .
Sigma-delta modulators
Figure 16 shows a rst-order sigma-delta modulator. It basically consists in a loop with a 1-bit analog to digital converter, which is the latched comparator, and a 1-bit digital
to analog converter in the feedback path. Provided the loop
has enough gain, the difference between the analog input
and the D/A converter output is minimized and the average
value of the D/A output must approach that of the input
signal. Similarly, the averaging of the comparator output
provides a digital word that represents the analog input.
This averaging is described later as part of the processing
required to obtain a digital word from the frequency output of a VFC (digital processing in Section 7), thus sigmadelta modulators can be viewed as synchronized chargebalancing voltage to frequency converters (the charge is
injected into or removed from the integrator for the DAC
13
Figure 14. Exponential-ramp-type one-shot timer
Figure 15. Synchronized charge balancing converter. The one-shot timer is replaced by externally
clocked circuitry. The charging phase length is now xed by the clock period, which is supposed very
accurate.
Figure 16. Sigma-delta modulator as synchronized charge-balancing voltage to frequency converter.
output to track the input voltage). In fact, the transfer function of the AD7740 (6) is f = 0.1 f CLK + 0.8(vin /vre f ) f CLK .
Since sigma-delta modulators can be built currently
with cheap CMOS technology, the main advantage of this
approach is the cost. The AD7741 (7) with 0.012% Nonlinearity at 2.75MHz of FoutMax costs ve times less than
the synchronous AD652 with 0.02% Nonlinearity at 2MHz
of FoutMax. In addition, they need fewer external components.
rived without taking into account the deviations from the

ideal case. Let us now show some real implementations of
formerly ideal blocks and discuss deviations from the ideal
case and their consequences in the output wave shape.
Implementation of the Voltage Controlled Current Sources
DESIGN ISSUES AND LIMITATIONS OF CONVERTERS

BASED ON MULTIVIBRATORS
Many VFCs described above use voltage controlled current

sources to implement the charging and discharging currents in Figure 8, where iC = KC vin and iD = KD vin . such
proportional relationship is obtained readily with a resistor, but a transistor is needed to isolate the control node
from the output, which is depicted in Figure 17(a). Such a
circuit provides the reference current ,
Although some design aspects relative to the design and

non-idealities have already been discussed in previous sections, for more clarity, some others regarding all the converters based on relaxation oscillators are examined here.
Specically, note that Eq. (15) and Eq. (18) have been de-
vin vBEon
(38)
iB
R
which acts as charging or discharging current. However,
note that this circuit provides a current that is not proportional to vin , but contains an offset that equals (vBEon /R +
ire f =
14
Figure 17. Controlled current source implementation
iB ). Besides modifying the intended proportional relationship, the added term has a strong temperature coefcient,
since the base to emitter voltage varies with the temperature as 2mV/ C.
A common alternative for eliminating the dependence
of iref on VBEon places the transistor in a negative feedback loop as Figure 17(b) illustrates. If we neglect the base
current, the output reference current for this circuit is
ire f =
A
vBEon
vin
(A + 1)R
(A + 1)R
(39)
Note that for A 1 the term depending on VBEon vanishes

and the reference current equals
ire f
vin
R
(40)
which is proportional to vin . The higher the value of the

input amplierA, the larger the accuracy of Eq. (40). In
addition, a MOS transistor can replace the bipolar one in
Figure 17(b) to reduce the error introduced by neglecting
the base current, due to the high impedance associated to
the gate of MOS transistors.
Finite output resistance of the constant currents
The current source depicted in Figure 17(b) still has a
non-ideal behavior that can introduce deviations in the expected output current. Specically, since the collector current depends on the collector-to-emitter voltage drop because of the B JT base length modulation, the current iref
varies with vo and this causes an error if the output node
directly drives the terminals of the capacitor in Figure 8,
because the voltage drop in the capacitor changes as a consequence of the circuit operation. We can model this dependence on the output node voltage by means of a resistor in parallel with an ideal current source whose value is
the expected one. This is illustrated in Figure 18, where
the Thevenin equivalent is also depicted. If the circuit is
now connected to the timing capacitor, the voltage vC does
not follow a linear behavior with t, but a well-known exponential behavior. As a consequence, the time to charge or
discharge the capacitor an amount vC is longer than expected, which is indicated in Figure 18(a) by means of the
error t . For the circuits described above, this means longer
periods or lower frequencies than those in the ideal case.
In order to improve the output impedance of the current
source, several cascode congurations have been reported
(13). Some usual ones are depicted in Figure 18(b), where
the current i ref equals iref , but the output resistance the
circuit in Figure 17.
Dependence of the Frequency on Temperature and

Voltage Supply Variations
The temperature stability of the converter output frequency depends on many factors, some of which have already been discussed above. First, in order to improve the
temperature stability we should avoid circuits like that in
Figure 10 whose output frequency depends directly on parameters like VBEon that has a large temperature coefcient. Other circuits described above must still be designed
carefully to avoid drift.
First, voltage references and charging currents have a
large inuence on the temperature stability. For instance,
the voltage source VR in the equation Eq. (32) must be designed to compensate the temperature dependence. Figure
19(a) shows an example circuit able to generate a reference
voltage VR with a low temperature coefcient (13). Such a
circuit takes advantage of the different sign of the temperature coefcients associated to the voltage-to-emitter drop
in a BJT and the thermal voltage VT . The box in Figure
19(b) encloses a circuit able to generate a current that is
proportional to VT . As long as the voltage at both output
terminals A1 and A2 is the same, i.e. VA1 = VA2 , the current
I2 is given by the equation
I2 =
(VBEQ1 VBEQ2 )
VT
nI1
=
ln(
)
R2
R2
I2
(41)
Two possible approaches to force VA1 =VA2 are illustrated in

Figure 19(b). The circuit at the top of Figure 19(b) uses a
negative feedback loop with an operational amplier that
ensures zero differential input, thus VA 1=VA2 . Taking into
account that the voltage drop across resistors R1 and R3 is
the same, we can derive the following expression for VR in
Figure 19(b)
VR = VBEQ1 +
R3 VT
nR3
ln(
)
R2
R1
(42)
Another strategy uses cascode transistors to get VA 1=VA2 .

This is the approach followed by the circuit at the bottom
of Figure 19(b), where the reference voltage is
VR = VBEQ3 +
R3 VT
ln(n)
R2
(43)
The reference voltage in Eq. (42) and Eq. (43) is made

independent of the temperature variations by choosing
properly the circuit and device parameters to achieve
dVR
|T =T0 = 0 (see (13) for more details).
dT
On the other hand, stable current sources can be obtained by driving the circuit in Figure 17(b) with a stable voltage source like that in Fig. 19, which is the strategy followed by many commercial VFCs like the AD537(2).
15
Figure 18. Current source nite output resistance: timing error (a) and common cascode strategies to increase the output resistance (b).
Figure 19. Band-gap reference voltage. (a) The voltages VBE and VT have temperature VTcoefcients of opposite sign, thus they compensate the global dependence of VR .(b) Two realizations of this approachVR.
Another element that affects the temperature stability is

the dependence on the temperature of the Schmitt trigger thresholds, specically that of the difference vC =
vCH vCL in Eq. (15). We have already discussed the particular case of the emitter-coupled circuit in Figure 11. With
regard to the direct implementation in Figure 9, the reference voltages and the comparator offsets are the main
sources of drift. Contributions to global drift are in both
cases a few tens of ppm /C.
In the case of the charge-balancing converters, note that
vC is determined mainly by the charging and discharging
currents and by the parameter T0 in Eq. (21). The drift of T0
in converters based on one-shot timers is due mainly to the
resistance R and the capacitance Cos in Eq. (36) and Figure
14, which are usually external in most commercial VFCs.
Finally, the stability of T0 is obviously improved when such
a parameter is determined by a precise external clock as
in the synchronous converters.
On the other hand, frequency drift with the supply voltage is also mainly due to variations of the charging and
discharging currents as well as that of vC . Band-gap references like those described above also provide supply independent biasing.
input stage, bias currents have to be provided from the input to the sensing nodes. Such currents are usually in the
range of a few hundred nano-amperes and add to or subtract from charging and discharging currents. MOS transistors can be used instead, to implement almost innite
input impedance thanks to the isolation provided by the
gate oxide. In addition, junction leakage or dielectric absorption currents can also be added to or subtracted from
charging currents, thus also affecting the output frequency.
However, these currents are commonly at a low nanoampere range. Hence, errors due to these base and leakage
currents can be usually tolerated as long as the charging
and discharging currents are in the range of 10A (14).
Finite Switching Times
The switching times in Figure 8 have been neglected in the
computation of the output frequency in Eq. (18). However,
they are not zero in real implementations and limit the
frequency capability of the converter. The elements that
contribute to the delay in switching are mainly the length
of the feedback path and the delays of the blocks along
it, while the gain of the active devices, the parasitic capacitances and the internal current levels determine such
delays.
Finite Sensing Node Impedance and Leakage Currents

Note that Figure 8 assumes innite impedance in the reversal block sensing input nodes. Real implementations
however do not usually present such a feature. Specically,
if the reversal block is built with bipolar transistors in the
Output Interfacing
The output stage of the commercial VFCs is designed to
allow easy interfacing to all digital logic families. Hence,
BJT open collector or both uncommitted collector and emit-
16
ter interfaces are provided. External pull-up resistors and

proper biasing complete the interface and determine the
rise times and output logic levels that are adequate for each
specic application. Internally, some interface circuitry is
needed to drive the base of the output BJT for circuits like
that on the right of Figure 11 (2), while it is not necessary
in one-shot based converters like those in (3, 25), whose
one-shot output drives it directly. Synchronous converters
also may use a one-shot timer to drive the output transistor, which allows the pulse width of the frequency output
to be controlled by means of an external capacitor.
External Component Selection
Timing capacitors in the converters described above, as
well as resistors in Figs. 12,14 and 15 are usually external components in commercially available VFCs. Special
care must be taken to select these components in order
to preserve performance. High linearity and low temperature coefcients for the resistors and the capacitors are
required. In addition, the latter should not leak, since dielectric absorption can affect the linearity and offset of the
transfer function.
FREQUENCY TO VOLTAGE CONVERSION
The complementary operation of that carried out by the
voltage to frequency converters is the frequency to voltage conversion, which is performed by the Frequency to
VoltageConverters (FVCs). This task can be often implemented by the same ICs that perform the V/F conversion,
and it is applicable in areas like telemetry, motor speed control, as well as to build interfaces for sensors whose output
is in the form of a variable frequency or pulse train (14,
19). Figure 20(a) shows how this conversion can be accomplished by means of an integrator and a current switch that
is driven by the frequency signal in the so called pulse integrating FVCs. The switch can also be driven by the output
of a one-shot timer (see Figure 20(a)), which reduces the
integration time, thus allowing small increments in the
output voltage without needing too large capacitors. The
average output voltage in Figure 20(a) can be calculated
by taking into account that in the steady state the whole
current will ow through the resistor R thus
vout = Ri0 tos f in
(44)
where i0 tos fin is the average current injected by thi0tosne

top current source in the integrator. A simpler passive integrator could be used instead of the active one in Figure
20(a), but the latter reduces the error caused by the nite
output impedance of the current source (this can also be
accomplished by means of cascode transistors (20)) and
speeds up the converter response time.
Since the current is injected in the integrator in packets
of size i0 tOs , the output voltage has an ac ripple riding on
it whose peak-to-peak value is
(vout ) p p =
io tos
C
(45)
It is possible to reduce the ripple by increasing the capacitance C in Eq. (45), but it also will enlarge the time constant
associated to the integrator = RC, thus the response time

is slow. This is a common trade-off in the design of these
circuits. One strategy to reduce the ripple while keeping
the dynamic response consists in ltering the output. As
reported in (20), a passive RC lter exhibits low ripple at
all frequencies, but it is slow. The lowest ripple at high frequencies is achieved by an active lter, which also provides
much faster step response than the RC lter. Finally, is it
possible to cascade a second active lter on the converters
output to reduce the ripple at moderate frequencies.
Another approach to convert a frequency into a voltage
is implemented by the phase-locked-loop type FVCs (21).
Figure 20(b) shows a phase-locked loop (PLL), which basically works as follows. The frequency and phase detector provides an output voltage which is proportional to the
difference between the output and input signal phases. After ltering, this voltage is converted back into the output
frequency by a V/F converter. The feedback loop locks the
phase difference between both input and output signals.
The lter output voltage is proportional to the deviations
of the frequency of the input signal, thus the circuit also
performs a frequency-to-voltage conversion. This converter
operates over a wide frequency range of 1 or 2 or 3 decades
and responds quickly to frequency changes while it does
not have ant inherent ripple.
Finally, pulse-integrating converters are open-loop versions of the charge balancing type converters in Section
5, thus their accuracy, linearity and stability are basically
the same as those obtainable from their related VFCs. With
respect to Figure 20(b), the high linearity provided by the
VFC, which is not required for most PLL applications, is
exploited here to get an ultra-linear FVC.
APPLICATIONS
This section describes some typical applications of the
voltage-to-frequency converters, which we classify in ve
major elds: telemetry and remote control, isolation, digital processing, communication and signal processing, and
articial neural networks.
Telemetry and remote control
Data acquisition of remote sensors is often not possible
without transforming the dc sensor output signal, because
it is physically unreachable or because of the vulnerability
of dc signals to be degraded when they have to travel long
distances through noisy channels (1019). Once converted
to frequency signals, they can be transmitted by wires, radio or ber optic links and processed digitally or converted
back into a dc signal to drive an actuator. The latter is performed by a frequency-to-voltage converter ,
Isolation
Some applications require galvanic isolation of the sensor
and the data processing unit in order to prevent unreasonable current leakage, or to prevent damage due to high
bias voltages which are often of several volts. Typical isolation interfaces use magnetic or optical couplers (10, 17)
like those depicted in Figure 22.
17
Figure 20. Frequency-to-Voltage converters. (a) Pulse-integrating converters, where the charge packets are injected in the integrator
at a rate determined by the input frequency. (b) Phase-locked-loop type converters, where the low-pass lter output voltage in a PLL is
proportional to the deviations of the input signal frequency.
Figure 21. Telemetry with VFC. The voltage signal from a sensor is translated into a frequency signal to be transmitted through twisted
pair, ber optic or radio links. On the other side, the signal can be converted back into a voltage to drive an analog display or an actuator,
or can be processed digitally.
Figure 22. Isolation with VFCs. The frequency signal from a VFC can be transmitted through magnetic (a) or optic (b) couplers to
implement galvanic isolation, thus preventing damage due to high bias voltages, or unreasonable current leakage.
Digital Processing
The output frequency signal from a VFC can be easily
translated into a digital code in order to be processed,
stored or displayed by conventional digital circuits and systems (928).
Actually, this procedure implements an A/D converter
while transforming the dc input of the VFC into a digital word. Figure 23(a) shows the simplest way to carry out
this A/D conversion, where a counter, which is only enabled
for a given interval TG named gate time, counts the transitions of the input signal. Thus, after this known period,
the counter stores the decimal value N = f TG , which
encodes the input voltage in a binary number. Another ap-
proach is illustrated in Figure 23(b) and uses two counters,

one to count the transitions of the input signal, and the
other to act as a timer that generates an interruption to a
microcomputer that reads the content of the rst counter
and resets it. In both cases, if FS = f max f min is the full
fmin frequency range, the resolution of the A/D converters
will be
n = log 2 (FS TG )
(46)
where n is the number of bits of the output digital word.

Hence, the larger the gate time, the larger the resolution,
which is also limited by the resolution of the VFC. Commercial VFCs provide resolutions as high as 13 bits. Note
18
Figure 23. Digital processing of the output frequency signal of a VFC. The transitions in a xed time interval TG (TG ) are counted and
the average frequency is computed.
that the input frequency must remain constant during the

gate time for proper conversion, otherwise an average of
the input frequency is computed. In addition, since the input signal and the clock are not synchronized, one cycle
of the frequency signal can be lost or counted if the transition is just after or before the edge of the clock signal, which
causes an error of 1 in the result stored in the counter.
Communication and Signal Processing
Since voltage-to-frequency converters are actually VCOs,
they can be used in phase-locked loops in applications such
as jitter reduction, skew suppression, frequency synthesis, clock recovery or FM modulation (23). However, special
care must be taken in order to reduce the jitter and enhance
the spectral purity of the output signal, like the use of differential mode in signal and control paths and guard rings
to isolate the noise sources. High linearity is not a key issue
in PLL applications, except that described in the Section
6, where nonlinearity degrades the loop stability but can
be of several tens of percent. Higher linearity is required
for FM modulation, where distortion in the detected signal
must be below 1%. Figure 20(b) depicts a PLL that uses a
voltage-to-frequency converter as a voltage controlled oscillator.
i In practice, the excitations may not strictly be dc, but ac signals

whose frequency is smaller, commonly much smaller, than that of
the signal generated by the circuit.
ii The dynamic range can be expressed in a number of equivalent bits b by using the following expression DR = 6.02 b + 1.76.
Thus, 60dB dynamic range corresponds to an equivalent resolution of 9.7bits.
iii In the electronic implementations of Figure 2(b) and (c), the
negative slope is caused by a negative resistance. This negative
resistance injects energy into the two other circuit components,
namely the inductor and the capacitor. Because these latter elements do not dissipate energy, but simply store it, the total energy
becomes increased and this manifests itself as an increase of the
signal amplitude. On the other hand, the positive slope is caused
by a positive resistance which dissipates part of the energy stored
in the energy-storage part of the circuit. At the equilibrium there
is balance between the energy injected and the energy dissipated,
the circuit energy remains unchanged and the movement is due
to a continuous interchange of this energy between the inductor
and the capacitor.
ACKNOWLEDGMENT
This work was supported in part by the spanish government under project TEC2006-12376-C02-01/MIC.
BIBLIOGRAPHY
1. A. A. Abidiand R. G. Meyer, Noise in relaxation oscillators,
IEEE Journal of Solid-State Circuits, Vol. CS-18, pp. 794802,
Dec. 1983.
2. AD537 Integrated circuits voltage-to-frequency converter,
Voltage-to-frequency converter in Analog to Digital Converters
in Product Catalogue, www.analog.com/en/
3. AD650 Voltage-to-frequency and frequency-to-voltage converter, Voltage-to-frequency converter in Analog to Digital Converters in Product Catalogue, www. analog. com/en/
4. ADVF32 Monolithic non-synchronous Voltage-to-frequency
and frequency-to-voltage converter, Voltage-to-frequency converter in Analog to Digital Converters in Product Catalogue,
http:www. analog. com/en/
5. AD652 Monolithic synchronous voltage-to-frequency converter, Voltage-to-frequency converter in Analog to Digital Converters in Product Catalogue, www. analog. com/en/
6. AD7740 3V/5V Low Power, Synchronous Voltage-to-Frequency
Converter, Voltage-to-frequency converter in Analog to Digital
Converters in Product Catalogue, www. analog. com/en/
7. AD7741/AD7742 Single and Multichannel, Synchronuos
Voltage-to Frequency Converters. Voltage-to-frequency converter in Analog to Digital Converters in Product Cata- logue,
www.analog.com/en/
8. M. Banu, MOS oscillators with multi-decade tuning range
and gigahertz maximum speed, IEEE Journal of Solid-State
Circuits, Vol. CS-23, pp. 474479, April 1988.
9. L. N. Beard, Use of a voltage-to-frequency converter to measure the integral JEdt, American Journal of Physics, Vol. 57,
Iss. 5, pp. 457476, 1989.
10. R. Carrand J. Yang, Current-to-frequency converter for electron multipliers, Review of Scientic Instruments, Vol. 60, Iss.
7, pp. 23212323, 1989.
11. L. M. DeVito, A versatile clock recovery architecture and
monolithic implementation, Monolithic phase-locked loops
and clock recovery circuits, Editor B. Razavi, IEEE Press, New
York 1996.
iv This device is obsolete, but its data are still a good example of
the performance obtained with this implementation.
12. S. W. Glowacki, A current-to-frequency converter, Nuclear
Instruments and Methods, Iss. 186, pp. 549551, NorthHolland Publishing Company 1981.
13. P. R. Grayand R. G. Meyer, Analysis and Design of Analog
Integrated Circuits, 2nd Edition. Ed. John Wiley & Sons. Singapore 1984.
14. A. B. Grebene, Bipolar and MOS analog integrated circuit design, John Wiley & Sons, New York 1984.
15. K. W. Kobayashi et al., A novel self-oscillating HEMT-HBT
cascode VCO-Mixer using an active tunable inductor, IEEE
Journal of Solid-State Circuits, Vol.33, No. 6, June 1998.
16. LM231A/LM231, LM331A/LM331 Precision voltage-tofrequency converters, National Semiconductor Other Analog
Product Catalogue www.national.com/catalog/
17. J-F Loude, Current-to-frequency isolation converter, Journal of Physics E-Scientic Instruments, Vol.18,Iss. 10, pp.
824826, 1985.
18. T. D. Moore, S. Turner, C.C. Speake, A capacitance displacement transducer with large dynamic-range, good linearity and
frequency read-out, Measurement Science & Technology, Vol.
4,Iss. 12,pp. 15191521, 1993.
19. R. Pallas-Arenyand
`
J. G. Webster, Sensors and Signal Conditioning, John Wiley and Sons, New York 1991.
20. R. A. Pease, Application Note C: V/F converter ICs handle frequency-to-voltage needs,National Semiconductor Application Notes, http://www.national.com/apnotes/Voltage-toFrequencyConverters v1. html
21. R. A. Pease, Application Note 210: New Phase-LockedLoops Have Advantages as Frequency to Voltage Converters (and more),National Semiconductor Application Notes, http://www. national. com/apnotes/Voltage-toFrequencyConverters v1. html
22. B. Razavi, Analysis, Modeling and Simulation of Phase Noise
in Monolithic Voltage-Controlled Oscillators, Proc. CICC, pp.
323326, May 1995.
23. B. Razavi, Design of Monolithic phase-locked loops and clock
recovery circuits- a tutorial, Monolithic phase-locked loops
and clock recovery circuits, Editor B. Razavi, IEEE Press, New
York 1996.
19
24. M. Soyuerand R. G. Meyer, High-frequency phase-locked

loops in monolithic bipolar technology, IEEE Journal of
Solid-State Circuits, Vol. CS-24, pp. 787795, June 1989.
25. VFC32 Voltage-to-frequency and frequency-to-voltage converter, Texas Instruments www.ti.com.
26. VFC110 High-frequency voltage-to-frequency converter, Texas
Instruments, www.ti.com.
27. VFC320 Voltage-to-frequency and frequency-to-voltage converter, Texas Instruments www.ti.com.
28. W. S. Wilburn, A high-accuracy beam current integrator for
fast spin-ip experiments, Nuclear Instruments and Methods in Physics Research Section A-Accelerators Spectrometers
Detectors and Associated Equipment, Vol.394, Iss. 12, pp.
257260, 1997.
FERNANDO VIDAL-VERDU 1
1
RAFAEL NAVAS-GONZALEZ
MANUEL DELGADO-RESTITUTO2
2,3
ANGEL RODRIGUEZVAZQUEZ
1 Dpto.
de Electrnica,
Universidad de Mlaga,
Campus de Teatinos,
Complejo Tecnologico,
Malaga,
Spain
2 Institute of Microelectronics of
Seville, CNM-CSIC, Edicio
CICA-CNM, C/ Tara sn,
Seville, Spain
3 School of Engineering,
University of Seville, Seville,
Spain

04 Automatic Control

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

04 Automatic Control

Uploaded by

Copyright:

Available Formats

then also the feedback system

where the matrix P is positive definite. It can be verified that

This problem was studied in Russia during the 1950s. In

THE POPOV CRITERION

THE CIRCLE CRITERION

R e[(1 + i)G(i)] >

and the corresponding transfer function is

Theorem 1 (Circle criterion). Suppose that the system

then the system

G(s) = C(sI A)1 B

Theorem 2 (Popov criterion). Suppose that : R R is

The Popov criterion can then be stated as follows.

Popov and his colleagues made their problem statements in

Figure 1. Absolute stability deals with linear time-invariant systems

stability can be concluded from the Popov criterion if and only

Example. To apply the Popov criterion to the system

x1 = 5x1 4x2 + (x1 )

GAIN AND PASSIVITY

Figure 4. The Popov criterion can be applied when there exists a

For this purpose, a dynamical system is viewed as a map

The only property of saturation that would be exploited by

for all T 0. The gain of F is denoted F and defined as the

for all w sat(v), T 0. However, the inequality will remain

do this is to introduce a function h(t) with the property

F G < 1

then the feedback equations

This is the same as Eq. (8), since is the gain of and

Theorem 3 (Small gain theorem). Suppose that the input

where H(i) ei h(t)dt. The factor 1 i H(i) is

for all T and all y = Fu

For example, if the input is a voltage and the output is a

ABSTRACT DATA TYPES

7. G. Zames and P. L. Falb, Stability conditions for systems with

ABSORBER. See ELECTROMAGNETIC FERRITE TILE ABSORBER.

J. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering

Fig. 1. Controller structure with adjustable controller gains.

Fig. 2. Gain scheduling.

Fig. 3. Indirect adaptive control.

Fig. 4. Direct adaptive control.

Fig. 5. Model reference control.

Adaptive Pole Placement Control

On-Line Parameter Estimation

The constants ai , bi are the plant parameters. We can express yn as

In a similar way we can express the plant dynamics as follows:

based on (t) is given by

and the estimation error is given by

0 is now obvious, namely, for

Using the above inequality, we can prove the following theorem.

then (t) exponentially fast.

The following theorem holds:

which yields the so-called nonrecursive least-squares algorithm

We can show that P, satisfy the differential equations

where ns is designed to satisfy

we can see that the following theorem holds.

Model Reference Adaptive Control

where Gp (s) is expressed in the form

which is expressed in the same form as Eq. (32), that is,

where Zm (s), Rm (s) are monic polynomials and km is a constant.

Reference Model Assumptions.

c 0 , 3 R1 , 1 , 2 Rn 1 are constant parameters to be designed, and (s) is an arbitrary monic Hurwitz

is to be chosen so that the transfer function from r to yp is equal to W m (s).

= [T 1 , T 2 , 3 ]T ; S is an (n + np 1) (2n 1) matrix that depends on the coefficients of Rp , kp Zp ,

whose state-space realization is given by

Adaptive Pole Placement Control

F G < 1