Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.

5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C.11 Bang-bang Control

11.1 Introduction ∂F d ⎛ ∂F ⎞
− ⎜⎜ ⎟=0 ⇒
⎟ p& 2 = − p1 (11.10)
∂x 2 dt ⎝ ∂x& 2 ⎠
This chapter deals with the control with restrictions: is
bounded and might well be possible to have discontinuities. ∂F d ⎛ ∂F ⎞
− ⎜ ⎟=0 ⇒ p 2 = µ ( β − α − 2u ) (11.11)
∂u dt ⎝ ∂u& ⎠
To illustrate some of the basic concepts involved when
controls are bounded and allowed to have discontinuities we ∂F d ⎛ ∂F ⎞
− ⎜ ⎟=0 2vµ = 0 (11.12)
start with a simple physical problem: Derive a controller such ∂v dt ⎝ ∂v& ⎠
that a car move a distance a with minimum time.
(11.12) ⇒ v = 0 or µ = 0 . We will consider these two
The motion equation of the car
cases.

d 2x (i) µ =0
=u (11.1)
dt 2 ⎧p = 0
⇒⎨ 1 ⇒ be impossible.
where ⎩ p2 = 0
(ii) v = 0
u = u (t ), − α ≤ u ≤ β (11.2) (11.5): v 2 = (u + α )( β − u ) ⇒ u = −α or u = β
Hence
represents the applied acceleration or deceleration (braking)
and x the distance traveled. The problem can be stated as
⎧β 0 ≤ t ≤τ
minimize x& 2 = ⎨
⎩− α τ <t ≤T
T
T = 1 dt
∫o
(11.3) the switch taking place at time τ . Integrating using
boundary conditions on x 2
subject to (10.11) and (10.12) and boundary conditions
⎧β t 0 ≤ t ≤τ
x(0) = 0, x& (0) = 0, x(T ) = a, x& (T ) = 0 (11.4) x 2 = x&1 = ⎨ (11.13)
⎩ − α (t − T ) τ <t ≤T
The methods we developed in the last chapter would be
Integrating using boundary conditions on x1
appropriate for this problem except that they cannot cope
with inequality constraints of the form (11.2). We can
change this constraint into an equality constraint by ⎧1 2
introducing another control variable, v, where ⎪⎪ β t 0 ≤ t ≤τ
x1 = ⎨ 2 (11.14)
⎪− 1 α (t − T ) 2 + a τ <t ≤T
v = (u + α )( β − u )
2
(11.5) ⎪⎩ 2

Since v is real, u must satisfy (11.2). We introduce th usual Both distance, x1 , and velocity, x 2 , are continuous at
state variable notation x1 = x so that
t = τ , we must have

x&1 = x2 x1 (0) = 0, x2 (T ) = a (11.6) (11.14) ⇒ βτ = −α (τ − T )

x& 2 = u x 2 (0) = 0, x 2 (T ) = 0 (11.7) 1 1
(11.15) ⇒ βτ 2 = a − α (τ − T ) 2
2 2
We now form the augmented functional
Eliminating T gives the switching time as
T
T* =
∫ 0
{1 + p1 ( x 2 − x&1 ) + p 2 (u − x& 2 ) + µ [v 2
2aα
−(u + α )( β − u )]}dt (11.8) τ= (11.15)
β (α + β )

where p1 , p 2 ,η are Lagrange multipliers associated with the

and the final time is
constraints (11.6), (11.7) and (11.5) respectively. The Euler
equations for the state variables x1 , x 2 and control variables u
2a (α + β )
and v are T= (11.16)
αβ
∂F d ⎛ ∂F ⎞
− ⎜ ⎟=0 ⇒ p& 1 = 0 (11.9) The problem now is completely solved and the optimal
∂x1 dt ⎜⎝ ∂x&1 ⎟⎠ control is specified by

Chapter 11 Bang-bang Control 53

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

⎧β 0 ≤ t ≤τ ∂H
u=⎨ (11.17) p& i = − (i = 1,2, L , n) (11.23)
⎩− α τ <t ≤T ∂xi

this is illustrated in Fig. 11.1 The Euler equations for the control variables, u i , do not
follow as in section 10.4 as it is possible that there are
control u discontinuities in u i , and so we cannot assume that the partial
β
derivaties ∂H / ∂u i exist. On the other hand, we can apply the
∂F
free end point condition (9.23): = 0 to obtain
∂x& i
0 τ T time t p k (T ) = 0 k = q + 1, L , n (11.24)

that is, the adjoint variable is zero at every end point where
−α the corresponding state variable is not specified. As before,
we refer to (11.24) as tranversality conditions.
Fig. 11.1 Optimal Control
Our difficulty now lies in obtaining the analogous equation to
This figure shows that the control: ∂H / ∂u i = 0 for continuous controls. For the moment, let us
- has a switch (discontinuity) at time t = τ assume that we can differentiate H with respect to u, and
- only take its maximum and minimum values consider a small variation δu in the control u such
that u + δu still belong to U , the admissible control region.
This type of control is called bang-bang control.
Corresponding to the small change in u , there will be small
11.2 Pontryagin’s Principle (early 1960s) change in x , say δx , and in p , say δp . The change in the
value of J * will be δJ * , where
Problem: We are seeking extremum values of the functional
n

∑ p x& }dt
T
δJ * = δ
∫
T
{H −
J =
∫ 0
f 0 (x, u, t )dt (11.18) o
i =1
i i

subject to state equations The small change operator, δ , obeys the same sort of
x& i = f i(x, u, t ) (i = 1,2, L , n) (11.19) properties as the differential operator d / dx .

initial conditions x = x 0 and final conditions on x1 , x 2 , L , x q Assuming we can interchange the small change operator, δ ,
and integral sign, we obtain
(q ≤ n) and subject to u ∈ U , the admissible control region.
For example, in the previous problem, the admissible control n

∑ p x& )]dt
T
region is defined by δJ * =
∫ 0
[δ ( H −
i =1
i i

U = {u : −α ≤ u ≤ β } n

∫ [δH − ∑ δ ( p x& )]dt

T
= i i
As in section 10.4, we form the augmented functional 0
i =1
⎡ ⎤ n n

∑ ∑ p δ x& ]dt
n T

∑ ∫
T
J* =
∫ 0
⎢ f0 +
⎢
⎣ i =1
pi ( f i − x& i )⎥ dt
⎥
⎦
(11.20) =
0
[δH −
i =1
x& i δ pi −
i =1
i i

and define the Hamintonian Using chain rule for partial differentiation

n m n n
H = f0 + ∑i =1
pi f i (11.21) δH = ∑
∂H
∂u j
δu j + ∑
∂H
∂xi
δxi + ∑ ∂p δp
∂H
i
i
j =1 i =1 i =1

For simplicity, we consider that the Hamintonian is a so that

function of the state vector x, control vector u, and adjoint
vector p, that is, H = H (x, u, p) . ⎡ m
∂H
∫ ∑ ∂u
T
We can express J * as δJ * = ⎢ δu j
0 ⎢ j
⎣ j =1
T⎛ n ⎞
∫ ⎜H −
∑ p x& ⎟⎟ dt ⎞⎤
n
J* = ⎛ ∂H
∑ ⎜⎜⎝ ∂p δ p
(11.22)
0 ⎜ i i
+ i − p& i δ xi − xi δ pi − pi δ x& i ⎟⎟⎥ dt
⎝ i =1 ⎠ i ⎠⎥⎦
i =1

and evaluating the Euler equations for xi , we obtain as in ∂H

section 10.4 the adjoint equations since p& i = −∂H / ∂xi . Also, from (11.21) = f i = x& i
∂pi

Chapter 11 Bang-bang Control 54

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

using (11.19): x& i = f i(x, u, t ), (i = 1,2, L , n) . Thus We first illustrate its use by examining a simple problem. We
required to minimize
⎡ m n ⎤
∂H
∫ ∑ ∑ ( p& iδ xi + piδ x&i )⎥⎥ dt
T T
δJ * = ⎢
0 ⎢
⎣ j =1
∂u j
δu j −
i =1 ⎦
J = 1dt
∫ 0

T⎡ m
∂H
n ⎤ subject to x&1 = x 2 , x& 2 = u where −α ≤ u ≤ β and x1 (T ) = a ,
∫ ∑ ∑ ( piδ xi )⎥⎥ dt
⎢ d
= δu j −
0 ⎢ ∂u j dt x2 (T ) = 0 , x1 (0) = x 2 (0) = 0 .
⎣ j =1 i =1 ⎦
Introducing adjoint variables p1 and p 2 , the Hamiltonian is
We can now integrate the second part of the integrand to
yield given by

H = 1 + p1 x 2 + p 2 u
T ⎛⎜ m ⎞
T
n
∂H
δJ * = − ∑ ( p δ x ) + ∫ ⎜⎜ ∑
i i
0 ∂u j
⎟
δu j ⎟ dt
⎟
(11.25)
We must minimize H with respect to u, and where
i =1 0 ⎝ j =1 ⎠ u ∈U = [−α , β ] , the admissible control region.. Since H is
linear in u, it clearly attains its minimum on the boundary of
At t = 0 : xi (i = 1,L, n) are specified ⇒ δxi (0) = 0
the control region, that is, either at u = −α or u = β . This
At t = T : xi (i = 1,L , q ) are fixed ⇒ δxi (T ) = 0 illustrated in Fig.11.3. In fact we can write the optimal
For i = q + 1, L, n, from the transversality conditions, (11,24) control as

pi (T ) δxi (T ) = 0 ⎧−α if p 2 > 0

u=⎨
⎩ β if p 2 < 0
for i = 1,2, L , q, q + 1, L , n . We now have
H

T ⎛⎜ m
∂H
⎞
∫ ∑
⎟
δJ * = ⎜ δu j ⎟ dt
0 ⎜ ∂u j ⎟
⎝ j =1 ⎠
control u
where δu j is the small variation in the j th component of the
control vector u . Since all these variations are independent, −α β
and we require δJ * = 0 for a turning point when the controls Fig. 11.3 The case p 2 > 0
are continuous, we conclude that
But p 2 will vary in time, and satisfies the adjoint equations,
∂H
= 0 ( j = 1,2, L , m) (11.26) ∂H
∂u j p& 1 = − =0
∂x1
But this is only valid when the controls are continuous and ∂H
p& 2 = − = − p1
not constrained. In our present case when u ∈ U , the ∂x 2
admissible control region and discontinuities in u are allowed.
The arguments presented above follow through in the same Thus p1 = A , a constant, and p 2 = − At + B , where B is
way, except that (∂H / ∂u j )du j must be replaced by constant. Since p 2 is a linear function of t, there will at most
be one switch in the control, since p 2 has at most one zero,
H (x; u1 , u 2 ,L, u j + δu j ,L, u m ; p) − H (x, u, p)
and from the physical situation there must be at least one
switch. So we conclude that
We thus obtain
(i) the control u = −α or u = β , that is, bang-bang control;
T m
δJ * =
∫∑0
j =1
[ H (x; u1 ,L, u j + δu j ,L, u m ; p) − H (x, u, p)] dt (ii) there is one and only one switch in the control.

Again, it is clear from the basic problem that initially u = β ,

In order for u to be a minimizing control, we must followed by u = −α at the appropriate time.
have δJ * ≥ 0 for all admissible controls u + δu . This implies
that 11.3 Switching Curves

H (x; u1 ,L, u j + δu j ,L, u m ; p) ≥ H (x, u, p)
for all admissible δu j and for j = 1,L, m . So we have
established that on the optimal control H is minimized with
respect to the control variables, u1 , u 2 ,L, u m . This is known
as Pontryagin’s minimum principle.
(11.27) In the last section we met the idea of a switch in a control.
The time (and position) of switching from one extremum
value of the control to another does of course depend on the
initial starting point in the phase plane. Byconsidering a
specific example we shall show how these switching
positions define a switching curve.

Chapter 11 Bang-bang Control 55

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Suppose a system is described by the state variables x1 , x 2 x 2 − B = − k log | ( x1 − k ) / A |

where
Now if u = 1 , that is, k = 1 , then the trajectories are of the
x&1 = − x1 + u (11.28) form
x& 2 = u (11.29)
x 2 − B = − log | ( x1 − 1) / A |
Here u is the control variable which is subject to the
constraints −1 ≤ u ≤ 1 . Given that at t = 0, x1 = a, x 2 = b , that is
we wish to find the optimal control which takes the system to x 2 = − log | x1 − 1 | +C (11.32)
x1 = 0, x 2 = 0 in minimum time; that is, we wish to minimize
where C is constant. The curves for different values of C are
T
J = 1dt
∫ 0
(11.30) illustrated in Fig.11.4

u =1 x2
while moving from (a, b) to (0,0) in the x1 − x 2 phase plane
and subject to (11.28), (11.29) and

−1 ≤ u ≤ 1 (11.31)

Following the procedure outline in section 11.2, we introduce

the adjoint variables p1 and p 2 and the Hamiltonian
0 x1
H = 1 + p1 (− x1 + u ) + p 2 u 1
= u ( p1 + p 2 ) + 1 − p1 x1

Since H is linear in u, and | u |≤ 1 , H is minimized with

respect to u by taking

⎧+1 if p1 + p 2 < 0 A
u=⎨
⎩− 1 if p1 + p 2 > 0

So the control is bang-bang and the number of switches will Fig. 11.4 Trajectories for u = 1
depend on the sign changes in p1 + p 2 . As the adjoint
equations, (11.23), are Follow the same procedure for u = −1 , giving

∂H x2 = log | x1 − 1 | +C (11.33)
p& 1 = − = p1
∂x1 p1 = Ae t
⇒ , A and B are constant and the curves are illustrated in Fig. 11.5.
∂H p2 = B
p& 2 = − =0
∂x 2
x2 B
and p1 + p 2 = Ae t + B , and this function has at most one
sign change.

So we know that from any initial point (a, b) , the optimal

control will be bang-bang, that is, u = ±1 , with at most one
switch in the control. Now suppose u = k , when k = ±1 ,
then the state equations for the system are x1
−1 0
x&1 = − x1 + k
x& 2 = k
u = −1
We can integrate each equation to give

x1 = Ae −t + k
, A and B are constants
x2 = k t + B

The x1 − x 2 plane trajectories are found by eliminating t ,

Fig. 11.4 Trajectories for u = −1
giving

Chapter 11 Bang-bang Control 56

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

The basic problem is to reach the origin from an arbitrary Following the usual procedure, we form the Hamiltonian
initial point. All the possible trajectories are illustrated in
Figs. 11.4 and 11.5, and we can see that these trajectories are H = 1 + p1 x2 + p 2 u (11.39)
only two possible paths which reach the origin, namely AO
in Fig. 11.4 and BO in Fig. 11.5. We minimize H with respect to u, where −1 ≤ u ≤ 1 , which
gives
x2 B
u = −1 ⎧+1 if p2 < 0
u=⎨
⎩− 1 if p2 > 0

The adjoint variables satisfy

⎧& ∂H
0 x1 ⎪ p1 = − ∂x = 0 ⎧p = A
⎪ 1
−1 1 ⎨ ⇒ ⎨ 1
⎪ p& = − ∂H ⎩ p 2 = − At + B
= − p1
⎪⎩ 2 ∂x2

Since x2 (T ) is not specified, the transversality condition

becomes p 2 (T ) = 0 . Hence 0 = − At + B and p 2 = A(T − t ) .
A u =1 For 0 < t < T , there is no change in the sign of p 2 , and hence
no switch in u. Thus either u = +1 or u = −1 , but with no
switches. We have
Fig. 11.4 Trajectories for u = −1
x22 = 2( x1 − B) when u =1 (11.40)
Combining the two diagrams we develop the Fig. 11.6. The
curve AOB is called switching curve. For initial points below x22 = −2( x1 − B) when u = −1 (11.41)
AOB, we take u = +1 until the switching curve is reached,
followed by u = −1 until the origin is reached. Similarly for These trajectories are illustrated in Fig. 11.7, the direction of
the points above AOB, u = −1 until the switching curve is the arrows being determined from x& 2 = u
reached, followed by u = +1 until the origin is reached. So
we have solved the problem of finding the optimal trajectory u = +1 x2 x2 u = −1
from an arbitrary starting point.

Thus the switching curve has equation

⎧ log(1 + x1 ) for x1 > 0 x1 x1

x2 = ⎨ (11.34) 0 0
⎩− log(1 + x1 ) for x1 < 0

11.4 Transversarlity conditions

To illustrate how the transversality conditions ( pi (T ) = 0 if

xi is not specified) are used, we consider the problem of
finding the optimum control u when | u | ≤ 1 for the system Fig. 11.7 Possible Optimal Trajectories
described by
x2
x&1 = x 2 (11.35) x1
x& 2 = u (11.36) 0
( a, b )
which takes the system from an arbitrary initial point
x1 (0) = a, x 2 (0) = b to any point on the x2 axis, that is,

{
x1 (T ) = 0 but x2 (T ) is not given, and minimize
T+
T T−{
∫
J = 1.dt
0
(11.37) ( a, b ) A

subject to (11.35), (11.36), the above boundary conditions Fig. 11.8 Initial point below OA
on x1 and x 2 , and such that
We first consider initial points (a, b) for which a > 0 . For
−1 ≤ u ≤ 1 (11.38) points above the curve OA, there is only one trajectory,

Chapter 11 Bang-bang Control 57

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

u = −1 which reaches the x 2 -axis, and this must be the

optimal curve. For points below the curve OA, there are two
possible curves, as shown in Fig. 11.8.

From (11.36): x& 2 = u , that is x& 2 = ±1 , and the integrating

between 0 and T gives

x 2 (T ) − x 2 (0) = ±T

that is

T = | x 2 (T ) − x 2 (0) | (11.42)

Hence the modulus of the difference in final and initial values

of x 2 given the time taken. This is shown in the diagram as
T+ for u = +1 and T− for u = −1 . The complete set of
optimal trajectories is illustrated in Fig. 11.9.

x2
u = +1 u = −1

0 x1

Fig. 11.9 Optimal Trajectories to reach x 2 -axis in minimum

time

11.5 Extension to the Boltza problem

Chapter 11 Bang-bang Control 58