LAGRANGE




.
(Barrier and interior point methods)
Lagrange (Penalty
methods and Augmented Lagrangian methods)
.


.

.

.
.

.




"
"
,

min imizef ( x)

gi ( x) 0,

i 1,..., m,

(B)

.
I (scalar)
:
1: I ( x)

R0 {x / gi ( x) 0, i 1,..., m}

2: {x }
xB gi ( xB ) 0 i,
R 0
k
lim I ( x ) .
k
s(r ) r
. r1 r2 0 , s(r1 ) s(r2 ) 0 .
{rk } lim r 0 ,
k
k

lim s(rk ) 0 .
k

:
:
1. U ( x, r1 ) f ( x) s(r1 ) I ( x) , r1
0
0
. x R .

.

0
2. x x( r1 )
U R {x / gi ( x) 0, i 1,..., m}.
x(r1 )
R 0 ,
U = + , x(r1 )

U R.

3. x(r1 ) U ( x, r2 ),
r1 r2 0 .
4. ,
U ( x, r ) x(rk 1 )
k {rk } .

minimize x1 x2
g1 x12 x2 0,
g 2 x1

0.
m

s(r)=r

I ( x) ln gi ( x)
i 1

I o .

L( x, r ) f ( x) r ln gi ( x)
i 1

f ( x) x1 x2

g1 ( x) x12 x2

g2 ( x) x1

L( x, r ) f ( x) r ln g1 ( x) r ln g2 ( x)
L( x, r ) x1 x2 r ln( x12 x2 ) r ln x1

.
L
x
1
L ( x , r )
=0
L
x
2

L 1 r
x1

r (2 x )
L
r
1 2 1
x1
x1 x2 x1
L
r
1
x2
x12 x2

L( x, r )

L( x, r ) 0
1
1
L
1
r
2

x
)

r
(

2
x
)

1
2
1
2
x12 x2
x1
x
x x
x
1

L
1
L
1
1 r
( x12 x2 )
1 r

2
2
x2
x1 x2
x2
x1 x2

r (2 x1 )
r

x 2 x x
1
2
1
L( x, r )

r
1

x12 x2

=0

r (2 x1 ) r
0
2
x1 x2 x1

(2)

(1)

r
0
2
x1 x2

(2)

x12 x2 r (1)

( x12 x2 )(2 x1 ) r
1
0
2
x1 x2
x1

1 2 x1

r
0
x1

x 0
1

r
x1 x1 2 x1 x1
0 x1 2 x12 r 0
x1

2 x x1 r 0
2
1

 1 8r

2 4 12 4 2 (r )
x1 (r )

x1 (r )

1 1 8r
4

x1 (r ) , x (r ) 1 1 8r
1

. x1 ( r )

x x2 r
2
1

x1 (r )

1 1 8r

x2 r

(2)

(1 1 8r )2
x2 (r )
r
16

x2 (r )

r.

 r=1

1 1 8 1 3
x1 (1)

0.5
4
4

(1 1 8)2
x2 (1)
1
16

(1 3) 2
1 1.25 .
16

r=0.5 x (0.5)
1

1 1 8 0.5 1 5

0.309
4
4

(1 1 8 0.5)2
(1 5)2
x2 (0.5)
0.5
0.5 0.595 .
16
16

r=0.25

1 1 8 0.25 1 3
x1 (0.25)

0.183
4
4

(1 1 8 0.25)2
(1 3)2
x2 (0.25)
0.25
0.25 0.283 .
16
16

r=0.1

1 1 8 0.1 1 1.8
x1 (0.1)

0.085
4
4

(1 1 8 0.1)2
(1 1.8)2
x2 (0.1)
0.1
0.1 0.107 .
16
16

x1 (r )

x2 (r )

1.000

0.500

1.250

r2

0.500

0.309

0.595

r3

0.250

0.183

0.283

r4

0.100

0.085

0.107

r1
r

 x1 ( r ) x2 ( r )
4 r .
r .
r
( x1 (r ), x2 (r )) (0, 0) .

a:

.
. :

min imizef ( x1 ,x2 )= x1 x2

g1 ( x1 ,x2 ) x12 x2 0,
g 2 ( x1 ,x2 ) x1

0.

MATHEMATICA


LAGRANGE (PENALTY AND AUGMENTED
LAGRANGIAN METHODS)



. c

,
.
c ,
.

.
.

min imizef ( x)
(2.1)

f: R n

h( x) 0 , x X
R, h : Rn Rm ,

Rn .


*
x* Lagrange
= R n.
Lc : Rn Rm
R ,

c
2
Lc ( x, ) f ( x) h( x) h( x) ,
2

(2.2)

c .

Lc (, )
x* :
*
() . , c
, > 0

> 0

* 2
Lc ( x, ) Lc ( x , ) x x , x x x* ,
2
*

(2.3)

x
*
Lc (, ) .
* ,
*
x
Lc (, ) .

() c .
c
.
Lc (, ) . Lc ( x, ) f ( x)
x, Lc ( x, ) f ( x)
x . ,
x *

Lc (, )

c .



min imizef ( x)

1 2
( x1 x22 )
2

x1 =1

(2.4)

x = (1,0) Lagrange
* = -1,

c
2
Lc ( x, ) f ( x) h( x) h( x) ,
2
1
f ( x) ( x1 x2 ) h( x) x1 1
2

1 2 2
c
Lc ( x, ) ( x1 x2 ) ( x1 1) ( x1 1)2 ,
2
2

(2.5)


x( , c)
L( x, r ) 0
L
x
1
L( x, r ) =0
L
x
2

L 1
c
L
2x1 2( x1 1)( x1 1) x1 c( x1 1)
x1 2
2
x1

L
x1 cx1 c
x1

L
x2
x2

L
1
2 x2
x2
2

x1 cx1 c
L( x, )
0
x2

c
x1 cx1 c 0 x1 (1 c) c x1
1 c

x2 0

x1 ( , c)

c
c 1

c > 0,

x2 ( , c) 0

lim* x1 ( , c) x1 ( * , c)

lim* x2 ( , c) x2 ( * , c)

= -1
Lagrange
*

lim* x1 ( , c) x1 ( * , c) x1 (1, c)

lim* x2 (, c) x2 ( , c) x2 (1, c)
*

(2.6)x1 ( , c)

c 1

x2 ( , c) 0

c (1) c 1
lim* x1 ( , c) x1 ( , c) x1 (1, c)

1 x1*

c 1 c 1
*

lim* x2 (, c) x2 ( * , c) x2 (1, c) 0 x2*

Lc ( x, )

(. 1.2

 1.2.1:

1
c
Lc ( x, ) ( x12 x22 ) ( x1 1) ( x1 1)2 ,
2
2

, c= 1 .
L ( x, )
c

x* (1,0) * = -1.

 (2.6) x1 ( , c)

c 1

x2 ( , c) 0

c c 1
1

lim x1 ( , c) lim
c

.
DE L HOSPITAL

c
(c )
1
lim
lim 1 x1*
c c 1
c (c 1)
c 1

lim x1 ( , c) lim
c

lim x2 ( , c) lim0 0 x2*

c

c ,
Lc ( x, )
(. 1.2.2).

 1.2.2:

1
c
Lc ( x, ) ( x12 x22 ) ( x1 1) ( x1 1)2 ,
2
2

, = 0 c.

Lc ( x, )

*
x (1, 0) c .

 -
X R n

min imizef ( x)
h( x) 0.


Lc (, ) , :
() Lagrange.
() c .

(2.25)


*
Lagrange. ,

ck

(ill conditioning). ,
k
c
,
k .

k 1

c h( x )
k

(2.26)

 1.2.2,
{ x k} x*
, { k c k h( x k )}
Lagrange * .

k
.
.

min imizef ( x)
x1 1,

1 2
( x1 x22 )
2

x* = (1,0) Lagrange * = -1.

c
2
Lc ( x, ) f ( x) h( x) h( x) ,

f ( x) 1 ( x12 x2 2 ) h( x) x1 1 ,
2

1
c
Lc ( x, ) ( x12 x22 ) ( x1 1) ( x1 1) 2 .
2
2
k

x
L k (, k )
c

ck k
x k
, 0 .
c 1

(2.27)

(2.27)
k 1
k
k
k
,

(2.26)

c
h
(
x
)
h( x) x 1
1

k
k
k
k
k

c
1
k 1
k
k
k
k
c k
1 c k
k
c 1

c 1 c 1

k
k
k k
k
k k
k
k
k

c
k k ck k
k

k
c 1
c 1
c 1
c 1

k 1

k 1

k ck

ck 1

Lagrange
= -1,
,

k ck *
.,
k
, * = -1
c 1
*

k
k
k
k
k
k
k
k
k

c
c

1
k 1
*
k
1 k
k
k
k
c 1
c 1 c 1
c 1
c 1

k 1 *

k *
ck 1

() k * 1 xk x* (1,0)
1
k
{ c } [ c 1
k * ,
].
() c k
, { k * }
c k .
k
c

k

1
min imize ( x12 x22 )
2
x1 1,

x = (1,0) Lagrange *

Lc ( x, ) f ( x) h( x)

c
2
h( x ) ,
2

1
f ( x) ( x12 x2 2 )
2

1 .

h( x) x1 1,

1
c
2
2
Lc ( x, ) ( x1 x2 ) ( x1 1) ( x1 1)2 .
2
2

 x k

Lck ( x, k )

k
k

k
x k
, 0 .
c 1
k
, , c 1.
k
c 1 ,
k
c
1 k 1 .

(2.28) h( x) x1 1 ,

(2.26)

k 1

c h( x )
k 1

k
k
ck k
ck 1
k
k c
c k
1 c k
k
c 1

c 1 c 1
k

k
k
k k
k
k k
k
k
k

c
k k ck k
k

k
c 1
c 1
c 1
c 1

(2.2

 k 1

k c k

ck 1

Lagrange = 1 ,

k 1

k c k
* *= 1

k
c 1
*

k
k
k
k
k
k
k
k
k

c
c

1
k 1
*
k
1 k
k
k
,
k
c 1
c 1 c 1
c 1
c 1
k
*

(2.29)
k 1 * k
.

c 1

,

. , c k
,
.
,
,
{ c k }, ,

 c k 1 . ,
k
(2.29) ,
c
2 [ 1
c k ].
1
k
,

.




Lagrange

.

.