EE609

Assignment 6

February 10, 2015

Weight: 1% if submitted by Feb 14, 2014.

Assignment will not be graded. No need to submit practice problems.

1 Assignment Problems
1. Show that the following two problems are duals of each other.
p = min max(PT u)i
i

s. t. u 0
m
X
ui = 1

(1)
(2)
(3)

i=1

and
d = max min(Pv)i

(4)

s. t. v 0
n
X
vj = 1

(5)

(6)

j=1

Does it hold that p = d ?

2. Consider the following unconstrained minimization problem:
min (kyk2 + xT y)

(7)

(a) Show that the problem is unbounded below if kxk2 > 1.

(b) Solve the problem for the case when kxk2 1.
3. Find the dual of the linear program:
min cT x

(8)

s. t. Gx h

(9)

Ax = b

(10)

2 Practice Problems
4. Find the dual of the penalty function approximation
min

m
X

(ri )

(11)

i=1

s. t. r = Ax b
where is the deadzone linear penalty function
(
(u) =

0
|u| 1

|u| 1
|u| > 1

(12)

(13)

EE609

Assignment 6, Page 2 of 5

February 10, 2015

5. Consider the following non-convex problem

p = min xT Ax

(14)

s. t. xi {1, 1}

(15)

where A Snn . Show that

nmin (A) p

Aij

(16)

i,j

(Hint: express the constraint as x2i = 1 and use weak duality).

6. Find the dual of the following convex optimization problem
min

n
X

xi log(xi )

(17)

i=1

s. t. Ax b
n
X
xi = 1

(18)
(19)

i=1

7. In this problem, we will show that strong duality holds for the following non-convex problem
min 2bT x xT x
T

s. t. x x 1

(20)
(21)

Prove the result using the following steps:

(a) Show that the solution of the primal problem always satisfies kx k = 1.
(b) Show that the primal optimum is p = 2 kbk 1.
(c) Show that the dual problem is given by
max bT b/( 1)
s. t. > 1

(22)
(23)

(d) Show that the dual optimum is d = 2 kbk 1.

8. Show that for any function f (u, v), it holds that
sup inf f (u, v) inf sup f (u, v)
u

(37)

Note that this result implies weak duality when f is the Lagrangian.
9. Consider an n n matrix A with rank(A) = n 1 and 1T A = 0. Find the dual of the following
problem and solve it explicitly.
min kAx bkp

Page 2

(38)

EE609

Assignment 6, Page 3 of 5

February 10, 2015

10. Consider the following problem

T

max b x +

n
X

ui 2

i=1

n
X

vi

(39)

i=1

s. t. Ax + c = u v

(40)

u0

(41)

v0

(42)

Show that it can equivalently be written as

max bT x 21T [Ax + c] + 1T [Ax + c]+

(43)

where y+ = max{y, 0} and y = max{y, 0}.

11. Find the dual of the convex piece-wise linear minimization problem:
min max (aTi x + bi )
i=1,...,m

(44)

12. Use the result from Question 2 to find the dual of the following problem
min

N
X

kAi x + bi k2 +

i=1

1
kx x0 k22
2

(45)

where Ai Rmi n , bi Rmi , and x0 Rn .

13. Find the dual of the following problem
max
x

m
X

log(bi aTi x).

(46)

i=1

14. Consider the following convex optimization problem:

min

m
X

exp(xi 1) + y

(47)

i=1

s. t. Ax b + y1 0

(48)

Use appropriate change of variables and elimination to show that it can equivalently be written as
m
X

eui )

(49)

s. t. Au b 0

(50)

min log(

i=1

if it holds that A1 = 0.
15. Consider the convex optimization problem
x = arg min f0 (x)

(51)

Page 3

EE609

Assignment 6, Page 4 of 5
s. t. fi (x) 0

February 10, 2015

i = 1, 2, . . . , m

(52)

where f0 (x) and fi (x) are differentiable and convex. Use KKT conditions to show that
f0 (x )T (x x ) 0

(53)

for all feasible x.

16. Find the dual of the penalty function approximation
m
X

min

(ri )

(54)

i=1

s. t. r = Ax b

(55)

where is the huber penalty function

(u) =

u2
2|u| 1

|u| 1
|u| > 1

(56)

17. Let E1 and E2 be two ellipsoids defined as

E1 = {x Rn |(x x1 )P1
1 (x x1 ) 1}

(57)

E2 = {x Rn |(x x2 )P1
2 (x x2 ) 1}

(58)

where P1 , P2 Sn++ . Show that E1 E2 = if and only if there exists z Rn such that


p
p




T
z
z
+
P
P


1
2 < z (x1 x2 )
2

(59)

Proceed along the following steps.

(a) Show that the distance between the two ellipsoids is the optimum value of the following problem
min kx yk2
x,y

p 1


s. t. P1 (x x1 ) 1

p 1


P2 (y x2 ) 1

(60)
(61)
(62)

(b) Show that (60) can equivalently be written as

(c) Show that its dual is given by

min kuk2

(63)

s. t. kvk 1

(64)

kwk 1
p
P1 v = x x 1
p
P2 w = y x 2 u = x y

p
p






max P1 z P2 z + zT (x1 x2 )
2

s. t. kzk 1

Page 4

(65)
(66)
(67)

(68)
(69)

EE609

Assignment 6, Page 5 of 5

February 10, 2015

18. Consider relative entropy minimization problem

min
x

n
X

xk log(xk /yk )

(70)

k=1

s. t. Ax = b
n
X
xk = 1

(71)
(72)

k=1

where the domain of the objective function is Rn++ . Show that its dual can be written as
!
n
X
T
T
yk exp(ak u)
max b u log
u

k=1

where ak is the k-th column of A.

Page 5

(73)