Toeplitz matrices and more

First studied by Toeplitz in 1901. tij = tij . Dened r0 r1 .. . r0 p1 T = . ... ... . . pn1 p1 by 2n 1 parameters. rn1 . . . r1 r0

Applications: computation of splines, time series analysis, Markov chains, queuing theory, signal and image processing, PDE solutions.

Properties
1. A Toeplitz matrix T is persymmetric, so is its inverse. Proof. Let En = f liplr(In ) (exchange matrix). Then T = En T T En (persymmetric). Then T 1 = En T T En . 2. All Toeplitz matrices commute asymptotically, or diagonalize in the same basis when n . 3. Their eigenvectors are sines and cosines. 4. T is closely related to FFT. Let a Fourier operator (DFT matrix) F = f f t(eye(n))/sqrt(n) F CF 1 = T D0 , D0 = diag(1, , 2 , . . . , n1 ), = ei/n 5. T can be converted into a circulant matrix with the size roughly doubled and 1st col v = (pT rT )T . A circulant matrix C can be diagonalized by F : F CF 1 = D = diag(F v ). 6. Circular convolution circulant Toeplitz matrix-vector product. 7. Toeplitz matrix vector in O(n log n). C 8. Displacement structure
T T Z1 T Z 1 T = u:1:2 v:,1:2 1 Z = . . . 1

x 0

T B B T

x 0

Tx Bx

Z can be diagonalized by FFT. D1 C CD1 = u :,1:2 v :T ,1:2 D1 = diag(1, 2 , 4 , . . . , 2(n1) ) D1 = D1 C=

T u i,1:2 v j, 1:2 di fj

Solution
Method Fast and stable Fast and unstable Superfast preconditioner Superfast and stable Superfast and unstable New superfast and stable Fast: Levinson-Durbin, Trench ... Fast stable: Nagy, Chandrasekaran, Sayed, Gohberg, Kailath, Olshevsky, Gu ... Superfast Preconditioners: Benzi, R. Chan, Nagy, Plemmons, T. Chan, Strang, Yeung, Di Benedetto, Jin, Kailath, Olshevsky, Ku, Kuo, Strela, Tyrtyshnikov, ... Superfast: Martinsson, Tygert, Rokhlin, Ammar, Gragg, Stewart, Codevico, van Barel, Heinig, Chandrasekaran, Gu, Xia, Zhu ... Cost 20n2 3n2 O(n log n) O(n2 + n log2 ) O(n log2 n) O(n log n) Storage 1 n2 2 4n O(n) O(n log2 ) O(n) O(n)

2.1

Levinson (1947)
Tk E k r r Ek 1
T

Consider SPD case rst. Scale diags to be 1. Write Tk+1 y = v = b1:k bk+1

1 Suppose Tk x = b1:k and y = Tk r are solved. Yule-Walker eq.:

Tk y = r
1 1 . Then Ek = Ek Tk Due to the persymmetry, Tk

Tk Ek r

v v

= b1:k =

1 1 1 r) = x + Ek y Ek r = x Ek (Tk (b1:k Ek r) = x Tk v = Tk T rT Ek 1

= bk+1 =

T T T x /(1 + rT y ) (x + Ek y ) = bk+1 rT Ek v = bk+1 rT Ek = bk+1 rT Ek

Here, 1 + rT y > 0 since Tk+1 is SPD (Golub p.194). Cost: 4n2 and can be improved.

2.2

Inversion (Trench)
1 Tn

A Er rT E 1

B v vT

Then A Er r E 1
T

0 1

Av = Er, rT Ev + = 1 Use the Yule-Walker system Ay = r. = 1 rT Ev = 1 + (Ay )T Ev = 1 + y T EAv = 1 y T EEr = 1 y T r 1 1 + rT y 1 v = A Er = EA1 r = rEy = =
1 These give the last column/row of Tn . Then

AB = Erv T = In1 B = A1 + bij = A1 = A1 vv T +

ij

vi vj (symmetry about antidiag) vnj vni vi vj = bnj,ni + .

nj,ni

vi vj

Thus, this gives the symmetric border. Repeat from boundary to center: row/cols n, 1, n 1, 2, . . . Cost: 13n2 /4.

2.3

Nonsymmetric case
T y = r, Tk w = p, Tk x = b. Tk

For T dened by vectors r, p, Use

2.4

Stability

Theorem 1 (Cybenko) The Levinson and Trench algorithms are weakly stable for SPD Toeplitz matrices. Let k be in Levinson for Yule-Walker, and
T ek = 1 + r1: k y1:k , k = T (rk+1 + r1: k Ek y1:k ) . ek

Then the condition number ||T

1

n1

||1
1

1 + | k | 1 |k |

Large for k close to 1. A rounding error committed at the j th stage becomes magnied by a factor of ej1 ej1en1 at the nth stage.

Factorization/Schur algorithms

Each time, make one lower/upper diagonal zero, from the rst super-/sub-diagonal to the nth.

Bareisss factorization algorithm [1] 4

QR algorithm
T = R= t 0 uT v T0 r11 z T 0 Rb = = T0 u v T t0 Rt z 0 rnn = QR

Set T T T = RT R
T T Rt + uuT v T v zT z Rb = Rt Rb

Write
T T R1 R1 = Rt Rt + uuT updating T T R2 R2 = R1 R1 v T v updating T T R2 z T z updating Rb = R2 Rb

Where R1 , R2 are upper triangular. For updating problems, there exists a product Q of Givens rotations s.t. Rt R1 Q = T u 0 For downdating problems, there exists a product Y of Hyperbolic ratations s.t. Y R1 v T = R2 0

References
[1] E. H. Bareiss, Numerical solution of linear equations with Toeplitz and vector Toeplitz matrices, Numer. Math. 13 (1969), pp. 404424. [2] G. Cybenko, The numerical stability of the Levinson-Durbin algorithm for Toeplitz systems of equations, SIAM I. Sci. Statist. Comput. 1 (1980), pp. 303319. [3] J. R. Bunch, The weak and strong stability of algorithms in numerical linear algebra, Linear Algebra Appl., 88-89 (1987), pp. 4966. [4] G. Golub and C. F. Van Loan, Matrix Computations, Johns Hopkins. [5] N. Levinson, The Wiener RMS error criterion in lter design and prediction, J. Math. Phys., 25 (1947), pp. 261278. [6] J. Nagy, Fast Inverse QR Factorization for Toeplitz Matrices, SIAM J. Sci. Comput., 14 (1993), pp. 11741193.