Linear Algebra Notes

Chapter 1
Systems of linear equations

1.1
1.1.1
Examples
Example
Consider the equation

x1 + 2x2 = 4
Lets assume that x1 and x2 are real numbers. That will be the case for much of this course.
We can solve for x1 in terms of x2 to get
x1 = 4 2x2
There are infinitely many solutions to this equation:
x1 = 4 2t,
1.1.2
x2 = t,
tR
Example
This time lets consider the solution of two equations
x1 + 2x2 = 4
x1 4x2 = 7
We can multiply the first equation by -1 and add to the second equation to get
x1 + 2x2 = 4
6x2 = 3
CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS
We can now divide the second equation by -6 to solve for x2 .
x1 + 2x2 = 4
x2 = 1/2
We can now multiply the second equation by -2 and add to the first.
x1
= 5
x2 = 1/2
Note that this time we get one unique solution and not infinitely many.
1.1.3
Example
x1 + 2x2 = 4
2x1 + 4x2 = 7
We can multiply the first equation by -2 and add to the second equation to get
x1 + 2x2 = 4
0+0 = 1
This is a contradiction and we conclude that the system of equations has no solution.
1.2
Idea
Note how simple the idea is: We multiply an equation by a scalar and then add to another
equation. This simplifies the system. We keep simplifying until we get a solution or we get
a contradiction.
1.3
Nonlinear equations
The situation with nonlinear equations is very different.
1.4. SOLVING SYSTEMS OF LINEAR EQUATIONS
1.3.1
Example
How do we solve
tan x = x
Do we know that this nonlinear equation has a solution? How many solutions does it have?
It actually has infinitely many solutions.
1.3.2
Example
tan x = x
x2 + y 2 = 1
How many solutions does this system have?
1.4
1.4.1
Solving systems of linear equations

Basic operations
The basic operations used to solve a system of linear equations are

scaling a row
interchanging rows
replacing a row by the sum of the row and a scalar multiple of another row
1.4.2
Matrix notation
Consider the system of linear equations

1x1 + 1x2 + 1x3 = 6
2x1 + 1x2 + 1x3 = 7
1x1 + 1x2 + 2x3 = 12
In matrix notation we would write this system

1 1 1
x1
6
2 1 1 x2 = 7
1 1 2
x3
12
1.4.3
Coefficient matrix
1
2
1
|

1 1
x1
6
1 1 x2 = 7
1 2
x3
12
{z
}
coefficient matrix
1.4.4
Augmented matrix
We also often represent the system by an augmented matrix
1 1 1 6
2 1 1 7
1 1 2 12
1.4.5
Apply elementary row operations to solve the system
-2 * Row 1 + Row 2 replaces Row 2
1 1
1
6
0 1 1 5
1 1
2 12
1 1
1
6
0 1 1 5
0 0
1
6
1 * Row 3 + Row 2 replaces Row 2
1 1 1 6
0 1 0 1
0 0 1 6
1 1 0 0
0 1 0 1
0 0 1 6
-1 * Row 2 replaces Row 2
1 1 0 0
0 1 0 1
0 0 1 6
1.5. REDUCED ROW ECHELON FORM
1 0 0 1
0 1 0 1
0 0 1 6
1.4.6
Solution
We now read off the solution

x2 = 1,
x1 = 1,
x3 = 6
which we might also write as a vector
1
1
6
1.5
1.5.1
Reduced row echelon form

Leading coefficient or pivot of a nonzero row
The first nonzero element from the left end of the row.
1.5.2
Example
Consider the matrix
1 1 1 6
0 2 1 5
1 1 2 12
The leading coefficient or pivot of the second row is 2.
1.5.3
Row echelon form
A matrix is in row echelon form if

All rows with at least one nonzero element are above any rows which only contain
zeros. Rows with all zeros should be at the bottom of the matrix.
The leading coefficient (pivot) of a nonzero row is always to the right of the leading
coefficient of the row above it.
All entries in a column below a leading coefficient are zeroes.
1.5.4
Example
Consider the matrix
1 1 1 6
0 2 1 5
1 1 2 12
This matrix is not in row echelon form because the leading coefficient 1 in the first row
does not have all zeros below it, equivalently the leading coefficient 1 in the third row is not
strictly to the right of the leading coefficient 1 in the first row.
1.5.5
Example
Consider the matrix
1
0
0
0
1
2
0
0
1
1
0
2
6
5
0
6
This matrix is not in row echelon form because there is a row of zeros above a nonzero row.
If we interchange the last two rows
1 1 1 6
0 2 1 5
0 0 2 6
0 0 0 0
the matrix is now in row echelon form.
1.5.6
Reduced row echelon form
A matrix is in reduced row echelon form if

It is in row echelon form
The leading coefficients of each nonzero row is 1 and there is no other nonzero entry
in the same column
1.5.7
Example
Consider the matrix
1
0
0
0
1
2
0
0
1
1
2
0
6
5
6
0
1.6. GAUSS-JORDAN ELIMINATION
This matrix is in row echelon form but not in reduced row echelon form because there are
nonzero rows which have pivots that are not 1 and there are nonzero entries above some of
the pivots.
1.5.8
Example contd
Consider the matrix
1 1 1 6
0 2 1 5
0 0 2 6
This matrix is not in reduced row echelon form. We can scale the second and third rows to
get
1 1 1
6
0 1 1/2 5/2
0 0 1
3
We can now multiply the third row by -1 and add to the first row and multiply the third
row by -1/2 and add to the second to get:
1 1 0 3
0 1 0 1
0 0 1 3
We can now add -1 times the second row to the first to get:
1 0 0 2
0 1 0 1
0 0 1 3
This matrix is in reduced row echelon form.
1.6
Gauss-Jordan elimination
The process of using elementary row operations:

scaling a row
interchanging rows
replacing a row by the sum of the row and a scalar multiple of another row
to put a matrix into row echelon form or reduced row echelon form is called Gauss-Jordan
elimination or Gaussian elimination.
1.7
1.7.1
Geometric interpretations
Hyperplanes
An equation of the form

c1 x 1 + c2 x 2 + + cn x n = d
with the ci and d constants defines an n 1 dimensional hyperspace in Rn .
1.7.2
Example
The equation
x+y+z = 7
defines a two dimensional plane in three (or perhaps higher) dimensional space.
1.7.3
Example
The equation
x+y = 7
defines a one-dimensional line in two (or perhaps higher) dimensional space.
1.7.4
Geometry and solutions
Solutions of systems of linear equations then can be interpreted as intersections of hyperplanes.
1.7.5
Example
Consider the system of equations
x+y = 3
xy = 1
If we draw the lines defined by these two equations then they intersect at exactly one point
(x = 1, y = 2) which is the solution to the system.
At how many points can two lines intersect?
1.8. EXERCISES
1.7.6
Example

x+y = 3
2x + 2y = 7
If we draw the one-dimensional lines defined by these two equations then they run parallel
without ever touching. There is no point of intersection on the graph and no solution to this
system of equations.
1.7.7
Example

x+y+z = 1
xy+z = 7
The two equations are of two-dimensional equations which intersect in a line and the solution
of system is
x = t,
y = 3,
z = 4 t,
tR
How many ways can two planes intersect (or not intersect) in three dimensions? How about
three planes?
1.8
1.8.1
Exercises
Exercise

x1 + x2 + x3 = 5
2x1 + 3x2 + 5x3 = 8
4x1 + 5x3 = 2
Write the augmented matrix that represents this system. Put the augmented matrix in reduced row echelon form. Find the solution.
Answer: x1 = 3 , x2 = 4 , x3 = 2.
10
1.8.2
Exercise
x1 + x2 + x3 = 5
2x1 + 3x2 + 5x3 = 8
Write the augmented matrix that represents this system. Put the augmented matrix in reduced row echelon form. Find the solution.
Answer: x1 = 7 + 2t , x2 = 2 3t , x3 = t, t R
Chapter 2
Matrices
2.1
Example
2.1.1
Problem
We consider solving a 3x3 linear system:

x+y+z = 1
2x + y + z = 2
x + 2y + 3z = 3
2.1.2
Gaussian elimination
Gaussian elimination is the process of putting the augmented matrix in row echelon form.
We will use this first:
The augmented matrix is
1 1 1 1
2 1 1 2
1 2 3 3
We get zeros below the first pivot of the first row by
-2*R1 + R2
-1*R1 + R3
1 1
1 1
0 1 1 0
0 1
2 2
11
12
CHAPTER 2. MATRICES
We get zeros below the first pivot of the second row by

1*R2 + R3
1 1
1 1
0 1 1 0
0 0
1 2
2.1.3
Row echelon form
The augmented matrix is now in row echelon form after performing Gaussian elimination
and the solutions can be found by scaling and back substitution:
The third row gives
z = 2
The second row gives
y = 0 + z y = z = (2) = 2
Then the first row gives
x = 1 y z = 1 2 (2) = 1
2.1.4
Gauss-Jordan elimination
Gauss-Jordan elimination puts the matrix in reduced row echelon form. We can think of it
as continuing on with Gaussian elimination until we get to reduced row echelon form.
2.1.5
Continuing
We had
1 1
1 1
0 1 1 0
0 0
1 2
We scale so that the pivots are all 1:
1 1 1 1
0 1 1 0
0 0 1 2
We can get zeros above the pivot of the second row by
2.2. MATRICES
13
-1*R2 + R1
1 0 0 1
0 1 1 0
0 0 1 2
We can get zeros above the pivot of the third row by
-1*R3 + R2
1 0 0 1
0 1 0 2
0 0 1 2
The augmented matrix is now in reduced row echelon form and the results can be read off
directly from the matrix:
x = 1,
2.2
Matrices
2.2.1
Definition
z = 2
y = 2,
Rectangular arrays of real numbers as follows.
2.2.2
Example
a11 a12
a21 a22
.
..
.
.
.
A =
ai1 ai2
.
..
..
.
am1 am2
a1n
a2n
..
ain
..
.
amn
is an m n matrix with m rows and n columns.

aij is the element of A that is in the ith row and the jth column.
2.2.3
Other examples

1 2
B =
4 5
14
CHAPTER 2. MATRICES
is a 2x2 matrix and

1
2

C =
3
4
5
is a column matrix with five elements.
2.2.4
Equality of matrices
Two matrices Amn and Bm0 n0 are equal if they have the same dimensions and each corresponding element is the same:
m = m0 ,
n = n0
aij = bij
for all 1 i m, 1 j n.
2.2.5
Addition of matrices
If two matrices
a11 a12
a21 a22
.
..
.
.
.
A =
ai1 ai2
.
..
..
.
am1 am2
a1n
a2n
..
ain
..
.
amn
and
b11 b12
b21 b22
.
..
.
.
.
B =
b
b
i1
i2
.
..
..
.
bm1 bm2
b1n
b2n
..
bin
..
.
bmn
are of the same dimension then their sum C = A + B is defined to be a matrix of the same
dimension whose elements are given by adding the corresponding elements of A and B:
2.2. MATRICES
15
c11 c12
c21 c22
.
..
.
.
.
C =
ci1 ci2
.
..
..
.
cm1 cm2
c1n
c2n
..
cin
..
.
cmn
with
cij = aij + bij
2.2.6
Example

1 1 1
1 1 1
2 2 2
+
=
2 2 2
1 1 1
3 3 3
2.2.7
Subtraction of matrices
If two matrices
a11 a12
a21 a22
.
..
.
.
.
A =
ai1 ai2
.
..
..
.
am1 am2
a1n
a2n
..
ain
..
.
amn
and
b11 b12
b21 b22
.
..
.
.
.
B =
bi1 bi2
.
..
..
.
bm1 bm2
b1n
b2n
..
bin
..
.
bmn
are of the same dimension then their difference C = A B is defined to be a matrix of the
same dimension whose elements are given by subtracting the corresponding elements of A
and B:
16
CHAPTER 2. MATRICES
c11 c12
c21 c22
.
..
.
.
.
C =
c
c
i1
i2
.
..
..
.
cm1 cm2
c1n
c2n
..
cin
..
.
cmn
with
cij = aij bij
2.2.8
Example

1 1 1
1 1 1
0 0 0
=
2 2 2
1 1 1
1 1 1
2.2.9
Multiplication by a scalar multiple
If Amn is a matrix
a11 a12
a21 a22
.
..
.
.
.
A =
a
a
i1
i2
.
..
..
.
am1 am2
a1n
a2n
..
ain
..
.
amn
and c R then the scalar multiple (cA)mn is given by multiplying each entry of A by c
(cA)ij = c aij
ca11 ca12
ca21 ca22
.
..
.
.
.
cA =
cai1 cai2
.
..
..
.
cam1 cam2
2.2.10
Example
If

1 1 1
A =
2 2 2
ca1n
ca2n
..
cain
..
.
camn
2.3. PRODUCTS OF MATRICES
17
then 4A is

4 4 4
4A =
8 8 8
2.3
2.3.1
Products of matrices
Definition
The product of matrix Amk and Bkn ( note that the number of columns of A must be the
same as the number of rows of B is defined by
(AB)ij =
k
X
aip bpj
p=1
We can think of this as taking dot products of the the ith row of A with the jth column of
B.
2.3.2
Example
Suppose

1 2 2
A =
0 5 1
and

3
B = 1
4
A has three columns and B has three rows so the product of A and B is defined. A has two
rows and B has one column so the product will have two rows and one column.
(AB)11 =
a1p bp1 = 1 3 + 2 1 + 2 4 = 13
(AB)21 =
a2p bp1 = 0 3 + 5 1 + 1 4 = 9
Then

13
AB =
9
18
2.3.3
CHAPTER 2. MATRICES
Lemma
If A and B are matrices and the product AB is defined then the jth column of AB is given
by
A[jth col of B]
2.3.4
Lemma
If A and B are matrices and the product AB is defined then the ith row of AB is given by
[ith row of A]B
2.3.5
Multiplication is not commutative
Given two matrices A and B, we may have that AB is defined and BA is not defined. If
both products are defined there is no requirement that BA = AB.
2.3.6
Exercise
Given

A =
1 0
2 3

1 2
B =
3 0
Show that AB 6= BA.
2.4
2.4.1
Transpose of a matrix
Definition
If A is a matrix
a11 a12
a21 a22
.
..
.
.
.
A =
ai1 ai2
.
..
..
.
am1 am2
then the transpose At of A is
a1n
a2n
..
ain
..
.
amn
2.4. TRANSPOSE OF A MATRIX
19
a11 a21
a12 a22
.
..
.
.
.
A =
a1i a2i
.
..
..
.
a1m a2m
an1
an2
..
ani
..
.
anm
or
(At )ij = (A)ji = aji
2.4.2
Example
Consider the matrix

1 2 2
A =
0 5 1
The transpose At of A is
1 0
At = 2 5
2 1
2.4.3
Transpose of a transpose
If A is a matrix then
(At )t = A
2.4.4
Transpose of a sum
If A and B are matrices of the same size then

(A + B)t = At + B t
2.4.5
Transpose of a scalar multiple
If A is a matrix and c is a real number then

(cA)t = cAt
20
CHAPTER 2. MATRICES
2.4.6
Transpose of a product
If A and B are matrices such that the product AB is defined then

(AB)t = B t At
2.5
2.5.1
Trace of a square matrix

Definition
If A is a square matrix
a21
a22
..
.
a2i
..
.
a11
a12
.
.
.
A =
a1i
.
..
a1n
a2n
an1
an2
..
ani
..
.
ann
the trace tr( A ) of A is the sum

tr A =
aii
2.5.2
Example
1 0 2
A = 2 5 7
2 1 4
tr A = 10.
2.6
Some properties of matrices
Suppose that in this section A, B, and C are matrices and a , b, and c are real numbers.
Suppose that the operations discussed in this section are defined.
2.6.1
Addition is commutative
A+B = B+A
2.6.2
Addition is associative
(A + B) + C = A + (B + C)
2.7. PROOFS OF THE PROPERTIES
2.6.3
21
Multiplication is associative
(AB)C = A(BC)
2.6.4
Distribution
A(B + C) = AB + AC
(B + C)A = BA + CA
2.6.5
Scalar multiplication
a(B + C) = aB + aC
a(bC) = (ab)C
a(BC) = (aB)C = B(aC)
2.7
Proofs of the properties
In this section the elements of A are denoted aij , the elements of B are denoted bij , and the
elements of C are denoted cij .
2.7.1
Proof
A+B = B+A
Proof.
(A + B)ij = aij + bij

= bij + aij
= (B + A)ij
2.7.2
Proof
Amn (Bnp + Cnp ) = Amn Bnp + Amn Cnp
Proof.
22
CHAPTER 2. MATRICES
(A[B + C])ij =
n
X
Aik (B + C)kj
k=1
=
=
n
X
k=1
n
X
(Aik Bkj + Aik Ckj )

Aik Bkj +
k=1
n
X
Aik Ckj
k=1
= (AB)ij + (AC)ij
2.7.3
Exercise
Using the previous two proofs as a model, give the proof for the other properties presented
in the previous section.
2.8
2.8.1
Identity matrix
Definition
Inn is the square matrix with all zeros except for ones on the main diagonal.
2.8.2
Example
1 0 0
I = 0 1 0
0 0 1
2.8.3
Property
If Ann is a square matrix then

Inn Ann = Ann Inn = Ann
2.8.4
Exercise
1
I = 0
0
2
A = 7
4
Confirm that IA = AI = A.
0 0
1 0
0 1
3 5
1 0
9 1
2.9. ELEMENTARY MATRICES
2.9
2.9.1
23
Elementary matrices
Definition
Any matrix formed from the identity matrix by an elementary row operation is called an
elementary matrix.
2.9.2
Example
From the identity matrix
1 0 0
I = 0 1 0
0 0 1
we can form the elementary matrix E by multiplying the second row by 3:
1 0 0
E = 0 3 0
0 0 1
Now if
2 3 5
A = 7 1 0
4 9 1
then EA is
2 3 5
EA = 21 3 0
4 9 1
E multiplies the third row of A by 3.
2.9.3
Example
1 0 0
I = 0 1 0
0 0 1
we can form the elementary matrix E by interchanging the first and second rows:
0 1 0
E = 1 0 0
0 0 1
Now if
24
CHAPTER 2. MATRICES
2 3 5
A = 7 1 0
4 9 1
then EA is
7 1 0
EA = 2 3 5
4 9 1
E exchanges the first and second rows of A.
2.9.4
Example
1 0 0
I = 0 1 0
0 0 1
we can form the elementary matrix E by multiplying the first row by -2 and adding to the
third row:
1 0 0
E = 0 1 0
2 0 1
Now if
2 3 5
A = 7 1 0
4 9 1
then EA is
2 3 5
EA = 7 1 0
0 3 9
E multiplies the first row of A and adds it to the third row of A, replacing the third row.
2.10
Exercises
2.10.1
Exercise
Solve the system of linear equations or show that it does not have a solution.
2.10. EXERCISES
25
2x1 + x2 = 7
x 1 x2 = 5
2.10.2
Exercise
2x1 5x2 + 4x3 = 8
2x1 + 2x3 = 4
x1 2x2 + x3 = 2
2.10.3
Exercise
2x1 5x2 + 4x3 = 8
2x1 + 2x3 = 4
2.10.4
Exercise
If

2 1 0
A =
0 1 5
then 3A = ?
2.10.5
Exercise
If

2
A =
0
0
B =
0
then AB =

1 0
1 5
1 1
1 1
0 1
26
2.10.6
CHAPTER 2. MATRICES
Exercise
If
1 1 1
A = 0 1 1
0 0 1
then tr A =
2.10.7
Exercise
Prove the properties of matrices given in this chapter.
2.10.8
Exercise
What does an elementary matrix do when it multiplies (from the left) a matrix A? Can you
prove this?
2.10.9
Exercise
x+y+z = 4
xy+z = 2
2x z = 0
Write the augmented matrix for this problem. What elementary matrices would be used to
reduce the augmented matrix to reduced row echelon form?
Chapter 3
Inverse of a matrix
3.1
3.1.1
Inverse of a matrix
Definition
If Ann is a square matrix, then A1

nn is that matrix (if such a matrix exists) so that
AA1 = A1 A = Inn
3.1.2
Example
Consider the matrix

A =
3 1
1 1
Then A has an inverse and

A
1
=
2

1 1
1 3
We can confirm that

AA
3.1.3

=
1 0
0 1
= A1 A
Invertible
A matrix A is said to be invertible if it has an inverse A1 .
3.2
3.2.1
Inverse of a two by two matrix

Inverses dont always exist
As an example of a matrix with no inverse, consider the zero matrix

27
28
CHAPTER 3. INVERSE OF A MATRIX
0 0
0 0
No matter what you multiply this matrix by, you would always get the zero matrix back and
could never get the identity matrix.
3.2.2
Inverse of a two by two matrix
Suppose that

a b
A =
c d
is such that ad bc 6= 0. Then
1
3.2.3
1
=
ad bc
d b
c a
Proof
The result can be proven by direct calculation:
AA
=
=
=
=

1
a b
d b
c d ad bc c a

1
a b
d b
c a
ad bc c d

1
ad bc
0
0
ad bc
ad bc

1 0
0 1
and

1
d b
a b
A A =
c d
ad bc c a

1
ad bc
0
=
0
ad bc
ad bc

1 0
=
0 1
1
3.3. INVERSE OF THE INVERSE
3.2.4
29
Example
The inverse of

A =
1 4
8 5
is
1
3.3
3.3.1
1
=
27

5 4
8 1
Inverse of the inverse

The inverse of the inverse is the original matrix
If A is an invertible matrix then

(A1 )1 = A
3.3.2
Proof
Follows directly from the definition of the inverse.
3.4
3.4.1
Inverse of a product of invertible matrices

Inverse of a product of invertible matrices
Suppose that A and B are both invertible. Then

(AB)1 = B 1 A1
3.4.2
Proof
(AB)(B 1 A1 ) = A(BB 1 )A1 = AIA1 = AA1 = I
(B 1 A1 )(AB) = B 1 (A1 A)B = B 1 IB = B 1 B = I
3.4.3
Extension
If A1 , A2 , ... , An are invertible matrices then what is the inverse of A1 A2 An ?
30
3.5
3.5.1
Powers of matrices
Definition
If A is a square matrix (invertible or otherwise) then

A0 I
A1 = A
A2 = AA
A}
An = AA
| {z
n times
3.5.2
Inverse of a power of a matrix
Suppose that A is invertible. Then

(An )1 = (A1 )n
3.5.3
Proof
By induction.
3.5.4
Notation
For an invertible matrix A and n a positive integer

An (A1 )n
3.5.5
Exponent rules
If A is an invertible square matrix and r, s Z

Ar As = Ar+s
(Ar )s = Ars
3.5.6
Proof
By induction.
3.6. INVERSE OF A TRANSPOSE
3.5.7
31
Polynomials of matrices
If A is a square matrix and p is a polynomial function defined by

p(x) = a0 + a1 x + a2 x2 + + an xn
then
p(A) = a0 I + a1 A + a2 A2 + + an An
3.5.8
Example
If

A =
0 1
1 0
and
f (x) = x3 + x2 + x
then
f (A) = I22
3.6
3.6.1
Inverse of a transpose
Inverse of a transpose of an invertible matrix
Suppose that A is an invertible matrix. Then At is also invertible and

(At )1 = (A1 )t
3.6.2
Proof
(A1 )t At = (AA1 )t = I t = I
At (A1 )t = (A1 A)t = I t = I
3.7
3.7.1
Inverses of elementary matrices

Inverse of scaling elementary matrix
One elementary row operation is to multiply the ith row by a real number c 6= 0. The elementary matrix for this operation is found by multiply the ith row of the identity matrix by
c. The inverse of this elementary matrix is found by multiplying the ith row of the identity
32
matrix by 1/c.
This can be confirmed by directly computing the product of the two matrices.
3.7.2
Inverse of an interchanging rows elementary matrix
One elementary row operation is to multiply the ith row and the jth row of a matrix. The
elementary matrix for this operation is found by interchanging the ith row and jth row of
the identity matrix. This matrix is its own inverse.
This can be confirmed by directly computing the product of the matrix with itself.
3.7.3
Inverse of a add multiple of a row to a row elementary

matrix
One elementary row operation is to multiply the ith row of a matrix by c 6= 0 and add it
to the jth row of a matrix. The elementary matrix for this operation is found by doing the
same operation to the identity matrix.
The inverse of this matrix is found by multiplying the ith row of the identity matrix by
c 6= 0 and adding it to the jth row of the identity matrix.
This can be confirmed by directly computing the product of the matrix with itself.
3.7.4
All elementary matrices are invertible
We see that all elementary matrices are invertible and the inverses are also elementary
matrices.
3.8
3.8.1
Inverses and solutions of systems

Example
x1 + x2 = 3
x1 x2 = 1
If we let

A =
Then we can write

1 1
1 1
3.9. FUNDAMENTAL THEOREM OF LINEAR ALGEBRA
33

x1
3
A
=
x2
1
The inverse of A is
1
1
=
2

1 1
1/2 1/2
=
1 1
1/2 1/2
Then we can multiply from the left on both sides to get

3
x1
1
1
A A
= A
1
x2
And so

x1
1/2 1/2
3
1
=
=
x2
1/2 1/2
1
2
3.8.2
Generalizing
If Ann is an invertible square matrix and xn1 is a column matrix of unknowns x1 , x2 , ..., xn
and bn1 is a column of real numbers then the equation
Ax = b
has the unique solution
x = A1 b
3.8.3
Proof
Multiply by A1 on both sides.
3.9
3.9.1
Fundamental theorem of linear algebra

Name
Its not actually called that. But its important enough that it should be. We will see at
different times in this course quite a few statements that are all equivalent and are extremely
important in linear algebra.
3.9.2
FTLA
The following are equivalent:

1) Ann is invertible.
34
2) The equation Ax = 0 has only the trivial solution. ( All entries of x are zero ).
3) The reduced row echelon form of A is the identity matrix I.
4) A is a product of elementary matrices.
3.9.3
Example
We know that the equation

1 2
x1
0
=
0 5
x2
0
has only the trivial solution

x1
0
=
x2
0
because the matrix on the left of the equation is invertible.
3.10
Calculating the inverse by hand
3.10.1
Method
Suppose Ann is a square matrix and we want to find the inverse. We set up a matrix
(A|Inn )
and then we perform elementary operations until we get
I|A1
If we cant get I on the left it means that the matrix A was not invertible.
3.10.2
Example
If we want to invert
1 1 1
A = 0 1 1
0 0 1
we start with the matrix
1 1 1 | 1 0 0
0 1 1 | 0 1 0
0 0 1 | 0 0 1
and perform elementary row operations until we get
3.11. PROOF OF THE FTLA
35
1 0 0 | 1 1 0
0 1 0 | 0 1 1
0 0 1 | 0 0
1
and so the inverse is
A1
1 1 0
= 0 1 1
0 0
1
3.11
Proof of the FTLA
3.11.1
Theorem

a) Ann is invertible
b) The equation Ax = 0 has only the trivial solution x = 0.
c) The reduced row echelon form of A is the I.
d) A is a product of elementary matrices.
3.11.2
Proof a) implies b)
If A is invertible then
Ax
A (Ax)
(A1 A)x
Ix
x
1
3.11.3
=
=
=
=
=
0
A1 0
0
0
0
Proof b) implies c)
Suppose that the equation Ax = 0 has only the solution x = 0. Then the augmented matrix
in rref looks like
36
1
0
0
1
0
0
0
1
..
.
0 | 0
0 | 0
0 | 0
1 | 0
and so the rref of A is I.
3.11.4
Proof c) implies d)
If the rref of A is I then A can be transformed to I by elementary row operations. Since the
elementary row operations can be done by elementary matrices, we have
En En1 E2 E1 A = I
for some elementary matrices E1 , E2 , ... , En . Then inverting gives
A = E11 E21 En1
3.11.5
Proof d) implies a)
If we can show that d) implies a) then we have completed the proof.

If A is a product of elementary matrices then A is a product of invertible matrices since
all elementary matrices are invertible. The product of invertible matrices is invertible so A
is invertible.
3.12
Even more FTLA
3.12.1
Theorem

c) The reduced row echelon form of A is the I.
NEW
e) Ax = b is consistent for all n 1 matrices b
3.12. EVEN MORE FTLA
37
f) Ax = b has exactly one solution for all n 1 matrices b
3.12.2
Proof so far
We have a) implies b) implies c) implies d) implies a).
3.12.3
e) implies a)
Consider
e1

1
0

= 0
..
.
0
There is some column matrix v 1 so that

Av 1 = e1
Consider
e2

0
1

= 0
..
.
0
There is some column matrix v 2 so that

Av 2 = e2
If ej is the column vector of n rows which has jth entry 1 and otherwise zero, then there are
column matrices v j so that
Av j = ej
for j = 1, 2, ..., n. If we write collect the v j as columns of a square matrix and the ej as the
columns of the identity matrix then we can rewrite the result as
A v1 v2 vn
A v1 v2 vn
and so
e1 e2 en
= I
38
A1 =
3.12.4
v1 v2 vn
Proof a) implies f )
Exercise for the student.
3.13
Exercises
3.13.1
Exercise
Consider the matrix
1 1 1
A = 1 0 1
0 1 1
Find the inverse of this matrix.
3.13.2
Exercise
x1 + x2 + x3 = 3
x1 + x3 = 4
x2 + x3 = 5
What is the solution of this system?
3.13.3
Exercise
We proved that for invertible matrices A1 and A2 of the same size that
1
(A1 A2 )1 = A1
2 A1
Given this fact show that

1 1
(A1 A2 A3 )1 = A1
3 A2 A1
Assume that for some positive integer k and any k invertible matrices of the same size that
1
1 1
(A1 A2 A3 Ak1 Ak )1 = A1
k Ak1 A2 A1
Prove that for another invertible matrix Ak+1 of the same size that
3.13. EXERCISES
39
1 1
1 1
(A1 A2 A3 Ak1 Ak Ak+1 )1 = A1
k+1 Ak Ak1 A2 A1
What can be concluded by mathematical induction?
3.13.4
Exercise
Consider the matrix
0 1 1
A = 0 0 1
0 0 0
Show that A3 = 0.
3.13.5
Exercise
Suppose that a square matrix A is such that A4 = 0. Prove that A does not have an inverse.
Hint: Assume that it does have an inverse and that this assumption leads to a contradiction.
3.13.6
Exercise
Consider the matrix
1 1 1
A = 1 0 1
0 1 1
Write A as the product of elementary matrices. What does the FTLA tell you?
3.13.7
Exercise
A square matrix is said to be diagonal if all the entries of the matrix not on the main diagonal
are zero. Then entries on the main diagonal may or may not be zero.
Show that if a diagonal matrix has no entries on the main diagonal that are zero then
the diagonal matrix is a product of elementary matrices. Find the inverse of such a matrix.
3.13.8
Exercise
What is the formula for a positive integer power of a diagonal matrix?
40
3.13.9
Exercise
Suppose that A is a square matrix. Consider the two equations

Ax = b,
Ax = c
Is it possible for the first equation to have exactly one solution and for the second equation
to have more than one solution?
3.13.10
Exercise
Suppose that there is a polynomial function p such that

p(x) = a0 + a1 x + a2 x2
You are given that
p(0) = 1,
p(1) = 3,
p(1) = 1
What is the polynomial function p?
3.13.11
Exercise
x1 + x2 = 0
cx1 + x2 = 0
where c is some real number. This system obviously has the trivial solution. For what values
of c does this system have only the trivial solution?
3.13.12
Exercise
Is this matrix A invertible?
4 0 0
0 3 0
0 0 0
3.13.13
Exercise
Go through the proof of the FTLA carefully.
Chapter 4
Some types of matrices
4.1
4.1.1
Diagonal matrices
Examples
The following square matrices are examples of diagonal matrices
1 0 0
I = 0 1 0
0 0 1
1 0 0
A = 0 5 0
0 0 7
0 0 0
B = 0 1 0
0 0 0
4.1.2
Definition
A square matrix which has the property that all entries not on the main diagonal are zero
is said to be diagonal. The entries on the main diagonal can be zero or not zero.
d1 0 0
0 d2 0
D = 0 0 d3
0 0
0 0
41
0
0
0
dn
0
0
0
...
42
4.2
4.2.1
CHAPTER 4. SOME TYPES OF MATRICES
Diagonal and elementary matrices

Elementary matrices that scale rows
Lets use 3x3 examples:
d1 0 0
E1 (d1 ) = 0 1 0
0 0 1
This elementary matrix performs the elementary row operation of multiplying the first row
of a matrix by d1 .
The inverse if d1 6= 0 is
1/d1 0 0
1 0
E1 (1/d1) = 0
0
0 1
This elementary matrix performs the elementary row operation of multiplying the first row
of a matrix by 1/d1 .
Now consider
1 0 0
E2 (d2 ) = 0 d2 0
0 0 1
This elementary matrix performs the elementary row operation of multiplying the second
row of a matrix by d2 .
1
0
0
E2 (1/d2 ) = 0 1/d2 0
0
0
1
This elementary matrix performs the elementary row operation of multiplying the second
row of a matrix by 1/d2 .
Now consider
1 0 0
E3 (d3 ) = 0 1 0
0 0 d3
This elementary matrix performs the elementary row operation of multiplying the third row
of a matrix by d3 .
4.3. INVERSES OF DIAGONAL MATRICES
43
1 0
0
0
E3 (1/d3 ) = 0 1
0 0 1/d3
This elementary matrix performs the elementary row operation of multiplying the third row
of a matrix by 1/d3 .
4.2.2
3x3 diagonal matrix
Then we can get
d1 0 0
D(d1 , d2 , d3 ) = 0 d2 0 = E1 (d1 )E2 (d2 )E3 (d3 )
0 0 d3
4.2.3
General result
Exercise.
4.3
4.3.1
Inverses of diagonal matrices

Inverses
If none of the di are zero then
D1
4.3.2
1/d1
0
0
0
0
1/d2
0
0
0
0
1/d
0
3
=
...
0
0
0
0

What if one of the di is zero?
Then the matrix is not invertible.
4.3.3
Example
If
1 0 0
A = 0 5 0
0 0 7
then
0
0
0
0
1/dn
44
A1
4.3.4
1 0
0
= 0 1/5 0
0 0 1/7
Product of elementary matrices
If a diagonal matrix has no zero entries on the main diagonal then it is the product of
elementary matrices.
4.3.5
Proof
Exercise. Hint: Consider performing a set of row operations on the identity matrix that will
eventually give a diagonal matrix with no zeros on the main diagonal.
4.3.6
Inverses
A diagonal matrix with no zeros on the main diagonal is invertible.
4.3.7
Proof
Previous result plus the FTLA.
4.4
4.4.1
Powers of diagonal matrices

Recall
When the diagonal matrix D(d1 , d2 , ..., dn ) multiplies another matrix A from the left it multiplies the ith row of A by di .
4.4.2
Repeated multiplication by a diagonal matrix
If we apply D twice the ith row of A will be multiplied by di twice or d2i . If we apply D to
A for a total of k times then the ith row of A will be multiplied by dki . Then
(D(d1 , d2 , . . . , dn ))k = D(dk1 , dk2 , . . . , dkn )
4.5
4.5.1
Multiplying from the right

Question
We said that if D is a diagonal matrix which multiplies A from the left, then it scales the
rows of A. What if instead of DA we had AB?
4.6. IMPORTANCE OF DIAGONAL MATRICES
4.5.2
45
Example
Consider
1 0 0
D = 0 2 0
0 0 3
and
1 0 1
A = 1 1 0
1 0 1
From what we have seen before, we can write
1 0 1
DA = 2 2 0
3 0 3
without even doing full matrix multiplication. We know that the rows of A will be scaled.
Now consider that
1 0 1
1 0 0
1 0 3
AD = 1 1 0 0 2 0 = 1 2 0
1 0 1
0 0 3
1 0 3
4.5.3
Exercise
Show that multiplication by a diagonal matrix from the right scales columns.
4.6
Importance of diagonal matrices
It is very easy to find the inverses and powers of a diagonal matrix. We will see later in the
course that this simplifies many calculations.
4.7
4.7.1
Symmetric matrices
Definition
A square matrix A is said to be symmetric if At = A.
4.7.2
Alternatively
If its entries aij are such that

aij = aji
46
4.7.3
Example
The identity matrix is symmetric.
4.7.4
Example
1 2 3
A = 2 0 4
3 4 7
is a symmetric matrix.
4.7.5
Transpose of a symmetric matrix
If A is symmetric then At is symmetric.
4.7.6
Proof
If A is symmetric then At = A. So
(At )t = At = A = At
4.7.7
Sum of symmetric matrices
If A and B are symmetric matrices then their sum A + B is symmetric.
4.7.8
Proof
A is symmetric so At = A. B is symmetric so B t = B. Then (A + B)t = At + B t by the

rules for transpose. Then (A + B)t = A + B.
4.7.9
Sum of symmetric matrices
If A and B are symmetric matrices then their difference A B is symmetric.
4.7.10
Proof
Exercise.
4.7.11
Scalar multiple of a symmetric matrix
If A is a symmetric matrix and k is a real number then the matrix kA is symmetric
4.7.12
Proof
If the elements aij of A have the property that aji = aij then the elements bij = kaij have
the property that bji = kaji = kaij = bij . Thus kA is symmetric.
4.8. TRIANGULAR MATRICES
4.7.13
47
Inverse of a symmetric matrix
If A is a symmetric matrix and is invertible then its inverse is also symmetric.
4.7.14
Proof
Suppose that A is symmetric. That is, At = A. Suppose that A is invertible. Then

(A1 )t At = (AA1 )t = I t = I
and
At (A1 )t = (A1 A)t = I t = I
But as A = At
(A1 )t A = I
and
A(A1 )t = I
And so
(A1 )t = A1
4.7.15
Product of a matrix and its transpose
Suppose Amn is a general matrix and Atnm . Then their product At A is symmetric.
4.7.16
Proof
(At A)t = At (At )t = At A
4.8
4.8.1
Triangular matrices
Upper triangular matrices
A square matrix is upper triangular if all the entries below the main diagonal are zero. Entries on the main diagonal may be zero or not.
That is, A is upper triangular if its entries aij are such that aij = 0 whenever i > j.
48
4.8.2
Example
0 1 2
A = 0 5 1
0 0 12
is an example of an upper triangular matrix.
4.8.3
Lower triangular matrices
A square matrix is lower triangular if all the entries above the main diagonal are zero. Entries on the main diagonal may be zero or not.
That is, A is lower triangular if its entries aij are such that aij = 0 whenever i < j.
4.8.4
Example
1 0 0
A = 2 3 0
4 5 6
is an example of a lower triangular matrix.
4.8.5
Identity matrix
The identity matrix is both upper triangular and lower triangular. In fact, diagonal matrices
are both lower triangular and upper triangular.
4.9
Properties of triangular matrices
4.9.1
Transpose of an upper triangular matrix
The transpose of an upper triangular matrix is lower triangular.
4.9.2
Proof
A is upper triangular if its entries aij are such that aij = 0 whenever i > j. Let the entries
of At be called bij . Then bij = aji and are such that bij is zero whenever j > i. Thus At is
lower triangular.
4.9.3
Transpose of an upper triangular matrix
The transpose of a lower triangular matrix is upper triangular.
4.9.4
Exercise.
Proof
4.9. PROPERTIES OF TRIANGULAR MATRICES
4.9.5
49
The sum of upper triangular matrices
If A and B are upper triangular matrices of the same size then their sum is upper triangular.
4.9.6
Proof
A is upper triangular if its entries aij are such that aij = 0 whenever i > j. B is upper
triangular if its entries bij are such that bij = 0 whenever i > j. The sum C = A + B has
entries cij = aij + bij . If i > j then both aij and bij are zero and so cij = 0 + 0 = 0 if i > j.
Then C is upper triangular.
4.9.7
The sum of lower triangular matrices
If A and B are lower triangular matrices of the same size then their sum is lower triangular.
4.9.8
Proof
Exercise.
4.9.9
The difference of upper triangular matrices
If A and B are upper triangular matrices of the same size then their difference A B is
upper triangular.
4.9.10
Proof
Exercise.
4.9.11
The difference of lower triangular matrices
If A and B are lower triangular matrices of the same size then their difference A B is lower
triangular.
4.9.12
Proof
Exercise.
4.9.13
Invertibility of upper triangular matrices
This depends on the the main diagonal.

If an upper triangular matrix has no zeros on the main diagonal then it is invertible.
50
4.9.14
Proof
We can use elementary matrices to scale the pivots of each row to 1. We can then use
elementary matrices to produce zeros above the pivots. Thus the rref of the matrix is the
identity matrix and then the matrix is invertible by the FTLA.
4.9.15
Invertibility of lower triangular matrices
A lower triangular matrix is invertible if and only if it has no zeros on the main diagonal.
4.9.16
Proof
Exercise.
4.9.17
Example
The matrix
1 1 1
A = 0 2 2
0 0 3
is invertible and the matrix
1
0
B =
0
0
1
0
0
0
1
2
3
0
4
5
7
1
is not invertible.
4.9.18
Form of inverses of triangular matrices
The inverse of an invertible upper triangular matrix is upper triangular and the inverse of
an invertible lower triangular is lower triangular.
4.9.19
The upper triangular case
Suppose that Unn is an upper triangular matrix that is invertible. We consider how to find
the inverse using
(U |I)
The matrix I is upper triangular. Every operation that we will perform will not change the
zeros below the main diagonal and so the resulting matrix will be upper triangular.
4.10.
LU DECOMPOSITION
4.9.20
51
The lower triangular case
Exercise.
4.10
LU decomposition
4.10.1
Idea
Suppose that A is an invertible matrix and we want to solve

Ax = b
We decompose A as the product of a lower triangular matrix L and an upper triangular
matrix U .
A = LU
Now we want to solve
LU x = b
If we write
y = Ux
then we can solve
Ly = b
for y and then solve
Ux = y
for x.
4.10.2
Benefits
In some applications, A will be fixed while b keeps changing. So one factors A only once
which is equivalent to Gaussian elimination.
4.10.3
Example
Suppose that we have the system

2 1 3
x1
1
4 5 0 x2 = 2
4
2 18
x3
0
One LU decomposition could be of the form
52
l21
L =
l31
u11
0
U =
0
0 0
1 0
l32 1
u12 u13
u22 u23
0 u33
Note that when we multiply L and U the first row of the result will be the first row of U , so
the first row of U should be the first row of A
1 0 0
L = l21 1 0
l31 l32 1
2 1 3
U = 0 u22 u23
0 0 u33
Consider that l21 2 = a21 = 4. Then l21 = 2. So we have
1 0 0
L = 2 1 0
l31 l32 1
2 1 3
U = 0 u22 u23
0 0 u33
Also (2)(1) + (1)(u22 ) = 5. So u22 = 3.
1 0 0
L = 2 1 0
l31 l32 1
2 1 3
U = 0 3 u23
0 0 u33
Also l31 (2) = 4 so l31 = 2.
1 0 0
L = 2 1 0
2 l32 1
2 1 3
U = 0 3 u23
0 0 u33
Also (2)(1) + l32 (3) = 2 so l32 = 4/3.
4.11. EXERCISES
53
1
0 0
L = 2 1 0
2 4/3 1
2 1 3
U = 0 3 u23
0 0 u33
The (2)(3) + (1)u23 = 0 so u23 = 6.
1
0 0
L = 2 1 0
2 4/3 1
2 1 3
6
U = 0 3
0 0 u33
Finally (2)(3) + (4/3)(6) + u33 = 18 so u33 = 4.
1
0 0
L = 2 1 0
2 4/3 1
2 1 3
6
U = 0 3
0 0 u33
It is left to the reader to solve the system.
4.10.4
Pivoting
A pivoting matrix P that exchanges rows is often used with a decomposition given by
A = P LU
We dont want to go too deeply into numerical linear algebra at this point and leave further
discussion to later courses.
4.11
Exercises
4.11.1
Exercise
Find the inverse of the diagonal matrix or show
1
D = 0
0
that the inverse does not exist.
0 0
2 0
0 4
54
1 0 0
D = 0 0 0
0 0 4
4.11.2
Exercise
The exponential function exp : R R is defined by

exp(x) 1 + x +
xk
x2 x3
+
+ + + +
2!
3!
k!
The exponential function for square matrices A is defined by

exp(A) I + A +
A2 A3
Ak
+
+ + +
+
2!
3!
k!
If
1 0 0
D = 0 2 0
0 0 4
then what is exp(D)?
4.11.3
Exercise
A square A matrix is said to be skew-symmetric if At = A. Prove that a skew-symmetric

matrix has all zeros on its main diagonal.
4.11.4
Exercise
Show that a square matrix A can always be written as the sum of a symmetry matrix and
a skew-symmetric matrix.
4.11.5
Exercise
Consider the lower triangular matrix
1 0 0
L = 2 1 0
2 2 1
Find the inverse of this matrix.
4.11. EXERCISES
4.11.6
55
Exercise
Consider the lower triangular matrix
1 0 0
L = 2 1 0
2 2 1
and the matrix
1 2 3
A = 2 6 7
2 8 11
Find an upper triangular matrix U so that A = LU .
56
Chapter 5
Determinants
5.1
Idea
The determinant is a measure of what a linear transformation represented by a square matrix does to a unit area. A determinant of two would mean that the area of the output of
the transformation is twice the area of the input. The sign represents a change in orientation.
In this course, we will be interested in determinants for what they can tell us about matrices.
5.2
For 2x2 systems
5.2.1
Definition
Given
A22

a b
=
c d
we define
|A| ad bc
or
det(A) ad bc
5.2.2
Example
If

A =
we define
57
1 2
3 4
58
CHAPTER 5. DETERMINANTS

1 2

= 1423 = 2
|A| =
3 4
5.3
For general square matrices
What if the matrix is bigger than a 2x2?
5.3.1
Minors
If the square matrix Ann has entries aij let mij be the determinant of the matrix given by
crossing out the row and column of aij .
5.3.2
Example
Suppose that we have a matrix
1 2 3
A = 4 5 6
7 8 9
Then
m11
m23
5.3.3

5
=
8

1
=
7

6
= 5968 = 3
9

2
= 1827 = 6
8
Cofactors
If A is a square matrix with entries aij and the associated minors are mij then the cofactor
cij is given by
cij = (1)i+j mij
5.3.4
Example
We calculated for
1 2 3
A = 4 5 6
7 8 9
the minors
m11 = 3
5.3. FOR GENERAL SQUARE MATRICES
59
m23 = 6
The associated cofactors are
c11 = (1)1+1 m11 = (1)(3) = 3
c23 = (1)2+3 m23 = (1)(6) = 6
5.3.5
Determinants
For a square matrix Ann we define

|A| = a11 c11 + a12 c12 + + a1n c1n
5.3.6
Example
For
1 2 3
A = 4 5 6
7 8 9

4 6
4 5
5 6

+ (2)
+ (3)
|A| = (1)
7 9
8 9
7 8
5.3.7
General definition
You can follow whatever row or column you like, just make sure to get the alternating signs
right. For a square matrix Ann expanding along the ith row, we have
n
X
|A| =
(1)i+j aij |Aij |
j=1
where Aij is the matrix resulting from crossing out the ith row and j column of A.
For a square matrix Ann expanding along the jth column, we have
n
X
|A| =
(1)i+j aij |Aij |
i=1
60
5.3.8
Example
For
1 0 3
A = 4 0 6
7 0 9
we can use
|A| = a12 c12 + a22 c22 + a32 c32 = 0 + 0 + 0 = 0
5.3.9
Example
What is the determinant of
1
0
A =
0
0
2
5
0
0
3
6
1
0
4
7
2
6
The result is immediate if you make the right choice for the rows and columns to use.
5.4
5.4.1
Some properties of determinants

Determinants of matrices with rows/cols of zeros
The determinant of a matrix with a row of all zeros is zero. The determinant of a matrix
with a column of all zeros is zero.
5.4.2
Examples
Consider a 2x2 matrix

a b
0 0
The determinant is clearly zero.
Consider a 3x3 matrix
a b c
A = 0 0 0
d e f
If we do the calculation using the second row we get
|A| = 0| | + 0| | + 0| | = 0
5.4. SOME PROPERTIES OF DETERMINANTS
61
Similarly, if we have a row of zeros in any n n matrix A we would get

|A| = 0| | + 0| | + 0| | + + 0| | = 0
The same reasoning holds for a column of zeros.
5.4.3
Multiplying a single row by a real
Suppose that Ann is a square matrix and B is the matrix given by multiplying each element
of a row by k. Then
|B| = k|A|
5.4.4
Example

1 2 0

2 4 0 = 0

3 5 0
and so

5 10 0

2 4 0 = 5 0 = 0

3 5 0
5.4.5
Proof
Suppose that Ann has elements aij . Let Aij be the (n 1) (n 1) matrix formed by
crossing out the ith row and jthe column of A.
Suppose that we multiply row s of A by a real number k to get B which has elements
bij . Let Bij be the (n 1) (n 1) matrix formed by crossing out the ith row and jthe
column of B.
Note that
bsj = kasj
from the definition of B and
Bsj = Asj
since A and B only differ in row s which is being crossed out in Asj and Bsj .
Then calculating the determinant of B by expanding along the sth row of B:
62
|B| = (1)s+1 bs1 |Bs1 | + (1)s+2 bs2 |Bs2 | + + (1)s+n bsn |Bsn |
= (1)s+1 kas1 |As1 | + (1)s+2 kas2 |As2 | + + (1)s+n kasn |Asn |

= k (1)s+1 as1 |As1 | + (1)s+2 as2 |As2 | + + (1)s+n asn |Asn |
= k|A|
5.4.6
Multiplying all elements by a real
Suppose that Ann is a square matrix and B is the matrix given by multiplying each element
of A by k. Then
|B| = |kA| = k n |A|
5.4.7
Proof
Apply the proof for a single row n times.
5.4.8
Result for multiplying a column by a real
Suppose that Ann is a square matrix and B is the matrix given by multiplying a column of
A by k. Then
|B| = |kA| = k|A|
5.4.9
Proof
Exercise
5.4.10
Determinant after interchanging two adjacent rows
Suppose that Ann is a square matrix and the matrix Bnn is obtained from A by swapping
two ADJACENT rows of A. Then
|B| = |A|
5.4.11
Example

1 2 3

0 4 5 = 24

0 0 6
so
63

1 2 3

0 0 6 = 24

0 4 5
5.4.12
Proof
Let A be the original matrix with entries ai,j . Let Ai,j be the matrix obtained by crossing
out the ith row of A and the jth column of A.
Let B be the matrix obtained by interchanging two adjacent rows of A with entries bi,j .
Let Bi,j be the matrix obtained by crossing out the ith row of B and the Bth column of B.
Lets suppose that B is obtained from A by interchanging row i and i + 1.
Note that
bi+1,k = ai,k
and that
Bi+1,k = Ai,k
from the definition of B.
We calculate the determinant of B by expansion along row i + 1
|B| = (1)i+1+1 bi+1,1 |Bi+1,1 | + (1)i+1+2 bi+1,2 |Bi+1,2 | + + (1)i+1+n bi+1,n |Bi+1,n |
= (1)i+1+1 ai,1 |Ai,1 | + (1)i+1+2 ai,2 |Ai,2 | + + (1)i+1+n ai,n |Ai,n |

= (1) (1)i+1 ai,1 |Ai,1 | + (1)i+2 ai,2 |Ai,2 | + + (1)i+n ai,n |Ai,n |
= (1)|A|
5.4.13
Determinant after swapping ANY two rows
Suppose that Ann is a square matrix and Bnn is the matrix obtained from A by interchanging any two rows of A. Then
|B| = |A|
5.4.14
Example

1 2 3

0 4 5 = 24

0 0 6
64
so

0 0 6

0 4 5 = 24

1 2 3
5.4.15
Proof
Suppose that row r and row s are the two rows to be interchanged with 1 r < s n.
We think of interchanging row r with the row below it repeatedly, including with row s and
then leaving it as the sth row of the new matrix. There were s r interchanges.
Now that row s is one row above row r do repeated interchanges with the row above it
until there row s is now the rth row of the new matrix. There were s r 1 interchanges.
Then by the result for interchanges of adjacent rows
|B| = (1)sr+sr1 |A| = (1)2(sr)1 |A| = |A|
5.4.16
Determinant after interchanging two columns
Suppose that Ann is a square matrix and Bnn is the matrix obtained from A by interchanging any two columns of A. Then
|B| = |A|
5.4.17
Proof
Exercise.
5.4.18
Determinant of a square matrix with two identical rows
The determinant of a square matrix with two identical rows is zero.
5.4.19
Example

1

1

4

1
1
2
3
1
1
3
2
1

1
4
= 0
1
1
5.4.20
65
Proof
Suppose that A is a square matrix with two rows that are the same. From the result on
row swaps, we can swap two rows and the determinant changes by a minus sign. Then if we
swap the two identical rows
|A| = |A| |A| = 0
5.4.21
Determinant of a square matrix with two identical columns
The determinant of a square matrix with two identical columns is zero.
5.4.22
Proof
Exercise.
5.4.23
Determinants and the third row operation
Let Ann be a square matrix. Let k be a real number. Let B be the result of adding k times
row r of A to another row s of A and replacing the original row s of A. Then
|B| = |A|
5.4.24
Example

1 1 1
1 1 1
1 1 1
1 1 1

2 1 3 = 0 1 1 = 0 1 1 = 0 1 1 = 1

3 2 5
3 2 5
0 1 2
0 0 1
5.4.25
Proof
Let Ann be a square matrix with entries aij . Let Aij be the matrix formed by crossing out
the ith row and jth column of A.
Let k be a real number.
Let B be the result of adding k times row r of A to row s of A and replacing the original row s of A. Let the entries of B be bij . Let Bij be the matrix formed by crossing out
the ith row and jth column of B.
Let C be the result of replacing row s of A by row r of A. Let the entries of C be cij .
Let Cij be the matrix formed by crossing out the ith row and jth column of B.
Note that
Asj = Bsj = Csj
66
and
arj = csj
and as C has two identical rows
|C| = 0
We calculate the determinant of B by expanding along row s:
|B| =
=
n
X
j=1
n
X
(1)s+j bsj |Bsj |

(1)s+j (asj + karj )|Bsj |
j=1
n
X
= k
(1)s+j arj |Bsj |
j=1
n
X
(1)s+j asj |Bsj |
= k
j=1
n
X
(1)s+j csj |Csj |
j=1
n
X
(1)s+j asj |Asj |
j=1
= k|C| + |A|
= |A|
5.5
5.5.1
Determinants of matrices and their transposes

Example
Consider the 2x2 case

a
A =
c

a
t
A =
b

b
d

c
d
The determinants are

|A| = ad bc
5.5. DETERMINANTS OF MATRICES AND THEIR TRANSPOSES
67
|At | = ad cb
So in the 2x2 case the determinant of a matrix A and its transpose At are the same.
5.5.2
Example
Consider the 3x3 case
a11
a21
A =
a31
a11
t
a12
A =
a13
a12 a13
a22 a23
a32 a33
a21 a31
a22 a32
a23 a33
We can find the determinant of A by going along its first row to get

a21 a22
a21 a23
a22 a23

+ a13
a12
|A| = a11
a31 a32
a31 a33
a32 a33
Now to get the determinant of the transpose we can expand along the first column to get
a11 a21 a31

At = a12 a22 a32
a13 a23 a33

a22 a32
a21 a31
a21 a31
t

|A | = a11
a12
+ a13
a23 a33
a23 a33
a22 a32
and we see that the result is exactly the same. Note that the determinants in the expansion
are the determinants of transposes of 2x2 matrices, for which we have already established
the result.
5.5.3
Theorem
If Ann is a square matrix then

|At | = |A|
5.5.4
Proof
Will do by induction.
Let Pn be the proposition to be proven. We already have P2 . Assume Pk , that for 2,...,k
|At22 | = |A|
68
|At33 | = |A|
..
.
|Atkk | = |A|
Now let A be a square (k + 1) (k + 1) matrix. Then
|A| =
k+1
X
(1)i+j aij |Aij |
j=1
where Aij is the k k matrix that results from crossing out the ith row and jth col of A.
But by the induction hypothesis, |Atij | = |Aij | so
|A| =
k+1
X
(1)i+j aij |Atij |
j=1
Note that the terms of the transpose matrix are given by

a0ji = aij
and so
|A| =
k+1
X
(1)i+j a0ji |Atji |
j=1
The RHS is the determinant of At expanding along the ith column so

|A| = |At |
As P2 and Pk implies Pk+1 then Pn by mathematical induction.
5.6
5.6.1
Determinants, row operations, and elementary matrices

Identity
|I| = 1
5.6.2
Determinant of row swapping elementary matrix
We find the matrix for row swapping by swapping rows of the identity matrix. By a previous
result the determinant of the elementary matrix will be -1.
5.6. DETERMINANTS, ROW OPERATIONS, AND ELEMENTARY MATRICES
5.6.3
69
Determinant of row multiplying elementary matrix
We find the matrix for row multiplying by multiplying a row of the identity matrix. By a
previous result the determinant of the elementary matrix will be the multiple.
5.6.4
Determinant of elementary matrix that adds a multiple of a

row to another row
We find the matrix for this operation by performing this operation on the identity matrix.
The determinant of the new matrix must be the same as the determinant of the identity by
a previous result, and so is 1.
5.6.5
Determinant of a product involving an elementary matrix

and a square matrix
Suppose that E is an elementary matrix and A is a square matrix of the same size. Then
|EA| = |E||A|
5.6.6
Proof
We consider the three types of elementary matrices and show that the result holds for each
type.
Suppose that E is an elementary matrix that interchanges two rows. Then
|EA| = |A|
from a previous result. But |E| = 1 so
|EA| = |A| = |E||A|
Suppose that E multiplies a row by a real number k. Then
|EA| = k|A| = |E||A|
Suppose that E multiplies a row and adds it to another row. Then
|EA| = |A| = 1 |A| = |E||A|
5.6.7
Determinant of an invertible matrix is nonzero
Suppose that a square matrix A is invertible. Then its determinant is non-zero.
70
5.6.8
Proof
By the FTLA A is invertible if and only if it is the product of elementary matrices

A = E1 E2 Es
Then
|A| = |E1 E2 Es | = |E1 ||E2 Es | = |E1 ||E2 | |Es |
The determinants of elementary matrices are not zero by our previous discussion so for A
invertible |A| =
6 0.
5.6.9
Determinant of a non-invertible (singular) matrix is zero
If a square matrix A is not invertible then its determinant is zero.
5.6.10
Proof
We do row reduction on A using elementary matrices to get its rref B. Note that B cannot
be I since A is not invertible and has at least one row of zeros.
B = Ek E2 E1 A
Taking the determinant of both sides and using a previous result
|B| = |Ek E2 E1 A| = |Ek | |E2 ||E1 ||A|
As B has at least one row of zeros by a previous result |B| = 0.
0 = |Ek | |E2 ||E1 ||A|
As elementary matrices have nonzero determinant we must have that |A| = 0.
5.6.11
Product of matrices with singular factor
Suppose that A and B are matrices of the same size and A is not invertible. Then the
product AB has determinant zero.
5.6.12
Proof
We apply a set of elementary matrices to reduce A to its rref

C = Ek Ek1 E2 E1 A
Note that as A is not invertible its rref has at least one row of zeros. Now multiply by B on
the right.
CB = Ek Ek1 E2 E1 AB
5.6. DETERMINANTS, ROW OPERATIONS, AND ELEMENTARY MATRICES
71
Since C has a row of zeros, so does CB. Then the determinant of CB is zero and
|CB| = |Ek Ek1 E2 E1 AB| = |Ek ||Ek1 | |E2 ||E1 ||AB|
0 = |Ek ||Ek1 | |E2 ||E1 ||AB|
As elementary matrices have nonzero determinants then |AB| = 0.
5.6.13
Product of matrices with singular factor
Suppose that A and B are matrices of the same size and A is not invertible. Then the
product AB has determinant zero.
5.6.14
Proof
Exercise.
5.6.15
Determinant of a product of matrices
Suppose that A and B are square matrices of the same size. Then
|AB| = |A||B|
5.6.16
Proof
Suppose that A is invertible. Then A is the product of elementary matrices

A = E1 E2 Ek
and we can write
AB = E1 E2 Ek B
and then
|AB| =
=
=
=
|E1 E2 Ek B|
|E1 ||E2 | |Ek |B|
|E1 E2 Ek |B|
|A||B|
The other possibility is that A is not invertible. Then by a previous result

|AB| = 0 = 0|B| = |A||B|
72
5.6.17
Corollary
If A is invertible then
|A1 | =
5.6.18
1
|A|
Proof
|I| = |AA1 | = |A||A1 |
and so
|A1 | =
5.6.19
|I|
1
=
|A|
|A|
Theorem
A square matrix A is invertible if an only if its determinant is not zero.
5.6.20
Proof
We saw that if A is invertible then its determinant is nonzero. Suppose that the determinant
of A is nonzero. If we apply elementary matrices Ei to get the rref B then
B = Ek Ek1 E2 E1 A
Taking the determinant of both sides
|B| = |Ek ||Ek1 | |E2 ||E1 ||A|
and so the determinant of the rref of A is not zero. If B had a row of zeros then |B| = 0
and so B must be I
I = Ek Ek1 E2 E1 A
and then A is invertible.
5.6.21
Theorem
A square matrix A is singular if and only if its determinant is zero.
5.6.22
Proof
Follows logically from previous theorem.
5.7. FTLA
5.7
FTLA
5.7.1
Before
73

c) The reduced row echelon form of A is I.
e) Ax = b is consistent for alln 1 matrices b.
5.7.2
Now add
f) |A| =
6 0
5.8
5.8.1
Adjoint
Definition - matrix of cofactors
Suppose that A is a square matrix with entries aij and cofactors cij . The matrix C whose
entries are the cofactors cij of A is called the matrix of cofactors of A.
5.8.2
Definition - adjoint
The transpose of the matrix of cofactors of A is called the adjoint of A and is often denoted
adj(A).
5.8.3
Example
For
3 2 1
3
A = 1 6
2 4 0
c11 = 12,
c21 = 4,
c12 = 6,
c22 = 2,
c13 = 16
c23 = 16
74
c32 = 10,
c31 = 12,
c33 = 16
The matrix of cofactors is
12 6 16
2
16
C = 4
12 10 16
and then taking the transpose of the matrix of cofactors gives us adj( A ):
12 4 12
2 10
adj(A) = 6
16 16 16
5.8.4
Adjoint and inverse
If the determinant of A is not zero then

A1 =
5.8.5
1
adj(A)
|A|
Proof
Will do later in class.
5.8.6
Example
For
3 2 1
3
A = 1 6
2 4 0
we found the cofactors and then can calculate the
|A| = 64
The adjoint matrix was
12 4 12
2 10
adj(A) = 6
16 16 16
and so
A1
12 4 12
1
6
2 10
=
64
16 16 16
5.9. EXERCISES
5.9
5.9.1
75
Exercises
Exercise
Evaluate the determinant

1 2

2 1
5.9.2
Exercise
Evaluate the minors of
1 2 1
A = 2 1 2
3 0 5
5.9.3
Exercise
Calculate the determinant of A.
1 2 1
A = 2 1 2
3 0 5
5.9.4
Exercise
0
1
2
3
0
2
1
0
6
0
2
3
7 12
0 0
1 2
0 5
1
0
A =
0
0
5.9.5
0
1
2
5
Exercise

1
0
A =
0
0
76
5.9.6
Exercise
1 6 7 12
2 12 14 24
A =
0 2 1 2
0 3 0 5
5.9.7
Exercise
6 7 12
1 14 24
0 1 2
0 0 5
6 7 12
1 14 24
0 0 5
0 1 2
6 7 12
1 14 24
0 0 5
0 5 10
6 7 12
1 14 24
0 0 5
0 5 10
1
0
A =
0
0
5.9.8
Exercise

1
0
A =
0
0
5.9.9
Exercise

1
0
A =
0
0
5.9.10
Exercise
Calculate the determinant of 2A.

1
0
A =
0
0
5.9. EXERCISES
5.9.11
77
Exercise
Calculate the determinant of 2A.
6 12 12
1 24 24
0 5 5
0 10 10
6 7 12
1 14 24
0 0 5
0 5 10
1
0
A =
0
0
5.9.12
Exercise
Given
1
0
A =
0
0
how many solutions can the equation Ax = 0 have?
5.9.13
Exercise
Suppose that for a given square matrix A the equation Ax = 0 has a single unique solution.
What can you say about the determinant of A?
5.9.14
Exercise
Suppose that for a given square matrix A there is a column matrix b so that the equation
Ax = b does not have any solution.
What can you say about the determinant of A?
5.9.15
Exercise
Find the adjoint of A.
1
0
A =
0
0
Use the adjoint to calculate the inverse of A.
6 7 12
1 14 24
0 0 5
0 1 2
78
Chapter 6
Vectors
6.1
A vector space
6.1.1
Defining the vector space
For the purposes of this course, the vector space R2 consists of column matrices of the form

x1
x2
where x1 and x2 are real numbers.
6.1.2
Examples
Some examples are

1
u =
5
and

2
v =
1
6.1.3
Zero vector
When we use 0 for a vector in R2 we mean

0
0
6.1.4
Addition
R2 consists of column matrices with addition being defined as for matrices as previously
discussed. So for
79
80
CHAPTER 6. VECTORS

u1
u =
u2
and

v1
v =
v2
the sum u + v is

u+v =
6.1.5
u1 + v1
u2 + v2
R2 consists of column matrices with scalar multiplication being defined as for matrices as
previously discussed. So for

u1
u =
u2
and k R

ku1
ku =
ku2
6.1.6
Coordinate geometry
We can associate a vector

x
r =
y
with the point (x, y) in the xy-plane. Or we can think of r as a directed line segment from
the origin to the point in the plane.
6.2
Properties of the vector space
6.2.1
Vectors are column matrices
We will think of the vectors of the vector space R2 as being a special class of matrices. They
inherit the properties of matrices that we have already discussed.
6.3.
ANOTHER VECTOR SPACE
6.2.2
81
Properties
Then the following properties are immediate:

a) For u and v vectors in R2
u+v = v+u
b) For u, v, and w vectors in R2
(u + v) + w = u + (v + w)
b) For u, in R2 and 0 the zero vector
u+0 = u = 0+u
c) For u in R2 we have
u + (u) = 0
d) For u and v in R2 and k a real number
k(u + v) = ku + kv
e) For u in R2
1u = u
6.3
Another vector space
6.3.1
R3 consists of column matrices of the form

x1
x2
x3
where x1 , x2 and x3 are real numbers.
6.3.2
Examples
Some examples are

1
5
u =
0
and
82
CHAPTER 6. VECTORS
1
v =
3
6.3.3
Zero vector
When we use 0 for a vector in R3 we mean

0
0
0
6.3.4
Addition
R3 consists of column matrices with addition being defined as for matrices as previously
discussed. So for

u1
u = u 2
u3
and

v1
v = v2
v3
the sum u + v is
u1 + v1
u + v = u2 + v2
u3 + v3
6.3.5
R3 consists of column matrices with scalar multiplication being defined as for matrices as

u1
u2
u =
u3
and k R
ku1
ku = ku2
ku3
6.4.
PROPERTIES OF THE VECTOR SPACE
6.3.6
83
Coordinate geometry
We can associate a vector

x
r = y
z
with the point (x, y, z) in three dimensional space . Or we can think of r as a directed line
segment from the origin to the point in the space.
6.4
6.4.1
We will think of the vectors of the vector space R3 as being a special class of matrices. They
6.4.2
Properties

a) For u and v vectors in R3
u+v = v+u
b) For u, v, and w vectors in R3
(u + v) + w = u + (v + w)
b) For u, in R3 and 0 the zero vector
u+0 = u = 0+u
c) For u in R3 we have
u + (u) = 0
d) For u and v in R3 and k a real number
k(u + v) = ku + kv
e) For u in R3
1u = u
84
CHAPTER 6. VECTORS
6.5
More vector spaces
6.5.1
Rn for n = 2, 3, 4, ... consists of column matrices of the form

x1
x2

..
.
xn
where x1 , x2 , ..., xn are real numbers.
6.5.2
Zero vector
When we use 0 for a vector in Rn we mean

0
0

..
.
0
6.5.3
Addition
Rn consists of column matrices with addition being defined as for matrices as previously
discussed. So for

u1
u2

u = ..
.
un
and

v1
v2

v = ..
.
vn
the sum u + v is
u1 + v1
u2 + v2
u + v = ..
.
un + vn
6.6.
PROPERTIES OF THE VECTOR SPACE
6.5.4
85
Rn consists of column matrices with scalar multiplication being defined as for matrices as

u1
u2

u = ..
.
un
and k R
ku1
ku2
ku = ..
.
kun
6.6
6.6.1
We will think of the vectors of the vector space Rn as being a special class of matrices. They
6.6.2
Properties

a) For u and v vectors in Rn
u+v = v+u
b) For u, v, and w vectors in Rn
(u + v) + w = u + (v + w)
b) For u, in Rn and 0 the zero vector
u+0 = u = 0+u
c) For u in Rn we have
u + (u) = 0
d) For u and v in Rn and k a real number
k(u + v) = ku + kv
86
CHAPTER 6. VECTORS
e) For u in Rn
1u = u
6.7
6.7.1
The euclidean inner product

Definition
For u and v in Rn we define

uv =
n
X
ui vi
i=1
where ui is the ith entry of u and vi is the ith entry of v.
6.7.2
Example
Given
1
u = 0
1

1
v = 0
1
The inner product of u and v is found by
u v = (1)(1) + (0)(0) + (1)(1) = 2
6.7.3
Magnitude of a vector
If we think of a with elements x1 , x2 , ..., xn as a directed line segment starting at the origin and going to the associated point then we might also ask what the length of the vector is.
In the two-dimensional case the magnitude squared of

u1
u =
u2
is
u u = u21 + u22
In the three-dimensional case the magnitude squared of
6.7. THE EUCLIDEAN INNER PRODUCT
87

u1
u2
u =
u3
is
u u = u21 + u22 + u23
In the n-dimensional case the magnitude squared of

u1
u2

u = ..
.
un
is
u u = u21 + u22 + + u2n
6.7.4
Norm
Often the term norm is used for the magnitude of the vector
kuk =
6.7.5
uu
Example
For
u =
kuk

=
1
2
1
2
2

+
2
= 1
and so
kuk = 1
6.7.6
Distance between two two-dimensional vectors
If we identify a vector

r1
r =
r2
with the point (r1 , r2 ) in the cartesian plane and another vector
88
CHAPTER 6. VECTORS

s1
s =
s2
with the point (s1 , s2 ) then we are used to using Pythagorus to find the distance between
the two points:
p
(s1 r1 )2 + (s2 r2 )2 = ks rk
We define the distance between the two vectors r and s by
ks rk
6.7.7
Distance between two three-dimensional vectors
If we identify a vector

r1
r2
r =
r3
with the point (r1 , r2 , r3 ) and another vector

s1
s2
s =
s3
with the point (s1 , s2 , s3 ) then we are used to using Pythagoras to find the distance between
the two points:
p
(s1 r1 )2 + (s2 r2 )2 + (s3 r3 )2 = ks rk
We define the distance between the two vectors r and s by

ks rk
6.7.8
Distance between vectors
Given two vectors r and s in Rn we define the distance between the two vectors to be
ks rk
6.7.9
Cauchy-Schwartz inequality
For u and v in Rn we have

|u v| kukkvk
6.8. PROPERTIES OF THE EUCLIDEAN NORM
6.8
Properties of the Euclidean norm
For u in Rn
1)
kuk 0
2)
kuk = 0 iff u = 0
3) For k a real number
kkuk = |k|kuk
4) Triangle inequality
ku + vk kuk + kvk
6.8.1
Proof of triangle inequality

ku + vk2 =
=
=
6.9
6.9.1
(u + v) (u + v)
u u + 2u v + v v
kuk2 + 2u v + kvk2
kuk2 + 2|u vk + kvk2
kuk2 + 2kukkvk + kvk2 by Cauchy-Schwartz
(kuk + kvk)2
Orthogonality
Definition
Two vectors u and v in Rn are said to be orthogonal if their dot product is zero
uv = 0
6.9.2
Example
The vectors

1
u =
0
89
90
CHAPTER 6. VECTORS

0
v =
1
are orthogonal.
6.10
Vectors and systems of linear equations
6.10.1
Example
Recall that we had systems that looked like
2x1 + x2 = 5
x 1 + x2 = 7
In the language of vectors and matrices we would write

2 1
x1
5
=
1 1
x2
7
If we then perform row reduction we

1
0
get

2
0
x1
=
x2
9
1
So there is a unique solution to the matrix-vector equation which is

2
x =
9
6.10.2
Example
Solve the equation Ax = b for

1 1
A =
2 2

1
b =
1
Performing row reduction gives us

1 1
x1
0
=
0 0
x2
1
The bottom row is a contradiction 0=1 so we conclude that this matrix-vector equation has
no solution.
6.10. VECTORS AND SYSTEMS OF LINEAR EQUATIONS
6.10.3
Example
Solve the equation Ax = b for

1 1
A =
2 2

1
b =
2
Performing row reduction gives us

1 1
x1
1
=
0 0
x2
0
So we conclude that x2 could be any real number and
x1 = 1 1 x2
So the vector solution to this matrix-vector equation is

1 x2
1
1
x =
=
+ x2
x2
0
1
6.10.4
Example
Consider the matrix-vector equation

1 1 1
x1
1
0 1 1 x2 = 2
0 0 1
x3
3
Performing row reduction gives

1
x1
1 0 0
0 1 0 x2 = 1
x3
3
0 0 1
The solution vector to the matrix-vector equation is

1
x = 1
3
6.10.5
Example

1 1 1
x1
1
0 1 1 x2 = 2
0 2 2
x3
4
91
92
CHAPTER 6. VECTORS

1 0 0
x1
1
0 1 1 x2 = 2
0 0 0
x3
0
We see that x3 could be any real number and
x2 = x3
and
x1 = 1
Then the solution vector to the matrix-vector equation is

1
1
0
2 x3
2 + x3 1
x =
=
x3
0
1
6.10.6
Example

1
0 1 1
x1
0 2 2 x2 = 2
3
0 3 3
x3

0 1 1
x1
1
0 0 0 x2 = 0
0 0 0
x3
0
We see that x1 and x3 could be any real numbers and
x2 = 1 x3
Then the solution vector to the matrix-vector equation is

x1
0
1
0
1 x3
1 + x1 0 + x3 1
x =
=
0
0
1
x3
6.11. EXERCISES
6.11
Exercises
6.11.1
Exercise
93
Consider the vectors

1
u =
5

1
v =
7
Calculate the vector w = 3u + 7v.
6.11.2
Exercise
Give examples of three vectors in R2 which are of magnitude 1.
6.11.3
Exercise
Give two vectors in R3 which are orthogonal to

1
1
u =
0
6.11.4
Exercise
What is the solution to the equation

x1
x2
1 1 0 0
x3 = 0
x4
94
CHAPTER 6. VECTORS
Chapter 7
Vector Spaces
7.1
The vector spaces already discussed
7.1.1
Examples of vector spaces
We have discussed some examples of vector spaces

R2 ,
R3 ,
Rn
which were column matrices with all real entries.
7.1.2
Zero vector
All of these vector spaces had a zero vector which is a column vector with all zero entries.

0
0

0 = ..
.
0
7.1.3
Property of the zero vector
For any vector u Rn and the zero vector we have

u+0 = 0+u = u
7.1.4
Addition
For Rn addition was defined entry-wise in such a way that that for u and v vectors in Rn we
have that
u + v is defined in Rn
95
96
7.1.5
CHAPTER 7. VECTOR SPACES
A scalar multiplication was defined so that for any R and any u Rn

u Rn
7.2
Properties we have seen
From the way we defined addition and scalar multiplication for vectors in Rn we can show:
7.2.1
Properties
1. For u and v vectors in Rn

u+v
is also a vector in Rn .
u+v = v+u
3. For u, v, and w vectors in Rn
(u + v) + w = u + (v + w)
4. For u, in Rn there is a zero vector 0 such that
u+0 = u = 0+u
5. For every vector u in Rn there is a vector u in Rn such that
u + (u) = 0
6. For every vector u in Rn and in R there is a vector
u
in Rn .
7. For every u and v in Rn and a real number
(u + v) = u + v
8. For every u in Rn and and real numbers
( + )u = u + u
7.3. PROPERTIES OF A VECTOR SPACE
97
(u) = ()u
10. For u in Rn
1u = u
7.3
7.3.1
Properties of a vector space

Definition
A set V is said to be a real vector space if the properties that we just recalled for the Rn
vector spaces hold.
7.3.2
Properties
1. For u and v vectors in V

u+v
is also a vector in V .
u+v = v+u
3. For u, v, and w vectors in V
(u + v) + w = u + (v + w)
4. For u, in V there is a zero vector 0 such that
u+0 = u = 0+u
5. For every vector u in V there is a vector u in Rn such that
u + (u) = 0
6. For every vector u in V and in R there is a vector
u
in V .
7. For every u and v in V and a real number
(u + v) = u + v
8. For every u in V and and real numbers
98
( + )u = u + u
(u) = ()u
10. For u in V
1u = u
7.3.3
Example
We checked previously that Rn is a vector space. It has all the properties required of a real
vector space.
7.3.4
Example
Consider the set of all polynomial functions of finite degree. If we define addition and scalar
multiplication for functions in the usually way then this is a real vector space.
7.4
7.4.1
A longer example
Potential vector space
Consider the set V of all elements of R2 of the form

0
where could be any real number.
7.4.2
Question
Is this subset of R2 a vector space?
7.4.3
Check the required properties one by one
Checking 1.
Is the sum of two elements of V in V ? Consider

0
u =
V
and

0
v =
V
7.4. A LONGER EXAMPLE
99
The sum

u+v =
0
+
is also in V .
Checking 2.
For vectors u and v in V is u + v = v + u?
Yes, dont really need to check this since it was already checked for R2 .
Checking 3.
For u, v, and w vectors in V is addition associative?
Yes, dont really need to check this since it was already checked for R2 .
Checking 4.
Is the zero vector in V ? Yes. Dont really need to check that adding the zero vector to a
vector doesnt change the vector since this is already known for all of R2 .
Checking 5.
For every vector u in V there is a vector u in V such that
u + (u) = 0
Consider

0
u =
V
Then

u =

V
and u + (u) = 0.
Checking 6.
If u is in V is u in V ? Consider

0
u =
V
b
Then
100

u =
0
b

V
Checking 7.
For every u and v in V and a real number is (u + v) = u + v?
Consider

0
u =
V
a
and

0
v =
V
b
Then

0
(u + v) =
a+b

=
0
a + b

=

0
0
0
0
+
=
+
= u + v
a
b
a
b
Checking 8.
For every u in V and and real numbers is it true that ( + )u = u + u?
Consider

0
u =
V
a
Then for and real numbers

0
0
0
0
0
0
(+)u = (+)
=
=
+
=
+
= u+u
a
a + a
a
a
a
a
Checking 9.
For every u in V and and real numbers is it true that (u) = ()u?
Consider

0
u =
V
a
Then for and real numbers

0
0
0
(u) =
=
= ()
= ()u
a
a
a
7.5. SUBSPACES
101
Checking 10.
For u in V is 1u = u? Dont really need to check this since we already know that it is true
for all vectors u in R2 .
7.5
Subspaces
7.5.1
Definition
If V is a real vector space, and W is a non-empty subset of V which is also itself a vector
space then W is said to be a subspace of V .
7.5.2
Example
The V R2 that we just discussed is a subspace of R2 .
7.5.3
Theorem - Checking if a subset is a subspace
If W is a non-empty subset of vectors of a vector space V then W is a subspace of V if and

only if
a) u, v W implies u + v W
b) R and u W implies u W .
7.5.4
Proof
Exercise.
7.5.5
Example
Consider the set of vectors W R2 which consists of all vectors whose first and second
elements are the same. A typical element looks like

u =
W
This is a non-empty subset of R2 . If we add two typical elements

u =
W

v =
W
we get
102

+
u+v =
+
which has first and second elements the same and so the sum of two elements of W is in W .
Next, consider

u =
W
and a scalar k. We get

k
ku =
k
which has equal first and second entries. Then the scalar multiple of an element of W is in W .
Both required conditions hold and we conclude that W is a subspace of R2 .
7.5.6
Example
Suppose that V is a real vector space. Consider the subset

W = {0}
which consists only of the zero vector. Then for two vectors u and v in W :
u+v = 0+0 = 0W
For any real number and any vector u in V :
u = 0 = 0 V
Then the required conditions have been checked and we have found a trivial subspace of
every vector space.
7.5.7
Example
Consider the vector space R2 . Consider the subset

1
W =
1
Is this subset W a subspace of R2 ?
Consider for an element u in W that
0u = 0
which is not in W . So some scalar multiples of elements of W are not in W and so W is not
a subspace.
7.6. SUBPSACES AND SOLUTIONS OF SYSTEMS OF EQUATIONS
7.6
7.6.1
103
Subpsaces and solutions of systems of equations

Theorem
If Amn x = 0 is a system of m linear equations in n unknowns then the set of solutions is a

subspace of Rn .
7.6.2
Proof
Let W be the set of all solutions of the equation Ax = 0. Let u and v be elements of W . As
they are solutions of Ax = 0 we have Au = 0 and Av = 0.
Does u + v belong to W ?
A(u + v) = Au + Av = 0 + 0 = 0
So if u is a solution and v is a solution then the sum u + v is a solution.
Suppose that u is a solution and is a real number. Then is u a solution?
A(u) = (Au) = 0 = 0
So if u is in W then u is in W .
Then both conditions have been checked and W , the set of all solutions to the equation
Ax = 0, is a subspace of Rn .
7.6.3
Example
Consider the system

0
1 2 3
x1
2 4 6 x2 = 0
0
3 6 9
x3
We form the augmented matrix and do row reduction to get the rref
1 2 3 | 0
0 0 0 | 0
0 0 0 | 0
from which we conclude that
7.6.4
Example
Consider the equation
104

1 2 3
x1
0
2 4 6 x2 = 0
3 6 9
x3
0
If we form the augmented matrix and do row reduction we get
1 2 3 |0
0 0 0 |0
0 0 0 |0
This gives us that x2 and x3 can be any real numbers ( call them s and t ) and
x1 = 2x2 3x3
Then the solution vector is

x1
2x2 3x3
2s 3t
2
3
= s = s 1 + t 0
x2
x = x2 =
x3
x3
t
0
1
with s and t in R.
According to the theorem, this set of solutions is in fact a subspace of R3 .
7.7
7.7.1
Linear combinations
Linear combination of vectors
A vector v in a real vector space V is said to be a linear combination of vectors v1 , v2 , ... ,

vn if there are real numbers 1 , 2 , ... , n such that
v = 1 v1 + 2 v2 + + n vn
7.7.2
Example
In R2 the vector

v =
can be written as a linear combination of the vectors

1
e1 =
0

0
e2 =
1
by
7.7. LINEAR COMBINATIONS
105
v = e1 + e2
7.7.3
Example
In R2 the vector

v =

1
v1 =
1

1
v2 =
1
by
v =
7.7.4
v1 +
v2
2
2
Example
In R3 all vectors can be written as linear combinations of the vectors

1
e1 = 0
0

0
1
e2 =
0

0
0
e3 =
1
7.7.5
Example
In Rn all vectors can be written as linear combinations of the vectors

1
0

e1 = ..
.
0
106
e2

0
1

= ..
.
0
..
.
en
7.7.6

0
0

= ..
.
1
Theorem - linear combinations form a vector space
Suppose v1 , v2 , ... , vk are vectors in a vector space V .

a) Then the set W of all linear combinations of v1 , v2 , ... , vk is a subspace of V .
b) It is the smallest subspace that contains all the vi .
7.7.7
Proof of a)
Suppose that p and q are elements of W . Then p and q are both linear combinations of the
vi :
p = 1 v1 + 2 v2 + + k vk
q = 1 v1 + 2 v2 + + k vk
The sum is
p + q = (1 + 1 )v1 + (2 + 2 )v2 + + (k + k )vk
which is a linear combination of the vi and so in W . So the sum of two vectors in W is also
in W .
Now consider a scalar multiple of p:
p = (1 )v1 + (2 )v2 + + (k )vk
And then this is also a linear combination of the vi and so is in W . So a scalar multiple
of a vector in W is in W . The two required conditions have been checked and so W is a
subspace.
7.7.8
107
Proof of b)
Suppose a subspace contains vectors v1 , v2 , ... , vk .

One of the conditions for a subspace is that all scalar multiples of vectors in the subspace
must also be in the subspace. Then for all real numbers 1 , 2 , ... , k we must have than
1 v1 , 2 v2 , ... , k vk are also in the subspace.
The other condition for a subspace is that sum of any two vectors in the subspace must
also be in the subspace. Then
1 v1 + 2 v2
is in the subspace. Then
(1 v1 + 2 v2 ) + 3 v3
is in the subspace. And so on till
1 v1 + 2 v2 + + k vk
is also in the subspace.
Thus v1 , v2 , ... , vk in a subspace implies that all linear combinations of the vi are in
the subspace.
7.7.9
Span
Suppose a subspace V contains vectors v1 , v2 , ... , vk . The subspace W consisting of all the
linear combinations of the vi is called the space spanned by v1 , v2 , ... , vk and v1 , v2 , ... ,
vk are said to span W .
7.7.10
Example
e1 , e2 , and e3 span R3 .
7.7.11
Example
The vectors

1
0
0

1
1
0
108

1
1
1
span R3 .
7.7.12
Example
The vectors

3
2
1

2
1
0

1
1
1
do NOT span R3 .
7.8
7.8.1
Linear independence and dependence

Linear independence
Suppose that v1 , v2 , ... , vk are vectors in a real vector space V . The vi are said to be
linearly dependent if we can find real numbers 1 , 2 , .. , k , not all zero so that
1 v1 + 2 v2 + + k vk = 0
Otherwise the set is said to be linearly independent.
7.8.2
Alternatively
Another way of putting this is that the vi are linearly independent if

1 v1 + 2 v2 + + k vk = 0
only for all the i = 0. Then if the vectors are not linearly independent they are said to be
linearly dependent.
7.8. LINEAR INDEPENDENCE AND DEPENDENCE
7.8.3
109
Example
The vectors e1 and e2 in R2 are linearly independent because the only way to get
e1 + e2 = 0
is for both and to be zero.
7.8.4
Example
Are the vectors

2
1
u =
2

2
3
v =
6

2
w = 2
4
linearly independent?
We need to check whether or not the equation
xu + yv + zw = 0
has only the trivial solution x = y = z = 0 or whether there are other solutions. Lets write
this out. We want to check if there is a solution other than x = y = z = 0 for

2
2
2
0
x 1 + y 3 + z 2 = 0
2
6
4
0
We can rewrite this as

2 2 2
x
0
1 3 2 y = 0
2 6 4
z
0
If we form the augmented matrix then we get the rref
1 0 1/2 | 0
0 1 1/2 | 0
0 0 0 | 0
and so we get infinitely many solutions and conclude that the vectors considered are linearly
dependent.
110
7.9
7.9.1
Back to the FTLA

Recall

a) Ann is invertible.
e) Ax = b is consistent for all n 1 matrices b.
f) |A| =
6 0.
7.9.2
Add to the FTLA
g) The columns of A considered as vectors in Rn are linearly independent.
7.9.3
Proof
Suppose that A is invertible. Then the only solution to Ax = 0 is x = 0. Then the only
linear combination of the columns of A that gives zero is one where the scalar coefficients of
the sum are zero. Then the columns are linearly independent.
If the columns are linearly independent then the only linear combination that gives zero
is the one with all scalar coefficients zero. Then Ax = 0 has only the trivial solution and so
by the FTLA then A is invertible.
7.10
Dimension
7.10.1
Example

2
u =
0

1
v =
1

1
w =
1
7.10. DIMENSION
111
in R2 . We can consider the vector space W which is spanned by all of these vectors. The
elements of W look like
p = u + v + w
where , and can be any real numbers. But the set of vectors {u, v, w} are not linearly
independent since
uv+w = 0
So we can write
w = u+v
Then vectors in the space W can be written
p = u + v + w = u + v + (u + v) = 0 u + 0 v
We can write every vector in W as a linear combination of u, v, and w but we can also just
write the vector as a linear combination of u and v.
It is helpful to use as few vectors as are actually needed to span the space that one is
interested in.
7.10.2
General idea
Suppose that we are interested in a space W that we know is spanned by a set of vectors
B = {v1 , v2 , . . . , vm }
If the set of vectors is linearly independent, then none of the vectors can be written as a
linear combination of the other vectors. If the set is linearly dependent, then some vector
can be written as a linear combination of the others. Say, vm . We can drop vm from the set
and just use
B0 = {v1 , v2 , . . . , vm1 }
We keep going till we get a linearly independent set.
7.10.3
Basis
Such a minimal set is called a basis.
7.10.4
Minimality of the basis
For a given vector space W with two bases, the number of vectors in both bases is the same.
112
7.10.5
Example
The vectors
e1

1
=
0
e2

0
=
1
are a basis for R2 . So is

B = {e1 , e1 + e2 }
Both bases have two vectors.
7.10.6
Dimension
The dimension of a vector space V is the number of vectors in its basis.
7.10.7
Example
Rn has dimension n.
7.10.8
Example
The space of polynomials on R is of infinite dimension.
7.11
Exercises
7.11.1
Exercise
Show that the set of polynomial functions is a vector space.
7.11.2
Exercise
Give an example of a one-dimensional subspace in R3 .
7.11.3
Exercise
Given an example of a two-dimensional subspace in R3 .
7.11. EXERCISES
7.11.4
113
Exercise
The standard basis for R3 is

B = {e1 , e2 , e3 }
where
e1
e2
e3
0
=
0

0
= 1
0

0
= 0
1
Find another basis for R3 in which all the vectors have length 1.
7.11.5
Exercise
Can a set of five vectors in R4 be linearly independent?
114
Chapter 8
Euclidean vector spaces
8.1
8.1.1
Euclidean vector spaces

Definition
The vector spaces Rn with the inner/scalar/dot product that we have already discussed are
called Euclidean vector spaces.
8.1.2
Review
If you dont recall the dot product and its properties you should review them at this time.
8.2
8.2.1
Dot product and matrix multiplication

Transpose and dot product
Recall that for u and v in Rn we defined

u v = u1 v1 + u2 v2 + + un vn
Note that
u v = vtu
by the rules of matrix multiplication.
8.2.2
Example

1
1
u =
1
and
115
116
CHAPTER 8. EUCLIDEAN VECTOR SPACES

1
v = 2
3
The dot product can be calculated by
u v = vtu =
8.2.3

1
1 2 3 1 = 6
1
Matrix transpose and dot product
By the previous rule, for a square matrix Ann and vectors u and v in Rn
Au v = v t (Au) = (v t A)u = (At v)t u = u At v
and
u Av = (Av)t u = (v t At )u = v t (At u) = At u v
8.2.4
Example

1
u =
1
and

2
v =
5
and the matrix

1 2
A =
3 4
Then

1
1 3
2
1
16
Au v = u A v =
= 40
1
2 4
5
1
24
t
8.2.5
Matrix multiplication and vectors
One way of looking at the rule for matrix multiplication is to think of the first matrix as
being made up of row vectors
8.3. FUNCTIONS/MAPS
117
Amk
R1
R2
= ..
.
Rm
and the second matrix as being made up of columns

Bkn =
C1 C2
Cn
and then the product looks like
(AB)mn
8.3
8.3.1
R1 C1
R2 C1
= ..
.
R1 C2
R2 C2
..
.
R1 Cn
R2 Cn
Rm C1 Rm C2 Rm Cn
..
.
Functions/maps
Domain of functions
Suppose that f : U V is a function that maps elements of the set A to elements of the
set B. We call A the domain of f .
8.3.2
Example
Suppose that a function f : (0, 1) R is defined by

1
f (x) =
1 x2
The domain of this function is (0, 1).
8.3.3
Example
Suppose that a function f : R2 R is defined by

x
f
= x2 + y 2
y
The domain of this function is R2 .
8.3.4
Range of a function
Suppose that f : U V is a function that maps elements of the set A to elements of the
set B. We call the subset of B of all the elements that f maps to the range of f .
118
8.3.5
Example
Suppose that a function f : R2 R is defined by

x
f
= x2 + y 2
y
The domain of this function is R2 . The range is [0, ).
8.3.6
Example
Suppose that a function f : U R3 is defined by

x
2
2
U =
|x +y 1
R2
y

x
f
=
y
x
y
2
x + y2
The domain of this function is the unit disk U and the range can be graphed as a paraboloid
in 3-space.
8.4
8.4.1
Linear transformation
Linearity
Suppose that there is a function L : U V that maps between real vector spaces U and V .
Then L is said to be a linear transformation if the following conditions hold:
For any vectors x and y in U
L(x + y) = L(x) + L(y)
and for a real scalar
L(x) = L(x)
8.4.2
Question
What do these two conditions remind you of?
8.4.3
Exercise
Show that the range of L is a subspace of V .
8.5. EXAMPLE
8.4.4
119
Exercise
Consider a function f : R2 R3 defined by

x1
2x1 + x2
f
=
x2
3x2 + 3x2
Confirm that

1
3
f
=
6
1
Show that this f is a linear transformation by checking the two requirements for a linear
transformation.
8.5
8.5.1
Example
Transformation
Suppose that we have a linear transformation L : R2 R3 defined by

x1 + x2
x
= x1 x2
L 1
x2
x1
8.5.2
Linearity
We confirm that the transformation L is linear because for vectors u and v in R2 and scalar
:
L(u + v) = Lu + Lv
L(u) = Lu
120
8.5.3
First condition for linearity

x1
y
L
+ 1
x2
y2
=
=
=
=
8.5.4

x1 + y1
L
x2 + y2
(x1 + y1 ) + (x2 + y2 )
(x1 + y1 ) (x2 + y2 )
(x1 + y1 )
x1 + x 2
y1 + y2
x1 x2 + y1 y2
x1
y1

x
y
L 1 +L 1
x2
y2
Second condition for linearity

x1 + x2
x1 + x2
x1
x1
x
L
= L
= x1 x2 = x1 x2 = L 1
x2
x2
x2
x1
x1
8.5.5
Associated matrix
Note that

1 1
x
x
L
= 1 1
y
y
1 0
So we can calculate the result of the linear transformation by doing a matrix-vector multiplication.
8.6
General linear transformation
8.6.1
Transformation
Suppose L : Rn Rm is defined by
x1
a11 x1 + a12 x2 + + a1n xn
a11 a12
x2
a21 x1 + a22 x2 + + a2n xn
a21 a22

L .. =
= ..
..
.
.
.
xn
am1 x1 + am2 x2 + + amn xn
am1 am2

a1n
x1
x2
a2n

..
.
amn
xn
8.7. SOME LINEAR TRANSFORMATIONS
121
We can make a connection between the linear transformation L and the matrix A
a11
a21
A = ..
.
a12
a22
am1 am2
a1n
a2n
amn
which yields the same result as L:

L(x) = Ax
8.6.2
Standard matrix
A is often called the standard matrix of L.
8.7
8.7.1
Some linear transformations

Zero transformation
This is the transformation that maps every vector in Rn to the zero vector in Rm .
8.7.2
Exercise
What is the standard matrix for this linear transformation?
8.7.3
Identity operator
The identity operator L : Rn Rn is defined by

L(x) = x
8.7.4
Exercise
What is the standard matrix for the identity operator?
8.7.5
Reflection operators
2D reflection in the y axis

Consider the operator L : R2 R2 defined by

L(x) =
x1
x2
122
Example

1
1
L
=
1
1
Exercise
What is the standard matrix for this reflection operator?
2D reflection about x axis

L(x) =
x1
x2
Example

2
2
L
=
6
6
Exercise
What is the standard matrix for this operator?
8.7.6
Generally
Operators that reflect vectors in lines in 2-space and 3-space are called reflection operators.
8.7.7
Projection operators
In lower dimensions
In lower dimensions the projection operators are

x
x
L
=
y
0

x
0
L
=
y
y

x
x
L y = y
z
0
8.8. ROTATION OPERATORS
123

x
x
L y = 0
z
z

x
0
y
L y
=
z
z
Example
The projection of the vector

2
4
v =
7
onto the yz-plane is

0
4
Lv =
7
Exercise
What are the standard matrices for these operators?
8.8
Rotation operators
8.8.1
Rotations in 2-space
Suppose that one identifies the vector

x
r =
y
with the point (x, y) in the Cartesian plain or with a directed line segment running from the
origin to the point. Suppose that we want to keep the tail of the line segment at the origin
and rotate the tip (without changing the magnitude of the line segment) through an angle
in the counterclockwise direction.
Where does the tip of the line segment end up?
8.8.2
Rotation operators in 2-space
Rotations counterclockwise through an angle are defined by
124

x
x cos y sin
R
=
y
x sin + y cos
8.8.3
Example
Let

1
r =
0
Then

cos /4
R/4 (r) =
=
sin /4
8.8.4
1
2
1
2
Exercise
What is the standard matrix for a rotation operator in 2-space?
8.8.5
Exercise
Show that this operator does not change the distance from the origin. That is
krk = kR (r)k
8.8.6
Rotation operators in 3-space
Rotation about positive x axis

x
x
L y = y cos z sin
z
y sin + z cos
Rotation about positive y axis

x
x cos + z sin
y
L y =
z
x sin + z cos
Rotation about positive z axis

x
x cos y sin
L y = x sin + y cos
z
z
8.9. DILATION/CONTRACTION OPERATORS
8.8.7
125
Exercise
What are the standard matrices for the rotations about the axes in 3-space?
8.9
8.9.1
Dilation/contraction operators
Idea
We think of a vector

x
r =
y
as lying in the Cartesian plane between the origin and the point (x, y). We then consider
stretching or contracting the vector but leaving its direction unchanged.
8.9.2
Example
Consider the vector

2
r =
3
The vector that has double the magnitude and points in the same direction as r is

4
2r =
6
8.9.3
Operator
The operator that contracts or dilates by a factor k > 0 without changing the length of the
vector is

kx
Lr =
ky
in 2-space and
kx
Lr = ky
kz
in 3-space.
8.9.4
Exercise
What is the standard matrix for this operator?
126
8.9.5
Exercise
What is the extension to n-space?
8.10
Composition of linear transformations
8.10.1
Composition of linear transformations
Suppose that there is a linear transformation P : Rn Rk and another linear transformation

Q : Rk Rm . The composition is written
QP
and is defined by
(Q P )(x) = Q(P (x))
8.10.2
The composition of linear functions is linear
Proof.
Q(P (x + y)) = Q(P (x) + P (y)) = Q(P (x)) + Q(P (y))
Q(P (x)) = Q(P (x)) = Q(P (x))
8.10.3
Exercise
What is the standard matrix for a composition of linear functions?
8.10.4
Exercise
Show that composition is not commutative.
Chapter 9
Linear transformations of euclidean
spaces
9.1
9.1.1
Recall
Linearity
The map L : Rn Rm is said to be a linear transformation if for any u and v in the domain
and a real number we have
L(u + v) = L(u) + L(v)
L(u) = L(u)
9.1.2
Significance
The range of a linear transformation is a vector space.
9.1.3
Standard matrix
A linear transformation L : Rn Rm has a matrix Amn associated with it such that for u
in the domain of L
L(u) = Au
9.2
one-to-one functions
9.2.1
one-to-one
A function f : U V is said to be one-to-one (injective) if for x and y in U

f (x) = f (y) = x = y
127
128
9.2.2
CHAPTER 9. LINEAR TRANSFORMATIONS OF EUCLIDEAN SPACES
Example
Consider the function f : [1, 1] R defined by

f (x) = x2
This function is NOT one-to-one (injective) since f (x) = f (y) does not necessarily imply
that x = y. For example, if f (x) = 1 = f (y) it could be that x = 1 and y = 1.
9.2.3
Example
Consider the function f : [0, 1] R defined by

f (x) = x2
This function is one-to-one (injective) since f (x) = f (y) implies that x2 = y 2 and x = y
since all the possible values of x and y are non-negative.
9.3
9.3.1
Onto functions
Onto functions
A function f : U V is said to be onto V (surjective) if for any y in V there is an x in U

so that
y = f (x)
9.3.2
Example
Consider the function f : (1, 1) R defined by

f (x) = x2
This function is not onto R because 12 is in R and there is no x in (1, 1) so that x2 = 12.
9.3.3
Example
Consider the function f : [0, 1] [0, 1] defined by

f (x) = x2
This function is onto [0, 1].
9.3.4
Exercise
Consider the identity operator I : R3 R3 . Show that this function is both one-to-one and
onto.
9.4. INVERTIBLE FUNCTIONS
9.4
9.4.1
129
Invertible functions
Condition
A function f : U V has a well-defined inverse if f is both one-to-one and onto ( injective

and surjective ). That is because for every v in V there is a u in U so that f (u) = v (and
only one such u) so it makes sense to define the inverse function by
f 1 (v) = u
9.5
FTLA
9.5.1
Recall
We had that the following are equivalent for a square matrix Ann :
a) A is invertible.
c) The rref of A is I.
f) |A| =
6 0.
g) The columns of A considered as vectors in
9.5.2
Rn are linearly independent.
Linear transformation and standard matrix
Recall that a linear transformation L : Rn Rn has a square matrix associated with it and
every square matrix can be thought of as the standard matrix for a linear transformation
L : Rn Rn .
What does the invertibility of A tell us about L and how does it relate to the FTLA?
9.5.3
Theorem
Suppose Ann is a square matrix and L : Rn Rn is multiplication by A. If A is invertible

then the range of L is Rn .
130
9.5.4
Proof
Suppose that A is invertible. We want to show that for any y in Rn there is a x in Rn so

that
Ax = y
Then
x = A1 y
gives us the desired x for any given y. Thus L is onto Rn .
9.5.5
Theorem
Suppose Ann is a square matrix and L : Rn Rn is multiplication by A. If the range of L

is Rn then A is invertible.
9.5.6
Proof
Consider the n standard basis vectors for Rn . The ith one is ei which has all zeros except
for a 1 in the ith entry. As L is onto, for each ei there is some vector xi so that
Axi = ei
Now consider the matrix B whose columns are the xi :
B = ( x1 x2 xn1 xn )
Then
AB = A( x1 x2 xn1 xn ) = ( e1 e2 en1 en ) = I
and so B is the inverse of A.
9.5.7
Theorem
Suppose Ann is a square matrix and L : Rn Rn is multiplication by A. If A is invertible

then L is one-to-one.
9.5.8
Proof
If x and y are vectors in Rn and

L(x) = L(y)
then
Ax = Ay
9.6. INVERSE OF MATRIX AND INVERSE OF FUNCTION
131
and multiplying by the inverse of A on both sides gives

x = y
Then
L(x) = L(y) = x = y
9.5.9
Theorem
Suppose Ann is a square matrix and L : Rn Rn is multiplication by A. If L is one-to-one

then A is invertible.
9.5.10
Proof
Suppose that L is one-to-one. Then

L(x) = L(0) = x = 0
Ax = 0 = x = 0
So the equation Ax = 0 has only the trivial solution and A must be invertible by the FTLA.
9.5.11
Additions to the FTLA
h) The range of the linear transformation which is multiplication by A is Rn .

i) The linear transformation which is multiplication by A is one-to-one.
9.5.12
Example
A projection operator P : R2 R2 which projects onto the x-axis has standard matrix

1 0
A =
0 0
A is not invertible so this operator P is not one-to-one and the range of P is not R2 .
9.6
9.6.1
Inverse of matrix and inverse of function

Recall
If Ann is invertible then the operator L : Rn Rn which is multiplication by A is one-to-one

and onto and so L has an inverse.
132
9.6.2
Standard matrix for the inverse linear transformation
The standard matrix for the inverse linear transformation is A1 .
9.7
9.7.1
Standard matrix for a linear transformation

Question
Given a linear transformation L : Rn Rm how do we find its standard matrix?
9.7.2
Consider how linear transformation acts on basis vectors
Suppose that L acts on the basis vectors of Rn as

L(ei ) = xi
Once we have that we know what happens to any vector in the domain since it is a linear
combination of the ei :
X
L(
i ei ) = i xi
i
Now construct Amn as the matrix whose columns are the xi .
x11 x21 xn1

x1 x2 xn
2
2
2
A = ..
..
..
.
.
.
1
2
xm xm x6m
Note that
Aei = xi = L(ei )
so multiplication by A has the same result as L on the basis vectors and so the same is true
for any vector in the domain of L. Then A is the standard matrix for L.
9.7.3
Example
Consider the linear operator L : R2 R2 which is defined by

x
x+y
L
=
y
xy
Then

1
1+0
1
L
=
=
0
10
1
and
9.7. STANDARD MATRIX FOR A LINEAR TRANSFORMATION

0
0+1
1
L
=
=
1
01
1
Then the standard matrix for L is

A =

1 1
1 1
133
134
Chapter 10
Least squares
10.1
Orthogonal bases
10.1.1
Given a subspace
Suppose that W is an m-dimensional subspace of Rn with a basis

S = { w1 , w 2 , . . . w m }
Every vector in W can be expressed as a linear combination of the basis vectors:
w = 1 w1 + 2 w2 + + n wm
What are the coefficients i ?
10.1.2
Orthogonal basis
A basis is said to be orthogonal if each basis vector is orthogonal to all of the other basis
vectors.
10.1.3
Example
The standard basis for Rn is an example of an orthogonal basis.
10.1.4
Linear combinations with an orthogonal basis
Suppose that a subspace W of a euclidean vector space has an orthogonal basis. Then any
vector w in W can be written as a linear combination of the basis vectors. Now if we take
the dot product with the ith basis vector
w wi = i wi wi
and so
i =
w wi
wi wi
135
136
CHAPTER 10. LEAST SQUARES
Then any vector in W can be written

w =
10.2
Example
10.2.1
Problem
w w2
w wm
w w1
w1 +
w2 + +
wm
w1 w1
w2 w2
wm wm
Consider the subspace W spanned by the vectors

1
0
p =
0
and

1
1
q =
1
Any vector in W can be written as a linear combination of these vectors
w = p + q
Note that
p q = 1 6= 0
10.2.2
Finding an orthogonal basis
We will find a new orthonormal basis

O = { r, s }
for W . Let
r = p
be the first basis vector. Now lets find a new basis vector in W
s = q + r
that is orthogonal to r.
r s = r q + r r = 0
Then
=
rq
rr
10.3. GRAM-SCHMIDT ORTHOGONALIZATION PROCESS

Then the new orthogonal basis vectors for W are

1
0
r = p =
0
and

1
1
0
1
rq
r = 1 0 = 1
s = q + r = q
rr
1
1
0
1
10.3
Gram-Schmidt orthogonalization process
10.3.1
Problem
Given a basis
S = { w1 , w2 , . . . , wm }
of a subspace of Rn we would like to find an orthogonal basis
O = { u1 , u2 , . . . , um }
for W .
10.3.2
Gram-Schmidt orthogonalization process
1) Let
u1 = w1
2) Let
u2 = w2
w 2 u1
u1 u1
2) Let
u3 = w 3
w 3 u1 w 3 u2
u1 u1
u2 u2
..
.
2) Let
um = wm
wm u1 wm u2
wm um1

u1 u1
u2 u2
um1 um1
This results in an orthogonal basis for W .
137
138
10.3.3
Example again using Gram-Schmidt
Consider the subspace W of R3 spanned by the vectors

1
w1 = 0
0
and
w2
1
=
1
These vectors are not orthogonal. We will use the Gram-Schmidt process to create an
orthogonal basis for the same subspace.
10.3.4
Applying Gram-Schmidt
Let
u2
0
u1 = w1 =
0

1
1
0
1
w2 u1
0
1
1
u1 =
=
= w2
u1 u1
1
0
1
1
Then an orthogonal basis for the subspace W is

1
0
0
1
O =
u =
, u2 =
1
0
1
10.4
Using a basis to approximate a vector
10.4.1
A vector as a linear combination of basis vectors
Suppose that we have a vector v R3

1
v = 2
5
We can construct this vector ( and every other vector in R3 ) as a linear combination of the
standard basis vectors
v = 1e1 + 2e2 + 5e3
10.4. USING A BASIS TO APPROXIMATE A VECTOR
10.4.2
139
Question
Can the vector v be constructed as the linear combination of only two of the standard basis
vectors e1 and e2 ? No.
10.4.3
What is the best that you could do?
Suppose that you wanted to find a vector

w = 1 e 1 + 2 e 2
in the span of e1 and e2 closest ( in some sense ) to v.
Note that the basis vectors for this span are already orthogonal.
The distance squared between v and w is given by
kw vk2 = (1 1)2 + (2 2)2 + (0 5)2
Differentiating wrt 1 and 2 and setting the partial derivatives to zero gives
2(1 1) = 0
2(2 2) = 0
implying that the closest vector to v in the span of e1 and e2 is

1
2
1e1 + 2e2 =
0
10.4.4
What about a different span of vectors?
Suppose that we have vectors
w1
w2

1
= 0
0

1
1
=
1
and we want the closest vector w to v.

We previously saw that after the Gram-Schmidt orthogonalization process we get a new
basis vectors for the same subspace W :
140
u1
u2
0
=
0

0
= 1
1
Now we express the approximation w for the vector v in terms of the new basis vectors

1
2
w = 1 u1 + 2 u2 =
2
The distance squared to v is
kw vk2 = (1 1)2 + (2 2)2 + (2 5)2
If we seek to minimize this distance then we get two equations from the differentiation
2(1 1) = 0
2(2 2) + 2(2 5) = 0
and a bit of algebra gives
1 = 1,
2 = 7/2
Then the closest vector in the span of w1 and w2 to the vector v is

1
7
7/2
w = 1u1 + u2 =
2
7/2
10.5
Geometric viewpoint
10.5.1
Geometric form of problem
We have a vector in R3

1
v = 2
5
which we identify with the point (1, 2, 5) in Cartesian 3-space.
We also have the span of the vectors
10.5. GEOMETRIC VIEWPOINT
141
w1
0
=
0
w2

1
= 1
1
and
which we view geometrically as a plane in Cartesian 3-space. We want to project v onto this
plane.
Life is a little easier when working with an orthogonal basis so we use the orthogonal basis
vectors
u1

1
= 0
0
1
u2 =
1
instead. After projecting onto the plane we get the shadow of v in W
1
= 7/2
7/2
which we identify with the point in the plane (1,7/2,7/2).
10.5.2
Orthogonality to the plane
We can imagine drawing line segments from the original point v to different points in the
plane. Intuitively, the shortest line segment will be the one that is perpendicular to the
plane. This corresponds to the difference between v and w .
We confirm that w v is orthogonal to any vector in W :

0
1
3/2 2 = 0
(w v) (1 u1 + 2 u2 ) =
3/2
2
142
10.6
The least squares problem
10.6.1
The least squares problem in Rn
Let W be an m-dimensional subspace of Rn . Given a vector v in Rn find a vector w in W

so that
kw vk kw vk
10.6.2
for all w W
Best least squares approximation
The vector w in W is called the best least squares approximation to v.
10.6.3
Example
We have already done an example, with v R3 and w in a vector space which is the span
of two vectors w1 and w2 . The vector that we found is the best least squares approximation
to v.
10.7
Orthogonality result
10.7.1
Theorem
Suppose that v is a vector in Rn . Suppose that W is an m-dimensional subspace of Rn and

w in W is such that
(w v)t w = 0
for all w in W . Then w is the best least-squares approximation to v.
10.7.2
In other words ..
The vector connecting v to w is orthogonal to the subspace W .
10.7.3
Proof
Given the conditions of the theorem
10.8. TESTING FOR ORTHOGONALITY
kw vk2 =
=
=
=
=
=
143
kw w + w vk2
[(w w ) + (w v)]t [(w w ) + (w v)]
[(w w )t + (w v)t ][(w w ) + (w v)]
(w w )t (w w ) + (w w )t (w v) + (w v)t (w w ) + (w v)t (w v)
(w w )t (w w ) + 2(w v)t (w w ) + (w v)t (w v)
kw w k2 + 2(w v)t (w w ) + kw vk2
Now as w and w are in W by the assumption of the theorem

(w v)t w = 0
and
(w v)t w = 0
Then
kw vk2 =
= kw w k2 + kw vk2
Now as the magnitude of a vector is greater than or equal to zero we have

kw vk2 kw vk2
So
kw vk kw vk
for all vectors w in W .
10.8
Testing for orthogonality
10.8.1
Lemma
Suppose that we have a m-dimensional subspace W of Rn and we want to find out if a given
vector n of Rn is orthogonal to every vector in W . Then n is orthogonal to every vector w
in W if and only if it is orthogonal to every basis vector of W .
10.8.2
Proof
Suppose that W has a set of basis vectors

S = {w1 , w2 , . . . wm }
and
144
n wi = 0
for all the wi in S. Then as S is a set of basis vectors for W any vector w in S can be written
as a linear combination of the vectors in S:
w = 1 w1 + 2 w2 + + m wm
Taking the dot product with n gives
nw =
=
=
=
=
n (1 w1 + 2 w2 + + m wm )
n 1 w1 + n 2 w2 + + n m wm
1 n w1 + 2 n w2 + + m n wm
0 + 0 + 0
0
Conversely, if n is orthogonal to every vector in W then it is orthogonal in particular to the

basis vectors in W .
10.9
Existence and uniqueness of best approximations
10.9.1
Recall
Any best approximation w in a subspace W for v in Rn will have the property that for any
w in W
(w v)t w = 0
and to find such a w v it is sufficient to find one that is orthogonal to very basis vector
of W .
10.9.2
Resulting equations
If W has a set of basis vectors

S = {w1 , w2 , . . . wm }
then the best approximation w for v in Rn has the property that the following equations
hold
(w v)t w1 = 0
(w v)t w2 = 0
..
.
(w v)t wm = 0
10.9. EXISTENCE AND UNIQUENESS OF BEST APPROXIMATIONS
10.9.3
145
Uniqueness
If these equations have a unique solution then the best approximation exists and must be
unique.
10.9.4
Solving the system of equations
Suppose that
S = {w1 , w2 , . . . wm }
is a set of orthogonal basis vectors, i.e. each basis vector is orthogonal to the others. We
can always find such a basis for any subspace of Rn .
We write w as a linear combination of the orthonormal basis vectors for W
w = 1 w1 + 2 w2 + + m wm
Then the system of equations becomes
(1 w1 + 2 w2 + + m wm v)t w1 = 0
(1 w1 + 2 w2 + + m wm v)t w2 = 0
..
.
(1 w1 + 2 w2 + + m wm v)t wm = 0
and using orthogonality gives
1 kw1 k2 v t w1 = 0
2 kw2 k2 v t w2 = 0
..
.
m kwm k2 v t wm = 0
and then
v t wi
i =
kwi k2
for i = 1, 2, . . . , m.
146
10.9.5
A best approximation
The best approximation for v then is

w
10.9.6
m
X
v t w1
v t w2
v t wm
v t wi
=
w
+
w
+
+
w
=
wi
1
2
m
kw1 k2
kw2 k2
kwm k2
kwi k2
i=1
Uniqueness
The best approximation w is unique, no other vector in W can be a better least squares
approximation.
10.9.7
Proof
Suppose that w is the best approximation as we have just constructed. Let w+ be some
other best approximation in W . Then
kw+ vk2 =
=
=
=
kw+ w + w vk2
[(w+ w ) + (w v)]t [(w+ w ) + (w v)]
(w+ w )t (w+ w ) + (w+ w )t (w v) + (w v)t (w+ w ) + (w v)t (w v)
kw+ w k2 + 2(w+ w )t (w v) + kw vk2
As w v was orthogonal to any vector in W

kw+ vk2 = kw+ w k2 + kw vk2
But w and w+ are both best approximations so
kw+ vk2 = kw vk2
Then
kw+ w k2 = 0
and these best approximations are in fact the same vector.
10.10
Example
10.10.1
Problem
Consider the subspace W spanned by the vectors

1
w1 = 1
0
10.10. EXAMPLE
147
and
w2
0
=
1
We want to find the best least squares approximation in W to the vector

1
2
v =
4
The spanning vectors are linearly independent so
S = {w1 , w2 }
is a basis. As the basis is not orthogonal, we construct an orthogonal basis for W using the
Gram-Schmidt process:
Let
u2
1
u1 = w1 = 1
0

1
1
2
2
w 2 u1
1
1
0
=
u1 =
= w2
u1 u1
2
1
1
0
So an orthogonal basis for W is

1
1
O =
u1 = 1 , u2 = 1
0
1
Then the best least squares approximation in W is given by

v u1
v u2
u1 +
u2
u1 u1
u2 u2
3
3
=
u1 + u2
2 3
1
1
3
1 + 1
=
2
0
1
5/2
= 1/2
1
w =
148
10.10.2
Check
Recall that w v needs to be orthogonal to the basis vectors of W . We have

5/2
1
3/2
w v = 1/2 2 = 3/2
1
4
3
We see that
(w v)t w1 = 0
(w v)t w2 = 0
10.11
Finding a line that best fits data points
10.11.1
Experiment
I believe that a quantity y varies with time according to some law

y = mt + b
and now wish to conduct an experiment to find m and t.
10.11.2
Data points
I measure y at various times t and the measurements are recorded as

y
t
1 1
2 5
4 7
5 11
10.11.3
Trying to find the line
Using my model of y = mt + b and assuming that if I plotted this points they would lie on
the graph of this equation, I get the equations
m1 + b
m2 + b
m4 + b
m5 + b
=
=
=
=
1
5
7
11
10.11. FINDING A LINE THAT BEST FITS DATA POINTS
149
This can be written
1
2
4
5
10.11.4

1
1
5
1
m
=
7
1 b
1
11
Inconsistency
If I form the augmented matrix and do row reduction it turns out that the system is inconsistent. Now what?
10.11.5
Strategy
We will have some measurement in any experiment. So, lets try and find an m and a b that
will give a best fit of the line to the data points.
10.11.6
Seek to minimize square errors
We then seek an m and a b so as to minimize the sum of the differences for each t of mt + b
and the actual measured value:

1
(m + b 1)2 + (2m + b 5)2 + (4m + b 7)2 + (5m + b 11)2
2
Taking partial derivatives
I =
Im = (m + b 1) + 2(2m + b 5) + 4(4m + b 7) + 5(5m + b 11) = 0

Ib = (m + b 1) + (2m + b 5) + (4m + b 7) + (5m + b 11) = 0
or
(m + b) + 2(2m + b) + 4(4m + b) + 5(5m + b) = 1 + 2 5 + 4 7 + 5 11
and
(m + b) + (2m + b) + (4m + b) + (5m + b) = 1 + 5 + 7 + 11
This can be rewritten in terms of matrices as

1 1

1
1 2 4 5
1 2 4 5
2 1 m
5
=
1 1 1 1 4 1 b
1 1 1 1 7
5 1
11
Note that we now have the original system multiplied on both sides by the transpose of the
original matrix. Then we get
150

46 12
m
94
=
12 4
b
24
and so

1

m
46 12
94
11/5
=
=
b
12 4
24
3/5
10.12
Least squares solutions to inconsistent systems
10.12.1
Problem
Suppose that the equation

Ax = b
is inconsistent. We cannot find an x which satisfies the equation so we will try to find one
that is the best fit. That is, we want to minimize the magnitude of the vector
r = Ax b
10.12.2
Solution
We seek to minimize
F (x) = (Ax b)t (Ax b)
Let h be a small deviation about x
F (x + h) =
=
=
=
[A(x + h) b]t [A(x + h) b]

[(Ax b) + Ah]t [(Ax b) + Ah]
(Ax b)t (Ax b) + (Ah)t (Ax b) + O(h2 )
F (x) + ht At (Ax b) + O(h2 )
The term At (Ax b) is a sort of derivative which we will set equal to zero to get a minimum.
So, we want to solve
At (Ax b) = 0 = At Ax = At b
10.12.3
Normal equations
The equations
At Ax = At b
are called the normal equations.
10.13. EXERCISES
10.12.4
151
Properties
a) The normal equations are always consistent.
b) The solutions of the normal equations are the least-squares solutions of Ax = b.
c) If A is m n the solutions of the normal equations are unique if and only if A has
rank n.
10.13
Exercises
10.13.1
Exercise
Fit a line with an equation of the form y = mx + b to the following data:

t y
1 1
2 3
4 3
5 6
10.13.2
Exercise
Fit a a curve with an equation of the form y = ax2 + bx + c to the following data:
t
y
1 3
2 6
3 14
5 30
10.13.3
Exercise
Consider the vector v in R4

1
2
v =
3
4
Let W R4 be the span of the vector
152

4
3
w =
2
1
Find the best least-squares approximation to v in the subspace W .
Chapter 11
Vector Spaces
11.1
Examples
We will illustrate the properties of vector spaces using three example spaces.
11.1.1
Euclidean spaces
Note that linear combinations of two vectors in Rn give vectors in Rn .
11.1.2
Finite degree polynomials
Note that linear combinations of polynomials result in a polynomial.
11.1.3
Solutions to some differential equations
Consider the solutions to the differential equation

y 0 (x) = 0,
x [0, 1]
Note that linear combinations of two solutions is another solution.
11.2
Zero vector
11.2.1
Zero vector in euclidean spaces
The Rn vector spaces have a zero vector which is a column vector with all zero entries.

0
0

0 = ..
.
0
153
154
11.2.2
Zero vector in the polynomial vector space
What would be the zero vector in the polynomial vector space?
11.2.3
Zero vector in solutions of the differential eqn
What would be the zero vector in the vector space of the solutions of
y 0 (x) = 0,
11.2.4
x [0, 1]
Property of the zero vector
For any vector u in the vector space

u+0 = 0+u = u
11.2.5
Exercise
Confirm that the zero vector in each of the three example spaces has the desired property.
11.3
Addition
11.3.1
Addition
If u and v vectors in a vector space V then

u + v also in V
11.3.2
Example
Euclidean spaces.
11.3.3
Example
Adding two polynomials gives another polynomial.
11.3.4
Example
If u and v solutions to
y 0 (x) = 0
x [0, 1]
u0 (x) = 0
x [0, 1]
then
11.4. SCALAR MULTIPLICATION
155
v 0 (x) = 0
x [0, 1]
and
(u + v)0 (x) = u0 (x) + v 0 (x) = 0 + 0 = 0
11.4
11.4.1
x [0, 1]
If u is a vector in a real vector space then for any real number we have that u is a member
of the vector space.
11.4.2
Example
A scalar multiplication was defined so that for any R and any u Rn

u Rn
11.4.3
Example
The scalar multiplication of polynomials is defined as

(c0 + c1 x + c2 x2 + + cn xn ) = c0 + c1 x + c2 x2 + + cn xn
which is a vector in the space of polynomials.
11.4.4
Example
If u is a solution to
y 0 (x) = 0
x [0, 1]
u0 (x) = 0
x [0, 1]
then
and
(u)0 (x) = (u)0 (x) = 0 = 0
x [0, 1]
Then u is also a solution and a vector in the vector space of solutions of the differential
equation
y 0 (x) = 0
x [0, 1]
156
11.5
Properties we have seen for Euclidean spaces
From the way we defined addition and scalar multiplication for vectors in Rn we can show:
11.5.1
Properties

u+v
is also a vector in Rn .
u+v = v+u
3. For u, v, and w vectors in Rn
(u + v) + w = u + (v + w)
4. For u, in Rn there is a zero vector 0 such that
u+0 = u = 0+u
5. For every vector u in Rn there is a vector u in Rn such that
u + (u) = 0
6. For every vector u in Rn and in R there is a vector
u
in Rn .
7. For every u and v in Rn and a real number
(u + v) = u + v
( + )u = u + u
(u) = ()u
10. For u in Rn
1u = u
11.6
Properties of a vector space
11.6.1
Definition
157
A set V is said to be a real vector space if the properties that we just recalled for the Rn
vector spaces hold.
11.6.2
Property 1

u+v
is also a vector in V .
11.6.3
Example
We checked this for the space of polynomials and the space of solutions to the differential
equation previously.
11.6.4
Property 2

u+v = v+u
11.6.5
Example
Addition of polynomial functions is commutative and addition of functions is commutative.
11.6.6
Property 3
3. For u, v, and w vectors in V

(u + v) + w = u + (v + w)
11.6.7
Example
Addition of functions in general is associative so this property holds in particular for polynomial functions and solutions of the differential equation.
158
11.6.8
Property 4
4. For u, in V there is a zero vector 0 such that

u+0 = u = 0+u
11.6.9
Example
We already checked this for the example spaces.
11.6.10
Property 5
5. For every vector u in V there is a vector u in V such that

u + (u) = 0
11.6.11
Example
For a polynomial p defined by

p(x) = c0 + c1 x + c2 x2 + + cn xn
the polynomial p is defined by
(p)(x) = c0 + c1 x + c2 x2 + + cn xn
which is another polynomial and p + p is such that
(p + p)(x) = c0 c0 + (c1 c1 )x + (c2 c2 )x2 + + (cn cn )xn 0
11.6.12
Example
If u is a solution to the differential equation then

u0 (x) = 0
and
(u)p rime(x) = u0 (x) = 0 = 0
so u is also a vector in the space of solutions and
11.6.13
Property 6
6. For every vector u in V and in V there is a vector

u
in V .
11.6.14
159
Example
Already checked these for the example spaces.
11.6.15
Property 7
7. For every u and v in V and a real number

(u + v) = u + v
11.6.16
Example
True for the Euclidean spaces because of the properties of the real numbers. True in general
for functions so specifically for the polynomial functions and the functions that are members
of the solution space.
11.6.17
Property 8

( + )u = u + u
11.6.18
Examples
True for Euclidean spaces by the properties of real numbers. True in general for functions
and so true in particular for the polynomial functions and the functions that are solutions
to the differential equation.
11.6.19
Property 9

(u) = ()u
11.6.20
Examples
True for the Euclidean spaces by the properties of real numbers. True for real valued functions
in general and hence true in particular for the polynomial functions and the functions that
are solutions to the differential equation.
11.6.21
Property 10
10. For u in V
1u = u
160
11.6.22
Examples
True for Euclidean spaces by the properties of real numbers. True in general for real valued
functions by the definition of scalar multiplication.
11.7
Subspaces
11.7.1
Definition
If V is a real vector space, and W is a non-empty subset of V which is also itself a vector
space then W is said to be a subspace of V .
11.7.2
Theorem - Checking if a subset is a subspace
If W is a non-empty subset of vectors of a vector space V then W is a subspace of V if and

only if
a) u, v W implies u + v W
b) R and u W implies u W .
11.7.3
Proof
Exercise.
11.7.4
Example
Consider the subset W of R2 which consists of all of the vectors of the form

x
x2
Is this a subspace?
Consider the addition of two vectors

1
2
3
+
=
/W
1
4
5
W is not a subspace since the sum of two vectors in W is not necessarily in W .
11.7.5
Example
Consider the subset W of the space of all polynomials which consists of all polynomials of
degree 3 or less. This is a subspace because: 1) the sum of two polynomials of degree three
or less is a polynomial of degree three or less; 2) the scalar multiple of a polynomial of degree
three or less is a polynomial of degree three or less.
11.7.6
161
Example
Every vector space has a trivial subspace which is not very interesting.
11.7.7
Example
Solution space of the homogeneous equation Ax = 0.
11.8
Linear combinations
11.8.1
Linear combination of vectors
A vector v in a real vector space V is said to be a linear combination of vectors v1 , v2 , ... ,

vn if there are real numbers 1 , 2 , ... , n such that
v = 1 v1 + 2 v2 + + n vn
11.8.2
Example
In R2 the vector

v =

1
e1 =
0

0
e2 =
1
by
v = e1 + e2
11.8.3
Example
In R2 the vector

v =

1
v1 =
1
162

1
v2 =
1
by
v =
11.8.4
v1 +
v2
2
2
Example
The polynomial function

p(x) = 2 + 5x + 7x2
is a linear combination of the vectors
1, x, x2
11.8.5
Example
In Rn all vectors can be written as linear combinations of the vectors

1
0

e1 = ..
.
0

0
1

e2 = ..
.
0
..
.
en
11.8.6

0
0

= ..
.
1
Theorem - linear combinations form a subspace
Suppose v1 , v2 , ... , vk are vectors in a vector space V .

a) Then the set W of all linear combinations of v1 , v2 , ... , vk is a subspace of V .
b) It is the smallest subspace that contains all the vi .
11.8.7
163
Proof of a)
Suppose that p and q are elements of W . Then p and q are both linear combinations of the
vi :
p = 1 v1 + 2 v2 + + k vk
q = 1 v1 + 2 v2 + + k vk
The sum is
p + q = (1 + 1 )v1 + (2 + 2 )v2 + + (k + k )vk
which is a linear combination of the vi and so in W . So the sum of two vectors in W is also
in W .
Now consider a scalar multiple of p:
p = (1 )v1 + (2 )v2 + + (k )vk
And then this is also a linear combination of the vi and so is in W . So a scalar multiple
of a vector in W is in W . The two required conditions have been checked and so W is a
subspace.
11.8.8
Proof of b)
Suppose a subspace contains vectors v1 , v2 , ... , vk .

One of the conditions for a subspace is that all scalar multiples of vectors in the subspace
must also be in the subspace. Then for all real numbers 1 , 2 , ... , k we must have than
1 v1 , 2 v2 , ... , k vk are also in the subspace.
The other condition for a subspace is that sum of any two vectors in the subspace must
also be in the subspace. Then
1 v1 + 2 v2
is in the subspace. Then
(1 v1 + 2 v2 ) + 3 v3
is in the subspace. And so on till
1 v1 + 2 v2 + + k vk
is also in the subspace.
Thus v1 , v2 , ... , vk in a subspace implies that all linear combinations of the vi are in
the subspace.
164
11.8.9
Example
The set of all of the linear combinations of

1
u =
1
and

v =

1
1
is a subspace. In fact, it is all of R2 .
11.8.10
Span
Suppose a subspace V contains vectors v1 , v2 , ... , vk . The subspace W consisting of all the
linear combinations of the vi is called the space spanned by v1 , v2 , ... , vk and v1 , v2 , ... ,
vk are said to span W .
11.8.11
Example
The span of the polynomial functions p, q, and r defined

p(x) = 1, q(x) = 1 + x, r(x) = 1 + x + x2
is a subspace of the space of polynomial functions.
11.9
Linear independence and dependence
11.9.1
Linear independence
Suppose that v1 , v2 , ... , vk are vectors in a real vector space V . The vi are said to be
linearly dependent if we can find real numbers 1 , 2 , .. , k , not all zero so that
1 v1 + 2 v2 + + k vk = 0
Otherwise the set is said to be linearly independent.
11.9.2
Alternatively
Another way of putting this is that the vi are linearly independent if

1 v1 + 2 v2 + + k vk = 0
only for all the i = 0. Then if the vectors are not linearly independent they are said to be
linearly dependent.
11.10. BACK TO THE FTLA
11.9.3
165
Example
The standard basis vectors in Rn are linearly independent.
11.9.4
Example
The vectors p , q , r in the space of polynomial functions defined by

p(x) = 1 + x
q(x) = 1 x
r(x) = x
are linearly dependent because
p q 2r = 0
11.10
Back to the FTLA
11.10.1
Recall

a) Ann is invertible.
f) |A| =
6 0.
11.10.2
Add to the FTLA
g) The columns of A considered as vectors in Rn are linearly independent.
11.10.3
Proof
Suppose that A is invertible. Then the only solution to Ax = 0 is x = 0. Then the only
linear combination of the columns of A that gives zero is one where the scalar coefficients of
the sum are zero. Then the columns are linearly independent.
166
If the columns are linearly independent then the only linear combination that gives zero
is the one with all scalar coefficients zero. Then Ax = 0 has only the trivial solution and so
by the FTLA then A is invertible.
11.11
Dimension
11.11.1
Example
We saw that the vectors p , q , r in the space of polynomial functions defined by

p(x) = 1 + x
q(x) = 1 x
r(x) = x
are linearly dependent because
p q 2r = 0
Then
1
1
p q
2
2
Then the span of p, q, and r is the same as the span of just p and q.
r =
11.11.2
General idea
Suppose that we are interested in a space W that we know is spanned by a set of vectors
B = {v1 , v2 , . . . , vm }
If the set of vectors is linearly independent, then none of the vectors can be written as a
linear combination of the other vectors. If the set is linearly dependent, then at least one
vector can be written as a linear combination of the others. Say, vm . We can drop vm from
the set and just use
B0 = {v1 , v2 , . . . , vm1 }
We keep going till we get a linearly independent set.
11.11.3
Basis
Such a minimal set is called a basis.
11.12. EXERCISES
11.11.4
167
Minimality of the basis
For a given vector space W with two bases, the number of vectors in both bases is the same.
11.11.5
Example
B = {p, q}
and
B0 = {p, r}
are both bases for the same subspace of the space of polynomial functions.
11.11.6
Dimension
The dimension of a vector space V is the number of vectors in its basis.
11.11.7
Example
Rn has dimension n.
11.11.8
Example
The space of polynomial functions defined on R is of infinite dimension.
11.12
Exercises
11.12.1
Exercise
Give an example of a two-dimensional subspace in the space of polynomial functions.
11.12.2
Exercise
How many different bases are there for R3 ?
11.12.3
Exercise
Can a set of n + 1 vectors in Rn be linearly independent?
168
Chapter 12
Some particular vector spaces
12.1
Domain and range
12.1.1
Recall
Recall that if f is a map from a set U to set V then U is called the domain and the set of
values that f takes are called the range of f .
12.1.2
Example
Suppose f : R R is defined by
f (x) = x2
then the domain of f is R and the range of f is all the non-negative reals.
12.2
Example
12.2.1
Question
Consider the matrix A33
1 4 7
A = 2 5 8
3 6 9
A acts on vectors in R3 and sends them to R3 . Let L : R3 R3 be the linear operator
defined by
L(u) = Au
What is the range of A?
169
170
12.2.2
CHAPTER 12. SOME PARTICULAR VECTOR SPACES
What does the matrix do to the standard basis vectors?
2
L(e1 ) = Ae1 =
3
2
L(e2 ) = Ae2 =
3
2
L(e3 ) = Ae3 =
3
12.2.3

4 7
1
1
5 8
0
2
=
6 9
0
3

4 7
0
4
5 8
1
5
=
6 9
0
6

4 7
0
7
5 8
0
8
=
6 9
1
9
What does the matrix do to a general vector?
Where does A send a general vector? A typical vector in R3 looks like

v = 1 + 2 + 3
so

1
4
7
L(v) = Av = 2 + 5 + 8
3
6
9
12.2.4
Answer
The range of L is the set of vectors that look like

1
4
7
2 + 5 + 8
3
6
9
where , , and are any real numbers.
12.2.5
Note
The range of L is a vector space spanned by the column vectors of A. Is this true in general?
12.3
Column space
12.3.1
Matrix and column vectors
Suppose that we have a matrix Amn . Let L : Rn Rm be the linear transformation defined
by
L(u) = Au
12.3. COLUMN SPACE
171
We write
a11
a21
A = ..
.
a12
a22
..
.
am1 am2
a1n
a2n
..
.
amn
We can think of the matrix as being made up of n column vectors
v1
a11
a21
= ..
.
am1
v2
a12
a22
= ..
.
am2
..
.
vn
a1n
a2n
= ..
.
amn
12.3.2
Action of matrix on basis vectors
The standard basis vectors for Rn are
e1

1
0

= ..
.
0
e2

0
1

= ..
.
0
..
.
172
en

0
0

= ..
.
1
Now we consider the products

L(e1 ) = Ae1 = v1
L(e2 ) = Ae2 = v2
..
.
L(en ) = Aen = vn
12.3.3
Matrix acting on a typical vector
Any vector u Rn can be written as

u = 11 + 22 + + nn
and so
L(u) = Au = 1 v1 + 2 v2 + + n vn
12.3.4
Column space
The range of L is a vector space, namely the span of the column vectors of A. This space is
called the column space of A.
12.3.5
Column space and solutions
For a matrix Amn and column vectors x and b, the matrix-vector equation
Ax = b
is consistent if and only if the vector b is in the column space of A.
12.3. COLUMN SPACE
12.3.6
173
Proof
Suppose that the columns of A are

{v1 , v2 , v3 , , vn }
If b is in the column space of A then b is a linear combination of the columns of A
b = 1 v1 + 2 v2 + + n bn
Let
1
2

x = ..
.
n
Then
(1)
(1)
(1)
(1)
v1
v2
v3
vn
v (2) v (2) v (2) v (2)
n
1
1
2
3
.
.
.
..
2
..
..
..
.
Ax =
..
v (k) v (k) v (k) v (k)
n .
1
2
3
.
..
..
..
..
.
.
. n
(m)
(m)
(m)
(m)
v1
v2
v3
vn
(1)
(1)
(1)
(1)
1 v1 + 2 v2 + 3 v3 + + n vn
v (2) + v (2) + v (2) + + v (2)
2 2
3 3
n n
1 1
.
..
=
(k)
(k)
(k)
(k)
v + v + v + + v
n
1
2
3
n
1
2
3
..
.
(m)
(m)
(m)
(m)
1 v1 + 2 v2 + 3 v3 + + n vn
(1)
(1)
(1)
(1)
1 v1
2 v 2
3 v3
n vn
v (2) v (2) v (2)
v (2)
1 1 2 2 3 3
n n
. . .
.
.. .. ..
..
=
+
+
+
+
(k)
v (k) v (k)
v (k)
v
n
1 1 2 2 3 3
n
. . .
.
.. .. ..
..
(m)
(m)
(m)
(m)
1 v 1
2 v2
3 v3
n vn
= 1 v1 + 2 v2 + + n vn
= b
174
So b in the column space of A implies that Ax = b has a solution.

Now, suppose that Ax = b is consistent. Then there is a vector

1
2

x0 = ..
.
n
so that Ax0 = b. But
Ax0 = 1 v1 + 2 v2 + + n vn = b
so b can be written as a linear combination of the vectors that span the column space of A.
So b is in the column space of A.
12.4
Null space
12.4.1
Null space
Given a matrix Amn , the subspace of all vectors x Rn so that

Ax = 0
is called the null space of A.
12.4.2
The null space is a subspace
The null space is a subspace of Rn . If there are two vectors u and v such that
Au = 0
and
Av = 0
then for any linear combination
w = u + v
of u and v we have that
Aw = A(u + v) = Au + Av = 0 + 0 = 0
so any linear combination of two vectors in the null space is also in the null space. Then the
null space is a subspace.
12.5. SOLUTIONS OF HOMOGENEOUS AND INHOMOGENEOUS SYSTEMS
12.4.3
175
Example
Consider the matrix

A =
1 2
3 4
The null space of A is only the zero vector and nothing else. Why?
12.4.4
Example
Consider the matrix

A =
1 2
2 4
What is the null space of A?

The null space of A consists of all vectors of the form

2s
s

2
= s
1
for any real number s. Then the basis for the null space of A is

2
1
12.5
Solutions of homogeneous and inhomogeneous systems
12.5.1
Homogeneous linear system
Given matrix Amn , the linear system

Ax = 0
is called homogeneous.
12.5.2
Inhomogeneous linear system
Given matrix Amn and vector b 6= 0, the linear system

Ax = b
is called inhomogeneous.
176
12.5.3
Theorem - solns of the inhomogeneous system
If the equation Ax = b is consistent then the solutions can be written in the form
x = xp + 1 v1 + 2 v2 + + k vk
where
B = {v1 , v2 , , vk }
is a basis for the null space of A and xp is a particular solution to the equation Ax = b.
12.5.4
In other words
The general solution to Ax = b is any particular solution plus the null space of A.
12.5.5
Proof in one direction
Suppose that
B = {v1 , v2 , , vk }
is a basis for the null space of A and xp is a solution of Ax = b and x is any solution. Then
Axp = b
and
Ax = b
and so subtracting gives
A(x xp ) = 0
Then as xx0 is in the null space of A we must be able to write it as some linear combination
of the basis vectors of the null space:
x xp = 1 v1 + 2 v2 + + k vk
This implies that every solution of the equation can be written in the form
x = xp + 1 v1 + 2 v2 + + k vk
12.5. SOLUTIONS OF HOMOGENEOUS AND INHOMOGENEOUS SYSTEMS
12.5.6
Proof in the other direction
Suppose that xp is a solution of Ax = b and that

B = {v1 , v2 , , vk }
is a basis for the null space of A. Let
x = xp + 1 v1 + 2 v2 + + k vk
Then
Ax = Axp + 1 Av1 + 2 Av2 + + k Avk = b + 1 0 + 2 0 + + k 0 = b
so a vector of the form
x = xp + 1 v1 + 2 v2 + + k vk
is a solution of the equation.
12.5.7
Particular solution
We call xp a particular solution of Ax = b.
12.5.8
General solution
The vector
x = xp + 1 v1 + 2 v2 + + k vk
is called the general solution of Ax = b.
12.5.9
General solution of the homogeneous equation
The linear combination

1 v1 + 2 v2 + + k vk
is called the general solution to Ax = 0.
12.5.10
Example
Consider the solution of

x1

1 1 1 1
10
0 1 1 1 x2 = 9
x3
0 0 1 1
7
x4
177
178
The rref of the augmented matrix is
1 0 0 0 1
0 1 0 0 2
0 0 1 1 7
which gives a solution

x1
1
1
0
1
0
x2
2
2 0
2
0

x =
x3 = 7 x4 = 7 + x4 = 7 + t 1
x4
x4
0
x4
0
1
where t could be any real number. In the notation that we have been using, the particular
solution of the inhomogeneous problem is

1
2
xp =
7
0
and the general solution of the homogeneous problem is

0
0
xh = t
1
1
and the general solution of the inhomogeneous problem is
x = xp + xh
12.6
Null space and elementary row operations
12.6.1
Question
We might use elementary row operations on a matrix to find the null space of the matrix.
There would be a problem if the null space of the original matrix and the null space of the
row reduced matrix were not the same.
It turns out that the null spaces of a matrix and the matrix multiplied by an elementary
matrix from the left are the same.
12.6.2
Theorem
If E is an elementary matrix and A is a general matrix so that their product EA is defined

then the null space of EA and the null space of A are the same.
12.7. COLUMN SPACE AND ELEMENTARY OPERATIONS
12.6.3
179
Proof
Suppose that u is in the null space of A. That is, Au = 0. Then

(EA)u = E(Au) = E 0 = 0
So, every vector in the null space of A is in the null space of EA. So, the null space of A is
a subset of the null space of EA.
Now, suppose that there is a vector v in the null space of EA. That is, (EA)v = 0. Then
(EA)v = E(Av) = 0
The elementary matrices are all invertible so
Av = E 1 0 = 0
So, every vector in the null space of EA is a vector in the null space of A. Then the null
space of EA is a subset of the null space of A.
Since the null space of A is a subset of the null space of EA and the null space of EA
is a subset of the null space of A, the two spaces are the same.
12.7
Column space and elementary operations
12.7.1
Linear independence of column spaces
Suppose that we have a matrix Amn and B is obtained by A through elementary row
operations. Then the column vectors of A are linearly independent if and only if the column
vectors of B are linearly independent.
12.7.2
Proof
Sufficient to show this for

B = EA
with E an elementary matrix.
Suppose that the column vectors of A are linearly independent. Consider

1
2

u = ..
.
n
Then the vector Au is
180
Au = 1 c1 + 2 c2 + + n cn
where the ci are the column vectors of A.
If the column vectors of A are linearly independent then Au is not zero for any nonzero
u. No elementary matrix can make a nonzero vector zero, so
(EA)u = E(Au) 6= 0
Suppose that A has linearly dependent column vectors. Then there is a nonzero u so that
Au = 0. Then
(EA)u = E(Au) = 0
and so the column vectors of EA are linearly dependent.
12.7.3
Example
Consider the matrix
1
3
A =
2
5
2
8
1
3
5
6
9
1
Its rref is
1
0
0
0
0
1
0
0
0
0
1
0
As the column vectors of the rref are linearly independent the column vectors of the original
matrix A are linearly independent.
12.7.4
Related theorem
A given set of column vectors of A is linearly independent if and only if the corresponding
column vectors of B are linearly independent.
12.7.5
Proof
Work with A modified by dropping any columns not in the set.
12.7. COLUMN SPACE AND ELEMENTARY OPERATIONS
12.7.6
181
Basis of the column space
Suppose that A is a matrix and B is given by performing elementary row operations on A.

A given set of column vectors of A forms a basis for the column space of A if and only
if the corresponding column vectors of B form a basis for the column space of B.
12.7.7
Proof
Sufficient to show this for B = EA where E is an elementary matrix.

Suppose that a given set of column vectors are a basis for the column space of A. Then
that set of column vectors is linearly independent, by the definition of a basis. Then the
corresponding column vectors in B are linearly independent by the previous theorem. Now,
we would like to show that the corresponding column vectors are a basis.
We need to show that every vector in the column space of EA can be written as a linear combination of the corresponding vectors.
Suppose, without loss of generality, that that the first k vectors are a basis for the column space of A and the remaining n k can be written as linear combinations of the first k.
Then any vector in the column space is the image of a vector u mapped by the associated
linear transformation:
Au = 1 c1 + 2 c2 + + k ck + k ck+1 + + n cn
= 1 c1 + 2 c2 + + k ck + k
k
X
ik ci
+ + n
i=1
k
X
in cn
i=1
for some scalars bji . Then a typical vector in the column space of EA is
(EA)u = E(Au) = 1 Ec1 + 2 Ec2 + + k Eck + k
k
X
ik Eci + + n
i=1
k
X
in Ecn
i=1
so the images of the original basis vectors also span the column space of EA.
For the proof in the opposite direction, B = EA means E 1 B = A and note that the
inverse of an elementary matrix is an elementary matrix. Then the same proof can be used
again.
182
12.7.8
RREF and column space
If a matrix A is in rref, then the column vectors with the leading ones of the row vectors
form a basis for the column space of A.
12.7.9
Proof
Exercise.
12.7.10
Example
Consider the matrix
1 1 1 1 1
A = 2 2 3 4 4
3 3 1 3 4
The rref is
1 1 0 0 1/4
0 0 1 0 1/2
0 0 0 1 5/4
So a basis of the column space is

1
0
c1 =
0

0
c3 = 1
0

0
c4 = 0
1
Since this basis would span all of R3 we can say that the range of A is all of R3 .
12.8
Row space
12.8.1
Row space
The row space of a matrix A is the space spanned by the row vectors of A.
12.9. ELEMENTARY OPERATIONS AND THE ROW SPACE
12.8.2
183
Example
Consider the matrix
1 1 1 1
A = 0 0 1 1
0 0 0 2
The row vectors of A are
r1 =

1 1 1 1
r2 =

0 0 1 1
r3 =

0 0 0 2
The row space consists of all the linear combinations

r1 + r2 + r3
12.9
Elementary operations and the row space
12.9.1
Elementary operations do not change the row space
If A is a matrix and B is obtained by A by multiplication by elementary matrices then the

row space of A and the row space of B are the same.
12.9.2
Proof
Sufficient to show for B = EA where E is single elementary matrix.

Suppose that E interchanges matrices. Then the rows of A and B are the same, so the
row spaces do not change.
Suppose that E multiplies one row of A by a non-zero scalar. Then again, the row space
does not change.
Now, suppose that E multiplies the ith row of A by a non-zero scalar and adds it to
the jth row of A, replacing the jth row.
Then if the row space of A is spanned by
{r1 , r2 , , ri , , rj , , rm }
the row space of B is spanned by
184
{r1 , r2 , , ri , , rj + ri , , rm }
which will span the same space.
12.9.3
Row space and rref
If a matrix A is in rref, then the row vectors with the leading ones form a basis for the row
space of A.
12.9.4
Example
Consider the matrix
1
3
A =
2
5
2
8
1
3
5
6
9
1
Its rref is
1
0
0
0
0
1
0
0
0
0
1
0
and so the row space of A is spanned by the basis vectors

r1 =

1 0 0
r2 =

0 1 0
r3 =

0 0 1

Linear Algebra Notes

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Linear Algebra Notes

Uploaded by

Copyright:

Available Formats

Chapter 1

Systems of linear equations

Consider the equation

This time lets consider the solution of two equations

CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

We can now divide the second equation by -6 to solve for x2 .

The situation with nonlinear equations is very different.

1.4. SOLVING SYSTEMS OF LINEAR EQUATIONS

How many solutions does this system have?

Solving systems of linear equations

The basic operations used to solve a system of linear equations are

Consider the system of linear equations

CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

We also often represent the system by an augmented matrix

Apply elementary row operations to solve the system

-2 * Row 1 + Row 2 replaces Row 2

1.5. REDUCED ROW ECHELON FORM

We now read off the solution

which we might also write as a vector

Reduced row echelon form

Consider the matrix

Row echelon form

A matrix is in row echelon form if

CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

Consider the matrix

Consider the matrix

Reduced row echelon form

A matrix is in reduced row echelon form if

Consider the matrix

1.6. GAUSS-JORDAN ELIMINATION

Consider the matrix

The process of using elementary row operations:

CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

An equation of the form

Geometry and solutions

Solutions of systems of linear equations then can be interpreted as intersections of hyperplanes.

Consider the system of equations

Consider the system of equations

Consider the system of equations

Consider the system of linear equations

CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

Consider the system of linear equations

We consider solving a 3x3 linear system:

We get zeros below the first pivot of the second row by

Row echelon form

Rectangular arrays of real numbers as follows.

is an m n matrix with m rows and n columns.

is a 2x2 matrix and

Multiplication by a scalar multiple

2.3. PRODUCTS OF MATRICES

Multiplication is not commutative

2.4. TRANSPOSE OF A MATRIX

Consider the matrix

If A and B are matrices of the same size then

Transpose of a scalar multiple

If A is a matrix and c is a real number then

If A and B are matrices such that the product AB is defined then

Trace of a square matrix

the trace tr( A ) of A is the sum

Some properties of matrices

2.7. PROOFS OF THE PROPERTIES

a(BC) = (aB)C = B(aC)

Proofs of the properties

(A + B)ij = aij + bij