CSE 736: Markov Chains Lecturer: Hung Q. Ngo SUNY Buffalo, Spring 2003 Scribe: Hung Q. Ngo

Lecture 1: Linear Algebra

Standard texts on Linear Algebra and Algebra are [1, 2].

1 Preliminaries

1.1 Vectors and matrices

We shall use ℝ to denote the set of real numbers and ℂ to denote the set of complex numbers. For any c = a + bi ∈ ℂ, the complex conjugate of c, denoted by c̄ is defined to be c̄ = a − bi. The modulus of c, denoted by |c|, is \sqrt{a^{2} + b^{2}}. It is easy to see that |c|^{2} = cc̄.

If we mention the word “vector” alone, it is understood to be a column vector. An n-dimensional vector x has n entries in some field of numbers, such as ℝ or ℂ:

x = [x_{1} ; x_{2} ; ⋮ ; x_{n}] .

The set of all n-dimensional vectors over ℝ (respectively ℂ) is denoted by ℝ^{n} (respectively ℂ^{n}). They are also called real vectors and complex vectors, respectively.

Similar to vectors, matrices need an underlying field. We thus have complex matrices and real matrices just as in the case of vectors. In fact, an n-dimensional vector is a n × 1 matrix. In the discussion of the following paragraph, the concepts of complex conjugates, transposes, and conjugate transposes also apply to vectors in this sense.

Given an m × n matrix A = (a_{ij}), the complex conjugate Ā of A is a matrix obtained from A by replacing each entry a_{ij} of A by the corresponding complex conjugate ā_{ij}. The transpose A^{T} of A is the matrix obtained from A by turning its rows into columns and vice versa. For example,

A = [\begin{matrix}0 & 3 & 1 \\ −2 & 0 & 1 \end{matrix}] , A^{T} = [\begin{matrix}0 & −2 \\ 3 & 0 \\ 1 & 1 \end{matrix}] .

The conjugate transpose A^{∗} of A is defined to be Ā^{T}. A square matrix A is symmetric if A = A^{T}, and it is Hermitian if A = A^{∗}.

Given a real vector x ∈ ℝ^{n}, the length ∥x∥ of x is

∥x∥ = \sqrt{x_{1}^{2} + x_{2}^{2} + ⋯ + x_{n}^{2}}.  (1)

Notice that ∥x∥^{2} = xx^{T}. When x is a complex vector, we use x^{∗} instead of x^{T}. Hence, in general we define ∥x∥ = \sqrt{xx^{∗}} = \sqrt{x^{∗}x}. (You should check that xx^{∗} = x^{∗}x, and that it is a real number so that the square root makes sense.)

The length ∥x∥ is also referred to as the L_{2}-norm of vector x, denoted by ∥x∥_{2}. In general, the L_{p}-norm ∥x∥_{p} of an n-dimensional vector x, p = 1, 2, …, is defined to be

∥x∥_{p} := (|x_{1}|^{p} + ⋯ + |x_{n}|^{p})^{\frac{1}{p}} ,  (2)

and

∥x∥_{∞} := max_{i=1..n} |x_{i}|.  (3)

The following identities are easy to show, yet of great importance. Given an p × q matrix A and an q × r matrix B, we have

(AB)^{T} = B^{T} A^{T}  (4)
(AB)^{∗} = B^{∗}A^{∗}  (5)

(Question: what’s the dimension of (AB)^{T} and (AB)^{∗}?)

A square matrix A is said to be singular if there is no unique solution to the equation Ax = b. It does not matter what b is. The uniqueness of solutions to Ax = b is an intrinsic property of A alone. If there is one and only one x such that Ax = b, then A is said to be non-singular.

1.2 Determinant and trace

Given a square matrix A = (a_{ij}) of order n, the equation Ax = 0 has a unique solution x if and only if det A ≠ 0, where det A denotes the determinant of A, which is defined by

det A = ∑_{π∈S_{n}}(−1)^{sign(π)} ∏_{i=1}^{n} a_{iπ(i)}.  (6)

Here, S_{n} denotes the set of all permutations on the set [n] = {1, …, n}. (S_{n} is more often referred to as the symmetric group of order n.) Given a permutation π ∈ S_{n}, the number of inversions of π is the number of pairs (π(i), π(j)) for which i < j and π(i) > π(j). The sign of a permutation π, denoted by sign(π) is −1 if there are an odd number of inversions, and 1 if there are an even number of inversions.

Let us take an example for n = 3. In this case S_{n} consists of 6 permutations:

S_{n} = {123, 132, 213, 231, 312, 321}.

Notationally, if π = 132, then π(1) = 1, π(2) = 3, and π(3) = 2; and, sign(π) = −1 since there is only one “out-of-order” pair (3, 2) in the permutation. To be more precise, sign(123) = 1, sign(132) = −1, sign(312) = 1, sign(213) = −1, sign(231) = 1, sign(321) = −1.

With

A = [\begin{matrix}0 & 3 & 1 \\ −2 & 0 & 1 \\ −1 & 2 & 2 \end{matrix}]

we have

det A = a_{11}a_{22}a_{33} + (−1)a_{11}a_{23}a_{32} + (−1)a_{12}a_{21}a_{33} + a_{12}a_{23}a_{31} +
a_{13}a_{21}a_{32} + (−1)a_{13}a_{22}a_{31}
= 0 · 0 · 2 + (−1) · 0 · 1 · 2 + (−1) · 3 · (−2) · 2 + 3 · 1 · (−1) +
1 · (−2) · 2 + (−1) · 1 · 0 · (−1)
= 5

The trace of a square matrix A, denoted by tr A is the sum of its diagonal entries. The matrix A above has

tr A = 0 + 0 + 2 = 2.

1.3 Combinations of vectors and vector spaces

A vector w is a linear combination of m vectors v_{1}, …, v_{m} if w can be written as

w = a_{1}v_{1} + a_{2}v_{2} + … a_{m}v_{m}.  (7)

The number a_{i} is called the coefficient of the vector v_{i} in this linear combination. Note that, as usual, we have to fix the underlying field such as ℝ or ℂ. If, additionally, we also have a_{1} + a_{2} + ⋯ + a_{m} = 1, then w is called an affine combination of the v_{i}.

A canonical combination is a linear combination in which λ_{j} ≥ 0, ∀j; and a convex combination is an affine combination which is also canonical. The linear (affine, canonical, convex) hull of {v_{1}, …, v_{m}} is the set of all linear (affine, canonical, convex) combinations of the v_{j}. Note that in the above definitions, m could be infinite. The convex hull of a finite set of vectors is called a cone, or more specifically a convex polyhedral cone.

A real vector space is a set V of real vectors so that a linear combination of any subset of vectors in V is also in V. In other words, vector spaces have to be closed under taking linear combinations. Technically speaking, this is an incomplete definition, but it is sufficient for our purposes. One can also replace the word “real” by “complex”. A subspace of a vector space V is a subset of V which is closed under taking linear combinations.

Given a set V = {v_{1}, …, v_{m}} of vectors, the set of all linear combinations of the v_{j} forms a vector space, denoted by span (V), or span (v_{1}, …, v_{m}). The column space of a matrix A is the span of its column vectors. The row space of A is defined similarly. Note that equation Ax = b (with A not necessarily a square matrix) has a solution if and only if b lies in the column space of A. The coordinates of x form the coefficients of the column vectors of A in a linear combination to form b.

A set V = {v_{1}, …, v_{m}} of (real, complex) vectors is said to be linearly independent if

a_{1}v_{1} + a_{2}v_{2} + … a_{m}v_{m} = 0 only happens when a_{1} = a_{2} = … a_{m} = 0.

Otherwise, the vectors in V are said to be (linearly) dependent.

The dimension of a vector space is the maximum number of linearly independent vectors in the space. The basis of a vector space V is a subset {v_{1}, …, v_{m}} of V which is linearly independent and span (v_{1}, …, v_{m}) = V. It is easy to show that m is actually the dimension of V. A vector space typically has infinitely many bases. All bases of a vector space V have the same size, which is also the dimension of V.

The dimensions of a matrix A’s column space and row space are equal, and is referred to as the rank of A. This fact is not very easy to show, but not difficult either. Gaussian elimination is of great use here.

The sets ℝ^{n} and ℂ^{n} are vector spaces by themselves. A set of m vectors in ℝ^{n} must be linearly dependent if m > n. Similarly, a set of m vectors in ℂ^{n} must be linearly dependent if m > n.

Given a basis B of V and some vector v ∈ V, there is exactly one way to write v as a linear combination of vectors in B. (Why?)

1.4 Inverses

Given a square matrix A, the inverse of A, denoted by A^{−1} is a matrix B such that

AB = BA = I, or AA^{−1} = A^{−1}A = I.

If A and B both have inverses, then the inverse of AB can be calculated easily:

(AB)^{−1} = B^{−1}A^{−1}.  (8)

The same rule holds for 3 or more matrices. For example,

(ABCD)^{−1} = D^{−1}C^{−1}B^{−1}A^{−1}.

If A has an inverse, it is said to be invertible. Not all matrices are invertible. There are many conditions to test if a matrix has an inverse, including: non-singularity, non-zero determinant, non-zero eigenvalues, linearly independent column vectors, linearly independent row vectors.

2 Eigenvalues and eigenvectors

In this section, we shall be concerned with square matrices only, unless stated otherwise. We shall use I to denote the identity matrix.

The eigenvalues of a matrix A are the numbers λ such that the equation Ax = λx, or (λI −A)x = 0, has a non-zero solution vector, in which case the solution vector x is the corresponding eigenvector.

Since the all-0 vector, denoted by 0, is always a solution, it would be the only solution if det(λI − A) ≠ 0. Hence, the eigenvalues are solutions to the equation det(λI − A) = 0, i.e. p_{A}(λ) = 0 where

p_{A}(λ) := det(λI − A)

is called the characteristic polynomial of A. For example, if

A = [\begin{matrix}2 & 1 \\ −2 & 3 \end{matrix}] ,

then,

p_{A}(λ) = det [\begin{matrix}λ − 2 & −1 \\ +2 & λ − 3 \end{matrix}] = (λ − 2)(λ − 3) + 2 = λ^{2} − 5λ + 8.

If we work on the complex numbers, then equation p_{A}(λ) = 0 always has n roots (up to multiplicities). However, we shall be concerned mostly with matrices which have real eigenvalues. We shall establish sufficient conditions for a matrix to have real eigenvalues, as shall be seen in later sections.

Theorem 2.1. Let λ_{1}, …, λ_{n} be the eigenvalues of an n × n complex matrix A, then

(i) λ_{1} + ⋯ + λ_{n} = tr A.

(ii) λ_{1} … λ_{n} = det A.

Proof. In the complex domain, p_{A}(λ) has n complex roots since it is a polynomial of degree n. The eigenvalues λ_{1}, …, λ_{n} are the roots of p_{A}(λ). Hence, we can write

p_{A}(λ) = ∏_{i}(λ − λ_{i}) = λ^{n} + c_{n−1}λ^{n−1} + ⋯ + c_{1}λ + c_{0}.

It is evident that

c_{n−1} = − (λ_{1} + ⋯ + λ_{n})
c_{0} = (−1)^{n}λ_{1} … λ_{n}.

On the other hand, by definition we have

p_{A}(λ) = det [\begin{matrix}λ − a_{11} & −a_{12} & … & −a_{1n} \\ −a_{21} & λ − a_{22} & … & −a_{2n} \\ ⋮ & … & … & ⋮ \\ −a_{n1} & −a_{n2} & … & λ − a_{nn} \end{matrix}] .

Expanding p_{A}(λ) in this way, the coefficient of λ^{n−1} (which is c_{n−1}) is precisesly −(a_{11} + a_{22} + ⋯ + a_{nn}); and the coefficient of λ^{0} (which is c_{0}) is (−1)^{n} det A (think carefully about this statement!).

2.1 The diagonal form

Proposition 2.2. Suppose the n × n matrix A has n linearly independent eigenvectors x_{1}, …, x_{n}. Let S be the matrix whose columns are the vectors x_{i}, then S^{−1}AS = Λ where Λ is a diagonal matrix with (Λ)_{ii} = λ_{i}.

Proof. The diagonal matrix with λ_{1}, …, λ_{n} as diagonal entries is often denoted by diag (λ_{1}, …, λ_{n}). Note that since the column vectors of S are independent, S is invertible and S^{−1} makes sense. We want to show S^{−1}AS = Λ, which is the same as showing AS = SΛ. Since Ax_{i} = x_{i}λ_{i}, it follows that

AS = A [\begin{matrix}| & … & | \\ x_{1} & … & x_{n} \\ | & … & | \end{matrix}] = [\begin{matrix}| & … & | \\ Ax_{1} & … & Ax_{n} \\ | & … & | \end{matrix}] = [\begin{matrix}| & … & | \\ λ_{1}x_{1} & … & λ_{n}x_{n} \\ | & … & | \end{matrix}] = SΛ.

In general, if a matrix S satisfies the property that S^{−1}AS is a diagonal matrix, then S is said to diagonalize A, and A is said to be diagonalizable. It is easy to see from the above proof that if A is diagonalizable by S, then the columns of S are eigenvectors of A; moreover, since S is invertible by definition, the columns of S must be linearly independent. In other words, a matrix is diagonalizable if and only if it has n inpdependent eigenvectors.

Proposition 2.3. If x_{1}, … x_{k} are eigenvectors corresponding to distinct eigenvalues λ_{1}, … λ_{k}, then x_{1}, … x_{k} are linearly independent.

Proof. When k = 2, suppose c_{1}x_{1} + c_{2}x_{2} = 0. Multiplying by A gives c_{1}λ_{1}x_{1} + c_{2}λ_{2}x_{2} = 0. Subtracting λ_{2} times the previous equation we get

c_{1}(λ_{1} − λ_{2})x_{1} = 0.

Hence, c_{1} = 0 since λ_{1} ≠ λ_{2} and x_{1} ≠ 0. The general case follows trivially by induction.

Proposition 2.4. If λ_{1}, … λ_{n} are eigenvalues of A, then λ_{1}^{k}, … λ_{n}^{k} are eigenvalues of A^{k}. If S diagonalizes A, i.e. S^{−1}AS = Λ, then S^{−1}A^{k}S = Λ^{k}

Proof. Trivial.

2.2 Symmetric and Hermitian matrices

For any two vectors x, y ∈ ℂ^{n}, the inner product of x and y is defined to be

x^{∗}y = x̄^{T} y = x̄_{1}y_{1} + ⋯ + x̄_{n}y_{n}

Two vectors are orthogonal to one another if their inner product is 0. The vector 0 is orthogonal to all vectors. Two orthogonal non-zero vectors must be linearly independent. For, if x^{∗}y = 0 and ax+by = 0, then 0 = ax^{∗}x + bx^{∗}y = ax^{∗}x. This implies a = 0, which in turns implies b = 0 also. With the same reasoning, one easily shows that a set of pairwise orthogonal non-zero vectors must be linearly independent.

If A is any complex matrix, recall that the Hermitian transpose A^{∗} of A is defined to be Ā^{T}, and that A is said to be Hermitian if A = A^{∗}. A real matrix is Hermitian if and only if it is symmetric. Also notice that the diagonal entries of a Hermitian matrix must be real, because they are equal to their respective complex conjugates. The next lemma lists several useful properties of a Hermitian matrix.

Lemma 2.5. Let A be a Hermitian matrix, then (i) for all x ∈ ℂ^{n}, x^{∗}Ax is real. (ii) every eigenvalue of A is real. (iii) the eigenvectors of A, if come from distinct eigenvalues, are orthogonal to one another.

Proof. It is straightforward that (i) (x^{∗}Ax)^{∗} = x^{∗}A^{∗}x^{∗∗} = x^{∗}Ax. (ii) Ax = λx implies λ = \frac{x^{∗}Ax}{x^{∗}x}. (iii) Suppose Ax = λ_{1}x, Ay = λ_{2}y, and λ_{1} ≠ λ_{2}, then

(λ_{1}x)^{∗}y = (Ax)^{∗}y = x^{∗}Ay = x^{∗}(λ_{2}y).

Hence, (λ_{1} − λ_{2})x^{∗}y = 0, implying x^{∗}y = 0.

2.3 Orthonormal and unitary matrices

A real matrix Q is said to be orthogonal if Q^{T} Q = I. A complex matrix U is unitary if U^{∗}U = I. In other words, the columns of U (and Q) are orthonormal. Clearly being orthogonal is a special case of being unitary. We state without proof a simple proposition. Proposition 2.6. Let U be a unitary matrix, then

(i) (Ux)^{∗}(Uy) = x^{∗}y, and ∥Ux∥^{2} = ∥x∥^{2}. (ii) Every eigenvalue λ of U has modulus 1 (i.e. |λ| = λ^{∗}λ = 1). (iii) Eigenvectors corresponding to distinct eigenvalues of U are orthogonal. (iv) If U′ is another unitary matrix, then UU′ is unitary.

2.4 The Spectral theorem

Two matrices A and B are said to be similar iff there is an invertible matrix M such that M^{−1}AM = B. Thus, a matrix is diagonalizable iff it is similar to a diagonal matrix. Similarity is obviously an equivalent relation. The following proposition shows what is common among matrices in the same similarity equivalent class. Proposition 2.7. If B = M^{−1}AM, then A and B have the same eigenvalues. An eigenvector x of A corresponds to an eigenvector M^{−1}x of B. Proof. Ax = λx implies (M^{−1}A)x = λM^{−1}x, or (BM^{−1})x = λ(M^{−1}x).

An eigenvector corresponding to an eigenvalue λ is called a λ-eigenvector. The vector space spanned by all λ-eigenvectors is called the λ-eigenspace. We shall often use V_{λ} to denote this space if the underlying matrix is clear from context. Corollary 2.8. If A and B are similar, then the corresponding eigenspaces of A and B have the same dimension.

Proof. Suppose B = M^{−1}AM, then the mapping ϕ : x → M^{−1}x is an invertible linear transformation from one eigenspace of A to the corresponding eigenspace of B.^{1}

If two matrices A and B are similar, then we can say a lot about A if we know B. Hence, we would like to find B similar to A where B is as “simple” as possible. The first “simple” form is the upper-triangular form, as shown by the following Lemma, which is sometime referred to as the Jacobi Theorem.

Lemma 2.9 (Schur’s lemma). For any n × n matrix A, there is a unitary matrix U such that B = U^{−1}AU is upper triangular. Hence, the eigenvalues of A are on the diagonal of B.

Proof. We show this by induction on n. The lemma holds when n = 1. When n > 1, over ℂ A must have at least one eigenvalue λ_{1}. Let x′_{1} be a corresponding eigenvector. Use Gram-Schmidt process to extend x′_{1} to an orthonormal basis {x_{1}, x_{2}, …, x_{n}} of ℂ^{n}. Let U_{1} be the matrix whose columns are these vectors in order. From the fact that U_{1}^{−1} = U_{1}^{∗}, it is easy to see that

U_{1}^{−1}AU_{1} = [\begin{matrix}λ_{1} & & ∗ & & ∗ & & … & & ∗ \\ 0 & & ∗ & & ∗ & & … & & ∗ \\ 0 & & ∗ & & ∗ & & … & & ∗ \\ . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . \\ 0 & & ∗ & & ∗ & & … & & ∗ \end{matrix}] .

Now, let A′ = (U_{1}^{−1}AU_{1})_{11} (crossing off row 1 and column 1 of U_{1}^{−1}AU_{1}). Then by induction there exists an (n − 1) × (n − 1) unitary matrix M such that M^{−1}A′M is upper triangular. Let U_{2} be the n × n matrix obtained by adding a new row and new column to M with all new entries equal 0 except (U_{2})_{11} = 1. Clearly U_{2} is unitary and U_{2}^{−1}(U_{1}^{−1}AU_{1})U_{2} is upper triangular. Letting U = U_{1}U_{2} completes the proof.

The following theorem is one of the most important theorems in elementary linear algebra, beside the Jordan form.

Theorem 2.10 (Spectral theorem). Every real symmetric matrix can be diagonalized by an orthogonal matrix, and every Hermitian matrix can be diagonalized by a unitary matrix:

(real case) Q^{−1}AQ = Λ (complex case) U^{−1}AU = Λ

Moreover, in both cases all the eigenvalues are real.

Proof. The real case follows from the complex case. Firstly, by Schur’s lemma there is a unitary matrix U such that U^{−1}AU is upper triangular. Moreover,

(U^{−1}AU)^{∗} = U^{∗}A^{∗}(U^{−1})^{∗} = U^{−1}AU,

i.e. U^{−1}AU is also Hermitian. But an upper triangular Hermitian matrix must be diagonal. The realness of the eigenvalues follow from Lemma 2.5.

References

[1] M. Artin, Algebra, Prentice-Hall Inc., Englewood Cliffs, NJ, 1991.

[2] G. Strang, Linear algebra and its applications, Academic Press [Harcourt Brace Jovanovich Publishers], New York, second ed., 1980.

1I have not define linear transformation yet. The thing to remember is that if there is an invertible linear transformation from one vector space to another, then the two vector spaces have the same dimension. Invertible linear transformations are like isomorphisms or bijections, in some sense. A curious student should try to prove this fact directly without using the term linear transformation.