Eigenvalues and eigenvectors

When a matrix multiplies a vector, it generally both rotates and stretches it. But there are some special vectors that the matrix does not rotate, it only multiplies by a number. These are the eigenvectors, and the multiplier is the eigenvalue. This week we learn to find them; on the next page, if there are enough eigenvectors, we will make the matrix diagonal. The computing side is completely familiar: determinants (weeks 4-5) and the null space (week 9).

Golden rules:

1. First the eigenvalue: $\det(\lambda I - A) = 0$.
2. Then for each eigenvalue the eigenspace: solve $(A - \lambda I)\mathbf{x} = \mathbf{0}$.
3. $A - \lambda I$ must come out singular. If there is no zero row, you have found the eigenvalue wrongly.
4. Check: the sum of the eigenvalues is the trace, their product is the determinant.

Definition

Let $A$ be $n \times n$. If, for a nonzero $\mathbf{v}$ and a number $\lambda$,

$A\mathbf{v} = \lambda\mathbf{v}$

then $\lambda$ is an eigenvalue of $A$ and $\mathbf{v}$ is an eigenvector corresponding to $\lambda$.

• $\mathbf{v} = \mathbf{0}$ is never counted as an eigenvector ($A\mathbf{0} = \lambda\mathbf{0}$ for every $\lambda$, which would be meaningless).
• $\lambda = 0$ can be an eigenvalue: it means $A\mathbf{v} = \mathbf{0}$.
• If $\mathbf{v}$ is an eigenvector, so is every $c\mathbf{v}$ ($c \neq 0$). Eigenvectors are not single, they come along a line (or subspace).

Geometry. $A\mathbf{v}$ lies on the same line as $\mathbf{v}$. If $\lambda > 1$ it stretches, if $0 < \lambda < 1$ it shrinks, if $\lambda < 0$ it reverses direction. For the projection onto the $x$-axis in the plane, $\mathbf{e}_1$ corresponds to eigenvalue 1 and $\mathbf{e}_2$ to eigenvalue 0.

The characteristic equation

$A\mathbf{v} = \lambda\mathbf{v} \iff (\lambda I - A)\mathbf{v} = \mathbf{0}$. For a nonzero solution to exist, $\lambda I - A$ must be singular:

$\det(\lambda I - A) = 0.$

The left side is a polynomial of degree $n$ in $\lambda$: the characteristic polynomial. Its roots are exactly the eigenvalues. (Using $\det(A - \lambda I)$ gives the same roots; the sign differs only if $n$ is odd.)

Example 1. $A = \begin{bmatrix} 2 & 3 \\ 2 & 1 \end{bmatrix}$.

$\det(\lambda I - A) = \begin{vmatrix} \lambda - 2 & -3 \\ -2 & \lambda - 1 \end{vmatrix} = (\lambda - 2)(\lambda - 1) - 6 = \lambda^2 - 3\lambda - 4 = (\lambda - 4)(\lambda + 1).$

The eigenvalues are $4$ and $-1$. The $2 \times 2$ shortcut: $\lambda^2 - (\operatorname{tr}A)\lambda + \det A$, here trace 3, det $2 - 6 = -4$.

The eigenspace

All the eigenvectors for a given $\lambda$ plus $\mathbf{0}$ form the null space of $A - \lambda I$. It is called the eigenspace $E_\lambda$ of $\lambda$; it is a subspace.

Example 1 continued:

$\lambda = 4: \quad A - 4I = \begin{bmatrix} -2 & 3 \\ 2 & -3 \end{bmatrix} \xrightarrow{R_2 \leftarrow R_2 + R_1} \begin{bmatrix} -2 & 3 \\ 0 & 0 \end{bmatrix} \Rightarrow -2x + 3y = 0, \quad E_4 = \operatorname{span}\{(3, 2)\}.$

$\lambda = -1: \quad A + I = \begin{bmatrix} 3 & 3 \\ 2 & 2 \end{bmatrix} \xrightarrow{R_2 \leftarrow R_2 - \frac23 R_1} \begin{bmatrix} 3 & 3 \\ 0 & 0 \end{bmatrix} \Rightarrow x + y = 0, \quad E_{-1} = \operatorname{span}\{(1, -1)\}.$

Check: $A(3, 2)^T = (6 + 6, 6 + 2) = (12, 8) = 4(3, 2)$. $A(1, -1)^T = (2 - 3, 2 - 1) = (-1, 1) = -1 \cdot (1, -1)$.

PITFALL · getting a unique solution. If solving $(A - \lambda I)\mathbf{x} = \mathbf{0}$ gives only $\mathbf{x} = \mathbf{0}$, there is a calculation error in the eigenvalue. At a true eigenvalue $A - \lambda I$ is always singular and at least one zero row appears. This is both a warning and a free check.

A 3×3 example and multiplicity

Example 2. $A = \begin{bmatrix} 3 & 2 & -2 \\ 2 & 3 & -2 \\ 2 & 2 & -1 \end{bmatrix}$.

Characteristic polynomial (the calculation is long; using the common structure in the first two columns or expanding along the first row helps): $\lambda^3 - 5\lambda^2 + 7\lambda - 3 = (\lambda - 1)^2(\lambda - 3)$. Check: trace $3 + 3 - 1 = 5 = 1 + 1 + 3$, det $= 1 \cdot 1 \cdot 3 = 3$.

$\lambda = 1: \quad A - I = \begin{bmatrix} 2 & 2 & -2 \\ 2 & 2 & -2 \\ 2 & 2 & -2 \end{bmatrix} \to \begin{bmatrix} 1 & 1 & -1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}: \quad x = -y + z,$

$E_1 = \operatorname{span}\{(-1, 1, 0),\ (1, 0, 1)\} \quad (\text{dimension } 2).$

$\lambda = 3: \quad A - 3I = \begin{bmatrix} 0 & 2 & -2 \\ 2 & 0 & -2 \\ 2 & 2 & -4 \end{bmatrix} \xrightarrow[R_1 \leftarrow \frac12R_1,\ R_2 \leftarrow R_2 - 2R_1]{R_1 \leftrightarrow R_3} \begin{bmatrix} 1 & 1 & -2 \\ 0 & -2 & 2 \\ 0 & 2 & -2 \end{bmatrix} \xrightarrow{R_3 \leftarrow R_3 + R_2} \begin{bmatrix} 1 & 1 & -2 \\ 0 & -2 & 2 \\ 0 & 0 & 0 \end{bmatrix}$

$y = z$, $x = -y + 2z = z$: $E_3 = \operatorname{span}\{(1, 1, 1)\}$.

The two multiplicities:

Always $1 \le \text{geometric} \le \text{algebraic}$. Whether the two are equal is the key to diagonalization (next page).

Shortcuts

If the eigenvalues of $A$ are $\lambda_1, \dots, \lambda_n$ (with multiplicities):

One more item for the invertible matrix theorem: $A$ is invertible $\iff$ 0 is not an eigenvalue of $A$.

There may be no real eigenvalue

The characteristic polynomial of the $90^\circ$ rotation matrix $\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}$ is $\lambda^2 + 1$: no real root. In this course we usually work with real eigenvalues; in week 14 we will see that the eigenvalues of symmetric matrices are always real.

Common mistakes