Transformation matrices, transition matrices and similarity

Every linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$ is multiplication by a matrix. But which matrix? The answer depends on the basis you choose. In the standard basis the matrix of a transformation may look complicated; in another basis the same transformation can be diagonal, that is, just “stretching the axes”. This week we learn to find the matrix, to convert coordinates between bases, and the similarity relation that links the matrices of the same transformation in different bases. Next week’s diagonalization is built directly on this.

Golden rules:

1. The columns of the transformation matrix are the images of the basis vectors.
2. In a composition the matrix on the right is applied first.
3. Transition matrix $B \to B'$: turn $[B' \mid B]$ into $[I \mid P]$ with row operations. The target basis is on the left.
4. $A' = P^{-1}AP$, where the columns of $P$ are the vectors of the new basis.

The standard matrix

If $T : \mathbb{R}^n \to \mathbb{R}^m$ is linear, then

$A = \big[\, T(\mathbf{e}_1) \mid T(\mathbf{e}_2) \mid \cdots \mid T(\mathbf{e}_n) \,\big]$

is the $m \times n$ matrix that satisfies $T(\mathbf{x}) = A\mathbf{x}$ for every $\mathbf{x}$. Why: $\mathbf{x} = x_1\mathbf{e}_1 + \dots + x_n\mathbf{e}_n$, linearity gives $T(\mathbf{x}) = x_1T(\mathbf{e}_1) + \dots + x_nT(\mathbf{e}_n)$; and this is the column-combination interpretation of $A\mathbf{x}$.

Example 1. $T(x, y, z) = (x - 2y,\ 2x + z)$.

$T(\mathbf{e}_1) = (1, 2), \quad T(\mathbf{e}_2) = (-2, 0), \quad T(\mathbf{e}_3) = (0, 1) \quad\Rightarrow\quad A = \begin{bmatrix} 1 & -2 & 0 \\ 2 & 0 & 1 \end{bmatrix}.$

Check: $A(1, 1, 1)^T = (-1, 3)$ and $T(1, 1, 1) = (1 - 2, 2 + 1) = (-1, 3)$.

Composition and inverse

Let $T_1 : \mathbb{R}^n \to \mathbb{R}^m$ (matrix $A_1$), $T_2 : \mathbb{R}^m \to \mathbb{R}^p$ (matrix $A_2$). The matrix of the composition $T_2 \circ T_1$, which applies $T_1$ first and then $T_2$, is

$A_2A_1.$

The order is as in function composition: $T_2(T_1(\mathbf{x})) = A_2(A_1\mathbf{x})$.

Example 2. First a $90^\circ$ rotation $R = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}$, then a reflection about $y = x$, $F = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$:

$FR = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix} \ (\text{reflection about the } x\text{-axis}), \qquad RF = \begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix} \ (\text{reflection about the } y\text{-axis}).$

When the order changed, the result changed. The geometric face of the non-commutativity of matrix multiplication.

Inverse transformation. If the standard matrix $A$ of $T$ is invertible, $T$ is invertible and the matrix of $T^{-1}$ is $A^{-1}$ (Q4).

Matrix relative to nonstandard bases

Let $T : V \to W$, with a basis $B = \{\mathbf{v}_1, \dots, \mathbf{v}_n\}$ of $V$ and a basis $B'$ of $W$. The matrix of $T$ relative to $B$ and $B'$ is:

$[T]_{B',B} = \big[\, [T(\mathbf{v}_1)]_{B'} \mid \cdots \mid [T(\mathbf{v}_n)]_{B'} \,\big], \qquad [T(\mathbf{v})]_{B'} = [T]_{B',B}\,[\mathbf{v}]_B.$

Recipe: transform each basis vector, write the result in the coordinates of the target basis, make it a column.

The same for polynomials: the derivative $D : P_2 \to P_1$ relative to the standard bases is $\begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 2 \end{bmatrix}$. The coordinates of $5 + 3x - x^2$ are $(5, 3, -1)$; multiplying gives $(3, -2)$, that is, the derivative $3 - 2x$. We turned calculus into matrix multiplication.

The transition matrix (changing coordinates)

How are the coordinates of the same vector $\mathbf{v}$ in two different bases related? Let $B$ and $B'$ be two bases of $\mathbb{R}^n$. The transition matrix from $B$ to $B'$ $P$ is the matrix that satisfies, for every $\mathbf{v}$,

$[\mathbf{v}]_{B'} = P\,[\mathbf{v}]_B.$

Its columns are the coordinates of the vectors of $B$ relative to $B'$.

Computation: $[B' \mid B] \to [I \mid P]$. Write the bases as column matrices (the target basis $B'$ on the left) and apply Gauss-Jordan.

Example 3. $B = \{(1, 2), (1, 3)\}$, $B' = \{(1, 0), (1, 1)\}$.

$[B' \mid B] = \left[\begin{array}{cc|cc} 1 & 1 & 1 & 1 \\ 0 & 1 & 2 & 3 \end{array}\right] \xrightarrow{R_1 \leftarrow R_1 - R_2} \left[\begin{array}{cc|cc} 1 & 0 & -1 & -2 \\ 0 & 1 & 2 & 3 \end{array}\right] \quad\Rightarrow\quad P = \begin{bmatrix} -1 & -2 \\ 2 & 3 \end{bmatrix}.$

Check: if $[\mathbf{v}]_B = (1, 1)$ then $\mathbf{v} = (1, 2) + (1, 3) = (2, 5)$. $P(1, 1)^T = (-3, 5)$ and $-3(1, 0) + 5(1, 1) = (2, 5)$. The same vector.

As a formula, $P = (B')^{-1}B$ (here $B$ and $B'$ are column matrices). The transition matrix in the opposite direction is $P^{-1} = \begin{bmatrix} 3 & 2 \\ -2 & -1 \end{bmatrix}$.

PITFALL · mixing up the direction. $[B' \mid B]$ gives the transition $B \to B'$; $[B \mid B']$ gives $B' \to B$. Memorize “the target basis on the left”. If you are not sure, check one column: column 1 of $P$ must be the $B'$-coordinates of the 1st vector of $B$.

Similar matrices

Let $A$ be the standard matrix of $T : \mathbb{R}^n \to \mathbb{R}^n$ and $A'$ its matrix relative to the basis $B$. If $P$ is the matrix whose columns are the vectors of $B$ (the transition from $B$ to the standard basis), then

$A' = P^{-1}AP.$

The reading order is right to left: convert the $B$-coordinates to standard ($P$), apply $T$ in the standard basis ($A$), convert the result back to $B$-coordinates ($P^{-1}$).

Definition. Square matrices $A$ and $A'$ are similar if there is an invertible $P$ with $A' = P^{-1}AP$. Similar matrices are photographs of the same transformation in different bases.

Example 4. $A = \begin{bmatrix} 3 & 1 \\ 0 & 2 \end{bmatrix}$, $B = \{(1, 0), (1, -1)\}$, $P = \begin{bmatrix} 1 & 1 \\ 0 & -1 \end{bmatrix}$, $P^{-1} = \begin{bmatrix} 1 & 1 \\ 0 & -1 \end{bmatrix}$.

$AP = \begin{bmatrix} 3 & 2 \\ 0 & -2 \end{bmatrix}, \qquad A' = P^{-1}(AP) = \begin{bmatrix} 3 + 0 & 2 - 2 \\ 0 & 2 \end{bmatrix} = \begin{bmatrix} 3 & 0 \\ 0 & 2 \end{bmatrix}.$

In this basis $T$ multiplies the first basis vector by 3 and the second by 2. Direct check: $A(1, -1)^T = (2, -2) = 2(1, -1)$. Next week we will call these basis vectors eigenvectors.

What similarity preserves

$\det(P^{-1}AP) = \det P^{-1}\det A\det P = \det A$. What is preserved in the same way:

To show matrices are not similar, one differing invariant (trace, det, rank) is enough (Q10). Equal trace and det, however, do not prove similarity.

Common mistakes