Matrix operations and their properties

This week we turn the matrix from a table of numbers into an algebraic object: we will add, multiply and take powers. Most of the rules are familiar from numbers. Exam questions come exactly from the places that are not familiar: when a product is defined, why $AB$ is not equal to $BA$, why $(AB)^T$ is $B^TA^T$.

Golden rules:

1. Before a product, write the sizes: $(m \times n)(n \times p) = m \times p$. If the inner dimensions are not equal, the product does not exist.
2. $(AB)_{ij}$ = row $i$ of $A$ · column $j$ of $B$. If only one entry is asked, compute only that.
3. $AB \neq BA$ in general. Leave the habit “order doesn’t matter” from numbers behind here.
4. Transpose and inverse reverse the order in a product: $(AB)^T = B^TA^T$.

Definitions and notation

A matrix with $m$ rows and $n$ columns is called an $m \times n$ matrix. The element in row $i$, column $j$ is called $a_{ij}$ (entry). The notation $A = [a_{ij}]$ expresses this.

Two matrices are equal only if they have the same size and every corresponding entry is equal. So a “matrix equation” is really as many numerical equations as there are entries (Q6).

Addition and scalar multiplication

Two matrices of the same size are added entry by entry. Multiplying by a number $c$ multiplies every entry by $c$.

$2\begin{bmatrix} 1 & -2 \\ 3 & 0 \end{bmatrix} - 3\begin{bmatrix} 2 & 1 \\ -1 & 4 \end{bmatrix} = \begin{bmatrix} 2 & -4 \\ 6 & 0 \end{bmatrix} - \begin{bmatrix} 6 & 3 \\ -3 & 12 \end{bmatrix} = \begin{bmatrix} -4 & -7 \\ 9 & -12 \end{bmatrix}$

These operations obey the same rules as numbers: $A + B = B + A$, $(A + B) + C = A + (B + C)$, $c(A + B) = cA + cB$, $(c + d)A = cA + dA$.

Matrix product

If $A$ is $m \times n$ and $B$ is $n \times p$, then $AB$ is an $m \times p$ matrix and

$(AB)_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + \dots + a_{in}b_{nj}.$

That is, you multiply row $i$ of $A$ with column $j$ of $B$ entry by entry and add. Example:

$A = \begin{bmatrix} 1 & 2 & -1 \\ 0 & 3 & 4 \end{bmatrix}_{2 \times 3}, \quad B = \begin{bmatrix} 2 & 1 \\ -1 & 0 \\ 3 & 5 \end{bmatrix}_{3 \times 2}$

AB (2x2):
  (1,1): (1, 2, -1)·(2, -1, 3) = 2 - 2 - 3 = -3
  (1,2): (1, 2, -1)·(1, 0, 5)  = 1 + 0 - 5 = -4
  (2,1): (0, 3, 4)·(2, -1, 3)  = 0 - 3 + 12 = 9
  (2,2): (0, 3, 4)·(1, 0, 5)   = 0 + 0 + 20 = 20

$AB = \begin{bmatrix} -3 & -4 \\ 9 & 20 \end{bmatrix}, \qquad BA = \begin{bmatrix} 2 & 7 & 2 \\ -1 & -2 & 1 \\ 3 & 21 & 17 \end{bmatrix}$

Both are defined but one is $2 \times 2$ and the other $3 \times 3$. Even for square matrices they are generally not equal:

$C = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix},\ D = \begin{bmatrix} 1 & 0 \\ 1 & 1 \end{bmatrix}: \quad CD = \begin{bmatrix} 2 & 1 \\ 1 & 1 \end{bmatrix} \neq DC = \begin{bmatrix} 1 & 1 \\ 1 & 2 \end{bmatrix}.$

When $AB = BA$, $A$ and $B$ are said to commute. This is a special case, not the default.

Writing a system as a matrix

A system of $m$ equations in $n$ unknowns becomes a single matrix equation: $A\mathbf{x} = \mathbf{b}$. And $A\mathbf{x}$ has a nice interpretation: it is a linear combination of the columns of $A$, with the components of $\mathbf{x}$ as coefficients.

$\begin{bmatrix} 1 & 2 \\ 3 & -1 \\ 0 & 4 \end{bmatrix}\begin{bmatrix} 2 \\ -1 \end{bmatrix} = 2\begin{bmatrix} 1 \\ 3 \\ 0 \end{bmatrix} - 1\begin{bmatrix} 2 \\ -1 \\ 4 \end{bmatrix} = \begin{bmatrix} 0 \\ 7 \\ -4 \end{bmatrix}$

This view will come back in weeks 7 and 9 (span, column space): $A\mathbf{x} = \mathbf{b}$ is consistent if and only if $\mathbf{b}$ is a combination of the columns of $A$.

Properties of the product

What is true (when the sizes fit):

What is true for numbers but false for matrices:

PITFALL · cancelling on both sides. When you see $AB = AC$ you can’t “cancel the $A$‘s”. This is true only if $A$ is invertible (next week): you multiply from the left by $A^{-1}$. The $P$ above is not invertible ($\det P = 0$), which is why cancellation fails.

Transpose

The transpose $A^T$ of $A$ is the matrix with its rows made into columns: $(A^T)_{ij} = a_{ji}$. The transpose of an $m \times n$ matrix is $n \times m$.

$A = \begin{bmatrix} 1 & 2 & -1 \\ 0 & 3 & 4 \end{bmatrix} \quad\Rightarrow\quad A^T = \begin{bmatrix} 1 & 0 \\ 2 & 3 \\ -1 & 4 \end{bmatrix}$

Rules: $(A^T)^T = A$, $(A + B)^T = A^T + B^T$, $(cA)^T = cA^T$ and

$(AB)^T = B^TA^T.$

Why does the order reverse? If $AB$ is $m \times p$, then $(AB)^T$ is $p \times m$. $B^T$ is $p \times n$, $A^T$ is $n \times m$: only $B^TA^T$ gives this size. In the example above, $(AB)^T = \begin{bmatrix} -3 & 9 \\ -4 & 20 \end{bmatrix}$ and $B^TA^T$ gives the same.

Symmetric matrix: $A^T = A$. It must be square and $a_{ij} = a_{ji}$: a mirror about the diagonal. $A^TA$ and $AA^T$ are symmetric for every matrix, because $(A^TA)^T = A^T(A^T)^T = A^TA$. Symmetric matrices are the main topic of week 14.

Trace

The trace of a square matrix is the sum of its diagonal entries: $\operatorname{tr}A = a_{11} + a_{22} + \dots + a_{nn}$.

• $\operatorname{tr}(A + B) = \operatorname{tr}A + \operatorname{tr}B$, $\operatorname{tr}(cA) = c\operatorname{tr}A$, $\operatorname{tr}(A^T) = \operatorname{tr}A$.
• $\operatorname{tr}(AB) = \operatorname{tr}(BA)$, even when $AB$ and $BA$ have different sizes. In the example above both are $17$.

The trace will return in week 13 as the sum of the eigenvalues.

Powers

For square $A$, $A^2 = AA$, $A^3 = AAA$, and so on. When computing, look for a pattern:

$A = \begin{bmatrix} 1 & 2 \\ 0 & 1 \end{bmatrix}: \quad A^2 = \begin{bmatrix} 1 & 4 \\ 0 & 1 \end{bmatrix}, \quad A^3 = \begin{bmatrix} 1 & 6 \\ 0 & 1 \end{bmatrix}, \quad A^n = \begin{bmatrix} 1 & 2n \\ 0 & 1 \end{bmatrix}.$

The power of a diagonal matrix is the easiest: take the power of each number on the diagonal. $\operatorname{diag}(2, -1)^5 = \operatorname{diag}(32, -1)$. In week 13 the whole point of diagonalization will be to bring every matrix to this easy case.

TIP · a matrix inside a polynomial. If $p(x) = x^2 - 5x - 2$, then $p(A) = A^2 - 5A - 2I$. The constant term is multiplied by $I$; you can’t write "$-2$" and subtract a number from a matrix.

Common mistakes