Notes on Strang Lecture - 3

Matrix multiplication

For matrices $A$ and $B$ , the product $C = AB$ can be computed and interpreted in multiple ways.

Element-wise computation

The element $C_{ij}$ equals the dot product of the $i$ -th row of $A$ and the $j$ -th column of $B$ :

C_{ij} = \sum_{k=1}^{n} A_{ik} B_{kj}

Column and row perspectives

Column perspective: Matrix $C$ consists of linear combinations of the columns of $A$ , where the coefficients come from the columns of $B$ .
Row perspective: Matrix $C$ consists of linear combinations of the rows of $B$ , where the coefficients come from the rows of $A$ .

Sum of rank-one matrices

Another important interpretation: $AB$ is the sum of rank-one matrices formed by the outer product of the $i$ -th column of $A$ and the $i$ -th row of $B$ :

AB = \sum_{i=1}^{n} \vec{a}_i \vec{b}_i^T

where $\vec{a}_i$ denotes the $i$ -th column of $A$ and $\vec{b}_i^T$ denotes the $i$ -th row of $B$ .

Inverse matrices

For a square matrix $A$ , the inverse matrix $A^{-1}$ satisfies:

A^{-1}A = I, \quad \text{if } \exists A^{-1}

and equivalently:

AA^{-1} = I

Existence conditions

The inverse $A^{-1}$ exists if and only if there does not exists a non-zero vector $\vec{x}$ such that $A\vec{x} = \vec{0}$ . To see this, note that if such an $\vec{x}$ exists and $A^{-1}$ existed, we would have:

\vec{x} = A^{-1}A\vec{x} = \vec{0}

which contradicts the assumption that $\vec{x} \neq \vec{0}$ .

Similarly, $A^{-1}$ exist iff there is no non-zero row vector $\vec{x}^T$ such that $\vec{x}^T A = \vec{0}^T$ .

Computing the inverse

Suppose $A$ is invertible. One method to find $A^{-1}$ is to perform Gaussian-Jordan elimination on the augmented matrix $[A \mid I]$ , applying both forward and backward elimination steps to transform it into $[I \mid A^{-1}]$ .