Published on

Notes on Strang Lecture - 1

Lecture link

Systems of linear equations

A system of linear equations can be represented in multiple ways, each providing insights into the nature of solutions.

Example 1: two equations with two unknowns

Consider the following system:

2x−y=0−x+2y=3\begin{align*} 2x - y &= 0 \\ -x + 2y &= 3 \end{align*}

This system can be written in matrix form as:

[2−1−12][xy]=[03]\begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix}

Row picture

Each equation represents a line in the plane. A solution is a point where these two lines intersect. Solving the system, one finds that the lines meet at the point (x,y)=(1,2)(x, y) = (1, 2).

Column picture

Views the same system as a linear combination of column vectors:

x[2−1]+y[−12]=[03]x \begin{bmatrix} 2 \\ -1 \end{bmatrix} + y \begin{bmatrix} -1 \\ 2 \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix}

Here, one seeks coefficients xx and yy that combine the column vectors to produce the right-hand side vector. With x=1x = 1 and y=2y = 2, we have:

1⋅[2−1]+2⋅[−12]=[2−1]+[−24]=[03]1 \cdot \begin{bmatrix} 2 \\ -1 \end{bmatrix} + 2 \cdot \begin{bmatrix} -1 \\ 2 \end{bmatrix} = \begin{bmatrix} 2 \\ -1 \end{bmatrix} + \begin{bmatrix} -2 \\ 4 \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix}

Example 2: three equations with three unknowns

Consider another system:

2x−y=0−x+2y−z=−1−3y+4z=4\begin{align*} 2x - y &= 0 \\ -x + 2y - z &= -1 \\ -3y + 4z &= 4 \end{align*}

In matrix form:

[2−10−12−10−34][xyz]=[0−14]\begin{bmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -3 & 4 \end{bmatrix} \begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} 0 \\ -1 \\ 4 \end{bmatrix}

Row picture

In three dimensions, each equation represents a plane. A solution is a point that lies on the intersection of all three planes.

Column picture

x[2−10]+y[−12−3]+z[0−14]=[0−14]x \begin{bmatrix} 2 \\ -1 \\ 0 \end{bmatrix} + y \begin{bmatrix} -1 \\ 2 \\ -3 \end{bmatrix} + z \begin{bmatrix} 0 \\ -1 \\ 4 \end{bmatrix} = \begin{bmatrix} 0 \\ -1 \\ 4 \end{bmatrix}

Solvability

One question is: Can we solve Ax⃗=b⃗A\vec{x} = \vec{b} for every b⃗\vec{b}?

This question is equivalent to asking whether the linear combinations of the columns of AA fill the entire space. For an n×nn \times n matrix AA:

  • If the columns of AA are linearly independent and span Rn\mathbb{R}^n, then Ax⃗=b⃗A\vec{x} = \vec{b} has a unique solution for every b⃗∈Rn\vec{b} \in \mathbb{R}^n
  • The matrix AA is then called invertible or non-singular

Remark

If a matrix is invertible, the solution dimension decreases by 1 after intersecting with each additional linear constraint - when we have nn independent constraints in nn-dimensional space, we arrive at a single solution point.