Linear Algebra

A real vector space is a set $V$ together with two operations,

$+ \, : V \times V \to V, \qquad \cdot \, : \mathbb{R} \times V \to V,$

called addition and scalar multiplication, satisfying the following eight axioms for all $\mathbf{u}, \mathbf{v}, \mathbf{w} \in V$ and $a, b \in \mathbb{R}$:

1. (Commutativity of $+$) $\;\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$.
2. (Associativity of $+$) $\;(\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w})$.
3. (Zero vector) $\;$ There exists $\mathbf{0} \in V$ with $\mathbf{v} + \mathbf{0} = \mathbf{v}$ for every $\mathbf{v} \in V$.
4. (Additive inverses) $\;$ For every $\mathbf{v} \in V$ there exists $-\mathbf{v} \in V$ with $\mathbf{v} + (-\mathbf{v}) = \mathbf{0}$.
5. (Distributivity over vector addition) $\;a(\mathbf{u} + \mathbf{v}) = a\mathbf{u} + a\mathbf{v}$.
6. (Distributivity over scalar addition) $\;(a + b)\mathbf{v} = a\mathbf{v} + b\mathbf{v}$.
7. (Compatibility of scalar multiplication) $\;a(b\mathbf{v}) = (ab)\mathbf{v}$.
8. (Scalar identity) $\;1 \cdot \mathbf{v} = \mathbf{v}$.

Elements of $V$ are called vectors; real numbers in this context are called scalars.

The set $\mathbb{R}^n$ of $n$-tuples $(x_1, x_2, \ldots, x_n)$ of real numbers, with componentwise addition

$(x_1, \ldots, x_n) + (y_1, \ldots, y_n) = (x_1 + y_1, \ldots, x_n + y_n)$

and componentwise scalar multiplication

$c \cdot (x_1, \ldots, x_n) = (c x_1, \ldots, c x_n),$

is a real vector space. Verifying the eight axioms reduces to verifying them one coordinate at a time, where each verification is an instance of the familiar arithmetic of real numbers. The zero vector is $\mathbf{0} = (0, 0, \ldots, 0)$, and the additive inverse of $\mathbf{v}$ is $-\mathbf{v} = (-v_1, \ldots, -v_n)$. This is the prototype example, and most of the geometric intuition we will develop lives here.

Let $V = C[a, b]$ denote the set of continuous functions $f \colon [a, b] \to \mathbb{R}$. Define addition pointwise — $(f + g)(x) = f(x) + g(x)$ — and scalar multiplication pointwise — $(c f)(x) = c \cdot f(x)$. The eight axioms hold because each one reduces to the corresponding axiom for $\mathbb{R}$ applied at every point $x \in [a, b]$. Continuity is preserved because the sum of two continuous functions is continuous and a constant times a continuous function is continuous — facts established once and for all in the Sequences, Limits & Convergence topic. So $C[a, b]$ is a real vector space whose vectors are functions, not arrows.

Let $P_n$ denote the set of polynomials of degree at most $n$ with real coefficients,

$p(x) = a_0 + a_1 x + a_2 x^2 + \cdots + a_n x^n, \qquad a_i \in \mathbb{R}.$

Addition is the usual sum-of-coefficients; scalar multiplication multiplies every coefficient by the scalar. Both operations leave the degree at most $n$, so $P_n$ is closed under them, and the eight axioms hold by the same coefficient-by-coefficient argument as for $\mathbb{R}^{n+1}$. The zero vector is the zero polynomial. We will see in a moment that $P_n$ has exactly $n + 1$ “degrees of freedom” — the $n + 1$ coefficients — and that this is what it means for $P_n$ to be “$(n+1)$-dimensional.”

Three sets that look completely different — arrows in the plane, continuous functions on an interval, and polynomials — are all vector spaces, and we have just listed the eight properties they share. Every theorem we prove from those eight properties alone applies to all three sets at once, with no extra work. That is not a small bookkeeping convenience; it is the entire reason linear algebra is the most reusable piece of mathematics in the sciences. When you encounter a fourth or fifth example — and you will, in nearly every topic on this site — you do not have to redo the theory. You check the axioms, and then everything we are about to prove is yours.

Let $V$ be a vector space and let $\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_k \in V$ be a finite list of vectors. A linear combination of these vectors is any vector of the form

$c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \cdots + c_k \mathbf{v}_k, \qquad c_i \in \mathbb{R}.$

The span of $\{\mathbf{v}_1, \ldots, \mathbf{v}_k\}$, written $\mathrm{span}(\mathbf{v}_1, \ldots, \mathbf{v}_k)$, is the set of all linear combinations:

$\mathrm{span}(\mathbf{v}_1, \ldots, \mathbf{v}_k) = \bigl\{ c_1 \mathbf{v}_1 + \cdots + c_k \mathbf{v}_k \;\bigm|\; c_1, \ldots, c_k \in \mathbb{R} \bigr\}.$

The vectors $\mathbf{v}_1, \ldots, \mathbf{v}_k$ in a vector space $V$ are linearly independent if the only solution to the equation

$c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \cdots + c_k \mathbf{v}_k = \mathbf{0}$

is the trivial one, $c_1 = c_2 = \cdots = c_k = 0$. They are linearly dependent if some choice of coefficients that are not all zero produces the zero vector.

A basis of a vector space $V$ is a list of vectors $\{\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n\}$ that is

1. linearly independent, and
2. spans $V$, i.e., $\mathrm{span}(\mathbf{v}_1, \ldots, \mathbf{v}_n) = V$.

Let $\{\mathbf{v}_1, \ldots, \mathbf{v}_n\}$ be a basis of a vector space $V$. Then every $\mathbf{w} \in V$ has a unique expression as a linear combination of the basis vectors:

$\mathbf{w} = c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \cdots + c_n \mathbf{v}_n.$

The numbers $c_1, \ldots, c_n$ are called the coordinates of $\mathbf{w}$ with respect to the basis.

Existence. Because the basis spans $V$, every $\mathbf{w}$ is some linear combination of the basis vectors.

Uniqueness. Suppose $\mathbf{w}$ has two expressions:

$\mathbf{w} = c_1 \mathbf{v}_1 + \cdots + c_n \mathbf{v}_n = c_1' \mathbf{v}_1 + \cdots + c_n' \mathbf{v}_n.$

Subtracting the right side from the left,

$(c_1 - c_1') \mathbf{v}_1 + (c_2 - c_2') \mathbf{v}_2 + \cdots + (c_n - c_n') \mathbf{v}_n = \mathbf{0}.$

By linear independence of the basis, the only way a linear combination of $\mathbf{v}_1, \ldots, \mathbf{v}_n$ can equal $\mathbf{0}$ is if every coefficient is zero, so $c_i - c_i' = 0$ for each $i$, which is $c_i = c_i'$. The two expressions are the same.

If $\{\mathbf{v}_1, \ldots, \mathbf{v}_n\}$ and $\{\mathbf{w}_1, \ldots, \mathbf{w}_m\}$ are both bases of the same vector space $V$, then $n = m$.

We prove the theorem by establishing a stronger statement called the Exchange Lemma: if $\{\mathbf{v}_1, \ldots, \mathbf{v}_n\}$ spans $V$ and $\{\mathbf{w}_1, \ldots, \mathbf{w}_m\}$ is linearly independent in $V$, then $m \leq n$. Applying this lemma twice — first to the basis $\{\mathbf{v}_i\}$ (which spans) and the basis $\{\mathbf{w}_j\}$ (which is independent), giving $m \leq n$, and then with the roles reversed, giving $n \leq m$ — completes the proof.

We prove the lemma by induction on $k$, where at step $k$ we will have replaced $k$ of the $\mathbf{v}_i$ with the vectors $\mathbf{w}_1, \ldots, \mathbf{w}_k$ while preserving the spanning property.

Step 1. Because $\{\mathbf{v}_1, \ldots, \mathbf{v}_n\}$ spans $V$ and $\mathbf{w}_1 \in V$, we can write

$\mathbf{w}_1 = a_1 \mathbf{v}_1 + a_2 \mathbf{v}_2 + \cdots + a_n \mathbf{v}_n.$

At least one coefficient $a_i$ is nonzero — otherwise $\mathbf{w}_1 = \mathbf{0}$, which would contradict the linear independence of $\{\mathbf{w}_j\}$ (independent sets never contain the zero vector). Reordering the $\mathbf{v}_i$ if necessary, assume $a_1 \neq 0$. Then we can solve for $\mathbf{v}_1$:

$\mathbf{v}_1 = \frac{1}{a_1}\bigl(\mathbf{w}_1 - a_2 \mathbf{v}_2 - \cdots - a_n \mathbf{v}_n\bigr).$

So $\mathbf{v}_1$ is a linear combination of $\mathbf{w}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n$, which means $\{\mathbf{w}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n\}$ still spans $V$: anything previously reachable from the old list is still reachable from the new one.

Inductive step. Suppose after $k$ steps we have a spanning list of the form $\{\mathbf{w}_1, \ldots, \mathbf{w}_k, \mathbf{v}_{k+1}, \ldots, \mathbf{v}_n\}$. If $k = m$ we are done — the inductive procedure has consumed all the $\mathbf{w}_j$, so $m$ of the original $\mathbf{v}_i$ have been replaced and in particular $m \leq n$. Otherwise, $k < m$ and we have at least one more $\mathbf{w}_{k+1}$ to insert. Write

$\mathbf{w}_{k+1} = b_1 \mathbf{w}_1 + \cdots + b_k \mathbf{w}_k + c_{k+1} \mathbf{v}_{k+1} + \cdots + c_n \mathbf{v}_n,$

using the inductive spanning property. At least one of the $c_i$ must be nonzero, because if every $c_i$ were zero then $\mathbf{w}_{k+1}$ would be a linear combination of $\mathbf{w}_1, \ldots, \mathbf{w}_k$, contradicting the linear independence of $\{\mathbf{w}_j\}$. Reordering the remaining $\mathbf{v}_i$ if necessary, assume $c_{k+1} \neq 0$. Solving for $\mathbf{v}_{k+1}$ as in Step 1, we conclude that $\mathbf{v}_{k+1}$ is a linear combination of the new list $\{\mathbf{w}_1, \ldots, \mathbf{w}_{k+1}, \mathbf{v}_{k+2}, \ldots, \mathbf{v}_n\}$, so the new list still spans $V$. This completes the inductive step.

Termination. The induction can run for at most $n$ steps before we exhaust the $\mathbf{v}_i$. If at the end of the inductive procedure we still have $\mathbf{w}_j$ left over — that is, if $m > n$ — then we would have a contradiction at step $n + 1$: after replacing all $n$ of the $\mathbf{v}_i$ we would be trying to express $\mathbf{w}_{n+1}$ as a linear combination of $\{\mathbf{w}_1, \ldots, \mathbf{w}_n\}$, which would violate linear independence. So $m \leq n$, which is the lemma.

The dimension of a vector space $V$, written $\dim V$, is the number of vectors in any basis. By the Dimension Theorem, this number does not depend on the choice of basis, so the dimension is well-defined. A vector space is finite-dimensional if it has a basis with finitely many elements, and infinite-dimensional otherwise.

For each of our running examples:

• $\dim \mathbb{R}^n = n$, with the standard basis $\mathbf{e}_1, \mathbf{e}_2, \ldots, \mathbf{e}_n$.
• $\dim P_n = n + 1$, with basis $\{1, x, x^2, \ldots, x^n\}$.
• $\dim C[a, b] = \infty$. To see this, note that the functions $1, x, x^2, \ldots, x^k$ are linearly independent in $C[a, b]$ for every $k$ (a polynomial of degree at most $k$ has at most $k$ roots, so if a nontrivial linear combination of monomials were the zero function it would have infinitely many roots — impossible). So $C[a, b]$ contains $P_k$ as a subspace for every $k$, and no finite basis can suffice.

Most function spaces are infinite-dimensional. The space $C[a, b]$ above is just one example; the space of square-integrable functions $L^2(\mathbb{R})$ is another, and so is the sequence space $\ell^2$ of square-summable real sequences. The theorems we prove in this topic are about finite-dimensional spaces, but the language — span, basis, dimension, orthogonality — extends to infinite dimensions with appropriate care for convergence. That extension is the subject of functional analysis, the focus of Metric Spaces, Normed & Banach Spaces, and Inner Product & Hilbert Spaces elsewhere on this site. Finite-dimensional linear algebra is the warm-up; infinite-dimensional linear algebra is the real game.

A function $T \colon V \to W$ between real vector spaces is a linear map (or linear transformation) if it satisfies the two structure conditions

1. (Additivity) $\;T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v})$ for all $\mathbf{u}, \mathbf{v} \in V$.
2. (Homogeneity) $\;T(c \mathbf{v}) = c\, T(\mathbf{v})$ for all $c \in \mathbb{R}$ and all $\mathbf{v} \in V$.

Equivalently — and this single equation is sometimes used as the definition — $T(a \mathbf{u} + b \mathbf{v}) = a\, T(\mathbf{u}) + b\, T(\mathbf{v})$ for all scalars $a, b$ and all vectors $\mathbf{u}, \mathbf{v}$.

For a fixed angle $\theta \in \mathbb{R}$, define $R_\theta \colon \mathbb{R}^2 \to \mathbb{R}^2$ by

$R_\theta(x, y) = (\, x \cos\theta - y \sin\theta, \; x \sin\theta + y \cos\theta \,).$

Linearity is direct from the formula: each output coordinate is a real-linear combination of $x$ and $y$, so additivity and homogeneity in the input both go through. Geometrically, $R_\theta$ rotates every vector counter-clockwise by $\theta$ radians around the origin. The image of the standard basis is $R_\theta(\mathbf{e}_1) = (\cos\theta, \sin\theta)$ and $R_\theta(\mathbf{e}_2) = (-\sin\theta, \cos\theta)$ — a fact that will determine the matrix of $R_\theta$ in §5.

Let $\mathbf{u} \in \mathbb{R}^n$ be a unit vector. The orthogonal projection onto the line spanned by $\mathbf{u}$ is the map $P_\mathbf{u} \colon \mathbb{R}^n \to \mathbb{R}^n$ defined by

$P_\mathbf{u}(\mathbf{v}) = (\mathbf{v} \cdot \mathbf{u})\, \mathbf{u},$

where $\mathbf{v} \cdot \mathbf{u} = \sum_i v_i u_i$ is the standard dot product. Linearity follows because the dot product is linear in its first slot: $P_\mathbf{u}(a\mathbf{v} + b\mathbf{w}) = ((a\mathbf{v} + b\mathbf{w}) \cdot \mathbf{u})\, \mathbf{u} = (a\, \mathbf{v} \cdot \mathbf{u} + b\, \mathbf{w} \cdot \mathbf{u})\, \mathbf{u} = a P_\mathbf{u}(\mathbf{v}) + b P_\mathbf{u}(\mathbf{w})$. Geometrically, $P_\mathbf{u}$ sends every vector to its shadow on the line through the origin in the direction of $\mathbf{u}$. The image of $P_\mathbf{u}$ is exactly that line; its kernel is the hyperplane perpendicular to $\mathbf{u}$ — every vector orthogonal to $\mathbf{u}$ has $\mathbf{v} \cdot \mathbf{u} = 0$ and so projects to the origin.

For a fixed $\lambda \in \mathbb{R}$, define $S_\lambda \colon \mathbb{R}^n \to \mathbb{R}^n$ by

$S_\lambda(\mathbf{v}) = \lambda \mathbf{v}.$

Linearity is immediate: $S_\lambda(a\mathbf{u} + b\mathbf{v}) = \lambda(a\mathbf{u} + b\mathbf{v}) = a\, \lambda\mathbf{u} + b\, \lambda\mathbf{v} = a S_\lambda(\mathbf{u}) + b S_\lambda(\mathbf{v})$. The geometry depends on the sign of $\lambda$: for $\lambda > 0$ the map stretches every vector outward (or shrinks toward the origin if $0 < \lambda < 1$); for $\lambda < 0$ it stretches and flips through the origin; for $\lambda = 0$ it crushes the whole space to a single point.

For a fixed $k \in \mathbb{R}$, define $\mathrm{Sh}_k \colon \mathbb{R}^2 \to \mathbb{R}^2$ by

$\mathrm{Sh}_k(x, y) = (\, x + k y, \; y \,).$

Each output coordinate is a linear combination of the inputs, so $\mathrm{Sh}_k$ is linear. Geometrically, every point’s $y$-coordinate is preserved but its $x$-coordinate is bumped sideways by $k$ times $y$. The result is that horizontal lines slide rigidly, with the slide distance proportional to height: the $x$-axis stays put (everything on it has $y = 0$), and lines parallel to the $x$-axis stay parallel — they just shift sideways. This is the “deck of cards” picture from §1 made precise.

Let $T \colon V \to W$ be a linear map between vector spaces.

• The kernel (or nullspace) of $T$ is the subset of $V$ that $T$ sends to the zero vector of $W$:

$\ker T = \{\, \mathbf{v} \in V \mid T(\mathbf{v}) = \mathbf{0} \,\}.$

• The image (or range) of $T$ is the subset of $W$ consisting of all vectors that are hit by something in $V$:

$\mathrm{im}\, T = \{\, T(\mathbf{v}) \mid \mathbf{v} \in V \,\} \subseteq W.$

For any linear map $T \colon V \to W$, the kernel $\ker T$ is a subspace of $V$ and the image $\mathrm{im}\, T$ is a subspace of $W$.

We verify the two closure conditions and the zero-vector condition for each.

The kernel. Let $\mathbf{u}, \mathbf{v} \in \ker T$ and $c \in \mathbb{R}$. Then by linearity,

$T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v}) = \mathbf{0} + \mathbf{0} = \mathbf{0},$

so $\mathbf{u} + \mathbf{v} \in \ker T$, and

$T(c \mathbf{v}) = c\, T(\mathbf{v}) = c \cdot \mathbf{0} = \mathbf{0},$

so $c \mathbf{v} \in \ker T$. The zero vector $\mathbf{0}_V$ is in $\ker T$ because $T(\mathbf{0}_V) = \mathbf{0}_W$ (a consequence of homogeneity at $c = 0$).

The image. Let $\mathbf{w}_1, \mathbf{w}_2 \in \mathrm{im}\, T$, so $\mathbf{w}_1 = T(\mathbf{v}_1)$ and $\mathbf{w}_2 = T(\mathbf{v}_2)$ for some $\mathbf{v}_1, \mathbf{v}_2 \in V$, and let $c \in \mathbb{R}$. Then

$\mathbf{w}_1 + \mathbf{w}_2 = T(\mathbf{v}_1) + T(\mathbf{v}_2) = T(\mathbf{v}_1 + \mathbf{v}_2) \in \mathrm{im}\, T,$

and

$c \mathbf{w}_1 = c\, T(\mathbf{v}_1) = T(c \mathbf{v}_1) \in \mathrm{im}\, T.$

The zero vector $\mathbf{0}_W$ is in $\mathrm{im}\, T$ because $\mathbf{0}_W = T(\mathbf{0}_V)$.

Pure-mathematics texts say kernel; applied texts say nullspace. They denote the same set. We use both interchangeably, preferring kernel in abstract discussion of linear maps and nullspace when discussing computations with matrices, where “the nullspace of $A$” is the standard phrase. Likewise, range and image are synonyms, with the same prose-vs-computation split.

Let $V$ be a finite-dimensional vector space with basis $\{\mathbf{v}_1, \ldots, \mathbf{v}_n\}$, let $W$ be any vector space, and let $\mathbf{w}_1, \ldots, \mathbf{w}_n$ be any $n$ vectors in $W$ (chosen however you like — they need not be a basis, they need not be distinct, they need not even be nonzero). Then there is exactly one linear map $T \colon V \to W$ satisfying $T(\mathbf{v}_i) = \mathbf{w}_i$ for every $i$.

Why. Existence: every $\mathbf{v} \in V$ has a unique expansion $\mathbf{v} = c_1 \mathbf{v}_1 + \cdots + c_n \mathbf{v}_n$ by Proposition 1, and we are forced to define $T(\mathbf{v}) = c_1 \mathbf{w}_1 + \cdots + c_n \mathbf{w}_n$ in order for linearity to hold. Verifying that this definition is indeed linear is a mechanical check: linearity in the coefficient list follows from linearity in the coordinates. Uniqueness: if two linear maps $T$ and $T'$ both send $\mathbf{v}_i$ to $\mathbf{w}_i$ for every $i$, then they agree on the basis, and by linearity they agree on every linear combination of basis vectors, i.e. on every vector in $V$.

This single fact is the reason a linear map is recorded by a finite block of numbers — once you have specified where each of the $n$ basis vectors goes, the whole map is determined. The next section turns that block of numbers into a matrix.

Let $T \colon V \to W$ be a linear map between finite-dimensional vector spaces, let $\mathcal{B} = \{\mathbf{v}_1, \ldots, \mathbf{v}_n\}$ be a basis of $V$, and let $\mathcal{C} = \{\mathbf{w}_1, \ldots, \mathbf{w}_m\}$ be a basis of $W$. Write the image of each $\mathbf{v}_j$ in the basis $\mathcal{C}$ as

$T(\mathbf{v}_j) = \sum_{i=1}^{m} a_{ij} \mathbf{w}_i, \qquad j = 1, 2, \ldots, n.$

The matrix of $T$ relative to $\mathcal{B}$ and $\mathcal{C}$, written $[T]_{\mathcal{C} \leftarrow \mathcal{B}}$, is the $m \times n$ array $A = (a_{ij})$. Equivalently — and this is the picture worth carrying around — the $j$-th column of $[T]_{\mathcal{C} \leftarrow \mathcal{B}}$ is the coordinate vector of $T(\mathbf{v}_j)$ in the basis $\mathcal{C}$.

Let $A$ be an $m \times n$ matrix with columns $\mathbf{a}_1, \mathbf{a}_2, \ldots, \mathbf{a}_n \in \mathbb{R}^m$, and let $\mathbf{c} = (c_1, c_2, \ldots, c_n)^\top \in \mathbb{R}^n$ be a column vector. The matrix-vector product $A \mathbf{c}$ is the vector in $\mathbb{R}^m$ given by

$A \mathbf{c} = c_1 \mathbf{a}_1 + c_2 \mathbf{a}_2 + \cdots + c_n \mathbf{a}_n.$

That is, $A \mathbf{c}$ is the linear combination of the columns of $A$ with weights coming from $\mathbf{c}$.

With notation as in Definitions 8 and 9: if $\mathbf{v} \in V$ has coordinate vector $\mathbf{c} = (c_1, \ldots, c_n)^\top$ in the basis $\mathcal{B}$, then the coordinate vector of $T(\mathbf{v})$ in the basis $\mathcal{C}$ is the matrix-vector product $A \mathbf{c}$, where $A = [T]_{\mathcal{C} \leftarrow \mathcal{B}}$.

Start with the expansion of $\mathbf{v}$ in $\mathcal{B}$ and apply $T$, using linearity at each step:

$T(\mathbf{v}) = T\!\left( \sum_{j=1}^{n} c_j \mathbf{v}_j \right) = \sum_{j=1}^{n} c_j\, T(\mathbf{v}_j).$

By Definition 8, $T(\mathbf{v}_j) = \sum_{i=1}^{m} a_{ij} \mathbf{w}_i$. Substituting and swapping the order of the (finite) sums,

$T(\mathbf{v}) = \sum_{j=1}^{n} c_j \sum_{i=1}^{m} a_{ij} \mathbf{w}_i = \sum_{i=1}^{m} \!\left( \sum_{j=1}^{n} a_{ij} c_j \right) \mathbf{w}_i.$

The expression in parentheses is exactly the $i$-th entry of the matrix-vector product $A \mathbf{c}$, so the $i$-th coordinate of $T(\mathbf{v})$ in $\mathcal{C}$ is $(A \mathbf{c})_i$. That is the claim.

The rotation $R_\theta \colon \mathbb{R}^2 \to \mathbb{R}^2$ from Example 5 sends the standard basis to

$R_\theta(\mathbf{e}_1) = (\cos\theta, \sin\theta), \qquad R_\theta(\mathbf{e}_2) = (-\sin\theta, \cos\theta).$

By Definition 8, those two image vectors are the columns of $[R_\theta]$ in the standard basis:

$[R_\theta] = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \;\cos\theta \end{pmatrix}.$

Applying Theorem 2 with $\mathbf{c} = (x, y)^\top$ recovers the formula $R_\theta(x, y) = (x \cos\theta - y \sin\theta, \; x \sin\theta + y \cos\theta)$ from Example 5 — exactly as expected. The viz’s “Rotation by 30°” and “Rotation by 90°” presets are this matrix at $\theta = \pi/6$ and $\theta = \pi/2$ respectively.

Let $P \colon \mathbb{R}^2 \to \mathbb{R}^2$ be the map $P(x, y) = (x, 0)$. Then $P(\mathbf{e}_1) = (1, 0)$ and $P(\mathbf{e}_2) = (0, 0)$, so

$[P] = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}.$

The second column is all zeros, which is exactly the statement that $\mathbf{e}_2 \in \ker P$ — read columnwise. Whenever you spot a zero column in a matrix, you have spotted a basis vector in the kernel. This is the first of many “structural facts of the matrix that you read off geometrically without computing anything” the topic will accumulate.

The single most important fact of linear-algebra pedagogy — the one most often skipped over in undergraduate courses and the one that makes the viz above the centerpiece of this section — is that the $j$-th column of a matrix is the place the $j$-th basis vector lands. Once that is internalized, almost every matrix manipulation becomes a geometric statement:

• A column of zeros means the corresponding basis vector is in the kernel.
• Two equal columns mean the matrix sends two basis vectors to the same image, so the kernel contains the difference.
• A square matrix has linearly independent columns if and only if no two input basis vectors collide and none collapses to zero, which is exactly the condition that $T$ is invertible.
• The image of a matrix is the span of its columns — period. The “column space” of a matrix and the image of the underlying linear map are the same set.

Every one of those statements is just Definition 8 plus a moment’s thought about what linear-combination means. The standard order of pedagogy reverses this — first the row-by-column rule, then the geometric consequences — but reversing the ordering loses the reason the row-by-column rule looks the way it does. We will return to this picture repeatedly; treat it as the topic’s central image.

Let $T \colon V \to W$ and $S \colon W \to U$ be linear maps. Their composition $S \circ T \colon V \to U$ is the function

$(S \circ T)(\mathbf{v}) = S(T(\mathbf{v})), \qquad \mathbf{v} \in V.$

The composition of two linear maps is itself linear: $(S \circ T)(a \mathbf{u} + b \mathbf{v}) = S(T(a \mathbf{u} + b \mathbf{v})) = S(a T(\mathbf{u}) + b T(\mathbf{v})) = a S(T(\mathbf{u})) + b S(T(\mathbf{v})) = a (S \circ T)(\mathbf{u}) + b (S \circ T)(\mathbf{v})$, applying the linearity axioms of $T$ once and of $S$ once.

Let $B$ be an $\ell \times m$ matrix and $A$ be an $m \times n$ matrix. The matrix product $BA$ is the $\ell \times n$ matrix whose $(i, k)$ entry is

$(BA)_{ik} = \sum_{j=1}^{m} B_{ij}\, A_{jk}, \qquad 1 \leq i \leq \ell, \; 1 \leq k \leq n.$

Let $T \colon V \to W$ and $S \colon W \to U$ be linear maps between finite-dimensional vector spaces. Fix bases $\mathcal{B}$ of $V$, $\mathcal{C}$ of $W$, and $\mathcal{D}$ of $U$. Let

$A = [T]_{\mathcal{C} \leftarrow \mathcal{B}}, \qquad B = [S]_{\mathcal{D} \leftarrow \mathcal{C}}$

be the matrices of $T$ and $S$ in those bases. Then the matrix of the composition in the bases $\mathcal{B}$ and $\mathcal{D}$ is the matrix product

$[S \circ T]_{\mathcal{D} \leftarrow \mathcal{B}} = B A.$

We chase where each basis vector of $V$ ends up under the composition and read off the result columnwise, since by Definition 8 the $k$-th column of $[S \circ T]_{\mathcal{D} \leftarrow \mathcal{B}}$ is the coordinate vector of $(S \circ T)(\mathbf{v}_k)$ in the basis $\mathcal{D}$.

Write the bases as $\mathcal{B} = \{\mathbf{v}_1, \ldots, \mathbf{v}_n\}$, $\mathcal{C} = \{\mathbf{w}_1, \ldots, \mathbf{w}_m\}$, $\mathcal{D} = \{\mathbf{u}_1, \ldots, \mathbf{u}_\ell\}$, with $A = (a_{jk})$ an $m \times n$ matrix and $B = (b_{ij})$ an $\ell \times m$ matrix.

Step 1: apply $T$ to $\mathbf{v}_k$. By Definition 8 applied to $T$,

$T(\mathbf{v}_k) = \sum_{j=1}^{m} a_{jk}\, \mathbf{w}_j.$

Step 2: apply $S$ to the result. Using linearity of $S$ to pull the sum out, then Definition 8 applied to $S$ to expand each $S(\mathbf{w}_j)$,

$(S \circ T)(\mathbf{v}_k) = S(T(\mathbf{v}_k)) = S\!\left( \sum_{j=1}^{m} a_{jk}\, \mathbf{w}_j \right) = \sum_{j=1}^{m} a_{jk}\, S(\mathbf{w}_j) = \sum_{j=1}^{m} a_{jk} \sum_{i=1}^{\ell} b_{ij}\, \mathbf{u}_i.$

Step 3: swap the order of summation and read off the coordinates. The two sums are finite, so we may interchange them:

$(S \circ T)(\mathbf{v}_k) = \sum_{i=1}^{\ell} \!\left( \sum_{j=1}^{m} b_{ij}\, a_{jk} \right) \mathbf{u}_i.$

The parenthesized quantity is the $i$-th coordinate of $(S \circ T)(\mathbf{v}_k)$ in the basis $\mathcal{D}$. By Definition 8, that quantity is the $(i, k)$ entry of the matrix $[S \circ T]_{\mathcal{D} \leftarrow \mathcal{B}}$.

Step 4: recognize the row-by-column rule. Compare $\sum_{j=1}^{m} b_{ij}\, a_{jk}$ to the right-hand side of Definition 11:

$(BA)_{ik} = \sum_{j=1}^{m} B_{ij}\, A_{jk} = \sum_{j=1}^{m} b_{ij}\, a_{jk}.$

The two expressions are literally identical. So the $(i, k)$ entry of $[S \circ T]_{\mathcal{D} \leftarrow \mathcal{B}}$ equals the $(i, k)$ entry of the matrix product $BA$, for every $i$ and every $k$. The two matrices are equal entrywise, hence equal as matrices.

The transpose of an $m \times n$ matrix $A = (a_{ij})$ is the $n \times m$ matrix $A^\top$ whose entries are $(A^\top)_{ji} = a_{ij}$. Equivalently, the rows of $A$ are the columns of $A^\top$ and vice versa.

The identity matrix of size $n$, written $I_n$, is the $n \times n$ matrix whose entries are

$(I_n)_{ij} = \begin{cases} 1 & \text{if } i = j, \\ 0 & \text{otherwise.} \end{cases}$

It is the matrix of the identity linear map $\mathrm{id}_V \colon V \to V$ (sending every $\mathbf{v}$ to itself) in any basis.

A square matrix $A$ of size $n$ is invertible if there exists a matrix $A^{-1}$ of the same size satisfying

$A A^{-1} = A^{-1} A = I_n.$

When such an $A^{-1}$ exists it is unique, and it is called the inverse of $A$.

For matrices of compatible sizes and scalars $c \in \mathbb{R}$:

1. (Associativity of multiplication) $\;(AB)C = A(BC)$.
2. (Distributivity) $\;A(B + C) = AB + AC$ and $(A + B)C = AC + BC$.
3. (Transpose of a product) $\;(AB)^\top = B^\top A^\top$. Note the order reversal.
4. (Non-commutativity) In general, $AB \neq BA$. Matrix multiplication is not a commutative operation.

Let $A$ be the matrix of rotation by $90°$ and $B$ be the matrix of the horizontal shear with $k = 1$:

$A = \begin{pmatrix} 0 & -1 \\ 1 & \;0 \end{pmatrix}, \qquad B = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}.$