Unit 3 · Algebra II

The Adjoint

Every operator on an inner product space has a partner that moves it across the inner product: $\langle Tu,v\rangle=\langle u,T^*v\rangle$. Almost every theorem in the rest of the course is a statement about how $T$ and $T^*$ relate, so this is the pivot of the whole term.

By the end you can

State and prove the Riesz representation theorem.
Construct $T^*$ and show it is unique and linear.
Compute the matrix of $T^*$ as the conjugate transpose.
Derive the normal equations and solve a least-squares problem.

Why it matters & where it comes from

Why it exists

Transposing a matrix is a formula with no obvious meaning. The adjoint gives it one: $A^*$ is the unique operator that lets you slide $A$ from one side of the inner product to the other. Every structural class in this course — self-adjoint, unitary, normal — is defined by a relation between $T$ and $T^*$.

In the real world

The normal equations $A^*Ax=A^*b$ are how regression is actually computed. In quantum mechanics observables are self-adjoint operators, and the adjoint of the derivative operator (integration by parts) is what makes the whole formalism work.

Story & open questions

Riesz proved his representation theorem in 1907 for function spaces; the finite-dimensional case here is a shadow of it. In infinite dimensions the adjoint needs care about domains, and unbounded self-adjoint operators are the technical heart of quantum theory.

Riesz representation

Definition

A linear functional on $V$ is a linear map $f:V\to F$.

Theorem Riesz representation

Let $V$ be a finite-dimensional inner product space and $f$ a linear functional on $V$. Then there is a unique $y\in V$ with $f(v)=\langle v,y\rangle$ for all $v$.

Proof Build $y$ from an orthonormal basis

Take an orthonormal basis $\{e_1,\dots,e_n\}$ (Unit 2) and set $y=\sum_i\overline{f(e_i)}\,e_i$.

Then $\langle e_j,y\rangle =\sum_i\overline{\overline{f(e_i)}}\,\langle e_j,e_i\rangle=f(e_j)$, using conjugate-linearity in the second slot. Two linear maps agreeing on a basis are equal, so $f(v)=\langle v,y\rangle$ for all $v$.

Uniqueness: if $\langle v,y\rangle=\langle v,y'\rangle$ for all $v$, then $\langle v,y-y'\rangle=0$ for all $v$; take $v=y-y'$ to get $\|y-y'\|^2=0$. ∎

The conjugate in $y$ is not decoration. Because the second slot is conjugate-linear, the coefficients must be conjugated for $\langle\cdot,y\rangle$ to come out linear. Drop the bar and the theorem is false over $\mathbb{C}$.

Existence of the adjoint

Theorem The adjoint exists and is unique

For every linear $T:V\to V$ on a finite-dimensional inner product space there is a unique linear $T^*:V\to V$ with

$\langle Tu,v\rangle=\langle u,T^*v\rangle$ for all $u,v\in V$.

Proof One Riesz application per $v$

Fix $v$. The map $u\mapsto\langle Tu,v\rangle$ is a linear functional on $V$, so Riesz gives a unique vector — call it $T^*v$ — with $\langle Tu,v\rangle=\langle u,T^*v\rangle$ for all $u$.

That defines $T^*$ as a function. It is linear: for all $u$, $\langle u,T^*(v+cw)\rangle=\langle Tu,v+cw\rangle =\langle Tu,v\rangle+\overline{c}\langle Tu,w\rangle =\langle u,T^*v+cT^*w\rangle$, and a vector determined by all its inner products is unique. ∎

Theorem Properties

$(T+U)^*=T^*+U^*$; $(cT)^*=\overline{c}\,T^*$; $(TU)^*=U^*T^*$; $(T^*)^*=T$; and $I^*=I$. Each follows by sliding across the inner product and using uniqueness.

Example Order reverses

For $(TU)^*$: $\langle TUu,v\rangle=\langle Uu,T^*v\rangle =\langle u,U^*T^*v\rangle$. Since the adjoint is unique, $(TU)^*=U^*T^*$ — the same reversal as for matrix transpose, and for the same reason.

The matrix of the adjoint

Theorem Conjugate transpose

If $\beta$ is an orthonormal basis and $A=[T]_\beta$, then $[T^*]_\beta=A^*$, the conjugate transpose: $(A^*)_{ij}=\overline{A_{ji}}$.

In an orthonormal basis the entries are recovered by $A_{ij}=\langle Te_j,e_i\rangle$. So $[T^*]_{ij}=\langle T^*e_j,e_i\rangle=\overline{\langle e_i,T^*e_j\rangle} =\overline{\langle Te_i,e_j\rangle}=\overline{A_{ji}}$.

Only in an orthonormal basis

In a general basis the matrix of $T^*$ is not the conjugate transpose of $[T]$. The formula $A_{ij}=\langle Te_j,e_i\rangle$ used above already assumed orthonormality. This is the most common source of wrong answers in this unit.

Example Compute one

On $\mathbb{C}^2$ with the standard basis (orthonormal), let $A=\left(\begin{smallmatrix}1&i\\0&2\end{smallmatrix}\right)$. Then $A^*=\left(\begin{smallmatrix}1&0\\-i&2\end{smallmatrix}\right)$.

Check with $u=(1,0)$, $v=(0,1)$: $\langle Au,v\rangle=\langle(1,0),(0,1)\rangle=0$, and $\langle u,A^*v\rangle=\langle(1,0),(0,2)\rangle=0$. ✓

Four subspaces and least squares

Theorem Range and null space

$N(T^*)=R(T)^\perp$ and $R(T^*)=N(T)^\perp$.

For the first: $v\in N(T^*)$ iff $T^*v=0$ iff $\langle u,T^*v\rangle=0$ for all $u$ iff $\langle Tu,v\rangle=0$ for all $u$ iff $v\perp R(T)$. The second follows by replacing $T$ with $T^*$ and using $(T^*)^*=T$.

Theorem Normal equations

Let $A$ be an $m\times n$ real or complex matrix and $b\in F^m$. A vector $x$ minimises $\|Ax-b\|$ if and only if $A^*Ax=A^*b$. If the columns of $A$ are independent then $A^*A$ is invertible and the minimiser is unique.

Proof It is the projection theorem

$Ax$ ranges over the column space $R(A)$. By Unit 2 the closest point of $R(A)$ to $b$ is $P_{R(A)}b$, characterised by the residual being orthogonal to $R(A)$: $b-Ax\in R(A)^\perp$.

By the previous theorem $R(A)^\perp=N(A^*)$, so the condition is $A^*(b-Ax)=0$, that is $A^*Ax=A^*b$.

If the columns are independent, $Ax=0$ forces $x=0$; then $A^*Ax=0$ gives $\|Ax\|^2=\langle A^*Ax,x\rangle=0$, so $x=0$ and $A^*A$ is invertible. ∎

Example Fit a line

Fit $y=c_0+c_1t$ to the points $(0,1),(1,2),(2,4)$.

$A$ has rows $(1,0),(1,1),(1,2)$ and $b=(1,2,4)$. Then $A^*A=\left(\begin{smallmatrix}3&3\\3&5\end{smallmatrix}\right)$ and $A^*b=(7,10)$.

Solving gives $c_1=\tfrac32$ and $c_0=\tfrac56$: the least squares line is $y=\tfrac56+\tfrac32t$.

Common pitfalls

• Using the conjugate transpose in a non-orthonormal basis. The identity $[T^*]=[T]^*$ requires orthonormality.
• Writing $(cT)^*=cT^*$. The scalar conjugates: $(cT)^*=\overline{c}T^*$.
• Writing $(TU)^*=T^*U^*$. The order reverses.
• Assuming $T^*$ exists in infinite dimensions without checking boundedness. In this course $V$ is finite-dimensional, so Riesz always applies.

Practice

Original problems written for this guide in the style of Friedberg, Insel & Spence exercises. Write a full solution before opening the answer.

Practice 1 · Compute an adjoint

Find $A^*$ for $A=\left(\begin{smallmatrix}2&1-i\\0&3i\end{smallmatrix}\right)$ on $\mathbb{C}^2$.

Answer

Transpose and conjugate: $A^*=\left(\begin{smallmatrix}2&0\\1+i&-3i\end{smallmatrix}\right)$.

Practice 2 · Riesz vector

On $\mathbb{R}^3$, let $f(v)=v_1+2v_2-v_3$. Find the $y$ with $f(v)=\langle v,y\rangle$.

Answer

$y=(1,2,-1)$. Over $\mathbb{C}$ the coefficients would be conjugated.

Practice 3 · Adjoint of a projection

Prove that the orthogonal projection $P_W$ satisfies $P_W^*=P_W$.

Solution

Split $u=u_W+u_\perp$ and $v=v_W+v_\perp$. Then $\langle P_Wu,v\rangle=\langle u_W,v_W+v_\perp\rangle=\langle u_W,v_W\rangle$ since $u_W\perp v_\perp$, and symmetrically $\langle u,P_Wv\rangle=\langle u_W,v_W\rangle$. They agree, so $P_W^*=P_W$.

Practice 4 · Order reversal

Prove $(TU)^*=U^*T^*$ directly from the defining identity.

Solution

$\langle TUu,v\rangle =\langle Uu,T^*v\rangle=\langle u,U^*T^*v\rangle$ for all $u,v$. The adjoint of $TU$ is the unique operator with this property, so it equals $U^*T^*$.

Practice 5 · Null space identity

Prove $N(T^*T)=N(T)$.

Solution

If $Tx=0$ then $T^*Tx=0$. Conversely if $T^*Tx=0$ then $0=\langle T^*Tx,x\rangle=\langle Tx,Tx\rangle=\|Tx\|^2$, so $Tx=0$. This is exactly why $A^*A$ is invertible when $A$ has independent columns.

Practice 6 · Least squares

Fit a line to $(0,0),(1,1),(2,3)$.

Solution

$A$ has rows $(1,0),(1,1),(1,2)$ and $b=(0,1,3)$, so $A^*A=\left(\begin{smallmatrix}3&3\\3&5\end{smallmatrix}\right)$ and $A^*b=(4,7)$. Solving: $c_1=\tfrac32$, $c_0=-\tfrac16$, giving $y=-\tfrac16+\tfrac32t$.

Practice 7 · Trace and adjoint

Show $\langle A,B\rangle=\operatorname{tr}(B^*A)$ makes $\operatorname{tr}(A^*)=\overline{\operatorname{tr}(A)}$.

Solution

The diagonal entries of $A^*$ are $\overline{A_{ii}}$, so their sum is the conjugate of $\sum A_{ii}$.

Practice 8 · Basis warning

Give a basis of $\mathbb{R}^2$ and an operator where $[T^*]\ne[T]^{\,t}$.

Solution

Use the non-orthonormal basis $\beta=\{(1,0),(1,1)\}$ and let $T$ be the identity. Then $[T]_\beta=I$ and $[T^*]_\beta=I$, which happens to agree — so try $T(x,y)=(y,0)$. Computing $[T]_\beta$ and $[T^*]_\beta$ separately shows they are not transposes: the change-of-basis matrix is not unitary, and the transpose rule is lost.

Expanded topic checklist

Course coverage. This unit is part of the complete MAT247H1 topic map. It should cover the following items before you move on:

Riesz representation. Prove every linear functional is an inner product against a fixed vector.
Existence of $T^*$. Construct it and prove uniqueness and linearity.
Matrix form. Show $[T^*]=[T]^*$ in an orthonormal basis, and know why orthonormality is required.
Consequences. Derive $N(T^*)=R(T)^\perp$ and the normal equations.

Textbook/source alignment

Use the current UofT MAT247H1 calendar topics as the coverage spine, and pair lecture notes with a Friedberg-Insel-Spence-style linear algebra text for extra reading. The exact assigned textbook should still be checked against the instructor syllabus.

Core theorem map

Riesz gives the adjoint; the adjoint gives the four-subspace relations; those give least squares. From Unit 4 onward every operator class is a condition on $T$ versus $T^*$.

Study routine. Whenever a hypothesis mentions $T^*$, immediately rewrite it as an inner product identity. Most proofs in this course are one such rewrite away from finishing.

Deep practice extensions

Use these after the main practice set. They are designed to force definition-level reasoning, theorem selection, and mixed application rather than routine calculation only.

Extension 1 · Defining identity

State the equation that defines $T^*$.

Guide

$\langle Tu,v\rangle =\langle u,T^*v\rangle$ for all $u,v\in V$. Everything else is a consequence.

Extension 2 · Scalar rule

What is $(cT)^*$?

Guide

$\overline{c}\,T^*$ — the scalar moves out of the first slot and into the second, conjugating on the way.

Extension 3 · Orthogonality of subspaces

Express $R(T)^\perp$ in terms of $T^*$.

Guide

$R(T)^\perp=N(T^*)$. Dually $N(T)^\perp=R(T^*)$.

Extension 4 · When least squares is exact

When does the least-squares solution actually solve $Ax=b$?

Guide

Exactly when $b\in R(A)$, so the projection of $b$ onto the column space is $b$ itself and the residual is zero.

← Unit 02Orthogonality & Gram–Schmidt Unit 04 →Isometries