Overview / Outlines

A functional analysis primer

(Again, no proofs–see e.g. Stein and Shakarchi’s Real Analysis and Functional Analysis.)

Function spaces

Functional analysis deals with spaces of functions. Typically these are infinite-dimensional vector spaces with some sort of norm (Banach spaces), or even better, inner product (Hilbert spaces), and the additional structure has good analytic properties, i.e. the [induced] norm is complete.

If we do not assume completeness, we get pre-Banach or pre-Hilbert spaces; these can be completed to Banach or Hilbert spaces, and the completion is unique up to isomorphism.

The prototypical infinite-dimensional Hilbert space is the space of square-summable sequences \ell^2(\mathbb{Z}), with the inner product \langle (a_n), (b_n) \rangle = \sum_n a_n b_n; in fact, this is universal (for separable Hilbert spaces), and any L^2(X,\mu) (with the inner product \langle f, g \rangle = \int_X fg \,d\mu) is unitarily equivalent by the Fourier transform.

It is less clear what the universal Banach space is, but the L^p spaces (with 1 \leq p \leq \infty) are good prototypical examples.

Many other function spaces of interest are also Banach spaces.

Dual spaces

Linear maps between Banach spaces are also called linear operators. Given a linear operator T, we define its sup norm by \|T\| = \sup_{\|f\| = 1} \|T(f)\|. Operators with finite sup norm are called bounded. A linear operator T is bounded iff it is continuous.

Given any Banach space X, we can form the dual space X^* of all bounded linear functionals (i.e.. This is a Banach space with the sup norm \|\ell\| = \sup_{\|f\| = 1} \|\ell(f)\|.

Note that a linear functional (more generally, any linear map between Banach spaces) is bounded iff it is continuous.

Some examples

The dual space of L^p(X) for 1 \leq p < \infty is L^q(X), where q is the dual exponent which satisfies \frac 1p + \frac 1q = 1. Note (L^\infty)^* \supsetneq L^1—e.g. it contains the Dirichlet delta functionals, which are not representable by L^1-functions.

The dual space of the space of continuous functions C(X) is the space of all signed measures on X.

Hilbert spaces are self-dual, by the Riesz representation theorem. More precisely, given any continuous linear functional \ell on a Hilbert space \mathcal{H}, there exists a unique g = g(\ell) \in \mathcal{H} s.t. \ell(f) = \langle f, g \rangle for all f \in \mathcal{H}, with \|\ell\| = \|g\|, and this gives us an identification \mathcal{H}^* \to \mathcal{H} defined by \ell \mapsto g(\ell).

Building linear functionals

The Hahn-Banach theorem states that any linear functional defined on a linear subspace V_0 \subset V and bounded above by a sublinear function p on V_0 can be extended to a linear functional bounded above by p on the entire space V.

This allows us to define linear functionals by specifying values on some subspace, and then extending them using the general theorem.

Some applications / consequences:

  • convex subsets can be separated from points in complement of their closures
  • the natural injection V \to V^{**} into the double dual is isometric
    (L^\infty)^* \supsetneq L^1: the construction of the Dirac delta functionals uses the Hahn-Banach theorem

Topologies weak and strong

Note the unit ball in the norm topology is compact iff the space is finite-dimensional. In the infinite-dimensional case, there is no single “good” or “canonical” topology we could put on the space. There is the norm topology, which is in some sense natural if we are given a norm to start with; however it does not have the good properties that it does in the finite-dimensional case, whereas there are in fact some other topologies in which e.g. the unit ball is compact:

A sequence of points (x_n) in a Banach space \mathcal{B} is said to converge weakly to a point x \in \mathcal{B} if \ell(x_n) \to \ell(x) for every \ell \in \mathcal{B}^*. (If our Banach space is in fact a Hilbert space, then in view of the Riesz representation theorem we may re-express this condition as \langle x_n,y \rangle \to \langle x,y \rangle for every y \in \mathcal{B}.) The weak topology on \mathcal{B} is the (coarsest) topology for which sequences (or, more generally, nets) converge in the sense of weak convergence, or equivalently the coarsest topology in which bounded linear functionals remain continuous.

The closed unit ball in \mathcal{B} is compact iff \mathcal{B} is reflexive (this is the case if e.g. \mathcal{B} is a Hilbert space, or one of the L^p spaces.)

The weak* topology on the dual space \mathcal{B}^* is the coarsest topology such that the evaluation maps \varphi \mapsto \varphi(x) from \mathcal{H}^* to the base field remain continuous (for all x \in \mathcal{H}^*. It coincides with the topology of pointwise convergence of linear functionals.

The (sequential) Banach-Alaoglu theorem states that the closed unit ball of the dual space of a (separable) normed vector space is (sequentially) compact in the weak* topology (uses Tychonoff theorem, hence Choice.)

There are many other possible topologies, especially on the dual space, but that is a topic for another day.

Baire categories in Banach spaces

Baire’s two categories form a dichotomy of “size” based purely on topology, which is “in some sense a combination of [countability and density]” (quote adapted from Baire, as translated in Stein.)

First category (“meagre”) sets are countable unions of nowhere dense sets (i.e. sets whose closures have empty interior, such as discrete sets, or the Cantor set.) Complements of first category sets are generic. Anything which is not first category is second category.

Baire’s category theorem*states that any complete metric space X is of the second category (“the continuum is of the second category”.) One corollary of this is that generic sets are dense in a complete metric space (but note that there are also first category sets in [0,1] of full measure–e.g. [very] fat Cantor sets.)

Second category gives wriggle room

That wriggle room (usually in the form of “if we write a second category as a countable union of closed sets, at least one of them contains an open ball”) allows us to prove a bunch of useful analytic results for Banach spaces, which mostly extend our intuition for what happens in finite-dimensional spaces to the infinite-dimensional case:

  1. The uniform boundedness principle states that any set of continuous linear functionals on a Banach space \mathcal{B} which is pointwise bounded on some second category X \subset \mathcal{B} is uniformly bounded (i.e. bounded in the sup norm.)
  2. The open mapping theorem states that surjective continuous linear maps between Banach spaces are open mappings.
  3. The closed graph theorem states that linear maps between Banach spaces whose graphs are closed are continuous.

Hilbert space structures

The inner product that comes with a Hilbert space \mathcal{H} enables us to talk about orthogonal elements. We say that a (possibly infinite) tuple is an orthonormal basis for \mathcal{H} if it spans not necessarily the whole space, but a dense subspace.

e.g. (e^{inx})_{n=-\infty}^\infty is a orthonormal basis of L^2([-\pi, \pi]), by the theory of Fourier series.

Moreover, whenever we have a (topologically) closed subspace \mathcal{S} \subset \mathcal{H}, there is a well-defined notion of orthogonal projection onto \mathcal{S}, and thus (by subtracting the orthogonal projection) a well-defined orthogonal complement \mathcal{S}^\perp, which behave as we would expect it to from the case of finite-dimensional Hilbert spaces.


The inner product structure of a Hilbert space allows us to define these fun things called adjoints, which should be familiar from linear algebra: the adjoint of a linear operator T: \mathcal{H} \to \mathcal{H} is a linear operator T^*: \mathcal{H} \to \mathcal{H} satisfying \langle Tf, g \rangle = \langle f, T^*g \rangle for every f, g \in \mathcal{H}.

The construction of this adjoint goes through the Riesz representation theorem (see above), and so only works for operators from a Hilbert space to itself. With more care adjoints may be defined for operators between arbitrary pairs of Hilbert spaces (or even Banach spaces.) These adjoints really go between the dual spaces–in the first instance the distinction was blurred since Hilbert spaces are self-dual; without further assumptions, in the general case they may not be defined on the whole space and may not be unique.

Compact operators

A linear operator T: \mathcal{H} \to \mathcal{H} is compact if the image of the closed unit ball in \mathcal{H} under T is pre-compact (i.e. has [sequentially] compact closure.) Note compact operators are automatically bounded. “It turns out that dealing with compact operators provides us with the closest analogy to the usual theorems of (finite-dimensional) linear algebra.”

Some useful properties:

  • Pre- or post-composing a compact operator with a bounded operator yields a compact operator.
  • Limits of compact operators (in the sup norm) are compact.
  • Conversely, every compact operator is the limit of finite-rank operators (i.e. operators with finite-dimensional range.)
  • Compactness is preserved under taking adjoints.

Some useful examples:

  • Diagonalizable operators with eigenvalues |\lambda_k| \to 0
  • Hilbert-Schmidt operators

The Spectral Theorem for compact operators states that any compact symmetric operator T: \mathcal{H} \to \mathcal{H} has an orthonormal basis of eigenvectors, with top eigenvalue of norm \|T\|.

Spectral Theorem for bounded operators

There is a more general spectral theorem for bounded self-adjoint operators: given any bounded symmetric operator T: \mathcal{H} \to \mathcal{H}, there exists a measure space X and a real-valued f \in L^\infty(X) (representing the spectrum) s.t. A is unitarily conjugate to the “multiplication by f” operator on L^2(X) given by \varphi \mapsto (x \mapsto f(x)\varphi(x))

Alternatively this may be expressed in terms of a spectral resolution E_\lambda or projection-valued measure dE_\lambda, which allows us to write A = \int_{\sigma(A)} \lambda \, d E_\lambda.

In the case of compact operators the spectrum \sigma(A) is discrete (and the corresponding projection-valued measure a countable linear combination of atoms), and we recover the more specific statement above.


One thought on “A functional analysis primer

  1. Pingback: Metric, Norm, and Inner Product | Theories and Theorems

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s