(Again, no proofs–see e.g. Stein and Shakarchi’s Real Analysis and Functional Analysis.)
Functional analysis deals with spaces of functions. Typically these are infinite-dimensional vector spaces with some sort of norm (Banach spaces), or even better, inner product (Hilbert spaces), and the additional structure has good analytic properties, i.e. the [induced] norm is complete.
If we do not assume completeness, we get pre-Banach or pre-Hilbert spaces; these can be completed to Banach or Hilbert spaces, and the completion is unique up to isomorphism.
The prototypical infinite-dimensional Hilbert space is the space of square-summable sequences , with the inner product ; in fact, this is universal (for separable Hilbert spaces), and any (with the inner product ) is unitarily equivalent by the Fourier transform.
It is less clear what the universal Banach space is, but the spaces (with ) are good prototypical examples.
Many other function spaces of interest are also Banach spaces.
Linear maps between Banach spaces are also called linear operators. Given a linear operator T, we define its sup norm by . Operators with finite sup norm are called bounded. A linear operator T is bounded iff it is continuous.
Given any Banach space X, we can form the dual space of all bounded linear functionals (i.e.. This is a Banach space with the sup norm .
Note that a linear functional (more generally, any linear map between Banach spaces) is bounded iff it is continuous.
The dual space of for is , where is the dual exponent which satisfies . Note —e.g. it contains the Dirichlet delta functionals, which are not representable by -functions.
The dual space of the space of continuous functions C(X) is the space of all signed measures on X.
Hilbert spaces are self-dual, by the Riesz representation theorem. More precisely, given any continuous linear functional on a Hilbert space , there exists a unique s.t. for all , with , and this gives us an identification defined by .
Building linear functionals
The Hahn-Banach theorem states that any linear functional defined on a linear subspace and bounded above by a sublinear function p on can be extended to a linear functional bounded above by p on the entire space V.
This allows us to define linear functionals by specifying values on some subspace, and then extending them using the general theorem.
Some applications / consequences:
- convex subsets can be separated from points in complement of their closures
- the natural injection into the double dual is isometric
: the construction of the Dirac delta functionals uses the Hahn-Banach theorem
Topologies weak and strong
Note the unit ball in the norm topology is compact iff the space is finite-dimensional. In the infinite-dimensional case, there is no single “good” or “canonical” topology we could put on the space. There is the norm topology, which is in some sense natural if we are given a norm to start with; however it does not have the good properties that it does in the finite-dimensional case, whereas there are in fact some other topologies in which e.g. the unit ball is compact:
A sequence of points in a Banach space is said to converge weakly to a point if for every . (If our Banach space is in fact a Hilbert space, then in view of the Riesz representation theorem we may re-express this condition as for every .) The weak topology on is the (coarsest) topology for which sequences (or, more generally, nets) converge in the sense of weak convergence, or equivalently the coarsest topology in which bounded linear functionals remain continuous.
The closed unit ball in is compact iff is reflexive (this is the case if e.g. is a Hilbert space, or one of the spaces.)
The weak* topology on the dual space is the coarsest topology such that the evaluation maps from to the base field remain continuous (for all . It coincides with the topology of pointwise convergence of linear functionals.
The (sequential) Banach-Alaoglu theorem states that the closed unit ball of the dual space of a (separable) normed vector space is (sequentially) compact in the weak* topology (uses Tychonoff theorem, hence Choice.)
There are many other possible topologies, especially on the dual space, but that is a topic for another day.
Baire categories in Banach spaces
Baire’s two categories form a dichotomy of “size” based purely on topology, which is “in some sense a combination of [countability and density]” (quote adapted from Baire, as translated in Stein.)
First category (“meagre”) sets are countable unions of nowhere dense sets (i.e. sets whose closures have empty interior, such as discrete sets, or the Cantor set.) Complements of first category sets are generic. Anything which is not first category is second category.
Baire’s category theorem*states that any complete metric space X is of the second category (“the continuum is of the second category”.) One corollary of this is that generic sets are dense in a complete metric space (but note that there are also first category sets in [0,1] of full measure–e.g. [very] fat Cantor sets.)
Second category gives wriggle room
That wriggle room (usually in the form of “if we write a second category as a countable union of closed sets, at least one of them contains an open ball”) allows us to prove a bunch of useful analytic results for Banach spaces, which mostly extend our intuition for what happens in finite-dimensional spaces to the infinite-dimensional case:
- The uniform boundedness principle states that any set of continuous linear functionals on a Banach space which is pointwise bounded on some second category is uniformly bounded (i.e. bounded in the sup norm.)
- The open mapping theorem states that surjective continuous linear maps between Banach spaces are open mappings.
- The closed graph theorem states that linear maps between Banach spaces whose graphs are closed are continuous.
Hilbert space structures
The inner product that comes with a Hilbert space enables us to talk about orthogonal elements. We say that a (possibly infinite) tuple is an orthonormal basis for if it spans not necessarily the whole space, but a dense subspace.
e.g. is a orthonormal basis of , by the theory of Fourier series.
Moreover, whenever we have a (topologically) closed subspace , there is a well-defined notion of orthogonal projection onto , and thus (by subtracting the orthogonal projection) a well-defined orthogonal complement , which behave as we would expect it to from the case of finite-dimensional Hilbert spaces.
The inner product structure of a Hilbert space allows us to define these fun things called adjoints, which should be familiar from linear algebra: the adjoint of a linear operator is a linear operator satisfying for every .
The construction of this adjoint goes through the Riesz representation theorem (see above), and so only works for operators from a Hilbert space to itself. With more care adjoints may be defined for operators between arbitrary pairs of Hilbert spaces (or even Banach spaces.) These adjoints really go between the dual spaces–in the first instance the distinction was blurred since Hilbert spaces are self-dual; without further assumptions, in the general case they may not be defined on the whole space and may not be unique.
A linear operator is compact if the image of the closed unit ball in under T is pre-compact (i.e. has [sequentially] compact closure.) Note compact operators are automatically bounded. “It turns out that dealing with compact operators provides us with the closest analogy to the usual theorems of (finite-dimensional) linear algebra.”
Some useful properties:
- Pre- or post-composing a compact operator with a bounded operator yields a compact operator.
- Limits of compact operators (in the sup norm) are compact.
- Conversely, every compact operator is the limit of finite-rank operators (i.e. operators with finite-dimensional range.)
- Compactness is preserved under taking adjoints.
Some useful examples:
- Diagonalizable operators with eigenvalues
- Hilbert-Schmidt operators
The Spectral Theorem for compact operators states that any compact symmetric operator has an orthonormal basis of eigenvectors, with top eigenvalue of norm .
Spectral Theorem for bounded operators
There is a more general spectral theorem for bounded self-adjoint operators: given any bounded symmetric operator , there exists a measure space X and a real-valued (representing the spectrum) s.t. A is unitarily conjugate to the “multiplication by f” operator on given by
Alternatively this may be expressed in terms of a spectral resolution or projection-valued measure , which allows us to write .
In the case of compact operators the spectrum is discrete (and the corresponding projection-valued measure a countable linear combination of atoms), and we recover the more specific statement above.