Probability Theory — Appendices

 Appendix A — Measure Theory Needed for Probability

A.1 Semirings and π-systems

A semiring of sets is a primitive domain on which one can define a premeasure before extending it to a σ-algebra. A typical semiring is the family of half-open intervals $(𝑎,𝑏]$ in $\mathbb{𝑅}$, or rectangles of the form

$$({𝑎}_1,{𝑏}_1]\times \cdots \times ({𝑎}_{𝑑},{𝑏}_{𝑑}]$$

in ${\mathbb{𝑅}}^{𝑑}$. Semirings are useful because complicated measurable sets are built from simpler geometric blocks, while measures are often first defined on those blocks.

A π-system is a collection $\mathcal{𝑃}$ of sets closed under finite intersections:

$$𝐴,𝐵\in \mathcal{𝑃}\Rightarrow 𝐴\cap 𝐵\in \mathcal{𝑃}\mathrm{.}$$

Rectangles form a π-system. Cylinder sets in product spaces form a π-system. Half-lines $(-\mathrm{\infty },𝑡]$ form a π-system generating the Borel σ-algebra on $\mathbb{𝑅}$.

The value of π-systems is uniqueness. If two probability measures agree on a π-system $\mathcal{𝑃}$, and $\mathcal{𝑃}$ generates $\mathcal{𝐹}$, then under standard hypotheses the measures agree on all of $\mathcal{𝐹}$:

$$𝜇{\mathrm{\mid }}_{\mathcal{𝑃}}=𝜈{\mathrm{\mid }}_{\mathcal{𝑃}}\Rightarrow 𝜇{\mathrm{\mid }}_{𝜎(\mathcal{𝑃})}=𝜈{\mathrm{\mid }}_{𝜎(\mathcal{𝑃})}\mathrm{.}$$

This is why finite-dimensional distributions, interval probabilities, and rectangle probabilities can determine full laws.

A.2 Dynkin systems

A Dynkin system, or λ-system, is a collection $\mathcal{𝐷}\subseteq 2^{\mathrm{\Omega }}$ such that $\mathrm{\Omega }\in \mathcal{𝐷}$, if $𝐴,𝐵\in \mathcal{𝐷}$ and $𝐴\subseteq 𝐵$, then $𝐵\setminus 𝐴\in \mathcal{𝐷}$, and if ${𝐴}_1,{𝐴}_2,\dots \in \mathcal{𝐷}$ are disjoint, then

$$\bigcup _{𝑛=1}^{\mathrm{\infty }}{𝐴}_{𝑛}\in \mathcal{𝐷}\mathrm{.}$$

Every σ-algebra is a Dynkin system, but a Dynkin system need not be closed under arbitrary finite intersections.

The π-λ theorem says that if $\mathcal{𝑃}$ is a π-system, then the smallest Dynkin system containing $\mathcal{𝑃}$ is exactly $𝜎(\mathcal{𝑃})$. This is a proof engine: define

$$\mathcal{𝐷}=\{𝐴:\text{desired identity holds for }𝐴\},$$

prove $\mathcal{𝐷}$ is a Dynkin system, prove it contains a generating π-system, then conclude the identity holds on the whole σ-algebra.

This method proves uniqueness of product measures, independence extensions, equality of laws from CDFs, and many conditioning identities. It is a carrier-extension gate: local verification on generators becomes global verification on all measurable events.

A.3 Monotone class theorem

The monotone class theorem is another extension engine. A monotone class of sets is closed under increasing countable unions and decreasing countable intersections. If a monotone class contains an algebra $\mathcal{𝐴}$, then it contains $𝜎(\mathcal{𝐴})$.

The functional version is even more important in probability. If a class of bounded measurable functions is closed under bounded monotone pointwise limits and contains indicators of a generating algebra, then it contains all bounded measurable functions. This allows a standard proof pattern:

$$\text{indicators}\to \text{simple functions}\to \text{bounded measurable functions}\to \text{nonnegative/integrable functions}\mathrm{.}$$

For example, to prove an identity such as

$$\mathbb{𝐸}[𝑓(𝑋)𝑔(𝑌)]=\mathbb{𝐸}[𝑓(𝑋)]\mathbb{𝐸}[𝑔(𝑌)]$$

under independence, one first proves it for indicators, extends to simple functions, and then to bounded measurable functions by monotone class. The theorem licenses moving from event-level probability to function-level expectation.

A.4 Carathéodory extension theorem

Carathéodory’s extension theorem takes a premeasure defined on an algebra or semiring and extends it to a measure on the generated σ-algebra. If ${𝜇}_0$ is countably additive on the primitive class and σ-finite under suitable hypotheses, there exists a measure $𝜇$ on $𝜎(\mathcal{𝐴})$ extending ${𝜇}_0$.

This theorem is the construction engine behind Lebesgue measure, product measure, probability laws from distribution functions, and stochastic processes from finite-dimensional data. One usually defines mass on simple sets first:

$${𝜇}_0((𝑎,𝑏])=𝐹(𝑏)-𝐹(𝑎),$$

or

$$(𝜇\otimes 𝜈)(𝐴\times 𝐵)=𝜇(𝐴)𝜈(𝐵),$$

then extends to the full σ-algebra.

The hidden gate is consistency. A premeasure cannot assign contradictory values on overlapping decompositions. In stochastic processes, finite-dimensional distributions must be mutually consistent before Kolmogorov extension can build a process law. Extension theorems do not repair inconsistent local data; they only globalize coherent data.

A.5 Lebesgue–Stieltjes measures

A Lebesgue–Stieltjes measure is generated by a nondecreasing right-continuous function $𝐹:\mathbb{𝑅}\to \mathbb{𝑅}$. For a probability distribution function $𝐹$, define initially

$${𝜇}_{𝐹}((𝑎,𝑏])=𝐹(𝑏)-𝐹(𝑎)\mathrm{.}$$

Carathéodory extension then gives a Borel probability measure satisfying

$${𝜇}_{𝐹}((-\mathrm{\infty },𝑡])=𝐹(𝑡)\mathrm{.}$$

This construction is why every valid CDF corresponds to a unique probability law. It also unifies discrete, continuous, singular, and mixed distributions. If $𝐹$ has jumps, those jumps are atoms:

$$𝑃(𝑋=𝑡)=𝐹(𝑡)-𝐹({𝑡}^{-})\mathrm{.}$$

If $𝐹$ is absolutely continuous, then

$$𝐹(𝑡)={\int }_{-\mathrm{\infty }}^{𝑡}𝑓(𝑥)𝑑𝑥$$

for a density $𝑓$. If $𝐹$ is continuous singular, such as the Cantor distribution, there are no atoms and no Lebesgue density.

Thus the CDF is the universal one-dimensional representation. A density is a special case. A mass function is a special case. The measure is the carrier.

A.6 Radon measures

A Radon measure is a measure compatible with topology, usually locally finite and inner regular. On a locally compact Hausdorff space, inner regularity means

$$𝜇(𝐴)=\sup \{𝜇(𝐾):𝐾\subseteq 𝐴,𝐾\text{ compact}\}$$

for measurable $𝐴$, while outer regularity means

$$𝜇(𝐴)=\inf \{𝜇(𝑈):𝐴\subseteq 𝑈,𝑈\text{ open}\}\mathrm{.}$$

Radon measures are essential because probability on topological spaces requires approximation. Weak convergence, tightness, compactness, support, and regular conditional laws all rely on topological regularity. In Polish spaces, Borel probability measures are Radon, which is one reason Polish and standard Borel spaces dominate modern probability.

Radon regularity is the bridge between measurable events and geometric/topological reasoning. Without it, one can have a measure but lack useful approximation by compact or open sets. Probability would then lose many convergence and compactness tools.

A.7 Radon–Nikodym theorem

If $𝜈\ll 𝜇$, meaning $𝜇(𝐴)=0\Rightarrow 𝜈(𝐴)=0$, and the measures are σ-finite, then there exists a measurable function $𝑓$ such that

$$𝜈(𝐴)={\int }_{𝐴}𝑓𝑑𝜇$$

for every measurable $𝐴$. The function $𝑓$ is the Radon–Nikodym derivative:

$$𝑓=\frac{𝑑𝜈}{𝑑𝜇}\mathrm{.}$$

This theorem is the source of densities, likelihood ratios, conditional expectation, and change of measure. If $𝑄\ll 𝑃$, then

$${\mathbb{𝐸}}_{𝑄}[𝑋]={\mathbb{𝐸}}_{𝑃}\left[𝑋\frac{𝑑𝑄}{𝑑𝑃}\right]\mathrm{.}$$

In Bayesian statistics, likelihoods are Radon–Nikodym derivatives. In stochastic calculus, Girsanov densities are Radon–Nikodym derivatives between path measures.

Conditional expectation can also be constructed through Radon–Nikodym. For $𝑋\in {𝐿}^1$ and sub-σ-algebra $\mathcal{𝐺}$, define a signed measure on $\mathcal{𝐺}$:

$$𝜈(𝐺)={\int }_{𝐺}𝑋𝑑𝑃\mathrm{.}$$

Since $𝜈\ll 𝑃{\mathrm{\mid }}_{\mathcal{𝐺}}$, Radon–Nikodym gives

$$\frac{𝑑𝜈}{𝑑𝑃{\mathrm{\mid }}_{\mathcal{𝐺}}}=\mathbb{𝐸}[𝑋\mid \mathcal{𝐺}]\mathrm{.}$$

A.8 Disintegration theorem

Disintegration decomposes a measure into conditional measures along fibers. If $𝛾$ is a probability measure on $𝑋\times 𝑌$ and ${𝜇}_{𝑌}$ is its $𝑌$-marginal, then under standard Borel hypotheses there exists a Markov kernel $𝐾(𝑦,𝑑𝑥)$ such that

$$𝛾(𝑑𝑥,𝑑𝑦)=𝐾(𝑦,𝑑𝑥){𝜇}_{𝑌}(𝑑𝑦)\mathrm{.}$$

Equivalently,

$$𝛾(𝐴\times 𝐵)={\int }_{𝐵}𝐾(𝑦,𝐴){𝜇}_{𝑌}(𝑑𝑦)\mathrm{.}$$

This is the rigorous version of conditional distribution:

$$𝐾(𝑦,𝐴)=𝑃(𝑋\in 𝐴\mid 𝑌=𝑦)\mathrm{.}$$

It is essential because $𝑃(𝑌=𝑦)$ may be zero. The elementary formula

$$𝑃(𝐴\mid 𝐵)=\frac{𝑃(𝐴\cap 𝐵)}{𝑃(𝐵)}$$

does not handle conditioning on a continuous value. Disintegration does.

Disintegration is also the invariant form of Bayes’ theorem. Given a generative law

$$𝛾(𝑑𝜃,𝑑𝑥)=𝐿(𝜃,𝑑𝑥)𝜋(𝑑𝜃),$$

the posterior is the reverse disintegration:

$$𝛾(𝑑𝜃,𝑑𝑥)=\mathrm{\Pi }(𝑑𝜃\mid 𝑥){𝜇}_{𝑋}(𝑑𝑥)\mathrm{.}$$

Bayesian updating is therefore conditional-measure transport, not merely symbolic ratio manipulation.

A.9 Product measures

Given measures $𝜇$ on $(𝑆,\mathcal{𝑆})$ and $𝜈$ on $(𝑇,\mathcal{𝑇})$, the product measure $𝜇\otimes 𝜈$ is determined by

$$(𝜇\otimes 𝜈)(𝐴\times 𝐵)=𝜇(𝐴)𝜈(𝐵)\mathrm{.}$$

The product σ-algebra is

$$\mathcal{𝑆}\otimes \mathcal{𝑇}=𝜎\{𝐴\times 𝐵:𝐴\in \mathcal{𝑆},𝐵\in \mathcal{𝑇}\}\mathrm{.}$$

Product measure is the carrier of independent joint randomness. If $𝑋\sim 𝜇$, $𝑌\sim 𝜈$, and $𝑋,𝑌$ are independent, then

$$\mathcal{𝐿}(𝑋,𝑌)=𝜇\otimes 𝜈\mathrm{.}$$

Conversely, if the joint law factors as product, the variables are independent.

Tonelli and Fubini are the integration laws of product measure:

$$\int 𝑓𝑑(𝜇\otimes 𝜈)=\int \int 𝑓(𝑥,𝑦)𝜈(𝑑𝑦)𝜇(𝑑𝑥)$$

under nonnegativity or integrability hypotheses. Thus product measure supplies not only independence but also iterated expectation and multidimensional integration.

A.10 Weak convergence of measures

Weak convergence of probability measures on a metric space $𝑆$ is

$${𝜇}_{𝑛}\Rightarrow 𝜇$$

if

$$\int 𝑓𝑑{𝜇}_{𝑛}\to \int 𝑓𝑑𝜇$$

for every bounded continuous $𝑓:𝑆\to \mathbb{𝑅}$. This is convergence by continuous bounded probes, not by all measurable sets.

The portmanteau theorem gives equivalent gates. For closed $𝐹$,

$$\underset{𝑛}{\mathrm{lim\hspace{0.17em}sup}}{𝜇}_{𝑛}(𝐹)\le 𝜇(𝐹),$$

for open $𝐺$,

$$\underset{𝑛}{\mathrm{lim\hspace{0.17em}inf}}{𝜇}_{𝑛}(𝐺)\ge 𝜇(𝐺),$$

and for $𝜇$-continuity sets $𝐴$, meaning $𝜇(\mathrm{\partial }𝐴)=0$,

$${𝜇}_{𝑛}(𝐴)\to 𝜇(𝐴)\mathrm{.}$$

Weak convergence is the carrier of convergence in distribution. It is weaker than total variation and does not generally control expectations of unbounded functions. Tightness is the compactness condition. On Polish spaces, Prokhorov’s theorem says tightness is equivalent to relative compactness in weak topology. Weak convergence is therefore measure theory plus topology.

---

Appendix B — Functional Analysis Needed for Probability

B.1 Normed spaces

A normed vector space is a vector space $𝑉$ equipped with a function $\mathrm{\parallel }\cdot \mathrm{\parallel }$ satisfying positivity, homogeneity, and the triangle inequality:

$$\mathrm{\parallel }𝑥\mathrm{\parallel }\ge 0,\mathrm{\parallel }𝜆𝑥\mathrm{\parallel }=\mathrm{\mid }𝜆\mathrm{\mid }\mathrm{\parallel }𝑥\mathrm{\parallel },\mathrm{\parallel }𝑥+𝑦\mathrm{\parallel }\le \mathrm{\parallel }𝑥\mathrm{\parallel }+\mathrm{\parallel }𝑦\mathrm{\parallel }\mathrm{.}$$

In probability, the central examples are ${𝐿}^{𝑝}$ spaces:

$$\mathrm{\parallel }𝑋{\mathrm{\parallel }}_{𝑝}=(\mathbb{𝐸}\mathrm{\mid }𝑋{\mathrm{\mid }}^{𝑝}{)}^{1\mathrm{/}𝑝},𝑝\ge 1.$$

Norms measure error and convergence. ${𝐿}^1$ controls mean absolute error, ${𝐿}^2$ controls quadratic error, ${𝐿}^{\mathrm{\infty }}$ controls essential worst-case size, and ${𝐿}^{𝑝}$ interpolates between typical and tail behavior. Many probabilistic statements are norm statements:

$${𝑋}_{𝑛}\to 𝑋\text{ in }{𝐿}^{𝑝}\iff \mathrm{\parallel }{𝑋}_{𝑛}-𝑋{\mathrm{\parallel }}_{𝑝}\to 0.$$

The norm is also a stability certificate. If a random approximation is close in ${𝐿}^2$, it controls variance-scale error. If it is close in ${𝐿}^{\mathrm{\infty }}$, it controls all outcomes except null sets. The chosen norm determines what kind of probabilistic error is being certified.

B.2 Banach spaces

A Banach space is a complete normed space. Completeness means every Cauchy sequence converges to a point in the space. For $𝑝\ge 1$,

$${𝐿}^{𝑝}(\mathrm{\Omega },\mathcal{𝐹},𝑃)$$

is a Banach space after quotienting by almost sure equality.

Completeness is what makes approximation methods legitimate. One defines objects first for simple functions, step processes, bounded functions, or finite-dimensional approximations, then extends by completion. Itô integration is built this way: define the integral for simple predictable processes, prove an isometry, then complete in ${𝐿}^2$.

Banach-space-valued probability also appears in empirical processes, random series, stochastic PDE, and concentration in function spaces. A random element may take values in a Banach space, and convergence may be norm convergence:

$$\mathbb{𝐸}\mathrm{\parallel }{𝑋}_{𝑛}-𝑋{\mathrm{\parallel }}^{𝑝}\to 0.$$

This is stronger than finite-dimensional convergence and requires actual control of the full object.

B.3 Hilbert spaces

A Hilbert space is a complete inner-product space. The core probability example is ${𝐿}^2$, with

$$⟨𝑋,𝑌⟩=\mathbb{𝐸}[𝑋𝑌]$$

in the real case, and

$$⟨𝑋,𝑌⟩=\mathbb{𝐸}[𝑋\overline{𝑌}]$$

in the complex case. The norm is

$$\mathrm{\parallel }𝑋{\mathrm{\parallel }}_2^2=\mathbb{𝐸}[\mathrm{\mid }𝑋{\mathrm{\mid }}^2]\mathrm{.}$$

Hilbert geometry turns probability into orthogonal projection. Conditional expectation is the projection of $𝑋\in {𝐿}^2$ onto the closed subspace ${𝐿}^2(\mathcal{𝐺})$:

$$\mathbb{𝐸}[𝑋\mid \mathcal{𝐺}]={\mathrm{Proj}}_{{𝐿}^2(\mathcal{𝐺})}𝑋\mathrm{.}$$

The error is orthogonal to all $\mathcal{𝐺}$-measurable square-integrable random variables:

$$\mathbb{𝐸}[(𝑋-\mathbb{𝐸}[𝑋\mid \mathcal{𝐺}])𝑍]=0.$$

Gaussian processes are also Hilbert-space objects. Covariance kernels define inner products, and Gaussian Hilbert spaces encode linear Gaussian structure. Orthogonality and independence coincide for jointly Gaussian centered variables, making ${𝐿}^2$ geometry unusually powerful in Gaussian probability.

B.4 Duality

The dual space ${𝑉}^{\ast }$ consists of continuous linear functionals on $𝑉$. For $1<𝑝<\mathrm{\infty }$, the dual of ${𝐿}^{𝑝}$ is ${𝐿}^{𝑞}$, where

$$\frac{1}{𝑝}+\frac{1}{𝑞}=1,$$

via the pairing

$$⟨𝑋,𝑌⟩=\mathbb{𝐸}[𝑋𝑌]\mathrm{.}$$

Hölder’s inequality,

$$\mathrm{\mid }\mathbb{𝐸}[𝑋𝑌]\mathrm{\mid }\le \mathrm{\parallel }𝑋{\mathrm{\parallel }}_{𝑝}\mathrm{\parallel }𝑌{\mathrm{\parallel }}_{𝑞},$$

is the boundedness certificate for this pairing.

Duality is central to probability because laws are often identified by test functions:

$$𝜇\mapsto \int 𝑓𝑑𝜇\mathrm{.}$$

Weak convergence, total variation, Wasserstein duality, optimal transport, risk measures, hypothesis testing, and convex concentration all use dual formulations.

Duality also exposes what a convergence mode can see. If convergence is defined by a small class of test functionals, it may miss tail behavior, oscillation, or singular structure. The dual class is the sensor array. Too small a class gives weak information; too large a class may require stronger convergence.

B.5 Weak topology

The weak topology on a normed space is the coarsest topology making every continuous linear functional continuous. A sequence ${𝑥}_{𝑛}$ converges weakly to $𝑥$ if

$$\mathrm{\ell }({𝑥}_{𝑛})\to \mathrm{\ell }(𝑥)$$

for every $\mathrm{\ell }\in {𝑉}^{\ast }$. Weak convergence is generally weaker than norm convergence.

In probability laws, weak convergence means convergence against bounded continuous functions:

$${𝜇}_{𝑛}\Rightarrow 𝜇\iff \int 𝑓𝑑{𝜇}_{𝑛}\to \int 𝑓𝑑𝜇\mathrm{.}$$

This is not norm convergence of measures and not convergence on all measurable sets. It is topology-sensitive law convergence.

Weak topology is valuable because it supplies compactness. Bounded sets in reflexive spaces have weakly compact subsequences under appropriate conditions. Probability analogues include tightness and Prokhorov compactness. The price is weaker conclusions: weak convergence does not automatically control norms, tails, or expectations of unbounded functions.

B.6 Compactness criteria

Compactness criteria convert boundedness or regularity into subsequential convergence. In finite dimensions, closed bounded sets are compact. In infinite dimensions, this is false, so one needs additional structure.

In probability measures on Polish spaces, tightness is the main compactness criterion:

$$\mathrm{\forall }𝜀>0,\mathrm{\exists }𝐾\text{ compact}:\underset{𝑛}{\sup }{𝜇}_{𝑛}({𝐾}^{𝑐})<𝜀\mathrm{.}$$

By Prokhorov’s theorem, tightness gives relatively compact families in weak topology.

In function spaces, Arzelà–Ascoli gives compactness from uniform boundedness and equicontinuity. In stochastic-process convergence, one proves tightness by controlling oscillations:

$$𝑃({𝑤}_{𝑋}(𝛿)>𝜀)\to 0$$

as $𝛿\downarrow 0$, where ${𝑤}_{𝑋}$ is a modulus of continuity or Skorokhod oscillation functional.

The standard limit-proof architecture is:

$$\text{compactness/tightness}\to \text{subsequential limit}\to \text{identify every limit}\to \text{full convergence}\mathrm{.}$$

B.7 Separability

A topological space is separable if it has a countable dense subset. Separability is a countability gate. It lets one replace uncountable operations by countable approximations in controlled settings.

For stochastic processes, separability is often what makes suprema measurable. If ${𝑋}_{𝑡}$ has continuous paths on $[0,𝑇]$, then

$$\underset{𝑡\in [0,𝑇]}{\sup }{𝑋}_{𝑡}=\underset{𝑡\in \mathbb{𝑄}\cap [0,𝑇]}{\sup }{𝑋}_{𝑡}\mathrm{.}$$

The right side is a countable supremum of measurable variables, hence measurable. Without path regularity or separability, the uncountable supremum may not be measurable.

Standard Borel and Polish spaces owe much of their usefulness to separability. Regular conditional probabilities, disintegration, measurable selection, and weak convergence theory all behave better in separable carriers. Nonseparable spaces often generate version and measurability pathologies.

B.8 Riesz representation

Riesz representation theorems identify linear functionals with measures or inner products. For locally compact Hausdorff spaces, positive linear functionals on ${𝐶}_{𝑐}(𝑋)$ correspond to Radon measures:

$$𝐿(𝑓)=\int 𝑓𝑑𝜇\mathrm{.}$$

For Hilbert spaces, every continuous linear functional has the form

$$\mathrm{\ell }(𝑥)=⟨𝑥,𝑦⟩$$

for a unique $𝑦$.

In probability, Riesz representation explains why a law can be identified by its integrals against test functions. If

$$\int 𝑓𝑑𝜇=\int 𝑓𝑑𝜈$$

for a sufficiently rich class of $𝑓$, then $𝜇=𝜈$. This underlies weak convergence, distribution identification, and dual formulations of distances between measures.

It also links Markov semigroups and kernels. A positive linear operator $𝑃$ acting on test functions can often be represented by a transition kernel:

$$𝑃𝑓(𝑥)=\int 𝑓(𝑦)𝑃(𝑥,𝑑𝑦)\mathrm{.}$$

Thus operator action and probabilistic transition are dual descriptions.

B.9 Operators and semigroups

A semigroup of operators $({𝑃}_{𝑡}{)}_{𝑡\ge 0}$ satisfies

$${𝑃}_0=𝐼,{𝑃}_{𝑡+𝑠}={𝑃}_{𝑡}{𝑃}_{𝑠}\mathrm{.}$$

For a Markov process,

$${𝑃}_{𝑡}𝑓(𝑥)={\mathbb{𝐸}}_{𝑥}[𝑓({𝑋}_{𝑡})]\mathrm{.}$$

The semigroup is the expectation-evolution operator.

The generator is the infinitesimal operator

$$𝐴𝑓=\underset{𝑡\downarrow 0}{\lim }\frac{{𝑃}_{𝑡}𝑓-𝑓}{𝑡},$$

defined on a domain of functions for which the limit exists. For a diffusion,

$$𝐴𝑓=𝑏\cdot \mathrm{\nabla }𝑓+\frac{1}{2}\mathrm{Tr}(𝑎{\mathrm{\nabla }}^2𝑓)\mathrm{.}$$

The generator connects stochastic processes to PDE:

$${\mathrm{\partial }}_{𝑡}𝑢=𝐴𝑢\mathrm{.}$$

Semigroup theory is the analytic carrier of Markov probability. Transition kernels, invariant measures, spectral gaps, ergodicity, diffusion equations, and martingale problems can all be expressed through operators. The domain of the generator is not optional; it determines which functions the infinitesimal formula legally acts on.

---

Appendix C — Fourier and Transform Methods

C.1 Characteristic functions

The characteristic function of a real random variable $𝑋$ is

$${𝜙}_{𝑋}(𝑡)=\mathbb{𝐸}[{𝑒}^{𝑖𝑡𝑋}],𝑡\in \mathbb{𝑅}\mathrm{.}$$

It always exists because

$$\mathrm{\mid }{𝑒}^{𝑖𝑡𝑋}\mathrm{\mid }=1.$$

For a random vector $𝑋\in {\mathbb{𝑅}}^{𝑑}$,

$${𝜙}_{𝑋}(𝑡)=\mathbb{𝐸}[{𝑒}^{𝑖⟨𝑡,𝑋⟩}],𝑡\in {\mathbb{𝑅}}^{𝑑}\mathrm{.}$$

Characteristic functions determine laws:

They convert independent sums into products. If $𝑋,𝑌$ are independent,

$${𝜙}_{𝑋+𝑌}(𝑡)={𝜙}_{𝑋}(𝑡){𝜙}_{𝑌}(𝑡)\mathrm{.}$$

For iid sums ${𝑆}_{𝑛}={𝑋}_1+\cdots +{𝑋}_{𝑛}$,

$${𝜙}_{{𝑆}_{𝑛}}(𝑡)={𝜙}_{𝑋}(𝑡{)}^{𝑛}\mathrm{.}$$

Near zero, characteristic functions encode moments. If $\mathbb{𝐸}[𝑋]=𝜇$ and $\mathbb{𝐸}[{𝑋}^2]<\mathrm{\infty }$, then

$${𝜙}_{𝑋}(𝑡)=1+𝑖𝜇𝑡-\frac{1}{2}\mathbb{𝐸}[{𝑋}^2]{𝑡}^2+𝑜({𝑡}^2)\mathrm{.}$$

For centered variance-${𝜎}^2$ variables,

$${𝜙}_{𝑋}(𝑡)=1-\frac{{𝜎}^2{𝑡}^2}{2}+𝑜({𝑡}^2)\mathrm{.}$$

This local expansion is the analytic engine behind the CLT.

C.2 Moment generating functions

The moment generating function is

$${𝑀}_{𝑋}(𝑡)=\mathbb{𝐸}[{𝑒}^{𝑡𝑋}],$$

where finite. Unlike the characteristic function, it may not exist for all $𝑡$, or even for any nonzero $𝑡$. Heavy-tailed variables often have infinite ${𝑀}_{𝑋}(𝑡)$ for $𝑡>0$.

When ${𝑀}_{𝑋}$ exists near zero, it encodes moments:

$${𝑀}_{𝑋}^{(𝑘)}(0)=\mathbb{𝐸}[{𝑋}^{𝑘}]\mathrm{.}$$

For independent variables,

$${𝑀}_{𝑋+𝑌}(𝑡)={𝑀}_{𝑋}(𝑡){𝑀}_{𝑌}(𝑡)\mathrm{.}$$

The log-moment generating function

$${\mathrm{\Lambda }}_{𝑋}(𝑡)=\log {𝑀}_{𝑋}(𝑡)$$

is central in Chernoff bounds and large deviations. Chernoff gives

$$𝑃(𝑋\ge 𝑎)\le {𝑒}^{-𝑡𝑎}{𝑀}_{𝑋}(𝑡)$$

for $𝑡>0$, hence

$$𝑃(𝑋\ge 𝑎)\le \exp (-\underset{𝑡>0}{\sup }\{𝑡𝑎-{\mathrm{\Lambda }}_{𝑋}(𝑡)\})\mathrm{.}$$

The Legendre transform of $\mathrm{\Lambda }$ is the large-deviation rate function in Cramér-type theory.

C.3 Laplace transforms

For a nonnegative random variable $𝑋$, the Laplace transform is

$${\mathcal{𝐿}}_{𝑋}(𝜆)=\mathbb{𝐸}[{𝑒}^{-𝜆𝑋}],𝜆\ge 0.$$

It always exists for $𝜆\ge 0$ because $0\le {𝑒}^{-𝜆𝑋}\le 1$. For measures on $[0,\mathrm{\infty })$, the Laplace transform often determines the law.

Laplace transforms are especially effective for waiting times, subordinators, branching processes, renewal theory, hitting times, and nonnegative random variables. If $𝑋,𝑌\ge 0$ are independent, then

$${\mathcal{𝐿}}_{𝑋+𝑌}(𝜆)={\mathcal{𝐿}}_{𝑋}(𝜆){\mathcal{𝐿}}_{𝑌}(𝜆)\mathrm{.}$$

For a nonnegative variable,

$$\mathbb{𝐸}[𝑋]=-{\mathcal{𝐿}}_{𝑋}^{\mathrm{\prime }}(0+)$$

when the derivative exists and the expectation is finite. More generally, derivatives at zero encode moments with alternating signs:

$${\mathcal{𝐿}}_{𝑋}^{(𝑘)}(0+)=(-1{)}^{𝑘}\mathbb{𝐸}[{𝑋}^{𝑘}]\mathrm{.}$$

Laplace methods are one-sided transform methods suited to positive carriers.

C.4 Fourier inversion

Fourier inversion reconstructs a distribution from its characteristic function. If $𝑋$ has an integrable characteristic function, then $𝑋$ has a bounded continuous density

$${𝑓}_{𝑋}(𝑥)=\frac{1}{2𝜋}{\int }_{\mathbb{𝑅}}{𝑒}^{-𝑖𝑡𝑥}{𝜙}_{𝑋}(𝑡)𝑑𝑡\mathrm{.}$$

For lattice integer-valued variables,

$$𝑃(𝑋=𝑘)=\frac{1}{2𝜋}{\int }_{-𝜋}^{𝜋}{𝑒}^{-𝑖𝑡𝑘}{𝜙}_{𝑋}(𝑡)𝑑𝑡\mathrm{.}$$

Fourier inversion is the native engine for local limit theorems. Weak convergence may use pointwise convergence of characteristic functions, but local probabilities require integrating the characteristic function and controlling its behavior away from zero.

The standard local-limit proof splits the integral:

$$\int ={\int }_{\mathrm{\mid }𝑡\mathrm{\mid }\le 𝛿}+{\int }_{\mathrm{\mid }𝑡\mathrm{\mid }>𝛿}\mathrm{.}$$

Near zero, one uses Taylor expansion:

$$𝜙(𝑡\mathrm{/}\sqrt{𝑛}{)}^{𝑛}\to {𝑒}^{-{𝜎}^2{𝑡}^2\mathrm{/}2}\mathrm{.}$$

Away from zero, one needs decay or aperiodicity:

$$\mathrm{\mid }𝜙(𝑡)\mathrm{\mid }<1.$$

This second region is where lattice and smoothness obstructions live.

C.5 Lévy continuity theorem

Lévy’s continuity theorem states that if characteristic functions ${𝜙}_{𝑛}(𝑡)$ converge pointwise to a function $𝜙(𝑡)$ that is continuous at $0$, then $𝜙$ is the characteristic function of a probability law $𝜇$, and

$${𝜇}_{𝑛}\Rightarrow 𝜇\mathrm{.}$$

Conversely, if ${𝜇}_{𝑛}\Rightarrow 𝜇$, then

$${𝜙}_{𝑛}(𝑡)\to 𝜙(𝑡)$$

for every $𝑡$.

The continuity-at-zero condition prevents loss of mass. A pointwise limit of characteristic functions need not correspond to a probability law if mass escapes or the limit is discontinuous at zero. Thus Lévy’s theorem is a law-convergence gate, not merely an analytic convenience.

The CLT proof uses it directly. For centered iid variance-${𝜎}^2$ variables,

$${𝜙}_{{𝑆}_{𝑛}\mathrm{/}(𝜎\sqrt{𝑛})}(𝑡)={\left[{𝜙}_{𝑋}\left(\frac{𝑡}{𝜎\sqrt{𝑛}}\right)\right]}^{𝑛}\to {𝑒}^{-{𝑡}^2\mathrm{/}2}\mathrm{.}$$

Since ${𝑒}^{-{𝑡}^2\mathrm{/}2}$ is continuous at zero and is the standard normal characteristic function, convergence in distribution follows.

C.6 Smoothing inequalities

Smoothing inequalities bound distances between distribution functions using characteristic functions. A typical Berry–Esseen-style smoothing inequality has the form

$$\underset{𝑥}{\sup }\mathrm{\mid }𝐹(𝑥)-𝐺(𝑥)\mathrm{\mid }\le 𝐶{\int }_{-𝑇}^{𝑇}\mid \frac{{𝜙}_{𝐹}(𝑡)-{𝜙}_{𝐺}(𝑡)}{𝑡}\mid 𝑑𝑡+\frac{{𝐶}^{\mathrm{\prime }}}{𝑇},$$

where $𝐺$ has sufficient regularity.

The first term measures Fourier discrepancy on a bounded frequency range. The second term is smoothing error from truncating the integral. Optimizing $𝑇$ balances analytic approximation and tail of the transform integral.

Smoothing inequalities are the bridge from characteristic-function convergence to quantitative distributional error. Lévy’s theorem gives convergence but not rate. Smoothing inequalities supply rate, making them central in Berry–Esseen estimates, local limit bounds, and approximation theory.

C.7 Saddle-point estimates

Saddle-point methods evaluate probabilities or coefficients through complex or exponential integral representations. If a probability can be written as

$$𝑃({𝑆}_{𝑛}=𝑘)=\frac{1}{2𝜋𝑖}\int {𝑒}^{𝑛\mathrm{\Psi }(𝑧)}𝑎(𝑧)𝑑𝑧,$$

the dominant contribution often comes from a point ${𝑧}_0$ satisfying

$${\mathrm{\Psi }}^{\mathrm{\prime }}({𝑧}_0)=0.$$

Expanding around ${𝑧}_0$,

$$\mathrm{\Psi }(𝑧)\approx \mathrm{\Psi }({𝑧}_0)+\frac{1}{2}{\mathrm{\Psi }}^{\mathrm{\prime }\mathrm{\prime }}({𝑧}_0)(𝑧-{𝑧}_0{)}^2,$$

gives Gaussian-type prefactors multiplying the main exponential term.

In probability, saddle points appear in precise large deviations, local probabilities, random combinatorial structures, branching processes, occupancy models, and statistical mechanics. They refine exponential estimates by extracting the correct polynomial prefactor and local shape.

The method is powerful but gate-heavy. One needs analyticity, contour deformation legitimacy, nondegenerate saddle, control away from the saddle, and error bounds. Without those, saddle-point notation is only heuristic.

C.8 Transform methods in limit theory

Transform methods convert probabilistic operations into algebraic or analytic operations. Independent sums become products of transforms:

$${𝜙}_{𝑋+𝑌}={𝜙}_{𝑋}{𝜙}_{𝑌}\mathrm{.}$$

Scaling becomes argument rescaling:

$${𝜙}_{𝑎𝑋}(𝑡)={𝜙}_{𝑋}(𝑎𝑡)\mathrm{.}$$

Centering becomes multiplication by a phase:

$${𝜙}_{𝑋-𝜇}(𝑡)={𝑒}^{-𝑖𝑡𝜇}{𝜙}_{𝑋}(𝑡)\mathrm{.}$$

Different transforms fit different carriers. Characteristic functions are universal for real laws. Moment generating functions are powerful when exponential moments exist. Laplace transforms fit nonnegative variables. Probability generating functions

$${𝐺}_{𝑋}(𝑠)=\mathbb{𝐸}[{𝑠}^{𝑋}]$$

fit nonnegative integer-valued variables. Cauchy/Stieltjes transforms fit spectral measures and random matrix theory:

$${𝐺}_{𝜇}(𝑧)=\int \frac{1}{𝑧-𝑥}𝜇(𝑑𝑥)\mathrm{.}$$

Limit theory often follows the same pattern:

$$\text{normalize}\to \text{transform}\to \text{pointwise/analytic convergence}\to \text{inversion or continuity theorem}\to \text{law-level terminal}\mathrm{.}$$

The counterkernel is using a transform outside its domain: MGF without exponential moments, density inversion without integrability, moment matching without determinacy, or pointwise transform convergence without continuity/tightness.