Probability Theory — Chapters 21–29

Chapter 21 — Large Deviations

21.1 Exponential scales

Large deviation theory studies probabilities that decay exponentially in a scale parameter. Instead of asking whether ${𝑋}_{𝑛}\Rightarrow 𝑋$, it asks for asymptotics of the form

$$𝑃({𝑋}_{𝑛}\in 𝐴)\approx {𝑒}^{-𝑛\underset{𝑥\in 𝐴}{\inf }𝐼(𝑥)}\mathrm{.}$$

The scale $𝑛$ is not always the sample size; it may be $𝑡$, $1\mathrm{/}𝜀$, ${𝑛}^2$, volume, inverse noise, or another problem-specific growth parameter. The object $𝐼$ is the rate function. Events with small $𝐼$ are less expensive; events with large $𝐼$ are exponentially suppressed.

The exponential scale is invisible to the central limit theorem. A CLT sees fluctuations of size $\sqrt{𝑛}$, while large deviations often concern deviations of size $𝑛$. For iid ${𝑋}_{𝑖}$ with mean $𝜇$, the event

$$\frac{1}{𝑛}\sum _{𝑖=1}^{𝑛}{𝑋}_{𝑖}\approx 𝑎\mathrm{\ne }𝜇$$

usually has probability ${𝑒}^{-𝑛𝐼(𝑎)}$, not Gaussian-scale probability. The correct question is not “does the probability go to zero?” but “what exponential cost does the deviation pay?”

21.2 Chernoff bounds

Chernoff bounds use exponential moments to estimate tail probabilities. For any $𝑡>0$,

$$𝑃(𝑋\ge 𝑎)=𝑃({𝑒}^{𝑡𝑋}\ge {𝑒}^{𝑡𝑎})\le {𝑒}^{-𝑡𝑎}\mathbb{𝐸}[{𝑒}^{𝑡𝑋}]\mathrm{.}$$

Optimizing over $𝑡$ gives

$$𝑃(𝑋\ge 𝑎)\le \underset{𝑡>0}{\inf }{𝑒}^{-𝑡𝑎}\mathbb{𝐸}[{𝑒}^{𝑡𝑋}]\mathrm{.}$$

For sums of independent variables, moment generating functions factor:

$$\mathbb{𝐸}{𝑒}^{𝑡\sum _{𝑖}{𝑋}_{𝑖}}=\prod _{𝑖}\mathbb{𝐸}{𝑒}^{𝑡{𝑋}_{𝑖}}\mathrm{.}$$

Chernoff bounds are the finite-$𝑛$ ancestor of large deviation theory. They convert an event constraint into an exponential tilting problem. The bound is often sharp at exponential scale because the optimizing $𝑡$ defines a tilted probability measure under which the rare event becomes typical. The hidden debt is existence of the moment generating function in the needed range.

21.3 Cramér’s theorem

Cramér’s theorem is the fundamental large deviation theorem for iid sample means. Let ${𝑋}_1,{𝑋}_2,\dots$ be iid real variables and define

$${𝑆}_{𝑛}=\frac{1}{𝑛}\sum _{𝑖=1}^{𝑛}{𝑋}_{𝑖}\mathrm{.}$$

Let

$$\mathrm{\Lambda }(𝑡)=\log \mathbb{𝐸}[{𝑒}^{𝑡{𝑋}_1}]$$

be the log-moment generating function. Under standard hypotheses, ${𝑆}_{𝑛}$ satisfies a large deviation principle with rate function

$$𝐼(𝑥)=\underset{𝑡\in \mathbb{𝑅}}{\sup }\{𝑡𝑥-\mathrm{\Lambda }(𝑡)\}\mathrm{.}$$

Thus for suitable sets $𝐴$,

$$𝑃({𝑆}_{𝑛}\in 𝐴)\approx {𝑒}^{-𝑛\underset{𝑥\in 𝐴}{\inf }𝐼(𝑥)}\mathrm{.}$$

Cramér’s theorem says empirical means deviate from their expectation with exponential cost determined by the Legendre transform of $\mathrm{\Lambda }$. The mean $𝜇=\mathbb{𝐸}𝑋$ is the zero-cost point: $𝐼(𝜇)=0$. Away from $𝜇$, $𝐼(𝑥)>0$ when the law is nondegenerate. The theorem gives the correct rare-event regime for averages, not merely the CLT regime.

21.4 Rate functions

A rate function $𝐼:𝑆\to [0,\mathrm{\infty }]$ assigns exponential cost to states. It is usually required to be lower semicontinuous. A good rate function has compact level sets:

$$\{𝑥:𝐼(𝑥)\le 𝑐\}$$

compact for every $𝑐<\mathrm{\infty }$. Goodness is a compactness condition; it prevents minimizing sequences from escaping.

A large deviation principle has two inequalities. For closed $𝐹$,

$$\underset{𝑛\to \mathrm{\infty }}{\mathrm{lim\hspace{0.17em}sup}}\frac{1}{𝑛}\log 𝑃({𝑋}_{𝑛}\in 𝐹)\le -\underset{𝑥\in 𝐹}{\inf }𝐼(𝑥),$$

and for open $𝐺$,

$$\underset{𝑛\to \mathrm{\infty }}{\mathrm{lim\hspace{0.17em}inf}}\frac{1}{𝑛}\log 𝑃({𝑋}_{𝑛}\in 𝐺)\ge -\underset{𝑥\in 𝐺}{\inf }𝐼(𝑥)\mathrm{.}$$

The closed-set bound controls upper tails; the open-set bound guarantees enough mass near feasible points. Together they define exponential asymptotics at the topological level.

21.5 Legendre transform

The Legendre transform converts a convex log-moment generating function into a rate function:

$$𝐼(𝑥)={\mathrm{\Lambda }}^{\ast }(𝑥)=\underset{𝑡}{\sup }\{𝑡𝑥-\mathrm{\Lambda }(𝑡)\}\mathrm{.}$$

This operation is convex duality. The parameter $𝑡$ is the exponential tilt; the value $𝑥$ is the forced mean. At differentiability points, the optimizer satisfies

$${\mathrm{\Lambda }}^{\mathrm{\prime }}(𝑡)=𝑥\mathrm{.}$$

In large deviations, the Legendre transform is not formal decoration. It is the certificate that the cheapest way to force an atypical average is to tilt the underlying distribution. The original law $𝑃$ is replaced by

$$\frac{𝑑{𝑃}_{𝑡}}{𝑑𝑃}(𝑥)={𝑒}^{𝑡𝑥-\mathrm{\Lambda }(𝑡)}\mathrm{.}$$

Under ${𝑃}_{𝑡}$, the rare mean $𝑥$ becomes typical. The exponential cost of this tilt is the rate $𝐼(𝑥)$.

21.6 Sanov theorem

Sanov’s theorem is the large deviation principle for empirical measures. If ${𝑋}_1,\dots ,{𝑋}_{𝑛}$ are iid with law $𝜇$, the empirical measure

$${𝐿}_{𝑛}=\frac{1}{𝑛}\sum _{𝑖=1}^{𝑛}{𝛿}_{{𝑋}_{𝑖}}$$

satisfies an LDP with rate function

$$𝐻(𝜈\mid 𝜇)=\{\begin{array}{ll}{\displaystyle \int \log \left(\frac{𝑑𝜈}{𝑑𝜇}\right)𝑑𝜈,}& 𝜈\ll 𝜇,\\ +\mathrm{\infty },& \text{otherwise}\mathrm{.}\end{array}$$

This is relative entropy, or Kullback–Leibler divergence.

Sanov says the probability that the empirical distribution resembles $𝜈$ decays like

$${𝑒}^{-𝑛𝐻(𝜈\mid 𝜇)}\mathrm{.}$$

The theorem lifts large deviations from sample means to empirical laws. Cramér’s theorem follows by contraction: the empirical mean is the image of the empirical measure under $𝜈\mapsto \int 𝑥𝑑𝜈$.

21.7 Varadhan lemma

Varadhan’s lemma evaluates exponential integrals under an LDP. If ${𝑋}_{𝑛}$ satisfies an LDP with rate $𝐼$, then under suitable continuity and integrability hypotheses,

$$\underset{𝑛\to \mathrm{\infty }}{\lim }\frac{1}{𝑛}\log \mathbb{𝐸}\left[{𝑒}^{𝑛𝐹({𝑋}_{𝑛})}\right]=\underset{𝑥}{\sup }\{𝐹(𝑥)-𝐼(𝑥)\}\mathrm{.}$$

This is Laplace’s principle.

The lemma says exponential integrals are dominated by the best compromise between reward $𝐹(𝑥)$ and cost $𝐼(𝑥)$. It is the probabilistic analogue of saddle-point asymptotics. The supremum is not necessarily attained at the typical point; exponential reweighting selects a new dominant state.

21.8 Gärtner–Ellis theorem

The Gärtner–Ellis theorem derives an LDP from convergence of scaled log-moment generating functions. If

$${\mathrm{\Lambda }}_{𝑛}(𝑡)=\frac{1}{𝑛}\log \mathbb{𝐸}[{𝑒}^{𝑛𝑡{𝑋}_{𝑛}}]$$

converges to $\mathrm{\Lambda }(𝑡)$, and $\mathrm{\Lambda }$ satisfies suitable regularity conditions, then ${𝑋}_{𝑛}$ satisfies an LDP with rate

$$𝐼(𝑥)={\mathrm{\Lambda }}^{\ast }(𝑥)=\underset{𝑡}{\sup }\{𝑡𝑥-\mathrm{\Lambda }(𝑡)\}\mathrm{.}$$

This theorem generalizes Cramér’s theorem beyond iid sums. It is useful for dependent systems, statistical mechanics, Markov processes, and random matrices. The differentiability or exposed-point conditions are not cosmetic; without them, only partial lower bounds may hold. The moment-generating carrier may see convexified costs rather than all pathwise obstructions.

21.9 Moderate deviations

Moderate deviations sit between the CLT and large deviations. For iid centered variables with variance ${𝜎}^2$, consider scales ${𝑎}_{𝑛}$ satisfying

$${𝑎}_{𝑛}\to \mathrm{\infty },\frac{{𝑎}_{𝑛}}{\sqrt{𝑛}}\to 0.$$

Then

$$𝑃(\frac{\sum _{𝑖=1}^{𝑛}{𝑋}_{𝑖}}{𝜎\sqrt{𝑛}}\ge {𝑎}_{𝑛}𝑥)$$

often behaves like a Gaussian tail at exponential scale ${𝑎}_{𝑛}^2$:

$$\approx {𝑒}^{-{𝑎}_{𝑛}^2{𝑥}^2\mathrm{/}2}\mathrm{.}$$

The regime is larger than CLT fluctuations but smaller than order-$𝑛$ deviations. Moderate deviations show where Gaussian behavior remains valid into the tail. They require more tail control than the CLT but less than full Cramér-type large deviations.

21.10 Concentration versus large deviations

Concentration inequalities give nonasymptotic probability bounds such as

$$𝑃(\mathrm{\mid }𝑋-\mathbb{𝐸}𝑋\mathrm{\mid }\ge 𝑡)\le 2{𝑒}^{-𝑐{𝑡}^2}\mathrm{.}$$

Large deviation theory gives asymptotic exponential rates:

$$𝑃({𝑋}_{𝑛}\in 𝐴)\approx {𝑒}^{-𝑛\underset{𝐴}{\inf }𝐼}\mathrm{.}$$

Both study rare events, but their outputs differ. Concentration prioritizes finite-sample bounds, often with constants. Large deviations prioritize sharp exponential asymptotics and variational structure.

The two methods overlap but should not be conflated. A concentration bound may be nonsharp at exponential scale. An LDP may not give usable finite-$𝑛$ estimates. Concentration is a certificate of suppression; large deviations are a certificate of exponential cost geometry.

21.11 Rare-event regimes outside CLT

The CLT cannot control probabilities far from the central scale. If

$${𝑆}_{𝑛}=\sum _{𝑖=1}^{𝑛}{𝑋}_{𝑖},$$

then the CLT describes

$${𝑆}_{𝑛}-𝑛𝜇=𝑂(\sqrt{𝑛})\mathrm{.}$$

Events like

$${𝑆}_{𝑛}-𝑛𝜇\ge 𝑐𝑛$$

belong to large deviations. Events like

$${𝑆}_{𝑛}-𝑛𝜇\ge {𝑎}_{𝑛}\sqrt{𝑛},{𝑎}_{𝑛}\to \mathrm{\infty }$$

belong to moderate deviations. Events driven by one huge summand in heavy-tailed laws may follow subexponential rather than Cramér-type asymptotics.

The audit rule is scale first, theorem second. A Gaussian approximation at the wrong scale is a carrier error. Rare-event analysis must declare whether the deviation is collective tilting, one-big-jump behavior, boundary crossing, local lattice behavior, or path-level deviation.

---

Chapter 22 — Filtrations and Adapted Processes

22.1 Information over time

Probability over time requires an information structure. At time $𝑡$, the model should know which events are observable by then. This is encoded by a σ-algebra ${\mathcal{𝐹}}_{𝑡}$. If $𝑠\le 𝑡$, then information cannot decrease:

$${\mathcal{𝐹}}_{𝑠}\subseteq {\mathcal{𝐹}}_{𝑡}\mathrm{.}$$

The family $({\mathcal{𝐹}}_{𝑡})$ is a filtration.

Without a filtration, temporal statements like “known now,” “future,” “stopping time,” “martingale,” or “strategy” are undefined. A process alone gives values; a filtration tells which values and events are available at each time. Dynamic probability is probability plus information flow.

22.2 Filtrations

A filtration on $(\mathrm{\Omega },\mathcal{𝐹},𝑃)$ is an increasing family of sub-σ-algebras:

$$({\mathcal{𝐹}}_{𝑡}{)}_{𝑡\in 𝑇},{\mathcal{𝐹}}_{𝑠}\subseteq {\mathcal{𝐹}}_{𝑡}\subseteq \mathcal{𝐹}\text{for }𝑠\le 𝑡\mathrm{.}$$

The index set $𝑇$ may be discrete, such as $\mathbb{𝑁}$, or continuous, such as $[0,\mathrm{\infty })$.

The filtration is the carrier for conditional expectation over time. For an integrable random variable $𝑋$, $\mathbb{𝐸}[𝑋\mid {\mathcal{𝐹}}_{𝑡}]$ is the best estimate of $𝑋$ using information available at time $𝑡$. Increasing $𝑡$ refines the information. The tower property expresses temporal consistency.

22.3 Adapted processes

A process ${𝑋}_{𝑡}$ is adapted to $({\mathcal{𝐹}}_{𝑡})$ if

$${𝑋}_{𝑡}\text{ is }{\mathcal{𝐹}}_{𝑡}\text{-measurable}$$

for every $𝑡$. This means the value of ${𝑋}_{𝑡}$ is known at time $𝑡$. Adaptedness is the minimum legality condition for a process to be observable in real time.

A process can be adapted to one filtration and not another. For example, if ${\mathcal{𝐹}}_{𝑡}=𝜎({𝑋}_{𝑠}:𝑠\le 𝑡)$, then $𝑋$ is adapted to its natural filtration. If the filtration also contains future values, adaptedness may still hold but the model permits anticipatory information. That matters for stochastic integration and trading strategies.

22.4 Predictable processes

In discrete time, a process ${𝐻}_{𝑛}$ is predictable if ${𝐻}_{𝑛}$ is ${\mathcal{𝐹}}_{𝑛-1}$-measurable. The value used at time $𝑛$ must be chosen using information from before time $𝑛$. In continuous time, predictability is defined through the predictable σ-algebra, generated by left-continuous adapted processes.

Predictability is stronger than adaptedness and is essential for stochastic integration. An integrand in an Itô integral must not see the future increment it is integrating against. Predictability is the no-anticipation certificate. Without it, one can manufacture false gains or invalid integrals by using future noise.

22.5 Stopping times

A random time $𝜏$ is a stopping time if

$$\{𝜏\le 𝑡\}\in {\mathcal{𝐹}}_{𝑡}$$

for every $𝑡$. This means that by time $𝑡$, one can determine whether $𝜏$ has occurred. In discrete time, equivalently $\{𝜏=𝑛\}\in {\mathcal{𝐹}}_{𝑛}$.

Hitting times are standard examples:

$$𝜏=\inf \{𝑡:{𝑋}_{𝑡}\in 𝐴\}\mathrm{.}$$

Under suitable adaptedness and path regularity, such times are stopping times. Last exit times or times defined using future behavior are usually not stopping times. The stopping-time condition is the temporal measurability gate.

22.6 Optional times

The term “optional time” is often used synonymously with stopping time in many probability texts, though in finer continuous-time theory it connects to optional σ-algebras and optional processes. Optional processes are measurable with respect to the optional σ-algebra, generated by adapted càdlàg processes.

The distinction matters in stochastic calculus. Predictable objects are known just before time $𝑡$; optional objects are observable at time $𝑡$. Jump processes expose the difference. At a jump time, the jump itself may be optional but not predictable. This split is central to compensators, stochastic integration, and semimartingale theory.

22.7 Natural filtration

The natural filtration of a process ${𝑋}_{𝑡}$ is

$${\mathcal{𝐹}}_{𝑡}^{𝑋}=𝜎({𝑋}_{𝑠}:𝑠\le 𝑡)\mathrm{.}$$

It is the smallest filtration making $𝑋$ adapted. It represents exactly the information generated by observing the process up to time $𝑡$.

Natural filtrations are minimal carriers. In applications, one often enlarges them to include external information, initial randomness, completed null sets, or independent noise. Enlargement changes what counts as a stopping time or predictable strategy. The process is not the whole model; the filtration determines the information regime.

22.8 Completed and right-continuous filtrations

A filtration is complete if ${\mathcal{𝐹}}_0$ contains all $𝑃$-null subsets of $\mathcal{𝐹}$. It is right-continuous if

$${\mathcal{𝐹}}_{𝑡}=\bigcap _{𝑠>𝑡}{\mathcal{𝐹}}_{𝑠}\mathrm{.}$$

The usual conditions in continuous-time probability require completeness and right-continuity.

These conditions are technical but load-bearing. They improve behavior of stopping times, optional projections, martingale modifications, and debut theorems. Completing adds null information; right-continuity prevents infinitesimal information gaps immediately after $𝑡$. Many stochastic calculus theorems assume these conditions silently.

22.9 Information leakage errors

Information leakage occurs when a process, strategy, stopping rule, or integrand uses future data while being treated as nonanticipative. For example, defining ${𝐻}_{𝑛}=\mathrm{sign}({𝑋}_{𝑛+1}-{𝑋}_{𝑛})$ and using it as a trading strategy at time $𝑛$ is illegal because it depends on future movement.

The audit is always measurability against the correct σ-algebra. Adaptedness uses ${\mathcal{𝐹}}_{𝑡}$; predictability often uses ${\mathcal{𝐹}}_{𝑡-}$; stopping uses $\{𝜏\le 𝑡\}\in {\mathcal{𝐹}}_{𝑡}$. If the information gate fails, martingale fairness, optional stopping, and stochastic integration conclusions can collapse.

---

Chapter 23 — Martingales

23.1 Martingales, submartingales, supermartingales

A process $({𝑀}_{𝑡},{\mathcal{𝐹}}_{𝑡})$ is a martingale if ${𝑀}_{𝑡}$ is adapted, integrable, and

$$\mathbb{𝐸}[{𝑀}_{𝑡}\mid {\mathcal{𝐹}}_{𝑠}]={𝑀}_{𝑠},𝑠\le 𝑡\mathrm{.}$$

It is a submartingale if

$$\mathbb{𝐸}[{𝑀}_{𝑡}\mid {\mathcal{𝐹}}_{𝑠}]\ge {𝑀}_{𝑠},$$

and a supermartingale if the inequality is reversed.

A martingale is a fair process relative to the information filtration. It does not mean the path is constant or predictable. It means conditional future expectation equals present value. Submartingales have nonnegative conditional drift; supermartingales have nonpositive conditional drift.

23.2 Examples from sums, games, likelihood ratios

If ${𝑋}_{𝑖}$ are integrable martingale differences with

$$\mathbb{𝐸}[{𝑋}_{𝑖}\mid {\mathcal{𝐹}}_{𝑖-1}]=0,$$

then

$${𝑀}_{𝑛}=\sum _{𝑖=1}^{𝑛}{𝑋}_{𝑖}$$

is a martingale. Fair gambling capital under fair bets is the canonical example. If the bets have nonpositive expected gain, the capital is a supermartingale.

Likelihood ratios also produce martingales. If $𝑃$ and $𝑄$ are probability measures and ${\mathcal{𝐹}}_{𝑡}$ is increasing, the density process

$${𝐿}_{𝑡}=\frac{𝑑𝑄{\mathrm{\mid }}_{{\mathcal{𝐹}}_{𝑡}}}{𝑑𝑃{\mathrm{\mid }}_{{\mathcal{𝐹}}_{𝑡}}}$$

is a $𝑃$-martingale under suitable absolute continuity. This connects martingales to change of measure, statistics, and filtering.

23.3 Doob decomposition

In discrete time, any integrable adapted submartingale ${𝑋}_{𝑛}$ admits a decomposition

$${𝑋}_{𝑛}={𝑀}_{𝑛}+{𝐴}_{𝑛},$$

where ${𝑀}_{𝑛}$ is a martingale and ${𝐴}_{𝑛}$ is a predictable increasing process with ${𝐴}_0=0$. For a general adapted integrable process, one can decompose into martingale plus predictable drift increments.

The decomposition separates fair fluctuation from systematic drift. For a submartingale, the drift part is increasing. This is the prototype for compensators in continuous time, where counting processes are decomposed into martingale plus predictable intensity.

23.4 Martingale transforms

In discrete time, if ${𝑀}_{𝑛}$ is a martingale with increments $\mathrm{\Delta }{𝑀}_{𝑛}={𝑀}_{𝑛}-{𝑀}_{𝑛-1}$, and ${𝐻}_{𝑛}$ is predictable, then

$$(𝐻\cdot 𝑀{)}_{𝑛}=\sum _{𝑘=1}^{𝑛}{𝐻}_{𝑘}\mathrm{\Delta }{𝑀}_{𝑘}$$

is a martingale under integrability conditions. This is the martingale transform.

The predictability of ${𝐻}_{𝑘}$ is essential. It means the multiplier is chosen before observing $\mathrm{\Delta }{𝑀}_{𝑘}$. Martingale transforms model fair betting strategies and form the discrete ancestor of stochastic integrals. If ${𝐻}_{𝑘}$ can depend on future increments, the martingale property can fail.

23.5 Optional stopping theorem

The optional stopping theorem gives conditions under which

$$\mathbb{𝐸}[{𝑀}_{𝜏}]=\mathbb{𝐸}[{𝑀}_0]$$

for a martingale $𝑀$ and stopping time $𝜏$. Sufficient conditions include bounded $𝜏$, uniformly integrable stopped martingales, or suitable integrability and bounded increment conditions.

The theorem is not the false slogan “you cannot beat a martingale by stopping.” Without hypotheses, optional stopping can fail. Stopping times with infinite expectation, unbounded gains, or non-uniform integrability can create invalid expectations. The theorem is a gated result, not a universal gambling principle.

23.6 Optional sampling theorem

Optional sampling generalizes optional stopping to two stopping times $𝜎\le 𝜏$:

$$\mathbb{𝐸}[{𝑀}_{𝜏}\mid {\mathcal{𝐹}}_{𝜎}]={𝑀}_{𝜎}$$

under suitable hypotheses. For submartingales,

$$\mathbb{𝐸}[{𝑋}_{𝜏}\mid {\mathcal{𝐹}}_{𝜎}]\ge {𝑋}_{𝜎}\mathrm{.}$$

This theorem says martingale fairness persists between random observation times, provided those times are legitimate stopping times and integrability conditions hold. It is central for hitting probabilities, stopped processes, stochastic calculus, and Markov process arguments.

23.7 Maximal inequalities

Doob’s maximal inequality controls the running maximum of a submartingale. For a nonnegative submartingale ${𝑋}_{𝑘}$,

$$𝜆𝑃(\underset{𝑘\le 𝑛}{\max }{𝑋}_{𝑘}\ge 𝜆)\le \mathbb{𝐸}[{𝑋}_{𝑛}]\mathrm{.}$$

For martingales, applying this to $\mathrm{\mid }{𝑀}_{𝑘}\mathrm{\mid }$ or $\mathrm{\mid }{𝑀}_{𝑘}{\mathrm{\mid }}^{𝑝}$ gives pathwise maximum bounds.

Maximal inequalities convert terminal integrability into pathwise control over all earlier times. This is the martingale analogue of Kolmogorov’s inequality. It is indispensable for convergence theorems, stopping arguments, and tightness of stochastic processes.

23.8 Doob’s Lᵖ inequalities

For $𝑝>1$ and a martingale ${𝑀}_{𝑘}$,

$${\parallel \underset{𝑘\le 𝑛}{\max }\mathrm{\mid }{𝑀}_{𝑘}\mathrm{\mid }\parallel }_{𝑝}\le \frac{𝑝}{𝑝-1}\mathrm{\parallel }{𝑀}_{𝑛}{\mathrm{\parallel }}_{𝑝}\mathrm{.}$$

Equivalently,

$$\mathbb{𝐸}\left[\underset{𝑘\le 𝑛}{\max }\mathrm{\mid }{𝑀}_{𝑘}{\mathrm{\mid }}^{𝑝}\right]\le {\left(\frac{𝑝}{𝑝-1}\right)}^{𝑝}\mathbb{𝐸}\mathrm{\mid }{𝑀}_{𝑛}{\mathrm{\mid }}^{𝑝}\mathrm{.}$$

This inequality gives ${𝐿}^{𝑝}$ control of the entire path from ${𝐿}^{𝑝}$ control of the endpoint. It is one of the main reasons martingales are analytically tractable. The condition $𝑝>1$ matters; $𝑝=1$ has weaker forms and requires different handling.

23.9 Upcrossing inequality

For a submartingale ${𝑋}_{𝑛}$, the upcrossing inequality bounds the number of completed crossings of an interval $[𝑎,𝑏]$. If ${𝑈}_{𝑛}[𝑎,𝑏]$ is the number of upcrossings by time $𝑛$, then

$$(𝑏-𝑎)\mathbb{𝐸}[{𝑈}_{𝑛}[𝑎,𝑏]]\le \mathbb{𝐸}[({𝑋}_{𝑛}-𝑎{)}^{+}]-\mathbb{𝐸}[({𝑋}_0-𝑎{)}^{+}]\mathrm{.}$$

Upcrossings measure oscillation. If infinitely many upcrossings occur between rational levels $𝑎<𝑏$, the sequence cannot converge. The inequality is therefore a convergence engine: boundedness in ${𝐿}^1$ controls oscillation, forcing almost sure convergence of submartingales under suitable assumptions.

23.10 Martingale convergence theorem

If ${𝑀}_{𝑛}$ is a martingale bounded in ${𝐿}^1$ in the right way, such as a nonnegative supermartingale or an ${𝐿}^{𝑝}$-bounded martingale for $𝑝>1$, then ${𝑀}_{𝑛}$ converges almost surely to a limit ${𝑀}_{\mathrm{\infty }}$. Under uniform integrability,

$${𝑀}_{𝑛}=\mathbb{𝐸}[{𝑀}_{\mathrm{\infty }}\mid {\mathcal{𝐹}}_{𝑛}]$$

and convergence also holds in ${𝐿}^1$.

The theorem is a central pathwise compactness result. Martingales do not have deterministic monotonicity, but their conditional fairness plus integrability prevents endless profitable oscillation. Upcrossing control is the core mechanism.

23.11 Uniform integrability of martingales

A martingale $({𝑀}_{𝑛})$ is uniformly integrable if

$$\underset{𝐾\to \mathrm{\infty }}{\lim }\underset{𝑛}{\sup }\mathbb{𝐸}[\mathrm{\mid }{𝑀}_{𝑛}\mathrm{\mid }{1}_{\{\mathrm{\mid }{𝑀}_{𝑛}\mathrm{\mid }>𝐾\}}]=0.$$

Uniform integrability is exactly the condition that allows ${𝐿}^1$ terminal representation:

$${𝑀}_{𝑛}=\mathbb{𝐸}[{𝑀}_{\mathrm{\infty }}\mid {\mathcal{𝐹}}_{𝑛}]\mathrm{.}$$

Boundedness in ${𝐿}^{𝑝}$ for $𝑝>1$ implies uniform integrability. Without uniform integrability, a martingale can converge almost surely but lose mass in expectation. This is the martingale version of the general expectation convergence problem.

23.12 Reverse martingales

A reverse martingale is adapted to a decreasing sequence of σ-algebras ${\mathcal{𝐺}}_{𝑛}$, with

$$\mathbb{𝐸}[{𝑋}_{𝑛}\mid {\mathcal{𝐺}}_{𝑛+1}]={𝑋}_{𝑛+1}$$

when ${\mathcal{𝐺}}_{𝑛+1}\subseteq {\mathcal{𝐺}}_{𝑛}$. A standard example is

$${𝑋}_{𝑛}=\mathbb{𝐸}[𝑋\mid {\mathcal{𝐺}}_{𝑛}]\mathrm{.}$$

Reverse martingales converge almost surely and in ${𝐿}^1$ under integrability. They are used in exchangeability, de Finetti theory, ergodic theory, and tail σ-algebra arguments. They describe what remains when information is progressively forgotten rather than accumulated.

23.13 Martingale difference sequences

A martingale difference sequence satisfies

$$\mathbb{𝐸}[{𝐷}_{𝑛}\mid {\mathcal{𝐹}}_{𝑛-1}]=0.$$

Then

$${𝑀}_{𝑛}=\sum _{𝑘=1}^{𝑛}{𝐷}_{𝑘}$$

is a martingale. Martingale differences generalize independent centered variables; they allow dependence while preserving zero conditional drift.

Many limit theorems and concentration inequalities for independent sums extend to martingale differences with conditional variance and boundedness assumptions. The adapted conditional-mean-zero property is the replacement certificate for independence.

---

Chapter 24 — Martingale Limit and Concentration Tools

24.1 Azuma–Hoeffding inequality

If ${𝑀}_{𝑛}$ is a martingale with bounded increments

$$\mathrm{\mid }{𝑀}_{𝑘}-{𝑀}_{𝑘-1}\mathrm{\mid }\le {𝑐}_{𝑘},$$

then

$$𝑃({𝑀}_{𝑛}-{𝑀}_0\ge 𝑡)\le \exp (-\frac{{𝑡}^2}{2\sum _{𝑘=1}^{𝑛}{𝑐}_{𝑘}^2})\mathrm{.}$$

A two-sided version gives

$$𝑃(\mathrm{\mid }{𝑀}_{𝑛}-{𝑀}_0\mathrm{\mid }\ge 𝑡)\le 2\exp (-\frac{{𝑡}^2}{2\sum {𝑐}_{𝑘}^2})\mathrm{.}$$

Azuma–Hoeffding is a concentration theorem for bounded martingale differences. It converts conditional fairness plus bounded step size into subgaussian tails. It is especially useful when independence is absent but exposure martingales can be constructed.

24.2 Freedman inequality

Freedman’s inequality refines Azuma by using conditional variance. If ${𝑀}_{𝑛}$ has increments bounded above by $𝑏$, and predictable quadratic variation

$${𝑉}_{𝑛}=\sum _{𝑘=1}^{𝑛}\mathbb{𝐸}[(\mathrm{\Delta }{𝑀}_{𝑘}{)}^2\mid {\mathcal{𝐹}}_{𝑘-1}],$$

then

$$𝑃({𝑀}_{𝑛}\ge 𝑡,{𝑉}_{𝑛}\le 𝑣)\le \exp (-\frac{{𝑡}^2}{2(𝑣+𝑏𝑡\mathrm{/}3)})\mathrm{.}$$

This inequality is stronger when actual variance is much smaller than the worst-case squared increment bound. It is the martingale analogue of Bernstein concentration. It separates two debts: fluctuation variance $𝑣$ and maximum jump $𝑏$.

24.3 Burkholder–Davis–Gundy inequalities

The BDG inequalities relate moments of a martingale’s maximum to moments of its quadratic variation. In continuous time, for $𝑝>0$,

$$\mathbb{𝐸}\left[\underset{𝑡\le 𝑇}{\sup }\mathrm{\mid }{𝑀}_{𝑡}{\mathrm{\mid }}^{𝑝}\right]{\asymp }_{𝑝}\mathbb{𝐸}[[𝑀{]}_{𝑇}^{𝑝\mathrm{/}2}]$$

for suitable martingales. Here $[𝑀{]}_{𝑇}$ is quadratic variation.

BDG is a deep path-control theorem. It says the size of a martingale path is governed by accumulated quadratic variation. This is foundational for stochastic integration, SDE estimates, tightness, and regularity of martingale-driven processes.

24.4 Exponential martingales

Exponential martingales are built by exponentiating a martingale and subtracting compensating drift. For Brownian motion ${𝐵}_{𝑡}$,

$$\exp (𝜃{𝐵}_{𝑡}-\frac{1}{2}{𝜃}^2𝑡)$$

is a martingale. In discrete time, exponential supermartingales underlie Chernoff, Hoeffding, Bernstein, and Freedman bounds.

The compensation term is the log-moment correction. Exponential martingales turn tail events into optional stopping or maximal inequality problems. They are also the mechanism behind Girsanov change of measure: exponential martingales become Radon–Nikodym densities.

24.5 Stopping and localization

Localization handles processes that are well-behaved only up to random times. A process ${𝑀}_{𝑡}$ is a local martingale if there exists an increasing sequence of stopping times ${𝜏}_{𝑛}\uparrow \mathrm{\infty }$ such that

$${𝑀}_{𝑡\wedge {𝜏}_{𝑛}}$$

is a martingale for each $𝑛$.

Localization permits stochastic calculus for unbounded or explosive processes by proving statements on bounded stopped intervals and then passing to the limit. The liftback requires care: local martingales need not be true martingales, and expectation identities may fail without integrability or uniform integrability.

24.6 Quadratic variation

Quadratic variation measures accumulated squared increments. For a continuous martingale $𝑀$,

$$[𝑀{]}_{𝑡}$$

is the increasing process such that

$${𝑀}_{𝑡}^2-[𝑀{]}_{𝑡}$$

is a martingale. For Brownian motion,

$$[𝐵{]}_{𝑡}=𝑡\mathrm{.}$$

Quadratic variation is the second-order clock of martingale fluctuation. In stochastic calculus, it is why Itô’s formula has a second derivative term. Ordinary differentiable paths have zero quadratic variation; Brownian paths have finite nonzero quadratic variation. This is the analytic signature of stochastic motion.

24.7 Martingale CLT

A martingale central limit theorem gives Gaussian limits for martingale difference arrays. If ${𝐷}_{𝑛,𝑘}$ are martingale differences and the conditional variance sums converge:

$$\sum _{𝑘}\mathbb{𝐸}[{𝐷}_{𝑛,𝑘}^2\mid {\mathcal{𝐹}}_{𝑛,𝑘-1}]\to {𝜎}^2$$

in probability, and a conditional Lindeberg condition holds, then

$$\sum _{𝑘}{𝐷}_{𝑛,𝑘}\Rightarrow 𝑁(0,{𝜎}^2)\mathrm{.}$$

The theorem replaces independence with conditional mean-zero plus conditional variance stabilization. It is crucial in dependent asymptotics, stochastic approximation, statistics, Markov chains, and random environments. The conditional variance process is the carrier of limiting Gaussian scale.

24.8 Concentration via bounded differences

If a function $𝑓({𝑋}_1,\dots ,{𝑋}_{𝑛})$ changes by at most ${𝑐}_{𝑖}$ when only coordinate $𝑖$ is changed, and the ${𝑋}_{𝑖}$ are independent, define the Doob martingale

$${𝑀}_{𝑖}=\mathbb{𝐸}[𝑓({𝑋}_1,\dots ,{𝑋}_{𝑛})\mid {𝑋}_1,\dots ,{𝑋}_{𝑖}]\mathrm{.}$$

Its increments are bounded by ${𝑐}_{𝑖}$, so Azuma–Hoeffding yields

$$𝑃(𝑓-\mathbb{𝐸}𝑓\ge 𝑡)\le \exp (-\frac{2{𝑡}^2}{\sum _{𝑖}{𝑐}_{𝑖}^2})$$

up to convention-dependent constants.

This is McDiarmid’s bounded differences inequality. It converts low sensitivity to individual inputs into concentration. The martingale is an exposure process: reveal inputs one at a time and track the conditional expectation.

24.9 Algorithmic and combinatorial applications

Martingale concentration applies to random graphs, randomized algorithms, hashing, load balancing, random permutations, and combinatorial optimization. The typical method is to expose random choices sequentially, define a Doob martingale, bound the effect of each exposure, and apply Azuma, Freedman, or a refined inequality.

The key is not independence alone but controlled sensitivity under information revelation. If a single input can change the output drastically, bounded differences fail. If conditional variances are small, Freedman can improve bounds. The proof engine is filtration design plus increment audit.

---

Chapter 25 — Random Sequences and Processes

25.1 Process as random element in path space

A stochastic process $({𝑋}_{𝑡}{)}_{𝑡\in 𝑇}$ can be viewed as a random element

$$𝑋:\mathrm{\Omega }\to {𝑆}^{𝑇},𝑋(𝜔)=({𝑋}_{𝑡}(𝜔){)}_{𝑡\in 𝑇}\mathrm{.}$$

The path space ${𝑆}^{𝑇}$ carries a σ-algebra, often generated by coordinate maps. Under stronger regularity, the law may live on $𝐶[0,𝑇]$, $𝐷[0,𝑇]$, or another structured path space.

This viewpoint treats the whole trajectory as one random object. Finite-dimensional distributions become projections of the path law. Path properties such as continuity, bounded variation, hitting times, and oscillation become events in path space, requiring measurability and often regularity certificates.

25.2 Finite-dimensional distributions

The finite-dimensional distributions of a process are the laws of

$$({𝑋}_{{𝑡}_1},\dots ,{𝑋}_{{𝑡}_{𝑘}})$$

for finite choices ${𝑡}_1,\dots ,{𝑡}_{𝑘}$. These distributions describe all finite coordinate observations. They are necessary data for the process law.

For many process constructions, one first specifies finite-dimensional distributions and then checks consistency. But finite-dimensional distributions alone do not determine path regularity unless the path-space carrier is fixed. A process with continuous paths and a process with wild paths may share finite-dimensional distributions unless modifications are controlled.

25.3 Consistency conditions

Finite-dimensional distributions ${𝜇}_{{𝑡}_1,\dots ,{𝑡}_{𝑘}}$ must be compatible under permutation and marginalization. If one forgets a coordinate, the larger distribution must project to the smaller one:

$$(𝜋{)}_{\ast }{𝜇}_{{𝑡}_1,\dots ,{𝑡}_{𝑘}}={𝜇}_{{𝑠}_1,\dots ,{𝑠}_{𝑚}}\mathrm{.}$$

They must also be invariant under relabeling of repeated or reordered indices.

Consistency is the finite-data certificate for a global process law. Without it, no process can have those finite-dimensional distributions. With it, extension theorems can build a measure on a product path space under suitable assumptions.

25.4 Kolmogorov extension theorem

Kolmogorov’s extension theorem says that a consistent family of finite-dimensional distributions on suitable state spaces defines a probability measure on the infinite product space ${𝑆}^{𝑇}$. The coordinate process on that space then has the prescribed finite-dimensional laws.

The theorem provides existence of a process as a product-space random element. It does not guarantee continuous or measurable sample paths beyond the product σ-algebra. Regularity is a separate payload. This distinction is essential for Brownian motion: extension gives a process with Gaussian finite-dimensional distributions; continuity requires additional work.

25.5 Sample-path regularity

Sample-path regularity concerns properties of $𝑡\mapsto {𝑋}_{𝑡}(𝜔)$: continuity, càdlàg behavior, bounded variation, Hölder regularity, differentiability, or jump structure. These are not determined merely by one-time marginal laws.

Kolmogorov’s continuity criterion is a standard regularity tool. If

$$\mathbb{𝐸}\mathrm{\mid }{𝑋}_{𝑡}-{𝑋}_{𝑠}{\mathrm{\mid }}^{𝛼}\le 𝐶\mathrm{\mid }𝑡-𝑠{\mathrm{\mid }}^{1+𝛽}$$

for suitable $𝛼,𝛽>0$, then the process has a continuous modification with Hölder-type regularity below a threshold. Moment bounds on increments become path regularity certificates.

25.6 Separability

Separability reduces uncountable-time behavior to countable dense-time behavior. A process is separable, roughly, if its path properties can be determined by values on a countable dense subset, up to null sets. This prevents uncountable measurability traps.

In continuous-time probability, events like

$$\underset{𝑡\in [0,𝑇]}{\sup }{𝑋}_{𝑡}>𝑎$$

may not be measurable without path regularity or separability. If paths are continuous or càdlàg, the supremum over an interval can be reduced to a supremum over rational times. This is why path-space regularity and separability are not optional technicalities.

25.7 Modification and indistinguishability

Two processes ${𝑋}_{𝑡}$ and ${𝑌}_{𝑡}$ are modifications of each other if for every fixed $𝑡$,

$$𝑃({𝑋}_{𝑡}={𝑌}_{𝑡})=1.$$

They are indistinguishable if

$$𝑃({𝑋}_{𝑡}={𝑌}_{𝑡}\text{ for all }𝑡)=1.$$

Indistinguishability is stronger, especially when $𝑇$ is uncountable.

A fixed-time null set may depend on $𝑡$, and uncountably many null sets cannot be unioned safely. If processes have continuous paths, modification often implies indistinguishability under suitable conditions because equality on a countable dense set extends by continuity. Without regularity, the implication can fail.

25.8 Stationarity

A process is stationary if its finite-dimensional distributions are invariant under time shifts:

whenever the shifted indices are valid. Strict stationarity is full finite-dimensional invariance. Weak stationarity usually means constant mean and covariance depending only on time lag.

Stationarity is symmetry in time. It does not imply independence or ergodicity. A process may have the same distribution at all times while retaining long-range dependence or latent random environment. Stationarity supplies invariant law; ergodicity determines whether time averages collapse to constants.

25.9 Ergodicity

A stationary process is ergodic if shift-invariant events have probability $0$ or $1$. Equivalently, time averages converge to deterministic expectations rather than random invariant components. For an ergodic stationary sequence,

$$\frac{1}{𝑛}\sum _{𝑘=1}^{𝑛}𝑓({𝑋}_{𝑘})\to \mathbb{𝐸}[𝑓({𝑋}_1)]$$

almost surely under integrability.

Without ergodicity, the limit is conditional on the invariant σ-algebra:

$$\frac{1}{𝑛}\sum _{𝑘=1}^{𝑛}𝑓({𝑋}_{𝑘})\to \mathbb{𝐸}[𝑓({𝑋}_1)\mid \mathcal{𝐼}]\mathrm{.}$$

Thus ergodicity is the gate that turns ensemble averages into time averages.

25.10 Mixing

Mixing is asymptotic independence between distant time regions. A strong mixing condition may take the form

$$𝛼(𝑛)=\underset{𝐴\in {\mathcal{𝐹}}_{-\mathrm{\infty }}^0,𝐵\in {\mathcal{𝐹}}_{𝑛}^{\mathrm{\infty }}}{\sup }\mathrm{\mid }𝑃(𝐴\cap 𝐵)-𝑃(𝐴)𝑃(𝐵)\mathrm{\mid }\to 0.$$

Different mixing notions measure different strengths of dependence decay.

Mixing usually implies ergodicity, but it is stronger. It supports CLTs, concentration, empirical process results, and statistical consistency for dependent data. The rate of mixing matters: summable mixing coefficients give stronger limit theorems than mere convergence to zero.

25.11 Process-level carrier debt

A stochastic process claim must declare its path carrier, filtration, finite-dimensional laws, regularity, modification class, and convergence topology. A statement about ${𝑋}_{𝑡}$ at each fixed $𝑡$ is weaker than a statement about the whole path. A statement about finite-dimensional convergence is weaker than weak convergence in path space.

Common process-level errors include treating modifications as indistinguishable, assuming sample continuity from Gaussian marginals, using stopping times without a filtration, or applying continuous mapping arguments in the wrong path topology. Process probability is not coordinate probability repeated; it is probability on trajectories.

---

Chapter 26 — Markov Chains

26.1 Markov property

A discrete-time process ${𝑋}_{𝑛}$ is Markov if, conditional on the present, the future is independent of the past:

$$𝑃({𝑋}_{𝑛+1}\in 𝐴\mid {𝑋}_0,\dots ,{𝑋}_{𝑛})=𝑃({𝑋}_{𝑛+1}\in 𝐴\mid {𝑋}_{𝑛})\mathrm{.}$$

For time-homogeneous chains, this conditional law depends only on ${𝑋}_{𝑛}$, not on $𝑛$.

The Markov property is a conditional independence statement. It does not say the variables are independent; dependence passes through the current state. The present state is a sufficient statistic for future prediction. This is the carrier principle behind transition kernels and semigroups.

26.2 Transition kernels

A transition kernel $𝑃(𝑥,𝐴)$ gives the probability of moving from state $𝑥$ to a measurable set $𝐴$. For countable spaces, this is a matrix

$${𝑃}_{𝑖𝑗}=𝑃({𝑋}_{𝑛+1}=𝑗\mid {𝑋}_{𝑛}=𝑖)\mathrm{.}$$

The $𝑛$-step transition kernel is obtained by composition:

$${𝑃}^{𝑛}(𝑥,𝐴)=𝑃({𝑋}_{𝑛}\in 𝐴\mid {𝑋}_0=𝑥)\mathrm{.}$$

Kernels are probability-valued functions of the current state. They define the chain law once an initial distribution $𝜇$ is specified. The finite-dimensional law is

$$𝜇(𝑑{𝑥}_0)𝑃({𝑥}_0,𝑑{𝑥}_1)\cdots 𝑃({𝑥}_{𝑛-1},𝑑{𝑥}_{𝑛})\mathrm{.}$$

This factorization is the Markov certificate.

26.3 Finite-state chains

For a finite state space, the transition kernel is a stochastic matrix $𝑃$ with rows summing to one. If the distribution at time $𝑛$ is row vector ${𝜇}_{𝑛}$, then

$${𝜇}_{𝑛+1}={𝜇}_{𝑛}𝑃,{𝜇}_{𝑛}={𝜇}_0{𝑃}^{𝑛}\mathrm{.}$$

Finite chains reduce many questions to matrix analysis. Stationary distributions solve

$$𝜋=𝜋𝑃\mathrm{.}$$

Long-term behavior depends on irreducibility, periodicity, and spectral structure. Finite-state chains are the cleanest setting for mixing time and convergence to equilibrium.

26.4 Countable-state chains

Countable-state chains require more classification than finite chains because recurrence, transience, and invariant measures become subtle. Transition probabilities still form a stochastic matrix, but stationary distributions may fail to exist or may not be normalizable.

A simple random walk on $\mathbb{𝑍}$ is recurrent but has no probability stationary distribution. A biased random walk on $\mathbb{𝑍}$ is transient. Countable chains force separation between invariant measures, stationary probabilities, recurrence classes, and positive recurrence.

26.5 Communicating classes

State $𝑖$ reaches state $𝑗$, written $𝑖\to 𝑗$, if

$${𝑃}^{𝑛}(𝑖,𝑗)>0$$

for some $𝑛$. States communicate if $𝑖\to 𝑗$ and $𝑗\to 𝑖$. Communication is an equivalence relation; equivalence classes are communicating classes.

A class is closed if the chain cannot leave it once entered. Irreducibility means all states communicate. Long-run behavior is analyzed class by class. Transient states may lead into closed recurrent classes; closed classes trap the chain.

26.6 Recurrence and transience

A state $𝑖$ is recurrent if, starting from $𝑖$, the chain returns to $𝑖$ with probability one. It is transient if the return probability is less than one. Equivalently, recurrence can be characterized by

$$\sum _{𝑛=0}^{\mathrm{\infty }}{𝑃}^{𝑛}(𝑖,𝑖)=\mathrm{\infty },$$

while transience corresponds to finiteness of this sum.

In irreducible chains, recurrence or transience is a class property. Positive recurrence means expected return time is finite; null recurrence means return occurs almost surely but with infinite expected return time. Stationary probability distributions exist for irreducible positive recurrent chains.

26.7 Stationary distributions

A stationary distribution $𝜋$ satisfies

$$𝜋𝑃=𝜋\mathrm{.}$$

If ${𝑋}_0\sim 𝜋$, then ${𝑋}_{𝑛}\sim 𝜋$ for all $𝑛$. Stationarity is distributional invariance under the transition kernel.

For finite irreducible chains, a stationary distribution exists and is unique. For countable chains, existence of a probability stationary distribution corresponds to positive recurrence under irreducibility. Stationarity does not automatically mean convergence to stationarity; periodicity can obstruct convergence.

26.8 Reversibility

A chain with stationary distribution $𝜋$ is reversible if

$${𝜋}_{𝑖}{𝑃}_{𝑖𝑗}={𝜋}_{𝑗}{𝑃}_{𝑗𝑖}$$

for all states $𝑖,𝑗$. This detailed balance condition says the stationary flow from $𝑖$ to $𝑗$ equals the flow from $𝑗$ to $𝑖$.

Reversibility implies stationarity by summing over $𝑖$:

$$\sum _{𝑖}{𝜋}_{𝑖}{𝑃}_{𝑖𝑗}=\sum _{𝑖}{𝜋}_{𝑗}{𝑃}_{𝑗𝑖}={𝜋}_{𝑗}\mathrm{.}$$

Reversible chains are analytically tractable because the transition operator is self-adjoint in ${𝐿}^2(𝜋)$. This connects Markov chains to spectral theory.

26.9 Detailed balance

Detailed balance is the local equation

$$𝜋(𝑥)𝑃(𝑥,𝑑𝑦)=𝜋(𝑦)𝑃(𝑦,𝑑𝑥)$$

in general-state notation. It is stronger than stationarity, which only requires global balance:

$$\int 𝜋(𝑑𝑥)𝑃(𝑥,𝐴)=𝜋(𝐴)\mathrm{.}$$

Detailed balance is widely used to construct Markov chains with desired stationary distribution. In MCMC, Metropolis–Hastings transition rules are engineered to satisfy detailed balance. The cost is that reversible chains may be slower than nonreversible alternatives; reversibility is a certificate, not always an optimal design.

26.10 Hitting times

For a set $𝐴$, the hitting time is

$${𝜏}_{𝐴}=\inf \{𝑛\ge 0:{𝑋}_{𝑛}\in 𝐴\}\mathrm{.}$$

Hitting probabilities solve harmonic equations. For $ℎ(𝑖)={𝑃}_{𝑖}({𝜏}_{𝐴}<{𝜏}_{𝐵})$,

$$ℎ(𝑖)=\sum _{𝑗}{𝑃}_{𝑖𝑗}ℎ(𝑗)$$

on states outside $𝐴\cup 𝐵$, with boundary values $ℎ=1$ on $𝐴$, $ℎ=0$ on $𝐵$.

Expected hitting times solve Poisson equations. If $𝑔(𝑖)={\mathbb{𝐸}}_{𝑖}[{𝜏}_{𝐴}]$, then

$$𝑔(𝑖)=1+\sum _{𝑗}{𝑃}_{𝑖𝑗}𝑔(𝑗)$$

outside $𝐴$, with $𝑔=0$ on $𝐴$. Hitting times connect Markov chains to potential theory.

26.11 Absorbing chains

An absorbing state $𝑎$ satisfies ${𝑃}_{𝑎𝑎}=1$. Once entered, it cannot be left. More generally, a closed class acts as an absorbing region. Absorbing chains are analyzed by separating transient and absorbing states.

For finite absorbing chains with transient submatrix $𝑄$, the fundamental matrix is

$$𝑁=(𝐼-𝑄{)}^{-1}=\sum _{𝑛=0}^{\mathrm{\infty }}{𝑄}^{𝑛}\mathrm{.}$$

The entry ${𝑁}_{𝑖𝑗}$ gives the expected number of visits to transient state $𝑗$ starting from $𝑖$. Absorption probabilities and expected absorption times follow from $𝑁$.

26.12 Mixing time

For an irreducible aperiodic finite chain with stationary distribution $𝜋$, the mixing time measures how long until the law is close to $𝜋$. In total variation,

$${𝑡}_{\mathrm{m}\mathrm{i}\mathrm{x}}(𝜀)=\min \{𝑡:\underset{𝑥}{\sup }\mathrm{\parallel }{𝑃}^{𝑡}(𝑥,\cdot )-𝜋{\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}\le 𝜀\}\mathrm{.}$$

Mixing time quantifies convergence to equilibrium. It depends on bottlenecks, spectral gap, conductance, coupling, and geometry of the state space. Stationarity alone does not give speed. Mixing analysis is about finite-time convergence, not merely limiting invariance.

26.13 Coupling methods

A coupling of two chains constructs $({𝑋}_{𝑛},{𝑌}_{𝑛})$ on one probability space with each marginal following the chain. If they meet and stay together after time $𝑇$, then

$$\mathrm{\parallel }{𝑃}^{𝑛}(𝑥,\cdot )-{𝑃}^{𝑛}(𝑦,\cdot ){\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}\le 𝑃(𝑇>𝑛)\mathrm{.}$$

Coupling bounds mixing by controlling coalescence time.

Good couplings exploit contraction, monotonicity, shared randomness, or geometry. Coupling is not automatic; it is a designed joint carrier. The success of the method depends on how quickly the coupled chains meet under the chosen construction.

26.14 Spectral gap

For a reversible finite chain, the spectral gap is

$$𝛾=1-{𝜆}_2$$

where ${𝜆}_2$ is the second-largest eigenvalue in appropriate ordering. A positive spectral gap implies exponential convergence to equilibrium. Roughly,

$$\mathrm{\parallel }{𝑃}^{𝑛}(𝑥,\cdot )-𝜋{\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}$$

decays at a rate controlled by $(1-𝛾{)}^{𝑛}$, with constants depending on $𝜋$.

The spectral gap measures relaxation speed in ${𝐿}^2(𝜋)$. It is connected to Poincaré inequalities, conductance, and geometry. Small spectral gap indicates bottlenecks or slow modes. Spectral methods are powerful but usually require reversibility or operator control.

26.15 Markov chain Monte Carlo preview

MCMC constructs a Markov chain whose stationary distribution is a target law $𝜋$. After running the chain, samples are used to estimate

$$\int 𝑓𝑑𝜋\mathrm{.}$$

Metropolis–Hastings, Gibbs sampling, and Langevin algorithms are standard examples.

The validity of MCMC requires more than stationarity. One needs convergence to stationarity, adequate mixing, diagnostic control, and error estimates. A chain that has $𝜋$ as invariant law but mixes slowly may be computationally useless. The MCMC audit is invariant distribution plus convergence rate plus sampling error.

---

Chapter 27 — Markov Processes

27.1 Markov kernels on general spaces

On a measurable state space $(𝑆,\mathcal{𝑆})$, a Markov kernel $𝑃(𝑥,𝐴)$ satisfies: for each $𝑥$, $𝐴\mapsto 𝑃(𝑥,𝐴)$ is a probability measure, and for each $𝐴$, $𝑥\mapsto 𝑃(𝑥,𝐴)$ is measurable. It defines transition probabilities from states to measurable sets.

General kernels replace matrices. The Markov property becomes

$$𝑃({𝑋}_{𝑡+𝑠}\in 𝐴\mid {\mathcal{𝐹}}_{𝑡})={𝑃}_{𝑠}({𝑋}_{𝑡},𝐴)$$

for a transition family ${𝑃}_{𝑠}$. The measurable-space structure is essential; without it, the transition probability as a function of state may not be well-defined.

27.2 Semigroups

A Markov semigroup $({𝑃}_{𝑡}{)}_{𝑡\ge 0}$ acts on functions by

$${𝑃}_{𝑡}𝑓(𝑥)={\mathbb{𝐸}}_{𝑥}[𝑓({𝑋}_{𝑡})]\mathrm{.}$$

It satisfies

$${𝑃}_0=𝐼,{𝑃}_{𝑡+𝑠}={𝑃}_{𝑡}{𝑃}_{𝑠}\mathrm{.}$$

The second identity is the Chapman–Kolmogorov equation and encodes the Markov property.

Semigroups are operator carriers for Markov processes. Instead of tracking paths directly, one studies how expectations evolve. Semigroup methods connect probability to PDE, functional analysis, spectral theory, and generators.

27.3 Strong Markov property

The strong Markov property says the Markov property holds at stopping times. If $𝜏$ is a stopping time, then conditional on ${\mathcal{𝐹}}_{𝜏}$, the post-$𝜏$ process behaves like a fresh process started from ${𝑋}_{𝜏}$:

$$\mathbb{𝐸}[𝑓({𝑋}_{𝜏+𝑡})\mid {\mathcal{𝐹}}_{𝜏}]={𝑃}_{𝑡}𝑓({𝑋}_{𝜏})\mathrm{.}$$

This is stronger than the deterministic-time Markov property. It is essential for hitting times, excursions, renewal arguments, boundary problems, and potential theory. The stopping-time condition is load-bearing; arbitrary random times do not generally permit Markov restart.

27.4 Feller processes

A Feller semigroup acts on ${𝐶}_0(𝑆)$, the continuous functions vanishing at infinity, and satisfies

$${𝑃}_{𝑡}:{𝐶}_0(𝑆)\to {𝐶}_0(𝑆),\underset{𝑡\downarrow 0}{\lim }\mathrm{\parallel }{𝑃}_{𝑡}𝑓-𝑓{\mathrm{\parallel }}_{\mathrm{\infty }}=0.$$

A Markov process with such a semigroup is called a Feller process.

Feller structure connects stochastic processes to analytic generators. It gives regular dependence on initial conditions and supports construction through semigroup theory. Many diffusions, Lévy processes, and jump processes are Feller under suitable assumptions.

27.5 Generators

The generator $𝐴$ of a semigroup is

$$𝐴𝑓=\underset{𝑡\downarrow 0}{\lim }\frac{{𝑃}_{𝑡}𝑓-𝑓}{𝑡}$$

for functions $𝑓$ where the limit exists. It describes infinitesimal evolution. For a diffusion,

$$𝐴𝑓=𝑏(𝑥)\cdot \mathrm{\nabla }𝑓(𝑥)+\frac{1}{2}\mathrm{Tr}(𝑎(𝑥){\mathrm{\nabla }}^2𝑓(𝑥))\mathrm{.}$$

The generator is the local analytic fingerprint of a Markov process. It determines PDEs, martingale problems, transition behavior, and invariant measures. The domain of $𝐴$ is part of the data; writing a differential expression without specifying domain can be incomplete.

27.6 Resolvents

The resolvent of a semigroup is

$${𝑅}_{𝜆}𝑓={\int }_0^{\mathrm{\infty }}{𝑒}^{-𝜆𝑡}{𝑃}_{𝑡}𝑓𝑑𝑡,𝜆>0.$$

It satisfies

$${𝑅}_{𝜆}=(𝜆𝐼-𝐴{)}^{-1}$$

formally, when $𝐴$ is the generator.

Resolvents encode potential behavior: expected discounted occupation times. They are useful for constructing processes, solving boundary value problems, and analyzing recurrence. In Markov process theory, semigroup, generator, and resolvent are three equivalent analytic views when the carrier is well-behaved.

27.7 Invariant measures

A probability measure $𝜋$ is invariant if

$$\int {𝑃}_{𝑡}(𝑥,𝐴)𝜋(𝑑𝑥)=𝜋(𝐴)$$

for all $𝑡$ and measurable $𝐴$. Equivalently,

$$\int {𝑃}_{𝑡}𝑓𝑑𝜋=\int 𝑓𝑑𝜋\mathrm{.}$$

If ${𝑋}_0\sim 𝜋$, then ${𝑋}_{𝑡}\sim 𝜋$ for all $𝑡$.

Invariant measures describe equilibrium laws. Existence, uniqueness, and convergence to equilibrium require additional recurrence, irreducibility, Lyapunov, or compactness conditions. In continuous spaces, invariant measures may exist but convergence may fail or depend strongly on topology.

27.8 Harris recurrence

Harris recurrence is a recurrence notion for general-state Markov processes or chains. Roughly, a process is Harris recurrent if it visits every set of positive measure infinitely often with probability one, relative to an irreducibility measure.

This theory replaces communicating classes from countable chains. It provides the correct framework for ergodic theorems, invariant probability measures, and convergence in total variation on general spaces. Petite sets, minorization, and Lyapunov functions often appear as certificates for Harris recurrence and geometric ergodicity.

27.9 Regeneration

Regeneration occurs when a process probabilistically restarts, forgetting its past. A regeneration time $𝜏$ splits the path into independent or conditionally independent cycles. For regenerative processes, long-run averages reduce to renewal-reward ratios:

$$\underset{𝑡\to \mathrm{\infty }}{\lim }\frac{1}{𝑡}{\int }_0^{𝑡}𝑓({𝑋}_{𝑠})𝑑𝑠=\frac{\mathbb{𝐸}{\int }_0^{𝜏}𝑓({𝑋}_{𝑠})𝑑𝑠}{\mathbb{𝐸}𝜏}$$

under suitable conditions.

Regeneration is a powerful replacement for independence. It decomposes dependent paths into iid cycles. In Markov chains, regeneration can be constructed through atoms or minorization/splitting techniques.

27.10 Continuous-time Markov chains

A continuous-time Markov chain on a countable state space has transition rates ${𝑞}_{𝑖𝑗}$ for $𝑖\mathrm{\ne }𝑗$, with

$${𝑞}_{𝑖}=\sum _{𝑗\mathrm{\ne }𝑖}{𝑞}_{𝑖𝑗}\mathrm{.}$$

The chain waits an exponential time with rate ${𝑞}_{𝑖}$ in state $𝑖$, then jumps to $𝑗$ with probability ${𝑞}_{𝑖𝑗}\mathrm{/}{𝑞}_{𝑖}$.

The generator matrix $𝑄$ has entries

$${𝑄}_{𝑖𝑗}={𝑞}_{𝑖𝑗},𝑖\mathrm{\ne }𝑗,{𝑄}_{𝑖𝑖}=-{𝑞}_{𝑖}\mathrm{.}$$

Transition probabilities satisfy Kolmogorov equations:

$$\frac{𝑑}{𝑑𝑡}{𝑃}_{𝑡}={𝑃}_{𝑡}𝑄$$

or $𝑄{𝑃}_{𝑡}$, depending on forward/backward formulation. Explosion control is required if infinitely many jumps can occur in finite time.

27.11 Jump processes

Jump processes evolve by sudden discontinuities rather than continuous paths. A pure jump Markov process is specified by jump rates or a jump kernel. More generally, jump-diffusions combine continuous diffusion with jumps.

The generator often has the form

$$𝐴𝑓(𝑥)=𝑏(𝑥)\cdot \mathrm{\nabla }𝑓(𝑥)+\int (𝑓(𝑦)-𝑓(𝑥))𝑞(𝑥,𝑑𝑦),$$

possibly with compensation for small jumps. Jump processes require càdlàg path spaces and careful handling of predictable compensators, intensities, and explosion.

27.12 Birth-death processes

A birth-death process is a continuous-time Markov chain on $\mathbb{𝑁}$ with transitions only from $𝑛$ to $𝑛+1$ or $𝑛-1$. Birth rates are ${𝜆}_{𝑛}$, death rates are ${𝜇}_{𝑛}$. The generator is

$$𝐴𝑓(𝑛)={𝜆}_{𝑛}[𝑓(𝑛+1)-𝑓(𝑛)]+{𝜇}_{𝑛}[𝑓(𝑛-1)-𝑓(𝑛)]\mathrm{.}$$

Birth-death processes model queues, populations, branching-like systems, and chemical reactions. Stationary distributions, when they exist, often satisfy detailed balance:

$${𝜋}_{𝑛}{𝜆}_{𝑛}={𝜋}_{𝑛+1}{𝜇}_{𝑛+1}\mathrm{.}$$

Recurrence and explosion depend on the rate growth.

27.13 Martingale problems

Given an operator $𝐴$, the martingale problem asks for a process ${𝑋}_{𝑡}$ such that for all test functions $𝑓$ in the domain,

$$𝑓({𝑋}_{𝑡})-𝑓({𝑋}_0)-{\int }_0^{𝑡}𝐴𝑓({𝑋}_{𝑠})𝑑𝑠$$

is a martingale. Solving the martingale problem constructs a Markov process with generator $𝐴$.

This formulation avoids directly solving SDEs or transition densities. It is robust under weak convergence and is central in diffusion theory. Well-posedness means existence and uniqueness in law. If the martingale problem is well-posed, the generator determines the process law.

---

Chapter 28 — Brownian Motion

28.1 Construction of Brownian motion

Brownian motion ${𝐵}_{𝑡}$ is a process with ${𝐵}_0=0$, independent increments, Gaussian increments

$${𝐵}_{𝑡}-{𝐵}_{𝑠}\sim 𝑁(0,𝑡-𝑠),$$

and continuous paths. Kolmogorov extension constructs a process with the correct finite-dimensional Gaussian laws; continuity is then obtained through moment estimates and a continuity theorem.

The construction separates law specification from path regularity. Gaussian finite-dimensional distributions alone do not automatically give continuous paths. Brownian motion is the continuous modification of the Gaussian process with covariance

$$\mathbb{𝐸}[{𝐵}_{𝑠}{𝐵}_{𝑡}]=𝑠\wedge 𝑡\mathrm{.}$$

28.2 Gaussian processes

A Gaussian process is a process whose every finite linear combination is Gaussian. It is determined by its mean function

$$𝑚(𝑡)=\mathbb{𝐸}[{𝑋}_{𝑡}]$$

and covariance function

$$𝐾(𝑠,𝑡)=\mathrm{Cov}({𝑋}_{𝑠},{𝑋}_{𝑡})\mathrm{.}$$

Brownian motion is centered with $𝐾(𝑠,𝑡)=𝑠\wedge 𝑡$.

Gaussianity reduces finite-dimensional laws to linear algebra. Positive semidefiniteness of the covariance kernel is the consistency condition. Path properties, however, depend on metric behavior induced by

$$𝑑(𝑠,𝑡{)}^2=\mathbb{𝐸}\mathrm{\mid }{𝑋}_{𝑠}-{𝑋}_{𝑡}{\mathrm{\mid }}^2\mathrm{.}$$

Covariance determines law; regularity requires additional analysis of increments.

28.3 Independent increments

Brownian motion has independent increments: for

$$0\le {𝑡}_0<{𝑡}_1<\cdots <{𝑡}_{𝑛},$$

the variables

$${𝐵}_{{𝑡}_1}-{𝐵}_{{𝑡}_0},\dots ,{𝐵}_{{𝑡}_{𝑛}}-{𝐵}_{{𝑡}_{𝑛-1}}$$

are independent. Each increment has variance equal to its time length.

Independent increments make Brownian motion a continuous-time analogue of random walk. The Markov property, martingale property, and Gaussian transition densities follow. Independent increments are stronger than uncorrelated increments, though for Gaussian processes uncorrelated implies independent.

28.4 Continuous paths

Brownian motion has continuous paths almost surely, but those paths are almost surely nowhere differentiable. Continuity follows from moment bounds such as

$$\mathbb{𝐸}\mathrm{\mid }{𝐵}_{𝑡}-{𝐵}_{𝑠}{\mathrm{\mid }}^{𝑝}={𝐶}_{𝑝}\mathrm{\mid }𝑡-𝑠{\mathrm{\mid }}^{𝑝\mathrm{/}2}$$

and Kolmogorov continuity criteria.

The paths are rough: for small $ℎ$, increments are typically of size $\sqrt{ℎ}$, so difference quotients behave like ${ℎ}^{-1\mathrm{/}2}$. This roughness is why stochastic calculus is not ordinary calculus. Brownian paths have finite quadratic variation but infinite total variation on intervals.

28.5 Scaling

Brownian motion satisfies the scaling property

This follows from Gaussian increments: ${𝐵}_{𝑐𝑡}$ has variance $𝑐𝑡$, matching $\sqrt{𝑐}{𝐵}_{𝑡}$.

Scaling is a structural symmetry. It determines hitting-time distributions, fractal path behavior, local time scaling, and invariance principles. Brownian motion is self-similar with exponent $1\mathrm{/}2$. This exponent is the source of diffusive scaling.

28.6 Reflection principle

The reflection principle states, for Brownian motion and $𝑎>0$,

$$𝑃(\underset{0\le 𝑠\le 𝑡}{\sup }{𝐵}_{𝑠}\ge 𝑎)=2𝑃({𝐵}_{𝑡}\ge 𝑎)\mathrm{.}$$

Paths that cross level $𝑎$ are reflected after their first hitting time, giving a bijective probability argument.

This principle yields distributions of maxima and hitting times. For ${𝜏}_{𝑎}=\inf \{𝑡:{𝐵}_{𝑡}=𝑎\}$,

$$𝑃({𝜏}_{𝑎}\le 𝑡)=2𝑃({𝐵}_{𝑡}\ge 𝑎)\mathrm{.}$$

Reflection uses continuity and symmetry of Brownian increments; it is not available for arbitrary martingales or jump processes without modification.

28.7 Hitting times

For Brownian motion, the hitting time of level $𝑎$ is

$${𝜏}_{𝑎}=\inf \{𝑡\ge 0:{𝐵}_{𝑡}=𝑎\}\mathrm{.}$$

Its distribution satisfies

$$𝑃({𝜏}_{𝑎}\le 𝑡)=2(1-\mathrm{\Phi }\left(\frac{𝑎}{\sqrt{𝑡}}\right))$$

for $𝑎>0$. The density is

$${𝑓}_{{𝜏}_{𝑎}}(𝑡)=\frac{𝑎}{\sqrt{2𝜋{𝑡}^3}}{𝑒}^{-{𝑎}^2\mathrm{/}(2𝑡)}\mathrm{.}$$

Hitting times connect Brownian motion to boundary value problems. Probabilities of hitting one boundary before another solve harmonic equations. Expected hitting times solve Poisson equations when finite. In unbounded domains, expectations may be infinite even when hitting occurs almost surely.

28.8 Brownian bridge

A Brownian bridge from $0$ to $0$ over $[0,1]$ is Brownian motion conditioned on ${𝐵}_1=0$. It can be represented as

$${𝛽}_{𝑡}={𝐵}_{𝑡}-𝑡{𝐵}_1\mathrm{.}$$

It is a centered Gaussian process with covariance

$$\mathbb{𝐸}[{𝛽}_{𝑠}{𝛽}_{𝑡}]=𝑠\wedge 𝑡-𝑠𝑡\mathrm{.}$$

The Brownian bridge appears in empirical process limits, Kolmogorov–Smirnov statistics, conditioned diffusions, and Gaussian process theory. It is not Brownian motion because its increments are not independent; the endpoint constraint introduces global dependence.

28.9 Brownian motion in ℝᵈ

A $𝑑$-dimensional Brownian motion is

$${𝐵}_{𝑡}=({𝐵}_{𝑡}^{(1)},\dots ,{𝐵}_{𝑡}^{(𝑑)}),$$

where the coordinates are independent one-dimensional Brownian motions. Its transition density is

$${𝑝}_{𝑡}(𝑥,𝑦)=\frac{1}{(2𝜋𝑡{)}^{𝑑\mathrm{/}2}}\exp (-\frac{\mathrm{\mid }𝑦-𝑥{\mathrm{\mid }}^2}{2𝑡})\mathrm{.}$$

The generator is

$$𝐴=\frac{1}{2}\mathrm{\Delta }\mathrm{.}$$

Thus Brownian motion is probabilistically tied to the heat equation:

$${\mathrm{\partial }}_{𝑡}𝑢=\frac{1}{2}\mathrm{\Delta }𝑢\mathrm{.}$$

The dimension $𝑑$ changes recurrence, hitting probabilities, potential theory, and path intersection behavior.

28.10 Recurrence and transience

Brownian motion is recurrent in dimensions $𝑑=1,2$ in the sense that it returns arbitrarily close to points or neighborhoods, while it is transient in dimensions $𝑑\ge 3$. In $𝑑=1$, points are hit almost surely. In $𝑑=2$, points are polar in a finer sense, but neighborhoods are visited recurrently. In $𝑑\ge 3$, Brownian motion tends to infinity.

This dimension transition reflects potential theory. The Green function changes behavior by dimension. Recurrence means infinite occupation of neighborhoods; transience means finite expected occupation. The threshold at dimension two is fundamental.

28.11 Quadratic variation

Brownian motion has quadratic variation

$$[𝐵{]}_{𝑡}=𝑡\mathrm{.}$$

For partitions ${\mathrm{\Pi }}_{𝑛}$ of $[0,𝑡]$ with mesh going to zero,

$$\sum _{[𝑢,𝑣]\in {\mathrm{\Pi }}_{𝑛}}({𝐵}_{𝑣}-{𝐵}_{𝑢}{)}^2\to 𝑡$$

in probability, and along suitable partitions almost surely.

This property distinguishes Brownian motion from differentiable functions, whose quadratic variation is zero. Quadratic variation is why Itô calculus has correction terms. It is also the intrinsic clock of Brownian martingales.

28.12 Lévy characterization

Lévy’s characterization states that a continuous adapted process ${𝑀}_{𝑡}$ with ${𝑀}_0=0$ is Brownian motion if ${𝑀}_{𝑡}$ is a martingale and

$${𝑀}_{𝑡}^2-𝑡$$

is also a martingale. Equivalently, $𝑀$ is a continuous martingale with quadratic variation $[𝑀{]}_{𝑡}=𝑡$.

This theorem characterizes Brownian motion by martingale and quadratic variation properties rather than independent Gaussian increments. It is powerful in proving that transformed processes are Brownian under new measures or filtrations.

28.13 Donsker invariance principle

Donsker’s theorem says that rescaled random walks converge to Brownian motion. If ${𝑋}_{𝑖}$ are iid mean zero, variance one, and

$${𝑆}_{𝑛}(𝑡)=\frac{1}{\sqrt{𝑛}}\sum _{𝑖=1}^{\lfloor 𝑛𝑡\rfloor }{𝑋}_{𝑖}$$

with interpolation, then

$${𝑆}_{𝑛}\Rightarrow 𝐵$$

in a suitable path space.

This is the process-level CLT. It upgrades convergence of one-time sums to convergence of entire paths. The theorem explains Brownian motion as the universal scaling limit of many independent small-step systems. Tightness is the additional payload beyond finite-dimensional CLT.

28.14 Brownian motion as universal limit carrier

Brownian motion is universal because it is the canonical limit of centered finite-variance cumulative noise under diffusive scaling. It appears in random walks, empirical processes, martingales, queueing, statistical estimation, stochastic differential equations, and physics.

Universality has conditions. Heavy tails lead to stable Lévy processes; long-range dependence can lead to fractional Brownian motion; jumps lead to discontinuous Lévy limits; nonstandard dependence can change scaling. Brownian motion is the universal carrier only after finite-variance, weak-dependence, and diffusive-scaling debts are paid.

---

Chapter 29 — Stochastic Integration

29.1 Motivation

Ordinary integration fails for Brownian-driven equations because Brownian paths have infinite variation and are nowhere differentiable. Expressions like

$${\int }_0^{𝑡}{𝐻}_{𝑠}𝑑{𝐵}_{𝑠}$$

cannot be defined as Riemann–Stieltjes integrals in the usual pathwise sense for general adapted $𝐻$.

Stochastic integration builds an integral using probability, martingales, and ${𝐿}^2$ limits. The integrand must be nonanticipative, typically predictable. The integral is not just a pathwise area; it is a limit of sums where the integrand is evaluated using information before the Brownian increment.

29.2 Itô integral for simple processes

For a simple predictable process

$${𝐻}_{𝑡}=\sum _{𝑘=0}^{𝑛-1}{𝐻}_{𝑘}{1}_{({𝑡}_{𝑘},{𝑡}_{𝑘+1}]}(𝑡),$$

where ${𝐻}_{𝑘}$ is ${\mathcal{𝐹}}_{{𝑡}_{𝑘}}$-measurable, define

$${\int }_0^{𝑇}{𝐻}_{𝑡}𝑑{𝐵}_{𝑡}=\sum _{𝑘=0}^{𝑛-1}{𝐻}_{𝑘}({𝐵}_{{𝑡}_{𝑘+1}}-{𝐵}_{{𝑡}_{𝑘}})\mathrm{.}$$

Predictability ensures ${𝐻}_{𝑘}$ is known before the increment.

For such integrals,

$$\mathbb{𝐸}\left[{\int }_0^{𝑇}{𝐻}_{𝑡}𝑑{𝐵}_{𝑡}\right]=0,$$

and the Itô isometry holds:

$$\mathbb{𝐸}\left[{\left({\int }_0^{𝑇}{𝐻}_{𝑡}𝑑{𝐵}_{𝑡}\right)}^2\right]=\mathbb{𝐸}{\int }_0^{𝑇}{𝐻}_{𝑡}^2𝑑𝑡\mathrm{.}$$

29.3 Extension by isometry

The Itô isometry extends the integral from simple predictable processes to all predictable processes satisfying

$$\mathbb{𝐸}{\int }_0^{𝑇}{𝐻}_{𝑡}^2𝑑𝑡<\mathrm{\infty }\mathrm{.}$$

Choose simple ${𝐻}^{𝑛}\to 𝐻$ in ${𝐿}^2(\mathrm{\Omega }\times [0,𝑇])$, define

$$\int 𝐻𝑑𝐵={𝐿}^2\text{-}\underset{𝑛}{\lim }\int {𝐻}^{𝑛}𝑑𝐵\mathrm{.}$$

The isometry guarantees the definition is independent of approximating sequence. This is the measure-theoretic construction of stochastic integration: build on simple predictable objects, prove an exact norm identity, then complete the space.

29.4 Predictable integrands

An Itô integrand must be predictable or at least progressively measurable with suitable integrability, depending on formulation. Predictability encodes nonanticipation: ${𝐻}_{𝑡}$ may depend on past and present information but not future Brownian increments.

This condition is structural. If one uses right-endpoint values or future-dependent integrands, different integrals arise, such as Stratonovich-type integrals or anticipative integrals requiring Malliavin calculus. Itô integration is tied to adapted prediction and martingale structure.

29.5 Local martingales

The stochastic integral

$${𝑀}_{𝑡}={\int }_0^{𝑡}{𝐻}_{𝑠}𝑑{𝐵}_{𝑠}$$

is a martingale when $𝐻$ is square-integrable on finite horizons. Under local square integrability, it is a local martingale. Its quadratic variation is

$$[𝑀{]}_{𝑡}={\int }_0^{𝑡}{𝐻}_{𝑠}^2𝑑𝑠\mathrm{.}$$

Local martingales are the natural output of stochastic integration. They behave like martingales up to localization times but may fail to be true martingales globally. This distinction matters for expectation identities, change of measure, and financial modeling.

29.6 Quadratic variation

For Itô integrals,

$${\left[{\int }_0^{\cdot }{𝐻}_{𝑠}𝑑{𝐵}_{𝑠}\right]}_{𝑡}={\int }_0^{𝑡}{𝐻}_{𝑠}^2𝑑𝑠\mathrm{.}$$

For two integrals,

$${[\int 𝐻𝑑𝐵,\int 𝐾𝑑𝐵]}_{𝑡}={\int }_0^{𝑡}{𝐻}_{𝑠}{𝐾}_{𝑠}𝑑𝑠\mathrm{.}$$

Quadratic variation is the second-order bookkeeping of stochastic calculus. It replaces ordinary differential products. The symbolic rule is

$$(𝑑{𝐵}_{𝑡}{)}^2=𝑑𝑡,𝑑{𝐵}_{𝑡}𝑑𝑡=0,(𝑑𝑡{)}^2=0.$$

This rule is mnemonic for rigorous quadratic variation calculations.

29.7 Itô formula

If ${𝑋}_{𝑡}$ satisfies

$$𝑑{𝑋}_{𝑡}={𝑏}_{𝑡}𝑑𝑡+{𝜎}_{𝑡}𝑑{𝐵}_{𝑡}$$

and $𝑓\in {𝐶}^2$, then

$$𝑑𝑓({𝑋}_{𝑡})={𝑓}^{\mathrm{\prime }}({𝑋}_{𝑡})𝑑{𝑋}_{𝑡}+\frac{1}{2}{𝑓}^{\mathrm{\prime }\mathrm{\prime }}({𝑋}_{𝑡}){𝜎}_{𝑡}^2𝑑𝑡\mathrm{.}$$

Equivalently,

$$𝑓({𝑋}_{𝑡})=𝑓({𝑋}_0)+{\int }_0^{𝑡}{𝑓}^{\mathrm{\prime }}({𝑋}_{𝑠}){𝑏}_{𝑠}𝑑𝑠+{\int }_0^{𝑡}{𝑓}^{\mathrm{\prime }}({𝑋}_{𝑠}){𝜎}_{𝑠}𝑑{𝐵}_{𝑠}+\frac{1}{2}{\int }_0^{𝑡}{𝑓}^{\mathrm{\prime }\mathrm{\prime }}({𝑋}_{𝑠}){𝜎}_{𝑠}^2𝑑𝑠\mathrm{.}$$

The second derivative term is the Itô correction caused by quadratic variation. Ordinary chain rule fails because Brownian increments have square size comparable to $𝑑𝑡$. Itô formula is the fundamental calculus rule for diffusion processes.

29.8 Stochastic integration by parts

For semimartingales $𝑋,𝑌$,

$$𝑑({𝑋}_{𝑡}{𝑌}_{𝑡})={𝑋}_{𝑡}𝑑{𝑌}_{𝑡}+{𝑌}_{𝑡}𝑑{𝑋}_{𝑡}+𝑑[𝑋,𝑌{]}_{𝑡}\mathrm{.}$$

Integrated,

$${𝑋}_{𝑡}{𝑌}_{𝑡}={𝑋}_0{𝑌}_0+{\int }_0^{𝑡}{𝑋}_{𝑠}𝑑{𝑌}_{𝑠}+{\int }_0^{𝑡}{𝑌}_{𝑠}𝑑{𝑋}_{𝑠}+[𝑋,𝑌{]}_{𝑡}$$

with predictable convention refinements depending on exact formulation.

The extra quadratic covariation term is the stochastic correction. If one process has finite variation, its quadratic covariation with a continuous martingale is zero. Integration by parts is essential for exponential martingales, product processes, and SDE transformations.

29.9 Exponential martingales

For Brownian motion,

$${\mathcal{𝐸}}_{𝑡}(𝜃𝐵)=\exp (𝜃{𝐵}_{𝑡}-\frac{1}{2}{𝜃}^2𝑡)$$

is a martingale. More generally, for suitable $𝐻$,

$${\mathcal{𝐸}}_{𝑡}(\int 𝐻𝑑𝐵)=\exp ({\int }_0^{𝑡}{𝐻}_{𝑠}𝑑{𝐵}_{𝑠}-\frac{1}{2}{\int }_0^{𝑡}{𝐻}_{𝑠}^2𝑑𝑠)$$

is a local martingale and under conditions such as Novikov’s condition is a true martingale.

Exponential martingales are density processes for changing measure. They also yield concentration bounds and solve linear stochastic equations. The correction $-\frac{1}{2}\int {𝐻}^2$ compensates quadratic variation.

29.10 Girsanov theorem

Girsanov’s theorem describes how Brownian drift changes under an equivalent measure. If

$${𝑍}_{𝑡}=\exp (-{\int }_0^{𝑡}{𝜃}_{𝑠}𝑑{𝐵}_{𝑠}-\frac{1}{2}{\int }_0^{𝑡}{𝜃}_{𝑠}^2𝑑𝑠)$$

is a martingale and defines $𝑑𝑄={𝑍}_{𝑇}𝑑𝑃$, then

$${\widetilde{𝐵}}_{𝑡}={𝐵}_{𝑡}+{\int }_0^{𝑡}{𝜃}_{𝑠}𝑑𝑠$$

is Brownian motion under $𝑄$.

The theorem is a change-of-measure engine. It transforms drift into likelihood ratio. It is central in filtering, finance, diffusion theory, rare-event simulation, and stochastic control. The martingale condition on $𝑍$ is the integrability gate; without it, the measure change may fail.

29.11 Martingale representation theorem

In a Brownian filtration, every square-integrable martingale ${𝑀}_{𝑡}$ can be represented as

$${𝑀}_{𝑡}={𝑀}_0+{\int }_0^{𝑡}{𝐻}_{𝑠}𝑑{𝐵}_{𝑠}$$

for some predictable $𝐻$ with suitable square integrability. This says Brownian stochastic integrals span martingales in the Brownian information carrier.

The theorem is foundational for hedging in mathematical finance, filtering, and stochastic control. It is filtration-specific. In a larger filtration or with jump noise, representation may require additional martingale drivers. The theorem is not a universal statement about all martingales; it is a carrier theorem for Brownian filtrations.