Skip to content

DCT Lexicon · Governing Dynamics

Probability Kernels

A probability kernel is the measurable object that turns a deterministic map of the enterprise into a law of motion: from every state and decision it assigns a probability measure over the next state.

DCT Governing Dynamics

In depth

Formal statement

Let (X,X)(X,\mathcal{X}) be the state space and (U,U)(U,\mathcal{U}) the control space, both standard Borel. A transition kernel is a map P:X×U×X→[0,1]P : X \times U \times \mathcal{X} \to [0,1] such that B↦P(B∣x,u)B \mapsto P(B \mid x,u) is a probability measure for every (x,u)(x,u), and (x,u)↦P(B∣x,u)(x,u) \mapsto P(B \mid x,u) is measurable for every B∈XB \in \mathcal{X}. Kernels compose by integration, and a policy closes the loop.

(P∘Q)(B∣x)  =  ∫XQ(B∣y) P(dy∣x)(P \circ Q)(B \mid x) \;=\; \int_X Q(B \mid y)\, P(dy \mid x)Px0π(Xt+1∈B∣Ft)  =  P(B∣Xt,πt(X0,…,Xt))\mathbb{P}^{\pi}_{x_0}\big(X_{t+1} \in B \mid \mathcal{F}_t\big) \;=\; P\big(B \mid X_t, \pi_t(X_0,\dots,X_t)\big)

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

Ionescu-Tulcea extension

Given an initial law and a sequence of kernels, there is a unique probability measure on the path space XNX^{\mathbb{N}} whose finite-dimensional marginals are the iterated kernel compositions. Every simulated path of the enterprise, and every expectation taken over paths, rests on this construction.

Pμπ(dx0,dx1,… )  =  μ(dx0)∏t≥0P(dxt+1∣xt,πt)\mathbb{P}^{\pi}_{\mu}(dx_0, dx_1, \dots) \;=\; \mu(dx_0) \prod_{t \ge 0} P\big(dx_{t+1} \mid x_t, \pi_t\big)

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

In DCT

The kernel is the single source of randomness in the architecture. Bellman recursion integrates against it, belief-state filtering propagates through it, occupation measures are its time-averaged footprints, and Wasserstein ambiguity sets are balls around it. Declaring the kernel is what makes "the probability of a breach" a well-posed quantity rather than a figure of speech.

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

Limits

Measurability is not a technicality: without a measurable selection of policies, the value function need not be Borel and the expectation in Bellman's equation may be undefined. Universally measurable policies, or lower semi-analytic value functions, restore the theory.

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

Go next

  • Belief-State Control: Filtering propagates beliefs through the same kernel (Pathway 1).
  • Bellman Recursion: Bellman recursion integrates the value function against the kernel.
  • Occupation Measures: Occupation measures are the kernel's long-run footprint under a policy.

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

References

  • Ionescu-Tulcea (1949)
  • Bertsekas & Shreve (1978), Stochastic Optimal Control: The Discrete-Time Case
  • Kallenberg (2002), Foundations of Modern Probability

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

In the book: Vol. I, Ch. 8: Stochastic Enterprise Dynamics · Vol. II, Ch. 9: Stochastic Enterprise Optimization

Related: Belief-State Control · Bellman Recursion · Occupation Measures

GD.01