Skip to content

DCT Lexicon · Governing Dynamics

Bellman Recursion

Bellman's principle says an optimal plan stays optimal from every state it reaches. It turns a decision over a whole horizon into a recursion over one step.

DCT Governing Dynamics

In depth

Formal statement

For a discounted problem with reward rr, kernel PP and discount β∈(0,1)\beta \in (0,1), the value function is the unique fixed point of the Bellman operator.

(TV)(x)  =  max⁡u∈U(x){r(x,u)+β∫XV(y) P(dy∣x,u)}(\mathcal{T}V)(x) \;=\; \max_{u \in U(x)} \Big\{ r(x,u) + \beta \int_X V(y)\, P(dy \mid x,u) \Big\}∥TV−TW∥∞≤β ∥V−W∥∞,∥V⋆−Vk∥∞≤e1−β,    e=∥TVk−Vk∥∞\|\mathcal{T}V - \mathcal{T}W\|_\infty \le \beta\, \|V - W\|_\infty,\qquad \|V^\star - V_k\|_\infty \le \frac{e}{1-\beta},\;\; e = \|\mathcal{T}V_k - V_k\|_\infty

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

Contraction and verification

T\mathcal{T} is a β\beta-contraction on bounded functions, so value iteration converges geometrically, and a greedy policy for V⋆V^\star is optimal. The Bellman residual of any approximation bounds its distance to the true value.

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

In DCT

Schedules of automation rounds, hiring rules and the timing of a sale are dynamic programs. The Bellman residual ee is the dynamic certificate: it states how far from optimal a computed policy can be, in money.

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

Limits

The curse of dimensionality: grids grow exponentially with the state. Approximate dynamic programming trades exactness for scale, and its certificates weaken to bounds.

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

Go next

  • HJB Equations and Viscosity Solutions: In continuous time the recursion becomes the HJB equation (Pathway 3).
  • Robust Viability: Robust viability is a dynamic program with an adversary (Pathway 5).
  • Occupation Measures: The dual of the Bellman LP is a program over occupation measures.

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

References

  • Bellman (1957), Dynamic Programming
  • Puterman (1994), Markov Decision Processes
  • Bertsekas (2012), Dynamic Programming and Optimal Control

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

In the book: Vol. II, Ch. 5: Dynamic Enterprise Optimization · Vol. II, Ch. 7: Dynamic Programming for Enterprise Systems

Related: HJB Equations and Viscosity Solutions · Robust Viability · Occupation Measures

GD.11

Dynamic Corporate Transformation — Volume I: Foundations and Mathematical Framework; Volume II: Enterprise Optimization & Applications
Jump to any chapter

Volume I: Foundations and Mathematical Framework

  1. Ch. 1Introduction to Dynamic Corporate Transformation
  2. Ch. 2Enterprise Systems and Transformation
  3. Ch. 3Mathematical Foundations
  4. Ch. 4Enterprise Modeling Principles
  5. Ch. 5Enterprise State Representation
  6. Ch. 6Enterprise Transformation Operators
  7. Ch. 7Dynamic Enterprise Systems
  8. Ch. 8Stochastic Enterprise Dynamics
  9. Ch. 9Enterprise State Architecture
  10. Ch. 10Enterprise Capital Architecture
  11. Ch. 11Enterprise Performance Architecture
  12. Ch. 12Enterprise Risk, Resilience, and Robustness Architecture
  13. Ch. 13Unified Enterprise Transformation Architecture
  14. Ch. 14Mathematical Analysis of the Unified Enterprise Transformation Architecture
  15. Ch. 15Mathematical Formulation of Enterprise Optimization
  16. Ch. 16Synthesis, Future Directions, and Transition to Volume II

Volume II: Enterprise Optimization & Applications

  1. Ch. 1Introduction to Enterprise Optimization
  2. Ch. 2The General Enterprise Optimization Problem Revisited
  3. Ch. 3Convex Enterprise Optimization
  4. Ch. 4Nonlinear and Mixed-Integer Enterprise Optimization
  5. Ch. 5Dynamic Enterprise Optimization
  6. Ch. 6Optimal Control of Enterprise Systems
  7. Ch. 7Dynamic Programming for Enterprise Systems
  8. Ch. 8Hamilton–Jacobi–Bellman Enterprise Framework
  9. Ch. 9Stochastic Enterprise Optimization
  10. Ch. 10Neuro-Fuzzy Robust Enterprise Optimization
  11. Ch. 11Distributionally Robust Enterprise Optimization
  12. Ch. 12Multi-Objective Enterprise Optimization
  13. Ch. 13Machine Learning for Enterprise Transformation
  14. Ch. 14Artificial Intelligence for Enterprise Optimization
  15. Ch. 15Enterprise Digital Twins and Autonomous Enterprise Transformation
  16. Ch. 16Enterprise Applications and Integrated Transformation Case Studies

Dynamic Corporate Transformation

Springer Nature · Management for Professionals · Forthcoming 2027

  • Volume I — Foundations and Mathematical Framework
  • Volume II — Enterprise Optimization & Applications
Buy Volume I: Dynamic Corporate Transformation: Foundations and Mathematical FrameworkBuy Volume II: Dynamic Corporate Transformation: Enterprise Optimization & Applications
Jump to any chapter

Volume I: Foundations and Mathematical Framework

  1. Ch. 1Introduction to Dynamic Corporate Transformation
  2. Ch. 2Enterprise Systems and Transformation
  3. Ch. 3Mathematical Foundations
  4. Ch. 4Enterprise Modeling Principles
  5. Ch. 5Enterprise State Representation
  6. Ch. 6Enterprise Transformation Operators
  7. Ch. 7Dynamic Enterprise Systems
  8. Ch. 8Stochastic Enterprise Dynamics
  9. Ch. 9Enterprise State Architecture
  10. Ch. 10Enterprise Capital Architecture
  11. Ch. 11Enterprise Performance Architecture
  12. Ch. 12Enterprise Risk, Resilience, and Robustness Architecture
  13. Ch. 13Unified Enterprise Transformation Architecture
  14. Ch. 14Mathematical Analysis of the Unified Enterprise Transformation Architecture
  15. Ch. 15Mathematical Formulation of Enterprise Optimization
  16. Ch. 16Synthesis, Future Directions, and Transition to Volume II

Volume II: Enterprise Optimization & Applications

  1. Ch. 1Introduction to Enterprise Optimization
  2. Ch. 2The General Enterprise Optimization Problem Revisited
  3. Ch. 3Convex Enterprise Optimization
  4. Ch. 4Nonlinear and Mixed-Integer Enterprise Optimization
  5. Ch. 5Dynamic Enterprise Optimization
  6. Ch. 6Optimal Control of Enterprise Systems
  7. Ch. 7Dynamic Programming for Enterprise Systems
  8. Ch. 8Hamilton–Jacobi–Bellman Enterprise Framework
  9. Ch. 9Stochastic Enterprise Optimization
  10. Ch. 10Neuro-Fuzzy Robust Enterprise Optimization
  11. Ch. 11Distributionally Robust Enterprise Optimization
  12. Ch. 12Multi-Objective Enterprise Optimization
  13. Ch. 13Machine Learning for Enterprise Transformation
  14. Ch. 14Artificial Intelligence for Enterprise Optimization
  15. Ch. 15Enterprise Digital Twins and Autonomous Enterprise Transformation
  16. Ch. 16Enterprise Applications and Integrated Transformation Case Studies