Skip to content

DCT Lexicon · Governing Dynamics

Occupation Measures

An occupation measure records how often a policy visits each state–action pair. In these variables, stochastic control becomes a linear program, and nonconvex policy search becomes convex.

DCT Governing Dynamics

In depth

Formal statement

For a discounted problem with initial law ν\nu, the discounted occupation measure of a policy π\pi satisfies a linear balance equation, and every such measure comes from a policy.

μπ(B×C)=(1−β)∑t≥0βt Pνπ(Xt∈B, Ut∈C)\mu^{\pi}(B \times C) = (1-\beta) \sum_{t \ge 0} \beta^t\, \mathbb{P}^{\pi}_{\nu}\big(X_t \in B,\ U_t \in C\big)max⁡μ≥0∫r dμ    s.t.    μ(B×U)=(1−β) ν(B)+β∫P(B∣x,u) μ(dx,du)    ∀B\max_{\mu \ge 0} \int r\, d\mu \;\;\text{s.t.}\;\; \mu(B \times U) = (1-\beta)\,\nu(B) + \beta \int P(B \mid x,u)\, \mu(dx,du)\;\;\forall B

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

Linear programming duality for control

The occupation-measure program and the Bellman inequality program min⁡(1−β)∫V dν\min (1-\beta)\int V\,d\nu s.t. V≥r+βPVV \ge r + \beta P V are dual linear programs with equal values; an optimal measure disintegrates into an optimal (possibly randomised) stationary policy.

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

In DCT

Constraints on expected costs (an average draw on the line, a budget on hiring) are linear in the occupation measure, so constrained stochastic control stays convex. This is the route to certifying constrained policies with moment and semidefinite relaxations.

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

Limits

Constrained optima may require randomised policies. In continuous spaces the program is infinite-dimensional; moment–SOS hierarchies converge but grow quickly in size.

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

Go next

  • Dynamic Risk Measures: Risk constraints on the occupation measure lead to dynamic risk measures (Pathway 9).
  • Fenchel–Rockafellar Duality and Dual Bounds: Its dual is the Bellman inequality program.
  • Chance Constraints and Certificates: Chance constraints are measure constraints on the visited set.

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

References

  • Manne (1960)
  • Altman (1999), Constrained Markov Decision Processes
  • Lasserre (2010), Moments, Positive Polynomials and Their Applications

DCT Governing Dynamics: the element's drawer (/governing-dynamics)

In the book: Vol. II, Ch. 7: Dynamic Programming for Enterprise Systems · Vol. II, Ch. 5: Dynamic Enterprise Optimization

Related: Dynamic Risk Measures · Fenchel–Rockafellar Duality and Dual Bounds · Chance Constraints and Certificates

GD.15

Dynamic Corporate Transformation — Volume I: Foundations and Mathematical Framework; Volume II: Enterprise Optimization & Applications
Jump to any chapter

Volume I: Foundations and Mathematical Framework

  1. Ch. 1Introduction to Dynamic Corporate Transformation
  2. Ch. 2Enterprise Systems and Transformation
  3. Ch. 3Mathematical Foundations
  4. Ch. 4Enterprise Modeling Principles
  5. Ch. 5Enterprise State Representation
  6. Ch. 6Enterprise Transformation Operators
  7. Ch. 7Dynamic Enterprise Systems
  8. Ch. 8Stochastic Enterprise Dynamics
  9. Ch. 9Enterprise State Architecture
  10. Ch. 10Enterprise Capital Architecture
  11. Ch. 11Enterprise Performance Architecture
  12. Ch. 12Enterprise Risk, Resilience, and Robustness Architecture
  13. Ch. 13Unified Enterprise Transformation Architecture
  14. Ch. 14Mathematical Analysis of the Unified Enterprise Transformation Architecture
  15. Ch. 15Mathematical Formulation of Enterprise Optimization
  16. Ch. 16Synthesis, Future Directions, and Transition to Volume II

Volume II: Enterprise Optimization & Applications

  1. Ch. 1Introduction to Enterprise Optimization
  2. Ch. 2The General Enterprise Optimization Problem Revisited
  3. Ch. 3Convex Enterprise Optimization
  4. Ch. 4Nonlinear and Mixed-Integer Enterprise Optimization
  5. Ch. 5Dynamic Enterprise Optimization
  6. Ch. 6Optimal Control of Enterprise Systems
  7. Ch. 7Dynamic Programming for Enterprise Systems
  8. Ch. 8Hamilton–Jacobi–Bellman Enterprise Framework
  9. Ch. 9Stochastic Enterprise Optimization
  10. Ch. 10Neuro-Fuzzy Robust Enterprise Optimization
  11. Ch. 11Distributionally Robust Enterprise Optimization
  12. Ch. 12Multi-Objective Enterprise Optimization
  13. Ch. 13Machine Learning for Enterprise Transformation
  14. Ch. 14Artificial Intelligence for Enterprise Optimization
  15. Ch. 15Enterprise Digital Twins and Autonomous Enterprise Transformation
  16. Ch. 16Enterprise Applications and Integrated Transformation Case Studies

Dynamic Corporate Transformation

Springer Nature · Management for Professionals · Forthcoming 2027

  • Volume I — Foundations and Mathematical Framework
  • Volume II — Enterprise Optimization & Applications
Buy Volume I: Dynamic Corporate Transformation: Foundations and Mathematical FrameworkBuy Volume II: Dynamic Corporate Transformation: Enterprise Optimization & Applications
Jump to any chapter

Volume I: Foundations and Mathematical Framework

  1. Ch. 1Introduction to Dynamic Corporate Transformation
  2. Ch. 2Enterprise Systems and Transformation
  3. Ch. 3Mathematical Foundations
  4. Ch. 4Enterprise Modeling Principles
  5. Ch. 5Enterprise State Representation
  6. Ch. 6Enterprise Transformation Operators
  7. Ch. 7Dynamic Enterprise Systems
  8. Ch. 8Stochastic Enterprise Dynamics
  9. Ch. 9Enterprise State Architecture
  10. Ch. 10Enterprise Capital Architecture
  11. Ch. 11Enterprise Performance Architecture
  12. Ch. 12Enterprise Risk, Resilience, and Robustness Architecture
  13. Ch. 13Unified Enterprise Transformation Architecture
  14. Ch. 14Mathematical Analysis of the Unified Enterprise Transformation Architecture
  15. Ch. 15Mathematical Formulation of Enterprise Optimization
  16. Ch. 16Synthesis, Future Directions, and Transition to Volume II

Volume II: Enterprise Optimization & Applications

  1. Ch. 1Introduction to Enterprise Optimization
  2. Ch. 2The General Enterprise Optimization Problem Revisited
  3. Ch. 3Convex Enterprise Optimization
  4. Ch. 4Nonlinear and Mixed-Integer Enterprise Optimization
  5. Ch. 5Dynamic Enterprise Optimization
  6. Ch. 6Optimal Control of Enterprise Systems
  7. Ch. 7Dynamic Programming for Enterprise Systems
  8. Ch. 8Hamilton–Jacobi–Bellman Enterprise Framework
  9. Ch. 9Stochastic Enterprise Optimization
  10. Ch. 10Neuro-Fuzzy Robust Enterprise Optimization
  11. Ch. 11Distributionally Robust Enterprise Optimization
  12. Ch. 12Multi-Objective Enterprise Optimization
  13. Ch. 13Machine Learning for Enterprise Transformation
  14. Ch. 14Artificial Intelligence for Enterprise Optimization
  15. Ch. 15Enterprise Digital Twins and Autonomous Enterprise Transformation
  16. Ch. 16Enterprise Applications and Integrated Transformation Case Studies