Skip to content

Vol. II, Ch. 7

DCT Laboratory — Volume II, Chapter 7

Dynamic Programming for Enterprise Systems

Seed 26207 · Companion to Chapter 7 and AXIOM Lab 2.7 (module AXIOM-07) · Mirrored in DCT_V2_Ch07_Lab.xlsx

This laboratory reproduces Worked Example 7.1, How Much Margin Should Meridian Hold Above the Capability Floor?, with the book's declarations, and works the five steps of AXIOM Lab 2.7: look one quarter ahead, solve backward, iterate, read values and prices, approximate and shrink the period. Every Meridian number below is the book's (Table 7.3, Figures 7.1–7.3, Online Appendix 7.B.1); the final cell checks them. Money is in $ million.

The declarations (Worked Example 7.1, Step 1)

Workforce capability cc is the state; the decision hh is the points hired and trained in a quarter, at $4 million a point and at most 2.5 a quarter (Volume I, Chapter 10). Capability keeps 97.5 percent of itself each quarter and moves by an unplanned loss ww from the engineering office's record, so c′=0.975c+h−wc' = 0.975c + h - w. The record's law: capability ends where planned in half of all quarters, three-quarters of a point higher or lower in four quarters in ten, and a point and a half higher or lower in one quarter in ten, so ww is 1.5, 0.75, 0, −0.75-0.75 or −1.5-1.5 points with probabilities 0.05, 0.20, 0.50, 0.20 and 0.05. Below the floor of 55 the digital build slows by a fifth per point short, so a point short for a quarter costs 8/5=$1.68/5 = \$1.6 million. The board discounts at 10 percent a year, β=1.1−1/4\beta = 1.1^{-1/4} a quarter, over an open horizon. Capability is modeled between 30 and 80, with hiring limits at the two ends that keep it there (they never bind where the rules operate).

Python Code
import numpy as np
import scipy.sparse as sp
import scipy.sparse.linalg as spla
import matplotlib.pyplot as plt
plt.rcParams.update({"figure.dpi": 110, "axes.spines.top": False, "axes.spines.right": False})
SEED = 26207

BETA, ALPHA, K, CAP, FLOOR, P = 1.1 ** -0.25, 0.975, 4.0, 2.5, 55.0, 1.6
W  = np.array([1.5, 0.75, 0.0, -0.75, -1.5])        # unplanned loss (points)
PW = np.array([0.05, 0.20, 0.50, 0.20, 0.05])        # the engineering office's record
LO, HI, DX = 30.0, 80.0, 0.05

class Model:
    # The declared quarterly model on a grid; values are piecewise linear between nodes.
    def __init__(self, w=W, pw=PW, p=P, cap=CAP, lo=LO, hi=HI, dx=DX, charge=None):
        self.w, self.pw, self.p, self.cap = np.asarray(w, float), np.asarray(pw, float), p, cap
        self.lo, self.hi, self.dx = lo, hi, dx
        self.n = int(round((hi - lo) / dx)) + 1
        self.X = lo + dx * np.arange(self.n)
        self.s = np.round(self.w / dx).astype(int)          # events are whole grid steps
        self.g = -p * np.maximum(FLOOR - self.X, 0) if charge is None else charge(self.X)
        a = np.maximum(0.0, lo + self.w.max() - ALPHA * self.X)        # keep c' >= lo
        b = np.minimum(cap, hi + self.w.min() - ALPHA * self.X)        # keep c' <= hi
        self.a, self.b = np.minimum(a, cap), np.maximum(b, np.minimum(a, cap))
        self.L, self.U = ALPHA * self.X + self.a, ALPHA * self.X + self.b  # planned level y = 0.975c + h
    def interp(self, V, x):
        return np.interp(np.clip(x, self.lo, self.hi), self.X, V)
    def G(self, V, y):                       # -k y + beta E V(y - w): value of a planned level
        return -K * y + BETA * sum(pm * self.interp(V, y - wm) for wm, pm in zip(self.w, self.pw))
    def bellman(self, V):
        # (TV)(c) and the best planned level: exact for piecewise-linear V, because the value of a
        # decision is linear between grid nodes, so its maximum is at a node or at an end of [L, U].
        idx = np.arange(self.n)
        Gn = -K * self.X + BETA * sum(pm * V[np.clip(idx - sm, 0, self.n - 1)] for sm, pm in zip(self.s, self.pw))
        jlo = np.ceil((self.L - self.lo) / self.dx - 1e-9).astype(int)
        jhi = np.floor((self.U - self.lo) / self.dx + 1e-9).astype(int)
        J = jlo[:, None] + np.arange(int(round(self.cap / self.dx)) + 2)[None, :]
        ok = J <= jhi[:, None]
        vals = np.column_stack([self.G(V, self.L), np.where(ok, Gn[np.clip(J, 0, self.n - 1)], -np.inf), self.G(V, self.U)])
        ys = np.column_stack([self.L, np.where(ok, self.X[np.clip(J, 0, self.n - 1)], np.inf), self.U])
        best = vals.max(1)
        first = np.argmax(vals >= best[:, None] - 1e-10, axis=1)       # ties: least hiring
        return self.g + K * ALPHA * self.X + best, ys[np.arange(self.n), first]
    def evaluate(self, y, parts=False):
        # exact value of the stationary rule y(c): solve V = r + beta P V (Theorem 7.1(ii))
        rows, cols, vals = [], [], []
        for wm, pm in zip(self.w, self.pw):
            t = (np.clip(y - wm, self.lo, self.hi) - self.lo) / self.dx
            j = np.minimum(np.floor(t + 1e-12).astype(int), self.n - 2)
            rows += [np.arange(self.n)] * 2; cols += [j, j + 1]; vals += [pm * (1 - (t - j)), pm * (t - j)]
        Pm = sp.csr_matrix((np.concatenate(vals), (np.concatenate(rows), np.concatenate(cols))), shape=(self.n, self.n))
        lu = spla.splu((sp.identity(self.n, format="csc") - BETA * Pm.tocsc()))
        hire = -K * (y - ALPHA * self.X)
        return (lu.solve(self.g + hire), lu.solve(hire), lu.solve(self.g), Pm) if parts else lu.solve(self.g + hire)
    def target_rule(self, T):
        return np.clip(T, self.L, self.U)
    def at(self, V, c):
        return float(self.interp(V, c))
    def slope(self, V, c, side=+1):
        return (self.at(V, c + self.dx) - self.at(V, c)) / self.dx if side > 0 else (self.at(V, c) - self.at(V, c - self.dx)) / self.dx

m = Model()
print(f"beta = {BETA:.6f} a quarter; effective horizon 1/(1 - beta) = {1/(1-BETA):.1f} quarters")
print(f"unplanned loss: mean {PW @ W:.3f}, standard deviation {np.sqrt(PW @ W**2):.3f} points a quarter")
print(f"{m.n} grid nodes from {LO:.0f} to {HI:.0f}; the edge limits bind only below {(LO + W.max())/ALPHA:.2f} and above {(HI + W.min() - CAP)/ALPHA:.2f}")
beta = 0.976454 a quarter; effective horizon 1/(1 - beta) = 42.5 quarters
unplanned loss: mean 0.000, standard deviation 0.671 points a quarter
1001 grid nodes from 30 to 80; the edge limits bind only below 32.31 and above 77.95

Panel 1 — Solve backward (Theorem 7.1(i))

AXIOM Lab 2.7, step 2: a twelve-quarter horizon (Volume I's modeling horizon) solved by backward induction, V12=0V_{12} = 0 and Vk=TVk+1V_k = \mathcal{T} V_{k+1}, and its rule table: the hiring at each capability level in each quarter. Each quarter's rule is a target rule (Theorem 7.2 holds period by period); its target maximizes the value of a planned level, −ky+β E Vk+1(y−w)-ky + \beta\,\mathrm{E}\,V_{k+1}(y - w).

Python Code
V = np.zeros(m.n); idx = np.arange(m.n); levels = (53, 54, 55, 56, 57, 58); table = []
for k in range(11, -1, -1):
    G = -K * m.X + BETA * sum(pm * V[np.clip(idx - sm, 0, m.n - 1)] for sm, pm in zip(m.s, m.pw))
    Tk = float(m.X[np.argmax(G >= G.max() - 1e-10)])      # quarter k's target for planned capability
    V, y = m.bellman(V)                                    # V is now V_k
    table.append((k, Tk, [m.at(y - ALPHA * m.X, c0) for c0 in levels]))
print("quarter  target   hiring at capability " + "".join(f"{c0:7d}" for c0 in levels))
for k, Tk, hk in sorted(table):
    tgt = f"{Tk:6.2f}" if Tk >= ALPHA * 50 else "  none"
    print(f"  {k:2d}     {tgt}                        " + "".join(f"{h:7.2f}" for h in hk))
print("none: in the last quarters hiring is not worth its cost at any capability above 50")
team12 = m.at(V, 58) - m.at(V, 53)
print(f"value of the team over twelve quarters: V_0(58) - V_0(53) = {team12:.3f}")
quarter  target   hiring at capability      53     54     55     56     57     58
   0      55.75                           2.50   2.50   2.12   1.15   0.18   0.00
   1      55.75                           2.50   2.50   2.12   1.15   0.18   0.00
   2      55.75                           2.50   2.50   2.12   1.15   0.18   0.00
   3      55.75                           2.50   2.50   2.12   1.15   0.18   0.00
   4      55.75                           2.50   2.50   2.12   1.15   0.18   0.00
   5      55.75                           2.50   2.50   2.12   1.15   0.18   0.00
   6      55.75                           2.50   2.50   2.12   1.15   0.18   0.00
   7      55.65                           2.50   2.50   2.03   1.05   0.08   0.00
   8      54.85                           2.50   2.20   1.23   0.25   0.00   0.00
   9       none                           0.00   0.00   0.00   0.00   0.00   0.00
  10       none                           0.00   0.00   0.00   0.00   0.00   0.00
  11       none                           0.00   0.00   0.00   0.00   0.00   0.00
none: in the last quarters hiring is not worth its cost at any capability above 50
value of the team over twelve quarters: V_0(58) - V_0(53) = 23.838

Panel 2 — Iterate: value iteration and its stopping bound (Theorem 7.3(ii))

AXIOM Lab 2.7, step 3. Value iteration from V=0V = 0 shrinks the largest error by at least β\beta a sweep; once two sweeps differ by at most dd, the values are within βd/(1−β)\beta d/(1-\beta) of V∗V^*. The book stops at $1 (10−610^{-6} million): 813 sweeps (Figure 7.2(a)).

Python Code
def value_iteration(model, tol, V0=None, record=False):
    V = np.zeros(model.n) if V0 is None else V0.copy(); hist, diffs = [V.copy()], []
    while True:
        V1, y = model.bellman(V); d = float(abs(V1 - V).max()); V = V1; diffs.append(d)
        if record: hist.append(V.copy())
        if BETA / (1 - BETA) * d <= tol:
            return V, y, len(diffs), diffs, hist

Vref, _, n_ref, _, _ = value_iteration(m, 1e-12)
V, y, n_vi, diffs, hist = value_iteration(m, 1e-6, record=True)
_, _, n_vi3, _, _ = value_iteration(m, 1e-3)
errs = np.array([abs(h - Vref).max() for h in hist])
print(f"sweeps to $1,000 (1e-3): {n_vi3};  to $1 (1e-6): {n_vi}")
print(f"at the stop: stopping bound {BETA/(1-BETA)*diffs[-1]:.3e}, true error {errs[-1]:.3e}; starting error {errs[0]:.1f}")
nn = np.arange(len(errs))
fig, ax = plt.subplots(figsize=(7.6, 3.6))
ax.semilogy(nn, errs, color="#1F4E79", lw=2, label="error of the values")
ax.semilogy(nn, BETA**nn * errs[0], "--", color="#616161", lw=1, label="a priori bound  βⁿ × starting error")
ax.semilogy(nn[1:], BETA/(1-BETA)*np.array(diffs), ":", color="#E65100", lw=1.4, label="stopping bound  βd/(1−β)")
ax.axhline(1e-6, color="#999999", lw=0.6)
ax.set(xlabel="sweeps of value iteration", ylabel="largest error ($ million)", title="Value iteration from zero stops after 813 sweeps (seed 26207)")
ax.legend(frameon=False, fontsize=8); plt.tight_layout(); plt.show()
sweeps to $1,000 (1e-3): 523;  to $1 (1e-6): 813
at the stop: stopping bound 9.785e-07, true error 9.785e-07; starting error 611.2
Value iteration from zero stops after 813 sweeps (seed 26207)

Panel 3 — Iterate: policy iteration from Volume I's rule (Theorem 7.3(iii))

Evaluate a rule exactly (a sparse linear solve), make it greedy for its own values, repeat. From Volume I's rule, hold planned capability at 58, three evaluations reach the best rule; value iteration started from the same values creeps (Figure 7.2(b)).

Python Code
def policy_iteration(model, y0):
    y, out = y0.copy(), []
    while True:
        Vy = model.evaluate(y); out.append((y.copy(), Vy))
        TV, ynew = model.bellman(Vy)
        qcur = model.g + K * ALPHA * model.X + model.G(Vy, y)
        keep = TV - qcur <= 1e-9 * (1 + abs(TV))          # keep a decision that is already best
        if keep.all():
            return out
        y = np.where(keep, y, ynew)

def target_of(model, y):
    h = y - ALPHA * model.X
    inside = (h > model.a + 1e-9) & (h < model.b - 1e-9)
    return float(np.median(y[inside]))

pis = policy_iteration(m, m.target_rule(58.0))
for i, (yi, Vi) in enumerate(pis):
    print(f"evaluation {i}: target {target_of(m, yi):6.2f}   V(58) = {m.at(Vi, 58):9.3f}   V(53) = {m.at(Vi, 53):9.3f}")
Vv = pis[0][1].copy(); vi_track = []
for n in range(13):
    vi_track.append(m.at(Vv, 58)); Vv, _ = m.bellman(Vv)
print("value iteration from the same start, V(58) after 0, 1, 2, 12 sweeps:", [round(vi_track[i], 3) for i in (0, 1, 2, 12)])
ybest, Vbest = pis[-1]
print(f"policy iteration's value equals value iteration's to {abs(Vbest - Vref).max():.1e}")
evaluation 0: target  58.00   V(58) =  -246.151   V(53) =  -268.961
evaluation 1: target  55.35   V(58) =  -234.008   V(53) =  -257.797
evaluation 2: target  55.75   V(58) =  -230.755   V(53) =  -254.593
value iteration from the same start, V(58) after 0, 1, 2, 12 sweeps: [-246.151, -245.765, -245.233, -242.144]
policy iteration's value equals value iteration's to 2.2e-12

Panel 4 — Look one quarter ahead: the best rule and its margin (Theorem 7.2)

AXIOM Lab 2.7, step 1. At each capability level the value of a decision is what it earns now plus the discounted expected value of where it leads; the best decision maximizes it. Written in the planned level y=0.975c+hy = 0.975c + h, the best rule is a target rule: bring planned capability to 55.75, hiring at the cap below 54.6 and nothing above 57.2. The margin of 0.75 is the critical ratio's: holding a planned point costs 4(1−0.975β)4(1 - 0.975\beta) a quarter against 1.6β1.6\beta for a point short next quarter.

Python Code
T = target_of(m, ybest)
print(f"target {T:.2f} (margin {T - FLOOR:.2f}); hire nothing above {T/ALPHA:.2f}; at the cap below {(T - CAP)/ALPHA:.2f}")
hold = K * (1 - ALPHA * BETA); ratio = hold / (BETA * P)
print(f"holding a planned point: {hold:.5f} a quarter ({4*hold:.3f} a year); a point short next quarter: {BETA*P:.3f}; critical ratio {ratio:.4f}")
print(f"P(loss > 0.75) = {PW[W > 0.75].sum():.2f} <= {ratio:.4f} <= P(loss >= 0.75) = {PW[W >= 0.75].sum():.2f}  -> margin 0.75")
need = (1 - ALPHA) * T + ALPHA * W
print("hiring needed to restore the target after each event:", np.round(need, 3), "-> the cap (2.5) and no firing (0) bind at the extremes")
hs = np.linspace(0, CAP, 251)
fig, ax = plt.subplots(figsize=(7.6, 3.6))
for c0, col in ((53, "#B71C1C"), (56, "#1F4E79"), (58, "#E65100")):
    yv = ALPHA * c0 + hs
    q = m.g[np.argmin(abs(m.X - c0))] - K * hs + BETA * sum(pm * m.interp(Vbest, yv - wm) for wm, pm in zip(W, PW))
    ax.plot(hs, q - q.max(), color=col, lw=2, label=f"c = {c0}: best h = {hs[np.argmax(q)]:.2f}")
ax.set(xlabel="points hired this quarter, h", ylabel="value of the decision less its best ($ million)", title="One quarter ahead: the value of each hiring decision (seed 26207)")
ax.legend(frameon=False, fontsize=8); plt.tight_layout(); plt.show()
target 55.75 (margin 0.75); hire nothing above 57.18; at the cap below 54.62
holding a planned point: 0.19183 a quarter (0.767 a year); a point short next quarter: 1.562; critical ratio 0.1228
P(loss > 0.75) = 0.05 <= 0.1228 <= P(loss >= 0.75) = 0.25  -> margin 0.75
hiring needed to restore the target after each event: [ 2.856  2.125  1.394  0.663 -0.069] -> the cap (2.5) and no firing (0) bind at the extremes
One quarter ahead: the value of each hiring decision (seed 26207)

Panel 5 — Read values and prices; certify (Proposition 7.1, Theorems 7.2(iii), 7.3(ii))

AXIOM Lab 2.7, step 4. The slope of the value function prices a point of capability; its differences value lumps. The engineering team, five points, is worth V∗(58)−V∗(53)V^*(58) - V^*(53) under this chapter's declaration of the delay, bracketed by five points at the prices at either end. Two certificates: the Bellman residual of the computed values, taken at every breakpoint of the piecewise-linear function, and Theorem 7.2(iii)'s closed form, an upper bound on what any rule can achieve.

Python Code
print("    c        V*     price just below   price just above")
for c0 in (50, 53, 55, 56, 58, 60):
    print(f"{c0:5.1f}  {m.at(Vbest, c0):9.3f}  {m.slope(Vbest, c0, -1):12.3f}  {m.slope(Vbest, c0, +1):16.3f}")
team = m.at(Vbest, 58) - m.at(Vbest, 53)
print(f"team V(58) - V(53) = {team:.3f}, between 5 V'(58-) = {5*m.slope(Vbest, 58, -1):.2f} and 5 V'(53+) = {5*m.slope(Vbest, 53, +1):.2f}")
print(f"  twelve quarters (Panel 1) give {team12:.3f}: the team's value is earned in the first quarters, before the paths from 53 and 58 meet at the target")

def bellman_at(model, V, xs):          # (TV)(c) at any points, for the residual over the continuum
    a = np.maximum(0.0, model.lo + model.w.max() - ALPHA * xs); b = np.minimum(model.cap, model.hi + model.w.min() - ALPHA * xs)
    L, U = ALPHA * xs + a, ALPHA * xs + b
    best = np.maximum(model.G(V, L), model.G(V, U))
    jlo = np.ceil((L - model.lo) / model.dx - 1e-9).astype(int); jhi = np.floor((U - model.lo) / model.dx + 1e-9).astype(int)
    J = jlo[:, None] + np.arange(int(round(model.cap / model.dx)) + 2)[None, :]
    Gn = model.G(V, model.X)
    vals = np.where(J <= jhi[:, None], Gn[np.clip(J, 0, model.n - 1)], -np.inf).max(1)
    return -P * np.maximum(FLOOR - xs, 0) + K * ALPHA * xs + np.maximum(best, vals)
bps = np.concatenate([m.X, m.X / ALPHA, (m.X - CAP) / ALPHA, [(LO + W.max()) / ALPHA, (HI + W.min() - CAP) / ALPHA]])
bps = np.unique(np.round(bps[(bps >= LO) & (bps <= HI)], 12))
res = np.abs(bellman_at(m, Vbest, bps) - m.interp(Vbest, bps))
print(f"Bellman residual over the continuum: {res.max():.6f} at c = {bps[res.argmax()]:.3f} -> computed values within {res.max()/(1-BETA):.3f} of V*")

def upper_bound(x0, ym=55.75):         # Theorem 7.2(iii): g(x) + k alpha x + L(y_m)/(1 - beta)
    Eg = sum(pm * -P * max(FLOOR - (ym - wm), 0) for wm, pm in zip(W, PW))
    Lval = -K * (1 - ALPHA * BETA) * ym + BETA * Eg - BETA * K * ALPHA * (PW @ W)
    return -P * max(FLOOR - x0, 0) + K * ALPHA * x0 + Lval / (1 - BETA)
for x0 in (53, 55, 56, 58):
    print(f"upper bound at {x0}: {upper_bound(x0):9.3f}   best rule {m.at(Vbest, x0):9.3f}   gap {upper_bound(x0) - m.at(Vbest, x0):.3f}")

rng = np.random.default_rng(SEED); N, Q = 100_000, 900
c = np.full(N, 58.0); tot = np.zeros(N); disc = 1.0; cum = np.cumsum(PW)
for q in range(Q):
    h = np.clip(T - ALPHA * c, np.maximum(0, LO + W.max() - ALPHA * c), np.minimum(CAP, HI + W.min() - ALPHA * c))
    tot += disc * (-P * np.maximum(FLOOR - c, 0) - K * h)
    c = ALPHA * c + h - W[np.searchsorted(cum, rng.random(N))]; disc *= BETA
sim = float(tot.mean())
print(f"simulation of the rule, 100,000 paths x 900 quarters (seed {SEED}): {sim:.3f} +/- {tot.std()/np.sqrt(N):.3f}  (recursion {m.at(Vbest, 58):.3f})")
    c        V*     price just below   price just above
 50.0   -280.567        10.084            10.071
 53.0   -254.593         7.237             7.192
 55.0   -242.356         5.500             3.900
 56.0   -238.456         3.900             3.900
 58.0   -230.755         3.682             3.682
 60.0   -223.551         3.511             3.511
team V(58) - V(53) = 23.838, between 5 V'(58-) = 18.41 and 5 V'(53+) = 35.96
  twelve quarters (Panel 1) give 23.838: the team's value is earned in the first quarters, before the paths from 53 and 58 meet at the target
Bellman residual over the continuum: 0.004403 at c = 52.667 -> computed values within 0.187 of V*
upper bound at 53:  -253.185   best rule  -254.593   gap 1.408
upper bound at 55:  -242.185   best rule  -242.356   gap 0.172
upper bound at 56:  -238.285   best rule  -238.456   gap 0.172
upper bound at 58:  -230.485   best rule  -230.755   gap 0.270
simulation of the rule, 100,000 paths x 900 quarters (seed 26207): -230.782 +/- 0.039  (recursion -230.755)

Panel 6 — Rules against plans (Proposition 7.2; Table 7.3)

A rule responds to what happens; a plan fixed today cannot, so the record's swings accumulate under it. The best committed plan holds 3.5 points of margin in the long run and costs $23.9 million more than the best rule: the value of responding. Re-planning each quarter with the best plan recovers the best rule.

Python Code
def describe(y):
    tot, hire, short, Pm = m.evaluate(y, parts=True)
    pi = np.zeros(m.n); pi[np.argmin(abs(m.X - 58))] = 1.0; PT = Pm.T.tocsr()
    for _ in range(20000):
        new = PT @ pi
        if abs(new - pi).sum() < 1e-13: break
        pi = new
    return m.at(tot, 58), -m.at(hire, 58), -m.at(short, 58), float(pi[m.X < FLOOR - 1e-9].sum())

class Accumulated:     # S_k = sum_j 0.975^(k-1-j) w_j, the capability a plan loses to events by quarter k
    def __init__(self, ds=0.01, kmax=400):
        smax = abs(W).max() / (1 - ALPHA) + 2.0; self.S = np.arange(-smax, smax + ds / 2, ds); n = len(self.S)
        shift = np.round(W / ds).astype(int); p = np.zeros(n); p[np.argmin(abs(self.S))] = 1.0; self.tails = []
        for k in range(kmax + 1):
            A = np.cumsum((p * self.S)[::-1])[::-1]; B = np.cumsum(p[::-1])[::-1]; self.tails.append((np.append(A, 0), np.append(B, 0)))
            t = (ALPHA * self.S - self.S[0]) / ds; j = np.floor(t).astype(int); th = t - j
            qd = np.bincount(j, p * (1 - th), n + 1)[:n] + np.bincount(j + 1, p * th, n + 1)[:n]
            p = np.zeros(n)
            for sm, pm in zip(shift, PW):
                if sm >= 0: p[sm:] += pm * qd[:n - sm]
                else: p[:sm] += pm * qd[-sm:]
        self.p_long = p
    def shortfall(self, k, margin):        # E[(S_k - margin)^+]
        A, B = self.tails[min(k, len(self.tails) - 1)]; margin = np.atleast_1d(np.asarray(margin, float))
        j = np.searchsorted(self.S, margin, side="left"); return A[j] - margin * B[j]

acc = Accumulated()
def best_plan(c0=58.0, horizon=1200, dp=0.01):
    Pg = np.arange(40.0, 75.0 + dp / 2, dp); Wn = np.zeros(len(Pg)); Ts = np.zeros(horizon)
    for k in range(horizon - 1, -1, -1):
        Gk = -K * Pg + BETA * Wn; Ts[k] = Pg[np.argmax(Gk >= Gk.max() - 1e-12)]
        pen = P * acc.shortfall(k, Pg - FLOOR) if k > 0 else P * np.maximum(FLOOR - Pg, 0)
        Wn = -pen + K * ALPHA * Pg + np.interp(np.clip(Ts[k], ALPHA * Pg, ALPHA * Pg + CAP), Pg, Gk)
    path, hires = [c0], []
    for k in range(horizon):
        nxt = float(np.clip(Ts[k], ALPHA * path[-1], ALPHA * path[-1] + CAP)); hires.append(nxt - ALPHA * path[-1]); path.append(nxt)
    hire = sum(BETA**q * K * hires[q] for q in range(horizon))
    short = sum(BETA**q * P * float(acc.shortfall(q, path[q] - FLOOR)[0]) for q in range(1, horizon))
    return np.array(path), Ts, hire, short

path, Ts, phire, pshort = best_plan()
rent = (1 - ALPHA) * 58
rhire = sum(BETA**q * K * rent for q in range(1200)); rshort = sum(BETA**q * P * float(acc.shortfall(q, 3.0)[0]) for q in range(1, 1200))
rows = [("Best rule: top up to 55.75", "0.75") + describe(ybest),
        ("Re-plan each quarter (first target %.2f)" % Ts[0], "0.75") + describe(m.target_rule(Ts[0])),
        ("Chapter 6's 55, as a rule", "0") + describe(m.target_rule(55.0)),
        ("Cover the worst quarter: 56.5", "1.5") + describe(m.target_rule(56.5)),
        ("Volume I's 58, as a rule", "3") + describe(m.target_rule(58.0))]
best_total = rows[0][2]
print(f"{'rule or plan':42s} margin  total   hiring   delay  short%   extra")
for name, mg, tot, hire, short, below in rows:
    print(f"{name:42s} {mg:>6s} {-tot:7.1f} {hire:8.1f} {short:7.1f} {100*below:6.1f} {best_total - tot:7.1f}")
long_short = float(acc.p_long[acc.S > path[1000] - FLOOR + 1e-9].sum())
print(f"{'Best committed plan':42s} {path[1000]-FLOOR:6.2f} {phire+pshort:7.1f} {phire:8.1f} {pshort:7.1f} {100*long_short:6.1f} {phire+pshort+best_total:7.1f}")
print(f"{'Volume I budget: rent on 58 every quarter':42s} {'3':>6s} {rhire+rshort:7.1f} {rhire:8.1f} {rshort:7.1f} {100*float(acc.p_long[acc.S > 3 + 1e-9].sum()):6.1f} {rhire+rshort+best_total:7.1f}")
print("best committed plan, planned levels for quarters 0-6:", np.round(path[:7], 2), "... long run", round(path[1000], 2))
print(f"value of responding = {phire + pshort + best_total:.3f}")

fig, ax = plt.subplots(figsize=(7.6, 3.6))
tg = np.round(np.arange(54.5, 58.51, 0.05), 2)
ax.plot(tg, [-m.at(m.evaluate(m.target_rule(t)), 58) for t in tg], color="#1F4E79", lw=2)
ax.axhline(phire + pshort, color="#E65100", ls="--", lw=1, label=f"best committed plan {phire+pshort:.1f}")
ax.axhline(rhire + rshort, color="#E65100", ls=":", lw=1, label=f"Volume I's budget {rhire+rshort:.1f}")
ax.set(xlabel="target for planned capability", ylabel="PV of hiring and delay from 58 ($ million)", title="Every target rule against the plans; least at 55.75 (seed 26207)")
ax.legend(frameon=False, fontsize=8); plt.tight_layout(); plt.show()
rule or plan                               margin  total   hiring   delay  short%   extra
Best rule: top up to 55.75                   0.75   230.8    228.0     2.7    6.0     0.0
Re-plan each quarter (first target 55.75)    0.75   230.8    228.0     2.7    6.0     0.0
Chapter 6's 55, as a rule                       0   237.4    222.1    15.2   27.5     6.6
Cover the worst quarter: 56.5                 1.5   234.0    234.0     0.1    0.3     3.3
Volume I's 58, as a rule                        3   246.2    246.2     0.0    0.0    15.4
Best committed plan                          3.50   254.7    244.9     9.7   12.3    23.9
Volume I budget: rent on 58 every quarter       3   256.3    246.3    10.0   16.0    25.6
best committed plan, planned levels for quarters 0-6: [58.   56.55 55.75 56.46 56.46 56.48 56.98] ... long run 58.5
value of responding = 23.905
Every target rule against the plans; least at 55.75 (seed 26207)

Panel 7 — Approximate (Theorem 7.3(iv))

AXIOM Lab 2.7, step 5. A coarser grid, or a smooth fitted value, makes each sweep cheaper but inexact; Theorem 7.3 bounds the cost from the approximate values' Bellman residual alone. The bound is a guarantee, not an estimate, and a shape that cannot bend at the floor fails badly.

Python Code
mc = Model(dx=0.75)
Vc, _, nc, _, _ = value_iteration(mc, 1e-9)
Vcf = np.interp(m.X, mc.X, Vc); TVc, yc = m.bellman(Vcf); rc = float(abs(TVc - Vcf).max())
Vg = m.evaluate(yc)
print(f"grid of 0.75 points ({mc.n} nodes): values within {abs(Vcf - Vbest).max():.3f}; residual {rc:.3f};"
      f" greedy target {target_of(m, yc):.2f} loses {m.at(Vbest, 58) - m.at(Vg, 58):.3f} from 58 (bound {2*BETA*rc/(1-BETA):.1f})")
S = np.arange(30.0, 80.01, 1.0); feats = lambda x: np.column_stack([np.ones_like(x), x - 55, (x - 55) ** 2])
th = np.zeros(3)
for it in range(5000):
    TV, _ = m.bellman(feats(m.X) @ th)
    new = np.linalg.lstsq(feats(S), np.interp(S, m.X, TV), rcond=None)[0]
    if abs(new - th).max() < 1e-10: th = new; break
    th = new
Vq = feats(m.X) @ th; TVq, yq = m.bellman(Vq)
eps = float(abs(feats(m.X) @ np.linalg.lstsq(feats(S), np.interp(S, m.X, TVq), rcond=None)[0] - TVq).max())
print(f"quadratic fit: fitting error {eps:.2f} a sweep; values off by {abs(Vq - Vbest).max():.1f} (bound {eps/(1-BETA):.1f});"
      f" greedy target {target_of(m, yq):.2f} loses {m.at(Vbest, 58) - m.at(m.evaluate(yq), 58):.1f} from 58")
grid of 0.75 points (68 nodes): values within 2.533; residual 0.267; greedy target 56.25 loses 2.115 from 58 (bound 22.1)
quadratic fit: fitting error 5.01 a sweep; values off by 112.5 (bound 212.8); greedy target 67.00 loses 81.7 from 58

Panel 8 — Shrink the period: Chapter 6's problem on finer calendars

AXIOM Lab 2.7, step 5, second part. Run the same recursion on Chapter 6's deterministic problem — the build waits until capability first reaches 55 and must then stay at or above it; delay costs $32 million a year; hiring at most 10.13 points a year at $4 million; decay δ=4ln⁡(1/0.975)\delta = 4\ln(1/0.975); ρ=ln⁡1.1\rho = \ln 1.1 — on quarterly, monthly and weekly calendars (hiring held level within each period). As the period shrinks, the recursion approaches Worked Example 6.1's continuous-time values, V(53)=−254.89V(53) = -254.89, V(58)=−222.36V(58) = -222.36, team $32.53 million (the limit of Equation 7.4).

Python Code
DELTA, RHO, UBAR, D = 4 * np.log(1 / 0.975), np.log(1.1), 100 * 4 * np.log(1 / 0.975), 32.0
def chapter6_on_calendar(dt, dx=0.005, lo=40.0, hi=70.0):
    a, b, beta = np.exp(-DELTA * dt), (1 - np.exp(-DELTA * dt)) / DELTA, np.exp(-RHO * dt)
    X = lo + dx * np.arange(int(round((hi - lo) / dx)) + 1); n = len(X)
    kap = K * (1 - beta) / (RHO * b)                 # cost of moving the next state up by one point
    flow = D * (1 - beta) / RHO                      # a period's delay
    def step(V0, V1):
        Wn = np.where(X >= FLOOR - 1e-12, V1, V0)    # next period: started once capability is at the floor
        Gs = -kap * X + beta * Wn
        def best(L, U, Gv):                          # max over the reachable next states [L, U]
            jlo = np.ceil((L - lo) / dx - 1e-9).astype(int); jhi = np.floor((U - lo) / dx + 1e-9).astype(int)
            width = int(np.ceil(b * UBAR / dx)) + 2
            J = jlo[:, None] + np.arange(width)[None, :]
            vals = np.where(J <= jhi[:, None], Gv[np.clip(J, 0, n - 1)], -np.inf)
            ends = np.column_stack([np.interp(L, X, Gv), np.interp(U, X, Gv)])
            return np.maximum(vals.max(1), ends.max(1))
        L, U = np.clip(a * X, lo, hi), np.clip(a * X + b * UBAR, lo, hi)
        V0n = -flow + kap * a * X + best(L, U, Gs)                    # not started: pay the delay
        G1 = -kap * X + beta * V1
        L1 = np.clip(np.maximum(a * X, FLOOR), lo, hi)                # started: stay at or above the floor
        V1n = np.where(U >= L1 - 1e-12, kap * a * X + best(L1, np.maximum(U, L1), G1), -1e9)
        return V0n, V1n
    V0, V1 = np.zeros(n), np.zeros(n)
    for it in range(200000):
        N0, N1 = step(V0, V1)
        d = max(abs(N0 - V0).max(), abs(N1[X >= FLOOR] - V1[X >= FLOOR]).max()); V0, V1 = N0, N1
        if beta / (1 - beta) * d < 1e-5: break
    Vgone = float(np.interp(53.0, X, V0)); Vkept = float(np.interp(58.0, X, V1))
    c, n_wait = 53.0, 0                              # periods of full hiring before capability reaches 55
    while c < FLOOR - 1e-12:
        c, n_wait = a * c + b * UBAR, n_wait + 1
    return Vgone, Vkept, n_wait * dt
print("calendar      V(53)      V(58)     team   build starts after (years)")
for name, dt in (("quarterly", 0.25), ("monthly", 1 / 12), ("weekly", 1 / 52)):
    g, k_, wait = chapter6_on_calendar(dt)
    print(f"{name:10s} {g:9.3f}  {k_:9.3f}  {k_ - g:7.3f}   {wait:6.3f}")
print(f"{'continuous':10s} {-254.892:9.3f}  {-222.363:9.3f}  {32.529:7.3f}   {np.log(47 / 45) / DELTA:6.3f}   (Worked Example 6.1)")
calendar      V(53)      V(58)     team   build starts after (years)
quarterly   -256.969   -222.374   34.595    0.500
monthly     -256.950   -222.366   34.584    0.500
weekly      -255.276   -222.363   32.913    0.442
continuous  -254.892   -222.363   32.529    0.429   (Worked Example 6.1)

The value of keeping the build running, V(58)V(58), agrees with the continuous-time value to the third decimal on every calendar. The value from 53 converges more slowly, and not evenly, because on a calendar the build can start only at the beginning of a period: the wait for capability to reach 55 rounds up to whole periods. Two quarters and six months are both half a year, 23 weeks are 0.442 years, against 0.429 in continuous time. As the period shrinks, the rounding vanishes and the recursion's values approach Chapter 6's.

Validation — the book's numbers, and agreement with DCT_V2_Ch07_Lab.xlsx

Each checkpoint is a number printed in Chapter 7 (Worked Example 7.1, Table 7.3, Figures 7.1–7.3) or in Online Appendix 7.B.1. The workbook recomputes the ones marked workbook with Excel formulas and shows PASS against the same engine values on its Checks sheet.

Python Code
checks = [
    ("value iteration sweeps to 1e-6", n_vi, 813, 0),
    ("policy iteration evaluations", len(pis), 3, 0),
    ("target (planned capability)", T, 55.75, 1e-9),
    ("trigger: hire nothing above", T / ALPHA, 57.179, 5e-4),
    ("critical ratio (workbook)", ratio, 0.1228, 5e-5),
    ("holding cost of a planned point (workbook)", hold, 0.19183, 5e-6),
    ("V*(58) (workbook)", m.at(Vbest, 58), -230.755, 5e-4),
    ("V*(53) (workbook)", m.at(Vbest, 53), -254.593, 5e-4),
    ("team V(58) - V(53) (workbook)", team, 23.838, 5e-4),
    ("price of a point at 58 (workbook)", m.slope(Vbest, 58, -1), 3.682, 5e-4),
    ("price of a point at 56 (workbook)", m.slope(Vbest, 56, +1), 3.900, 5e-4),
    ("price of a point at 53 (workbook)", m.slope(Vbest, 53, +1), 7.192, 5e-4),
    ("Bellman residual over the continuum", res.max(), 0.004403, 5e-7),
    ("upper-bound gap at 58 (workbook)", upper_bound(58) - m.at(Vbest, 58), 0.270, 5e-4),
    ("simulation of the rule from 58 (seed 26207)", sim, -230.782, 5e-4),
    ("best rule, total cost from 58", -rows[0][2], 230.755, 5e-4),
    ("target 55: extra cost", best_total - rows[2][2], 6.615, 5e-4),
    ("target 56.5: extra cost", best_total - rows[3][2], 3.289, 5e-4),
    ("target 58: extra cost", best_total - rows[4][2], 15.396, 5e-4),
    ("best committed plan, total", phire + pshort, 254.660, 5e-4),
    ("Volume I's budget, total", rhire + rshort, 256.324, 5e-3),
    ("value of responding", phire + pshort + best_total, 23.905, 5e-4),
    ("re-planning's first target", Ts[0], 55.75, 1e-9),
    ("long-run share short, best rule (%)", 100 * rows[0][5], 6.0, 0.05),
    ("long-run share short, target 55 (%)", 100 * rows[2][5], 27.5, 0.05),
]
ok = 0
for name, got, want, tol in checks:
    status = "PASS" if abs(got - want) <= tol else "FAIL"; ok += status == "PASS"
    print(f"{name:45s} {got:12.4f}   book {want:10.4f}   {status}")
print(f"\n{ok} of {len(checks)} checkpoints PASS")
value iteration sweeps to 1e-6                    813.0000   book   813.0000   PASS
policy iteration evaluations                        3.0000   book     3.0000   PASS
target (planned capability)                        55.7500   book    55.7500   PASS
trigger: hire nothing above                        57.1795   book    57.1790   PASS
critical ratio (workbook)                           0.1228   book     0.1228   PASS
holding cost of a planned point (workbook)          0.1918   book     0.1918   PASS
V*(58) (workbook)                                -230.7550   book  -230.7550   PASS
V*(53) (workbook)                                -254.5929   book  -254.5930   PASS
team V(58) - V(53) (workbook)                      23.8379   book    23.8380   PASS
price of a point at 58 (workbook)                   3.6824   book     3.6820   PASS
price of a point at 56 (workbook)                   3.9000   book     3.9000   PASS
price of a point at 53 (workbook)                   7.1918   book     7.1920   PASS
Bellman residual over the continuum                 0.0044   book     0.0044   PASS
upper-bound gap at 58 (workbook)                    0.2703   book     0.2700   PASS
simulation of the rule from 58 (seed 26207)      -230.7822   book  -230.7820   PASS
best rule, total cost from 58                     230.7550   book   230.7550   PASS
target 55: extra cost                               6.6148   book     6.6150   PASS
target 56.5: extra cost                             3.2888   book     3.2890   PASS
target 58: extra cost                              15.3965   book    15.3960   PASS
best committed plan, total                        254.6598   book   254.6600   PASS
Volume I's budget, total                          256.3238   book   256.3240   PASS
value of responding                                23.9048   book    23.9050   PASS
re-planning's first target                         55.7500   book    55.7500   PASS
long-run share short, best rule (%)                 6.0235   book     6.0000   PASS
long-run share short, target 55 (%)                27.5282   book    27.5000   PASS

25 of 25 checkpoints PASS

Next. Chapter 8 lets the periods shrink to zero: the Hamilton–Jacobi–Bellman equation (Equation 7.4) and its verification theorems. AXIOM-07 runs the same model interactively (Lab 2.7); this notebook and the workbook are its reproducible record.

Dynamic Corporate Transformation — Volume I: Foundations and Mathematical Framework; Volume II: Enterprise Optimization & Applications
Jump to any chapter

Volume I: Foundations and Mathematical Framework

  1. Ch. 1Introduction to Dynamic Corporate Transformation
  2. Ch. 2Enterprise Systems and Transformation
  3. Ch. 3Mathematical Foundations
  4. Ch. 4Enterprise Modeling Principles
  5. Ch. 5Enterprise State Representation
  6. Ch. 6Enterprise Transformation Operators
  7. Ch. 7Dynamic Enterprise Systems
  8. Ch. 8Stochastic Enterprise Dynamics
  9. Ch. 9Enterprise State Architecture
  10. Ch. 10Enterprise Capital Architecture
  11. Ch. 11Enterprise Performance Architecture
  12. Ch. 12Enterprise Risk, Resilience, and Robustness Architecture
  13. Ch. 13Unified Enterprise Transformation Architecture
  14. Ch. 14Mathematical Analysis of the Unified Enterprise Transformation Architecture
  15. Ch. 15Mathematical Formulation of Enterprise Optimization
  16. Ch. 16Synthesis, Future Directions, and Transition to Volume II

Volume II: Enterprise Optimization & Applications

  1. Ch. 1Introduction to Enterprise Optimization
  2. Ch. 2The General Enterprise Optimization Problem Revisited
  3. Ch. 3Convex Enterprise Optimization
  4. Ch. 4Nonlinear and Mixed-Integer Enterprise Optimization
  5. Ch. 5Dynamic Enterprise Optimization
  6. Ch. 6Optimal Control of Enterprise Systems
  7. Ch. 7Dynamic Programming for Enterprise Systems
  8. Ch. 8Hamilton–Jacobi–Bellman Enterprise Framework
  9. Ch. 9Stochastic Enterprise Optimization
  10. Ch. 10Neuro-Fuzzy Robust Enterprise Optimization
  11. Ch. 11Distributionally Robust Enterprise Optimization
  12. Ch. 12Multi-Objective Enterprise Optimization
  13. Ch. 13Machine Learning for Enterprise Transformation
  14. Ch. 14Artificial Intelligence for Enterprise Optimization
  15. Ch. 15Enterprise Digital Twins and Autonomous Enterprise Transformation
  16. Ch. 16Enterprise Applications and Integrated Transformation Case Studies

Dynamic Corporate Transformation

Springer Nature · Management for Professionals · Forthcoming 2027

  • Volume I — Foundations and Mathematical Framework
  • Volume II — Enterprise Optimization & Applications
Buy Volume I: Dynamic Corporate Transformation: Foundations and Mathematical FrameworkBuy Volume II: Dynamic Corporate Transformation: Enterprise Optimization & Applications
Jump to any chapter

Volume I: Foundations and Mathematical Framework

  1. Ch. 1Introduction to Dynamic Corporate Transformation
  2. Ch. 2Enterprise Systems and Transformation
  3. Ch. 3Mathematical Foundations
  4. Ch. 4Enterprise Modeling Principles
  5. Ch. 5Enterprise State Representation
  6. Ch. 6Enterprise Transformation Operators
  7. Ch. 7Dynamic Enterprise Systems
  8. Ch. 8Stochastic Enterprise Dynamics
  9. Ch. 9Enterprise State Architecture
  10. Ch. 10Enterprise Capital Architecture
  11. Ch. 11Enterprise Performance Architecture
  12. Ch. 12Enterprise Risk, Resilience, and Robustness Architecture
  13. Ch. 13Unified Enterprise Transformation Architecture
  14. Ch. 14Mathematical Analysis of the Unified Enterprise Transformation Architecture
  15. Ch. 15Mathematical Formulation of Enterprise Optimization
  16. Ch. 16Synthesis, Future Directions, and Transition to Volume II

Volume II: Enterprise Optimization & Applications

  1. Ch. 1Introduction to Enterprise Optimization
  2. Ch. 2The General Enterprise Optimization Problem Revisited
  3. Ch. 3Convex Enterprise Optimization
  4. Ch. 4Nonlinear and Mixed-Integer Enterprise Optimization
  5. Ch. 5Dynamic Enterprise Optimization
  6. Ch. 6Optimal Control of Enterprise Systems
  7. Ch. 7Dynamic Programming for Enterprise Systems
  8. Ch. 8Hamilton–Jacobi–Bellman Enterprise Framework
  9. Ch. 9Stochastic Enterprise Optimization
  10. Ch. 10Neuro-Fuzzy Robust Enterprise Optimization
  11. Ch. 11Distributionally Robust Enterprise Optimization
  12. Ch. 12Multi-Objective Enterprise Optimization
  13. Ch. 13Machine Learning for Enterprise Transformation
  14. Ch. 14Artificial Intelligence for Enterprise Optimization
  15. Ch. 15Enterprise Digital Twins and Autonomous Enterprise Transformation
  16. Ch. 16Enterprise Applications and Integrated Transformation Case Studies