DCT Laboratory — Volume II, Chapter 7
Dynamic Programming for Enterprise Systems
Seed 26207 · Companion to Chapter 7 and AXIOM Lab 2.7 (module AXIOM-07) · Mirrored in DCT_V2_Ch07_Lab.xlsx
This laboratory reproduces Worked Example 7.1, How Much Margin Should Meridian Hold Above the Capability Floor?, with the book's declarations, and works the five steps of AXIOM Lab 2.7: look one quarter ahead, solve backward, iterate, read values and prices, approximate and shrink the period. Every Meridian number below is the book's (Table 7.3, Figures 7.1–7.3, Online Appendix 7.B.1); the final cell checks them. Money is in $ million.
The declarations (Worked Example 7.1, Step 1)
Workforce capability is the state; the decision is the points hired and trained in a quarter, at $4 million a point and at most 2.5 a quarter (Volume I, Chapter 10). Capability keeps 97.5 percent of itself each quarter and moves by an unplanned loss from the engineering office's record, so . The record's law: capability ends where planned in half of all quarters, three-quarters of a point higher or lower in four quarters in ten, and a point and a half higher or lower in one quarter in ten, so is 1.5, 0.75, 0, or points with probabilities 0.05, 0.20, 0.50, 0.20 and 0.05. Below the floor of 55 the digital build slows by a fifth per point short, so a point short for a quarter costs million. The board discounts at 10 percent a year, a quarter, over an open horizon. Capability is modeled between 30 and 80, with hiring limits at the two ends that keep it there (they never bind where the rules operate).
Python Code
import numpy as np
import scipy.sparse as sp
import scipy.sparse.linalg as spla
import matplotlib.pyplot as plt
plt.rcParams.update({"figure.dpi": 110, "axes.spines.top": False, "axes.spines.right": False})
SEED = 26207
BETA, ALPHA, K, CAP, FLOOR, P = 1.1 ** -0.25, 0.975, 4.0, 2.5, 55.0, 1.6
W = np.array([1.5, 0.75, 0.0, -0.75, -1.5]) # unplanned loss (points)
PW = np.array([0.05, 0.20, 0.50, 0.20, 0.05]) # the engineering office's record
LO, HI, DX = 30.0, 80.0, 0.05
class Model:
# The declared quarterly model on a grid; values are piecewise linear between nodes.
def __init__(self, w=W, pw=PW, p=P, cap=CAP, lo=LO, hi=HI, dx=DX, charge=None):
self.w, self.pw, self.p, self.cap = np.asarray(w, float), np.asarray(pw, float), p, cap
self.lo, self.hi, self.dx = lo, hi, dx
self.n = int(round((hi - lo) / dx)) + 1
self.X = lo + dx * np.arange(self.n)
self.s = np.round(self.w / dx).astype(int) # events are whole grid steps
self.g = -p * np.maximum(FLOOR - self.X, 0) if charge is None else charge(self.X)
a = np.maximum(0.0, lo + self.w.max() - ALPHA * self.X) # keep c' >= lo
b = np.minimum(cap, hi + self.w.min() - ALPHA * self.X) # keep c' <= hi
self.a, self.b = np.minimum(a, cap), np.maximum(b, np.minimum(a, cap))
self.L, self.U = ALPHA * self.X + self.a, ALPHA * self.X + self.b # planned level y = 0.975c + h
def interp(self, V, x):
return np.interp(np.clip(x, self.lo, self.hi), self.X, V)
def G(self, V, y): # -k y + beta E V(y - w): value of a planned level
return -K * y + BETA * sum(pm * self.interp(V, y - wm) for wm, pm in zip(self.w, self.pw))
def bellman(self, V):
# (TV)(c) and the best planned level: exact for piecewise-linear V, because the value of a
# decision is linear between grid nodes, so its maximum is at a node or at an end of [L, U].
idx = np.arange(self.n)
Gn = -K * self.X + BETA * sum(pm * V[np.clip(idx - sm, 0, self.n - 1)] for sm, pm in zip(self.s, self.pw))
jlo = np.ceil((self.L - self.lo) / self.dx - 1e-9).astype(int)
jhi = np.floor((self.U - self.lo) / self.dx + 1e-9).astype(int)
J = jlo[:, None] + np.arange(int(round(self.cap / self.dx)) + 2)[None, :]
ok = J <= jhi[:, None]
vals = np.column_stack([self.G(V, self.L), np.where(ok, Gn[np.clip(J, 0, self.n - 1)], -np.inf), self.G(V, self.U)])
ys = np.column_stack([self.L, np.where(ok, self.X[np.clip(J, 0, self.n - 1)], np.inf), self.U])
best = vals.max(1)
first = np.argmax(vals >= best[:, None] - 1e-10, axis=1) # ties: least hiring
return self.g + K * ALPHA * self.X + best, ys[np.arange(self.n), first]
def evaluate(self, y, parts=False):
# exact value of the stationary rule y(c): solve V = r + beta P V (Theorem 7.1(ii))
rows, cols, vals = [], [], []
for wm, pm in zip(self.w, self.pw):
t = (np.clip(y - wm, self.lo, self.hi) - self.lo) / self.dx
j = np.minimum(np.floor(t + 1e-12).astype(int), self.n - 2)
rows += [np.arange(self.n)] * 2; cols += [j, j + 1]; vals += [pm * (1 - (t - j)), pm * (t - j)]
Pm = sp.csr_matrix((np.concatenate(vals), (np.concatenate(rows), np.concatenate(cols))), shape=(self.n, self.n))
lu = spla.splu((sp.identity(self.n, format="csc") - BETA * Pm.tocsc()))
hire = -K * (y - ALPHA * self.X)
return (lu.solve(self.g + hire), lu.solve(hire), lu.solve(self.g), Pm) if parts else lu.solve(self.g + hire)
def target_rule(self, T):
return np.clip(T, self.L, self.U)
def at(self, V, c):
return float(self.interp(V, c))
def slope(self, V, c, side=+1):
return (self.at(V, c + self.dx) - self.at(V, c)) / self.dx if side > 0 else (self.at(V, c) - self.at(V, c - self.dx)) / self.dx
m = Model()
print(f"beta = {BETA:.6f} a quarter; effective horizon 1/(1 - beta) = {1/(1-BETA):.1f} quarters")
print(f"unplanned loss: mean {PW @ W:.3f}, standard deviation {np.sqrt(PW @ W**2):.3f} points a quarter")
print(f"{m.n} grid nodes from {LO:.0f} to {HI:.0f}; the edge limits bind only below {(LO + W.max())/ALPHA:.2f} and above {(HI + W.min() - CAP)/ALPHA:.2f}")beta = 0.976454 a quarter; effective horizon 1/(1 - beta) = 42.5 quarters unplanned loss: mean 0.000, standard deviation 0.671 points a quarter 1001 grid nodes from 30 to 80; the edge limits bind only below 32.31 and above 77.95
Panel 1 — Solve backward (Theorem 7.1(i))
AXIOM Lab 2.7, step 2: a twelve-quarter horizon (Volume I's modeling horizon) solved by backward induction, and , and its rule table: the hiring at each capability level in each quarter. Each quarter's rule is a target rule (Theorem 7.2 holds period by period); its target maximizes the value of a planned level, .
Python Code
V = np.zeros(m.n); idx = np.arange(m.n); levels = (53, 54, 55, 56, 57, 58); table = []
for k in range(11, -1, -1):
G = -K * m.X + BETA * sum(pm * V[np.clip(idx - sm, 0, m.n - 1)] for sm, pm in zip(m.s, m.pw))
Tk = float(m.X[np.argmax(G >= G.max() - 1e-10)]) # quarter k's target for planned capability
V, y = m.bellman(V) # V is now V_k
table.append((k, Tk, [m.at(y - ALPHA * m.X, c0) for c0 in levels]))
print("quarter target hiring at capability " + "".join(f"{c0:7d}" for c0 in levels))
for k, Tk, hk in sorted(table):
tgt = f"{Tk:6.2f}" if Tk >= ALPHA * 50 else " none"
print(f" {k:2d} {tgt} " + "".join(f"{h:7.2f}" for h in hk))
print("none: in the last quarters hiring is not worth its cost at any capability above 50")
team12 = m.at(V, 58) - m.at(V, 53)
print(f"value of the team over twelve quarters: V_0(58) - V_0(53) = {team12:.3f}")quarter target hiring at capability 53 54 55 56 57 58 0 55.75 2.50 2.50 2.12 1.15 0.18 0.00 1 55.75 2.50 2.50 2.12 1.15 0.18 0.00 2 55.75 2.50 2.50 2.12 1.15 0.18 0.00 3 55.75 2.50 2.50 2.12 1.15 0.18 0.00 4 55.75 2.50 2.50 2.12 1.15 0.18 0.00 5 55.75 2.50 2.50 2.12 1.15 0.18 0.00 6 55.75 2.50 2.50 2.12 1.15 0.18 0.00 7 55.65 2.50 2.50 2.03 1.05 0.08 0.00 8 54.85 2.50 2.20 1.23 0.25 0.00 0.00 9 none 0.00 0.00 0.00 0.00 0.00 0.00 10 none 0.00 0.00 0.00 0.00 0.00 0.00 11 none 0.00 0.00 0.00 0.00 0.00 0.00 none: in the last quarters hiring is not worth its cost at any capability above 50 value of the team over twelve quarters: V_0(58) - V_0(53) = 23.838
Panel 2 — Iterate: value iteration and its stopping bound (Theorem 7.3(ii))
AXIOM Lab 2.7, step 3. Value iteration from shrinks the largest error by at least a sweep; once two sweeps differ by at most , the values are within of . The book stops at $1 ( million): 813 sweeps (Figure 7.2(a)).
Python Code
def value_iteration(model, tol, V0=None, record=False):
V = np.zeros(model.n) if V0 is None else V0.copy(); hist, diffs = [V.copy()], []
while True:
V1, y = model.bellman(V); d = float(abs(V1 - V).max()); V = V1; diffs.append(d)
if record: hist.append(V.copy())
if BETA / (1 - BETA) * d <= tol:
return V, y, len(diffs), diffs, hist
Vref, _, n_ref, _, _ = value_iteration(m, 1e-12)
V, y, n_vi, diffs, hist = value_iteration(m, 1e-6, record=True)
_, _, n_vi3, _, _ = value_iteration(m, 1e-3)
errs = np.array([abs(h - Vref).max() for h in hist])
print(f"sweeps to $1,000 (1e-3): {n_vi3}; to $1 (1e-6): {n_vi}")
print(f"at the stop: stopping bound {BETA/(1-BETA)*diffs[-1]:.3e}, true error {errs[-1]:.3e}; starting error {errs[0]:.1f}")
nn = np.arange(len(errs))
fig, ax = plt.subplots(figsize=(7.6, 3.6))
ax.semilogy(nn, errs, color="#1F4E79", lw=2, label="error of the values")
ax.semilogy(nn, BETA**nn * errs[0], "--", color="#616161", lw=1, label="a priori bound βⁿ × starting error")
ax.semilogy(nn[1:], BETA/(1-BETA)*np.array(diffs), ":", color="#E65100", lw=1.4, label="stopping bound βd/(1−β)")
ax.axhline(1e-6, color="#999999", lw=0.6)
ax.set(xlabel="sweeps of value iteration", ylabel="largest error ($ million)", title="Value iteration from zero stops after 813 sweeps (seed 26207)")
ax.legend(frameon=False, fontsize=8); plt.tight_layout(); plt.show()sweeps to $1,000 (1e-3): 523; to $1 (1e-6): 813 at the stop: stopping bound 9.785e-07, true error 9.785e-07; starting error 611.2

Panel 3 — Iterate: policy iteration from Volume I's rule (Theorem 7.3(iii))
Evaluate a rule exactly (a sparse linear solve), make it greedy for its own values, repeat. From Volume I's rule, hold planned capability at 58, three evaluations reach the best rule; value iteration started from the same values creeps (Figure 7.2(b)).
Python Code
def policy_iteration(model, y0):
y, out = y0.copy(), []
while True:
Vy = model.evaluate(y); out.append((y.copy(), Vy))
TV, ynew = model.bellman(Vy)
qcur = model.g + K * ALPHA * model.X + model.G(Vy, y)
keep = TV - qcur <= 1e-9 * (1 + abs(TV)) # keep a decision that is already best
if keep.all():
return out
y = np.where(keep, y, ynew)
def target_of(model, y):
h = y - ALPHA * model.X
inside = (h > model.a + 1e-9) & (h < model.b - 1e-9)
return float(np.median(y[inside]))
pis = policy_iteration(m, m.target_rule(58.0))
for i, (yi, Vi) in enumerate(pis):
print(f"evaluation {i}: target {target_of(m, yi):6.2f} V(58) = {m.at(Vi, 58):9.3f} V(53) = {m.at(Vi, 53):9.3f}")
Vv = pis[0][1].copy(); vi_track = []
for n in range(13):
vi_track.append(m.at(Vv, 58)); Vv, _ = m.bellman(Vv)
print("value iteration from the same start, V(58) after 0, 1, 2, 12 sweeps:", [round(vi_track[i], 3) for i in (0, 1, 2, 12)])
ybest, Vbest = pis[-1]
print(f"policy iteration's value equals value iteration's to {abs(Vbest - Vref).max():.1e}")evaluation 0: target 58.00 V(58) = -246.151 V(53) = -268.961 evaluation 1: target 55.35 V(58) = -234.008 V(53) = -257.797 evaluation 2: target 55.75 V(58) = -230.755 V(53) = -254.593 value iteration from the same start, V(58) after 0, 1, 2, 12 sweeps: [-246.151, -245.765, -245.233, -242.144] policy iteration's value equals value iteration's to 2.2e-12
Panel 4 — Look one quarter ahead: the best rule and its margin (Theorem 7.2)
AXIOM Lab 2.7, step 1. At each capability level the value of a decision is what it earns now plus the discounted expected value of where it leads; the best decision maximizes it. Written in the planned level , the best rule is a target rule: bring planned capability to 55.75, hiring at the cap below 54.6 and nothing above 57.2. The margin of 0.75 is the critical ratio's: holding a planned point costs a quarter against for a point short next quarter.
Python Code
T = target_of(m, ybest)
print(f"target {T:.2f} (margin {T - FLOOR:.2f}); hire nothing above {T/ALPHA:.2f}; at the cap below {(T - CAP)/ALPHA:.2f}")
hold = K * (1 - ALPHA * BETA); ratio = hold / (BETA * P)
print(f"holding a planned point: {hold:.5f} a quarter ({4*hold:.3f} a year); a point short next quarter: {BETA*P:.3f}; critical ratio {ratio:.4f}")
print(f"P(loss > 0.75) = {PW[W > 0.75].sum():.2f} <= {ratio:.4f} <= P(loss >= 0.75) = {PW[W >= 0.75].sum():.2f} -> margin 0.75")
need = (1 - ALPHA) * T + ALPHA * W
print("hiring needed to restore the target after each event:", np.round(need, 3), "-> the cap (2.5) and no firing (0) bind at the extremes")
hs = np.linspace(0, CAP, 251)
fig, ax = plt.subplots(figsize=(7.6, 3.6))
for c0, col in ((53, "#B71C1C"), (56, "#1F4E79"), (58, "#E65100")):
yv = ALPHA * c0 + hs
q = m.g[np.argmin(abs(m.X - c0))] - K * hs + BETA * sum(pm * m.interp(Vbest, yv - wm) for wm, pm in zip(W, PW))
ax.plot(hs, q - q.max(), color=col, lw=2, label=f"c = {c0}: best h = {hs[np.argmax(q)]:.2f}")
ax.set(xlabel="points hired this quarter, h", ylabel="value of the decision less its best ($ million)", title="One quarter ahead: the value of each hiring decision (seed 26207)")
ax.legend(frameon=False, fontsize=8); plt.tight_layout(); plt.show()target 55.75 (margin 0.75); hire nothing above 57.18; at the cap below 54.62 holding a planned point: 0.19183 a quarter (0.767 a year); a point short next quarter: 1.562; critical ratio 0.1228 P(loss > 0.75) = 0.05 <= 0.1228 <= P(loss >= 0.75) = 0.25 -> margin 0.75 hiring needed to restore the target after each event: [ 2.856 2.125 1.394 0.663 -0.069] -> the cap (2.5) and no firing (0) bind at the extremes

Panel 5 — Read values and prices; certify (Proposition 7.1, Theorems 7.2(iii), 7.3(ii))
AXIOM Lab 2.7, step 4. The slope of the value function prices a point of capability; its differences value lumps. The engineering team, five points, is worth under this chapter's declaration of the delay, bracketed by five points at the prices at either end. Two certificates: the Bellman residual of the computed values, taken at every breakpoint of the piecewise-linear function, and Theorem 7.2(iii)'s closed form, an upper bound on what any rule can achieve.
Python Code
print(" c V* price just below price just above")
for c0 in (50, 53, 55, 56, 58, 60):
print(f"{c0:5.1f} {m.at(Vbest, c0):9.3f} {m.slope(Vbest, c0, -1):12.3f} {m.slope(Vbest, c0, +1):16.3f}")
team = m.at(Vbest, 58) - m.at(Vbest, 53)
print(f"team V(58) - V(53) = {team:.3f}, between 5 V'(58-) = {5*m.slope(Vbest, 58, -1):.2f} and 5 V'(53+) = {5*m.slope(Vbest, 53, +1):.2f}")
print(f" twelve quarters (Panel 1) give {team12:.3f}: the team's value is earned in the first quarters, before the paths from 53 and 58 meet at the target")
def bellman_at(model, V, xs): # (TV)(c) at any points, for the residual over the continuum
a = np.maximum(0.0, model.lo + model.w.max() - ALPHA * xs); b = np.minimum(model.cap, model.hi + model.w.min() - ALPHA * xs)
L, U = ALPHA * xs + a, ALPHA * xs + b
best = np.maximum(model.G(V, L), model.G(V, U))
jlo = np.ceil((L - model.lo) / model.dx - 1e-9).astype(int); jhi = np.floor((U - model.lo) / model.dx + 1e-9).astype(int)
J = jlo[:, None] + np.arange(int(round(model.cap / model.dx)) + 2)[None, :]
Gn = model.G(V, model.X)
vals = np.where(J <= jhi[:, None], Gn[np.clip(J, 0, model.n - 1)], -np.inf).max(1)
return -P * np.maximum(FLOOR - xs, 0) + K * ALPHA * xs + np.maximum(best, vals)
bps = np.concatenate([m.X, m.X / ALPHA, (m.X - CAP) / ALPHA, [(LO + W.max()) / ALPHA, (HI + W.min() - CAP) / ALPHA]])
bps = np.unique(np.round(bps[(bps >= LO) & (bps <= HI)], 12))
res = np.abs(bellman_at(m, Vbest, bps) - m.interp(Vbest, bps))
print(f"Bellman residual over the continuum: {res.max():.6f} at c = {bps[res.argmax()]:.3f} -> computed values within {res.max()/(1-BETA):.3f} of V*")
def upper_bound(x0, ym=55.75): # Theorem 7.2(iii): g(x) + k alpha x + L(y_m)/(1 - beta)
Eg = sum(pm * -P * max(FLOOR - (ym - wm), 0) for wm, pm in zip(W, PW))
Lval = -K * (1 - ALPHA * BETA) * ym + BETA * Eg - BETA * K * ALPHA * (PW @ W)
return -P * max(FLOOR - x0, 0) + K * ALPHA * x0 + Lval / (1 - BETA)
for x0 in (53, 55, 56, 58):
print(f"upper bound at {x0}: {upper_bound(x0):9.3f} best rule {m.at(Vbest, x0):9.3f} gap {upper_bound(x0) - m.at(Vbest, x0):.3f}")
rng = np.random.default_rng(SEED); N, Q = 100_000, 900
c = np.full(N, 58.0); tot = np.zeros(N); disc = 1.0; cum = np.cumsum(PW)
for q in range(Q):
h = np.clip(T - ALPHA * c, np.maximum(0, LO + W.max() - ALPHA * c), np.minimum(CAP, HI + W.min() - ALPHA * c))
tot += disc * (-P * np.maximum(FLOOR - c, 0) - K * h)
c = ALPHA * c + h - W[np.searchsorted(cum, rng.random(N))]; disc *= BETA
sim = float(tot.mean())
print(f"simulation of the rule, 100,000 paths x 900 quarters (seed {SEED}): {sim:.3f} +/- {tot.std()/np.sqrt(N):.3f} (recursion {m.at(Vbest, 58):.3f})")c V* price just below price just above 50.0 -280.567 10.084 10.071 53.0 -254.593 7.237 7.192 55.0 -242.356 5.500 3.900 56.0 -238.456 3.900 3.900 58.0 -230.755 3.682 3.682 60.0 -223.551 3.511 3.511 team V(58) - V(53) = 23.838, between 5 V'(58-) = 18.41 and 5 V'(53+) = 35.96 twelve quarters (Panel 1) give 23.838: the team's value is earned in the first quarters, before the paths from 53 and 58 meet at the target Bellman residual over the continuum: 0.004403 at c = 52.667 -> computed values within 0.187 of V* upper bound at 53: -253.185 best rule -254.593 gap 1.408 upper bound at 55: -242.185 best rule -242.356 gap 0.172 upper bound at 56: -238.285 best rule -238.456 gap 0.172 upper bound at 58: -230.485 best rule -230.755 gap 0.270 simulation of the rule, 100,000 paths x 900 quarters (seed 26207): -230.782 +/- 0.039 (recursion -230.755)
Panel 6 — Rules against plans (Proposition 7.2; Table 7.3)
A rule responds to what happens; a plan fixed today cannot, so the record's swings accumulate under it. The best committed plan holds 3.5 points of margin in the long run and costs $23.9 million more than the best rule: the value of responding. Re-planning each quarter with the best plan recovers the best rule.
Python Code
def describe(y):
tot, hire, short, Pm = m.evaluate(y, parts=True)
pi = np.zeros(m.n); pi[np.argmin(abs(m.X - 58))] = 1.0; PT = Pm.T.tocsr()
for _ in range(20000):
new = PT @ pi
if abs(new - pi).sum() < 1e-13: break
pi = new
return m.at(tot, 58), -m.at(hire, 58), -m.at(short, 58), float(pi[m.X < FLOOR - 1e-9].sum())
class Accumulated: # S_k = sum_j 0.975^(k-1-j) w_j, the capability a plan loses to events by quarter k
def __init__(self, ds=0.01, kmax=400):
smax = abs(W).max() / (1 - ALPHA) + 2.0; self.S = np.arange(-smax, smax + ds / 2, ds); n = len(self.S)
shift = np.round(W / ds).astype(int); p = np.zeros(n); p[np.argmin(abs(self.S))] = 1.0; self.tails = []
for k in range(kmax + 1):
A = np.cumsum((p * self.S)[::-1])[::-1]; B = np.cumsum(p[::-1])[::-1]; self.tails.append((np.append(A, 0), np.append(B, 0)))
t = (ALPHA * self.S - self.S[0]) / ds; j = np.floor(t).astype(int); th = t - j
qd = np.bincount(j, p * (1 - th), n + 1)[:n] + np.bincount(j + 1, p * th, n + 1)[:n]
p = np.zeros(n)
for sm, pm in zip(shift, PW):
if sm >= 0: p[sm:] += pm * qd[:n - sm]
else: p[:sm] += pm * qd[-sm:]
self.p_long = p
def shortfall(self, k, margin): # E[(S_k - margin)^+]
A, B = self.tails[min(k, len(self.tails) - 1)]; margin = np.atleast_1d(np.asarray(margin, float))
j = np.searchsorted(self.S, margin, side="left"); return A[j] - margin * B[j]
acc = Accumulated()
def best_plan(c0=58.0, horizon=1200, dp=0.01):
Pg = np.arange(40.0, 75.0 + dp / 2, dp); Wn = np.zeros(len(Pg)); Ts = np.zeros(horizon)
for k in range(horizon - 1, -1, -1):
Gk = -K * Pg + BETA * Wn; Ts[k] = Pg[np.argmax(Gk >= Gk.max() - 1e-12)]
pen = P * acc.shortfall(k, Pg - FLOOR) if k > 0 else P * np.maximum(FLOOR - Pg, 0)
Wn = -pen + K * ALPHA * Pg + np.interp(np.clip(Ts[k], ALPHA * Pg, ALPHA * Pg + CAP), Pg, Gk)
path, hires = [c0], []
for k in range(horizon):
nxt = float(np.clip(Ts[k], ALPHA * path[-1], ALPHA * path[-1] + CAP)); hires.append(nxt - ALPHA * path[-1]); path.append(nxt)
hire = sum(BETA**q * K * hires[q] for q in range(horizon))
short = sum(BETA**q * P * float(acc.shortfall(q, path[q] - FLOOR)[0]) for q in range(1, horizon))
return np.array(path), Ts, hire, short
path, Ts, phire, pshort = best_plan()
rent = (1 - ALPHA) * 58
rhire = sum(BETA**q * K * rent for q in range(1200)); rshort = sum(BETA**q * P * float(acc.shortfall(q, 3.0)[0]) for q in range(1, 1200))
rows = [("Best rule: top up to 55.75", "0.75") + describe(ybest),
("Re-plan each quarter (first target %.2f)" % Ts[0], "0.75") + describe(m.target_rule(Ts[0])),
("Chapter 6's 55, as a rule", "0") + describe(m.target_rule(55.0)),
("Cover the worst quarter: 56.5", "1.5") + describe(m.target_rule(56.5)),
("Volume I's 58, as a rule", "3") + describe(m.target_rule(58.0))]
best_total = rows[0][2]
print(f"{'rule or plan':42s} margin total hiring delay short% extra")
for name, mg, tot, hire, short, below in rows:
print(f"{name:42s} {mg:>6s} {-tot:7.1f} {hire:8.1f} {short:7.1f} {100*below:6.1f} {best_total - tot:7.1f}")
long_short = float(acc.p_long[acc.S > path[1000] - FLOOR + 1e-9].sum())
print(f"{'Best committed plan':42s} {path[1000]-FLOOR:6.2f} {phire+pshort:7.1f} {phire:8.1f} {pshort:7.1f} {100*long_short:6.1f} {phire+pshort+best_total:7.1f}")
print(f"{'Volume I budget: rent on 58 every quarter':42s} {'3':>6s} {rhire+rshort:7.1f} {rhire:8.1f} {rshort:7.1f} {100*float(acc.p_long[acc.S > 3 + 1e-9].sum()):6.1f} {rhire+rshort+best_total:7.1f}")
print("best committed plan, planned levels for quarters 0-6:", np.round(path[:7], 2), "... long run", round(path[1000], 2))
print(f"value of responding = {phire + pshort + best_total:.3f}")
fig, ax = plt.subplots(figsize=(7.6, 3.6))
tg = np.round(np.arange(54.5, 58.51, 0.05), 2)
ax.plot(tg, [-m.at(m.evaluate(m.target_rule(t)), 58) for t in tg], color="#1F4E79", lw=2)
ax.axhline(phire + pshort, color="#E65100", ls="--", lw=1, label=f"best committed plan {phire+pshort:.1f}")
ax.axhline(rhire + rshort, color="#E65100", ls=":", lw=1, label=f"Volume I's budget {rhire+rshort:.1f}")
ax.set(xlabel="target for planned capability", ylabel="PV of hiring and delay from 58 ($ million)", title="Every target rule against the plans; least at 55.75 (seed 26207)")
ax.legend(frameon=False, fontsize=8); plt.tight_layout(); plt.show()rule or plan margin total hiring delay short% extra Best rule: top up to 55.75 0.75 230.8 228.0 2.7 6.0 0.0 Re-plan each quarter (first target 55.75) 0.75 230.8 228.0 2.7 6.0 0.0 Chapter 6's 55, as a rule 0 237.4 222.1 15.2 27.5 6.6 Cover the worst quarter: 56.5 1.5 234.0 234.0 0.1 0.3 3.3 Volume I's 58, as a rule 3 246.2 246.2 0.0 0.0 15.4 Best committed plan 3.50 254.7 244.9 9.7 12.3 23.9 Volume I budget: rent on 58 every quarter 3 256.3 246.3 10.0 16.0 25.6 best committed plan, planned levels for quarters 0-6: [58. 56.55 55.75 56.46 56.46 56.48 56.98] ... long run 58.5 value of responding = 23.905

Panel 7 — Approximate (Theorem 7.3(iv))
AXIOM Lab 2.7, step 5. A coarser grid, or a smooth fitted value, makes each sweep cheaper but inexact; Theorem 7.3 bounds the cost from the approximate values' Bellman residual alone. The bound is a guarantee, not an estimate, and a shape that cannot bend at the floor fails badly.
Python Code
mc = Model(dx=0.75)
Vc, _, nc, _, _ = value_iteration(mc, 1e-9)
Vcf = np.interp(m.X, mc.X, Vc); TVc, yc = m.bellman(Vcf); rc = float(abs(TVc - Vcf).max())
Vg = m.evaluate(yc)
print(f"grid of 0.75 points ({mc.n} nodes): values within {abs(Vcf - Vbest).max():.3f}; residual {rc:.3f};"
f" greedy target {target_of(m, yc):.2f} loses {m.at(Vbest, 58) - m.at(Vg, 58):.3f} from 58 (bound {2*BETA*rc/(1-BETA):.1f})")
S = np.arange(30.0, 80.01, 1.0); feats = lambda x: np.column_stack([np.ones_like(x), x - 55, (x - 55) ** 2])
th = np.zeros(3)
for it in range(5000):
TV, _ = m.bellman(feats(m.X) @ th)
new = np.linalg.lstsq(feats(S), np.interp(S, m.X, TV), rcond=None)[0]
if abs(new - th).max() < 1e-10: th = new; break
th = new
Vq = feats(m.X) @ th; TVq, yq = m.bellman(Vq)
eps = float(abs(feats(m.X) @ np.linalg.lstsq(feats(S), np.interp(S, m.X, TVq), rcond=None)[0] - TVq).max())
print(f"quadratic fit: fitting error {eps:.2f} a sweep; values off by {abs(Vq - Vbest).max():.1f} (bound {eps/(1-BETA):.1f});"
f" greedy target {target_of(m, yq):.2f} loses {m.at(Vbest, 58) - m.at(m.evaluate(yq), 58):.1f} from 58")grid of 0.75 points (68 nodes): values within 2.533; residual 0.267; greedy target 56.25 loses 2.115 from 58 (bound 22.1) quadratic fit: fitting error 5.01 a sweep; values off by 112.5 (bound 212.8); greedy target 67.00 loses 81.7 from 58
Panel 8 — Shrink the period: Chapter 6's problem on finer calendars
AXIOM Lab 2.7, step 5, second part. Run the same recursion on Chapter 6's deterministic problem — the build waits until capability first reaches 55 and must then stay at or above it; delay costs $32 million a year; hiring at most 10.13 points a year at $4 million; decay ; — on quarterly, monthly and weekly calendars (hiring held level within each period). As the period shrinks, the recursion approaches Worked Example 6.1's continuous-time values, , , team $32.53 million (the limit of Equation 7.4).
Python Code
DELTA, RHO, UBAR, D = 4 * np.log(1 / 0.975), np.log(1.1), 100 * 4 * np.log(1 / 0.975), 32.0
def chapter6_on_calendar(dt, dx=0.005, lo=40.0, hi=70.0):
a, b, beta = np.exp(-DELTA * dt), (1 - np.exp(-DELTA * dt)) / DELTA, np.exp(-RHO * dt)
X = lo + dx * np.arange(int(round((hi - lo) / dx)) + 1); n = len(X)
kap = K * (1 - beta) / (RHO * b) # cost of moving the next state up by one point
flow = D * (1 - beta) / RHO # a period's delay
def step(V0, V1):
Wn = np.where(X >= FLOOR - 1e-12, V1, V0) # next period: started once capability is at the floor
Gs = -kap * X + beta * Wn
def best(L, U, Gv): # max over the reachable next states [L, U]
jlo = np.ceil((L - lo) / dx - 1e-9).astype(int); jhi = np.floor((U - lo) / dx + 1e-9).astype(int)
width = int(np.ceil(b * UBAR / dx)) + 2
J = jlo[:, None] + np.arange(width)[None, :]
vals = np.where(J <= jhi[:, None], Gv[np.clip(J, 0, n - 1)], -np.inf)
ends = np.column_stack([np.interp(L, X, Gv), np.interp(U, X, Gv)])
return np.maximum(vals.max(1), ends.max(1))
L, U = np.clip(a * X, lo, hi), np.clip(a * X + b * UBAR, lo, hi)
V0n = -flow + kap * a * X + best(L, U, Gs) # not started: pay the delay
G1 = -kap * X + beta * V1
L1 = np.clip(np.maximum(a * X, FLOOR), lo, hi) # started: stay at or above the floor
V1n = np.where(U >= L1 - 1e-12, kap * a * X + best(L1, np.maximum(U, L1), G1), -1e9)
return V0n, V1n
V0, V1 = np.zeros(n), np.zeros(n)
for it in range(200000):
N0, N1 = step(V0, V1)
d = max(abs(N0 - V0).max(), abs(N1[X >= FLOOR] - V1[X >= FLOOR]).max()); V0, V1 = N0, N1
if beta / (1 - beta) * d < 1e-5: break
Vgone = float(np.interp(53.0, X, V0)); Vkept = float(np.interp(58.0, X, V1))
c, n_wait = 53.0, 0 # periods of full hiring before capability reaches 55
while c < FLOOR - 1e-12:
c, n_wait = a * c + b * UBAR, n_wait + 1
return Vgone, Vkept, n_wait * dt
print("calendar V(53) V(58) team build starts after (years)")
for name, dt in (("quarterly", 0.25), ("monthly", 1 / 12), ("weekly", 1 / 52)):
g, k_, wait = chapter6_on_calendar(dt)
print(f"{name:10s} {g:9.3f} {k_:9.3f} {k_ - g:7.3f} {wait:6.3f}")
print(f"{'continuous':10s} {-254.892:9.3f} {-222.363:9.3f} {32.529:7.3f} {np.log(47 / 45) / DELTA:6.3f} (Worked Example 6.1)")calendar V(53) V(58) team build starts after (years) quarterly -256.969 -222.374 34.595 0.500 monthly -256.950 -222.366 34.584 0.500 weekly -255.276 -222.363 32.913 0.442 continuous -254.892 -222.363 32.529 0.429 (Worked Example 6.1)
The value of keeping the build running, , agrees with the continuous-time value to the third decimal on every calendar. The value from 53 converges more slowly, and not evenly, because on a calendar the build can start only at the beginning of a period: the wait for capability to reach 55 rounds up to whole periods. Two quarters and six months are both half a year, 23 weeks are 0.442 years, against 0.429 in continuous time. As the period shrinks, the rounding vanishes and the recursion's values approach Chapter 6's.
Validation — the book's numbers, and agreement with DCT_V2_Ch07_Lab.xlsx
Each checkpoint is a number printed in Chapter 7 (Worked Example 7.1, Table 7.3, Figures 7.1–7.3) or in
Online Appendix 7.B.1. The workbook recomputes the ones marked workbook with Excel formulas and shows
PASS against the same engine values on its Checks sheet.
Python Code
checks = [
("value iteration sweeps to 1e-6", n_vi, 813, 0),
("policy iteration evaluations", len(pis), 3, 0),
("target (planned capability)", T, 55.75, 1e-9),
("trigger: hire nothing above", T / ALPHA, 57.179, 5e-4),
("critical ratio (workbook)", ratio, 0.1228, 5e-5),
("holding cost of a planned point (workbook)", hold, 0.19183, 5e-6),
("V*(58) (workbook)", m.at(Vbest, 58), -230.755, 5e-4),
("V*(53) (workbook)", m.at(Vbest, 53), -254.593, 5e-4),
("team V(58) - V(53) (workbook)", team, 23.838, 5e-4),
("price of a point at 58 (workbook)", m.slope(Vbest, 58, -1), 3.682, 5e-4),
("price of a point at 56 (workbook)", m.slope(Vbest, 56, +1), 3.900, 5e-4),
("price of a point at 53 (workbook)", m.slope(Vbest, 53, +1), 7.192, 5e-4),
("Bellman residual over the continuum", res.max(), 0.004403, 5e-7),
("upper-bound gap at 58 (workbook)", upper_bound(58) - m.at(Vbest, 58), 0.270, 5e-4),
("simulation of the rule from 58 (seed 26207)", sim, -230.782, 5e-4),
("best rule, total cost from 58", -rows[0][2], 230.755, 5e-4),
("target 55: extra cost", best_total - rows[2][2], 6.615, 5e-4),
("target 56.5: extra cost", best_total - rows[3][2], 3.289, 5e-4),
("target 58: extra cost", best_total - rows[4][2], 15.396, 5e-4),
("best committed plan, total", phire + pshort, 254.660, 5e-4),
("Volume I's budget, total", rhire + rshort, 256.324, 5e-3),
("value of responding", phire + pshort + best_total, 23.905, 5e-4),
("re-planning's first target", Ts[0], 55.75, 1e-9),
("long-run share short, best rule (%)", 100 * rows[0][5], 6.0, 0.05),
("long-run share short, target 55 (%)", 100 * rows[2][5], 27.5, 0.05),
]
ok = 0
for name, got, want, tol in checks:
status = "PASS" if abs(got - want) <= tol else "FAIL"; ok += status == "PASS"
print(f"{name:45s} {got:12.4f} book {want:10.4f} {status}")
print(f"\n{ok} of {len(checks)} checkpoints PASS")value iteration sweeps to 1e-6 813.0000 book 813.0000 PASS policy iteration evaluations 3.0000 book 3.0000 PASS target (planned capability) 55.7500 book 55.7500 PASS trigger: hire nothing above 57.1795 book 57.1790 PASS critical ratio (workbook) 0.1228 book 0.1228 PASS holding cost of a planned point (workbook) 0.1918 book 0.1918 PASS V*(58) (workbook) -230.7550 book -230.7550 PASS V*(53) (workbook) -254.5929 book -254.5930 PASS team V(58) - V(53) (workbook) 23.8379 book 23.8380 PASS price of a point at 58 (workbook) 3.6824 book 3.6820 PASS price of a point at 56 (workbook) 3.9000 book 3.9000 PASS price of a point at 53 (workbook) 7.1918 book 7.1920 PASS Bellman residual over the continuum 0.0044 book 0.0044 PASS upper-bound gap at 58 (workbook) 0.2703 book 0.2700 PASS simulation of the rule from 58 (seed 26207) -230.7822 book -230.7820 PASS best rule, total cost from 58 230.7550 book 230.7550 PASS target 55: extra cost 6.6148 book 6.6150 PASS target 56.5: extra cost 3.2888 book 3.2890 PASS target 58: extra cost 15.3965 book 15.3960 PASS best committed plan, total 254.6598 book 254.6600 PASS Volume I's budget, total 256.3238 book 256.3240 PASS value of responding 23.9048 book 23.9050 PASS re-planning's first target 55.7500 book 55.7500 PASS long-run share short, best rule (%) 6.0235 book 6.0000 PASS long-run share short, target 55 (%) 27.5282 book 27.5000 PASS 25 of 25 checkpoints PASS
Next. Chapter 8 lets the periods shrink to zero: the Hamilton–Jacobi–Bellman equation (Equation 7.4) and its verification theorems. AXIOM-07 runs the same model interactively (Lab 2.7); this notebook and the workbook are its reproducible record.


