Skip to content

Vol. II, Ch. 13

DCT Laboratory — Volume II, Chapter 13

Machine Learning for Enterprise Transformation

This laboratory demonstrates

Seed 26213 · Companion to Chapter 13 and AXIOM Lab 2.13 (module AXIOM-13) · Mirrored in DCT_V2_Ch13_Lab.xlsx

This laboratory reproduces Worked Example 13.1, Is the Market Desk's Warning Worth Paying For?, with the book's declarations, and works the five steps of AXIOM Lab 2.13: read the record, train and validate, price the warning, check and certify, and compress and explain. The law, the levers and the floor are Chapter 9's; the new declarations are the market desk's five readings, its record of 1,000 windows, the two packages and the audit committee's confidence. The seeds are the book's (Online Appendix Remark 13.B.1): Chapter 9's 100,000 paths come from Volume I, Chapter 8's stream with seed 8 and its 100,000 fresh paths from seed 26209; the desk's record uses seed 1300, the book paths' readings seed 1309, the fresh paths' readings the lab's own seed, 26213, the 20,000 fresh programs of the boosting stage curve seed 26214, and the 200 records of the study seeds 913000–913199. The learners are scikit-learn's, as in the book's engine. The cells take about a minute and a half on two cores, most of it the study of 200 records. Every Meridian number below is the book's (Tables 13.2–13.3, Figures 13.1–13.3, Online Appendix Example 13.B.2); the final cell checks them. Money is in $ million.

The declarations (Worked Example 13.1, Step 1; Online Appendix Remark 13.B.1)

Chapter 9's law, levers and floor, unchanged. Execution and market deviations EE move Meridian's liquidity around the Chapter 1 plan, E′=0.6E+wE' = 0.6E + w with ww of spread $50 million a quarter; funding windows close at random, 0.10 a year, each removing a gamma-distributed amount (mean $120 million, standard deviation $50 million) that is never recovered. A standby line ll, on a $5 million grid from 0 to $400 million, covers up to ll of each closure, for fees of 3 percent of ll; a quarter of committed delay ϕ\phi makes the transformation's draw 4 index points shallower, at $8 million. The board's appetite: at most a 5 percent chance of a breach of the $350 million floor in some quarter of 12. A path survives a package when its critical depth, the deepest draw it can absorb without breaching the floor in any quarter, is at least 28−4ϕ28 - 4\phi. Volume I's package, a $170 million line and 0.668 quarter of delay, costs $10.44 million.

The perfect warning (Chapter 9). A path has a closure ahead when a funding window closes in its 12 quarters: under the law, with chance 1−e−0.3=25.921 - e^{-0.3} = 25.92 percent. Chapter 9 priced a perfect early warning of closures, known when the board signs its package: one package for the programs with a closure ahead and one for the rest, the 5 percent appetite counted over both (Proposition 9.3(ii)). It was worth $4.12 million.

The market desk's warning (new declarations). The treasury's market desk reads five funding-market indicators when the program starts, in standard units: Meridian's bond spread, the high-yield spread, equity volatility, banks' lending standards and the bank-funding spread. A common financial-conditions factor drives them with loadings λ=(0.9,0.85,0.8,0.6,0.3)\boldsymbol\lambda = (0.9, 0.85, 0.8, 0.6, 0.3); programs with a closure ahead start higher by s=(0.25,0.25,0.15,0.7,1.9)\mathbf s = (0.25, 0.25, 0.15, 0.7, 1.9); given closure or not, the readings say nothing more about the path, so Chapter 9's law is unchanged. Each program's readings are Fλ+e⊙1−λ2+YsF\boldsymbol\lambda + \mathbf e\odot\sqrt{1 - \boldsymbol\lambda^2} + Y\mathbf s, with the common factor FF and the noises e\mathbf e independent standard normal and Y=1Y = 1 for a closure ahead. The desk's record holds 1,000 three-year windows of comparable issuers, each with its readings and whether a funding window closed; it draws YY with chance 25.92 percent. The treasury and the transformation office will sign two packages in advance, one for "warning" and one for "all clear", with the appetite counted over both; the audit committee certifies at one-sided 95 percent confidence.

How a warning is designed and checked (Remark 13.B.1). A warning is a score and a threshold: programs scoring at or above it are warned. A group whose programs have a closure ahead with share π\pi is designed on the book's paths, weighted π/n1\pi/n_1 on the n1n_1 paths with a closure ahead and (1−π)/n0(1 - \pi)/n_0 on the others: at each line, the least delay keeps the weighted share of paths whose critical depth falls below 28−4ϕ28 - 4\phi within the group's chance of a breach, and the cheapest line and delay are kept. The all-clear group's chance is searched on a grid of 0.25 points up to 10 percent, the warned group's fills the 5 percent appetite, and the pair with the least expected cost is kept. The value of a warning is Volume I's $10.44 million less that expected cost. For each line the paths are sorted once by critical depth, and a group's package for any share and any chance is found by bisection, for every line and every chance at once (Section 13.4, Computational box). Each design is then applied to the fresh paths, each program with its own readings: its true chance of a breach and its expected cost are their fresh averages.

Python Code
import math
import itertools
import numpy as np
from scipy import stats
import matplotlib.pyplot as plt
import sklearn
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import StratifiedKFold
plt.rcParams.update({"figure.dpi": 110, "axes.spines.top": False, "axes.spines.right": False})
SEED = 26213           # the lab's seed: the book draws the fresh paths' readings with it (Remark 13.B.1)
BOOK_SEED = 8          # Volume I, Chapter 8's stream: the book's 100,000 paths (Online Appendix 1.B.4)
FRESH_SEED = 26209     # the book's 100,000 fresh paths (Chapter 9's lab seed)
RECORD_SEED = 1300     # the desk's record; also the folds' shuffle and the boosting's random state
BOOK_READ_SEED = 1309  # the readings of the book's paths
STUDY_SEED = 913000    # the study's 200 records: seeds 913000-913199

N, Q = 100_000, 12                          # paths; quarters of the program
# Chapter 9's law in USD billion, exactly as Volume I's engine draws it; every result is reported in $ million
FLOOR = 0.35                                # the board's floor
A_E, SD_W = 0.6, 0.05                       # E' = 0.6 E + w, w with spread $50m a quarter
LAM, J_MEAN, J_SD = 0.10, 0.12, 0.05        # closures a year; gamma sizes, mean $120m and sd $50m, never recovered
GAMMA = ((J_MEAN / J_SD) ** 2, J_SD ** 2 / J_MEAN)      # the sizes' shape and scale
DEPTH0, DPQ = 28.0, 4.0                     # the draw's depth (index points); a quarter of delay: 4 points shallower
FEE, VQ, ALPHA = 0.03, 8.0, 0.05            # line fees over the program; a quarter of delay ($m); the appetite
LINES = np.arange(0.0, 400.01, 5.0)         # the line on a $5m grid from 0 to $400m; the delay is continuous
P_GRID = np.arange(0.0, 0.1001, 0.0025)     # the all-clear group's chance of a breach: 0.25 points up to 10%

def relax(x0, x_inf, k, t):
    return x_inf + (x0 - x_inf) * np.exp(-k * t)

def trough(depth, tau, t):
    return depth * (t / tau) * np.exp(1.0 - t / tau)

TQ = np.arange(1, Q + 1) / 4.0                          # the twelve quarters, in years
BASE = 0.62 * relax(100.0, 112.0, 0.35, TQ) / 100.0     # planned liquidity without the draw (index 100 = $0.62bn)
SHAPE = 0.62 * trough(1.0, 2.0, TQ) / 100.0            # the draw per index point of depth, deepest at two years

def draws(seed, n=N):
    # Volume I's stream in its order: each quarter the execution shocks, the closure counts, then the sizes path by path
    rng = np.random.default_rng(seed)
    W, events = np.empty((n, Q)), []
    for k in range(Q):
        W[:, k] = rng.normal(0.0, SD_W, n)
        count = rng.poisson(LAM / 4.0, n)
        events.append((np.repeat(np.arange(n), count), rng.gamma(*GAMMA, int(count.sum()))))
    E, e = np.zeros((n, Q)), np.zeros(n)
    for k in range(Q):
        e = A_E * e + W[:, k]
        E[:, k] = e
    return E, events

class Paths:
    # Chapter 9's paths. A line l ($m) covers up to l of each closure; phi quarters of delay make the draw 28 - 4 phi
    # deep; a path stays above the floor in every quarter exactly when the draw is no deeper than its critical depth.
    def __init__(self, seed, n=N):
        self.E, self.events = draws(seed, n)
        self.seed, self.n = seed, n
        self.ahead = np.zeros(n, bool)          # a funding window closes in the program's 12 quarters
        for path, _ in self.events:
            self.ahead[path] = True
    def lost(self, line):                   # cumulative closures the line does not cover (USD bn), quarter by quarter
        J, j = np.zeros((self.n, Q)), np.zeros(self.n)
        for k, (path, size) in enumerate(self.events):
            j = j + np.bincount(path, np.maximum(size - line / 1000.0, 0.0), self.n)
            J[:, k] = j
        return J
    def crit(self, line):                   # each path's critical depth under the line
        return np.min((BASE[None, :] + self.E - self.lost(line) - FLOOR) / SHAPE[None, :], axis=1)

def cost(line, phi):
    return FEE * line + VQ * phi

class Split:
    # The book's paths split by closure ahead: for every line, the critical depths in order and the running count of
    # paths with a closure ahead. A group whose programs close with share pi is designed on the paths weighted pi / n1
    # on the n1 closure paths and (1 - pi) / n0 on the others (Remark 13.B.1).
    def __init__(self, P, lines=LINES):
        self.lines, self.y, self.n = lines, P.ahead, P.n
        self.n1 = int(self.y.sum()); self.n0 = self.n - self.n1
        L, n = len(lines), self.n
        self.C = np.empty((L, n))
        A1 = np.empty((L, n + 1), dtype=np.int32)   # running count of closure paths; a spare last column
        for i, l in enumerate(lines):
            c = P.crit(l)
            o = np.argsort(c, kind="stable")
            self.C[i], A1[i, :n] = c[o], np.cumsum(self.y[o])
        A1[:, n] = self.n1
        self.A1, self.off = A1.ravel(), (np.arange(L, dtype=np.int32) * (n + 1))[:, None]
        self.steps = [1 << s for s in range(16, -1, -1)]       # 65,536 + ... + 1 >= 100,000 paths
    def curve(self, pis, targets):
        # The cheapest package (cost, line, delay) for groups with closure shares pis and chances of a breach targets,
        # one column each. At every line the paths are in order of critical depth, and the draw may be as deep as the
        # critical depth of the first path whose weighted running share, pi a1 / n1 + (1 - pi) a0 / n0 (a1 and a0
        # paths up to it with and without a closure ahead), exceeds the chance. A binary search finds that path for
        # every line and column at once (k: paths known to stay within the chance). inf where no package keeps it.
        L, n = len(self.lines), self.n
        pis = np.asarray(pis, dtype=float)[None, :]
        thr = np.asarray(targets, dtype=float)[None, :] + 1e-12
        k = np.zeros((L, thr.shape[1]), dtype=np.int32)
        for step in self.steps:
            j = k + (step - 1)
            jc = np.minimum(j, n)
            a1 = np.take(self.A1, jc + self.off)
            cw = pis * a1 / self.n1 + (1.0 - pis) * (jc + 1 - a1) / self.n0
            k += ((j < n) & (cw <= thr)) * step
        d = np.where(k < n, np.minimum(DEPTH0, self.C[np.arange(L)[:, None], np.minimum(k, n - 1)]), DEPTH0)
        phi = (DEPTH0 - d) / DPQ
        c = np.where(d >= 0, FEE * self.lines[:, None] + VQ * phi, np.inf)
        i = np.argmin(c, axis=0)
        cols = np.arange(thr.shape[1])
        return c[i, cols], self.lines[i], phi[i, cols]
    def group(self, pi, pg):
        c, l, ph = self.curve([pi], [pg])
        return None if not np.isfinite(c[0]) else (float(c[0]), float(l[0]), float(ph[0]))
    def two(self, share_w, pi_w, pi_c, p=ALPHA, p_grid=P_GRID):
        # Two packages chosen together: the all-clear group's chance pc on the grid, the warned group's from the appetite
        if share_w <= 0.0:
            g = self.group(pi_c, p)
            return None if g is None else dict(cost=g[0], warned=0.0, clear=g, warn=None, pc=p, pw=None)
        if share_w >= 1.0:
            g = self.group(pi_w, p)
            return None if g is None else dict(cost=g[0], warned=1.0, clear=None, warn=g, pc=None, pw=p)
        pcs = np.asarray(p_grid, dtype=float)
        pws = (p - (1.0 - share_w) * pcs) / share_w
        ok = pws >= 0.0
        if not ok.any():
            return None
        pcs, pws = pcs[ok], np.minimum(pws[ok], 1.0)
        m = len(pcs)
        c_, l_, f_ = self.curve(np.r_[np.full(m, pi_c), np.full(m, pi_w)], np.r_[pcs, pws])   # both groups at once
        cc, lc, fc, cw, lw, fw = c_[:m], l_[:m], f_[:m], c_[m:], l_[m:], f_[m:]
        tot = share_w * cw + (1.0 - share_w) * cc
        if not np.isfinite(tot).any():
            return None
        j = int(np.argmin(tot))
        return dict(cost=float(tot[j]), warned=share_w, clear=(float(cc[j]), float(lc[j]), float(fc[j])),
                    warn=(float(cw[j]), float(lw[j]), float(fw[j])), pc=float(pcs[j]), pw=float(pws[j]))
    def breaches(self, g):
        # the book's paths that breach the package g = (cost, line, delay): (with a closure ahead, without)
        i = int(round(g[1] / 5.0))
        k = int(np.searchsorted(self.C[i], DEPTH0 - DPQ * g[2], side="left"))
        a1 = int(self.A1[i * (self.n + 1) + k - 1]) if k else 0
        return a1, k - a1
    def chance(self, pi, g):
        # the group's chance of a breach as the design weighs the paths: pi on closure paths, 1 - pi on the others
        b1, b0 = self.breaches(g)
        return pi * b1 / self.n1 + (1.0 - pi) * b0 / self.n0

# the desk's readings
NAMES = ["Meridian's bond spread", "high-yield spread", "equity volatility", "banks' lending standards",
         "bank-funding spread"]
SHORT = ["bond", "high yield", "volatility", "lending", "funding"]
LOAD = np.array([0.9, 0.85, 0.8, 0.6, 0.3])            # loadings on the common financial-conditions factor
SHIFT = np.array([0.25, 0.25, 0.15, 0.7, 1.9])         # how much higher programs with a closure ahead start
SIGMA = np.outer(LOAD, LOAD) + np.diag(1.0 - LOAD ** 2)  # the readings' covariance, given closure or not
P_CLOSE = 1.0 - math.exp(-0.10 * 3.0)                  # a closure in three years under Chapter 9's law

def readings(y, rng, shift=SHIFT):
    # five readings per program: the common factor, each reading's own noise, and the closure shift
    n = len(y)
    f = rng.normal(size=n)
    e = rng.normal(size=(n, 5))
    return f[:, None] * LOAD[None, :] + e * np.sqrt(1.0 - LOAD ** 2)[None, :] + y[:, None] * shift[None, :]

def record(seed=RECORD_SEED, n=1000):
    # a record of n windows: whether a funding window closed (chance 25.92%), then the readings, from one stream
    rng = np.random.default_rng(seed)
    y = rng.random(n) < P_CLOSE
    return y, readings(y.astype(float), rng)

BOOK = Paths(BOOK_SEED)
SPLIT = Split(BOOK)
PI1 = SPLIT.n1 / SPLIT.n                    # the book paths' share with a closure ahead
FRESH = Paths(FRESH_SEED)
FRESH_CRIT = np.array([FRESH.crit(l) for l in LINES])  # every fresh path's critical depth under every line
FRESH_X = readings(FRESH.ahead.astype(float), np.random.default_rng(SEED))   # the fresh programs' own readings

def check(warned, design):
    # a design on the fresh paths: each program gets its group's package; the true chance of a breach and expected cost
    breach, cst = np.zeros(N, bool), 0.0
    for mask, g in ((warned, design["warn"]), (~warned, design["clear"])):
        if mask.sum() == 0 or g is None:
            continue
        i = int(round(g[1] / 5.0))
        breach[mask] = FRESH_CRIT[i, mask] < DEPTH0 - DPQ * g[2]
        cst += mask.mean() * g[0]
    return float(breach.mean()), float(cst)

def fresh_breaches(warned, design):
    # breaches on the fresh paths, group by group: (warned, all clear)
    out = []
    for mask, g in ((warned, design["warn"]), (~warned, design["clear"])):
        out.append(0 if (g is None or mask.sum() == 0) else
                   int((FRESH_CRIT[int(round(g[1] / 5.0)), mask] < DEPTH0 - DPQ * g[2]).sum()))
    return tuple(out)

NONE = SPLIT.two(0.0, 0.0, PI1)             # no warning: one package for every program, Volume I's
C5 = NONE["cost"]
NONE_FRESH = check(np.zeros(N, bool), NONE)
PERFECT = SPLIT.two(PI1, 1.0, 0.0)          # Chapter 9's perfect warning: warned exactly when a closure is ahead
PERFECT_FRESH = check(FRESH.ahead.copy(), PERFECT)
print(f"scikit-learn {sklearn.__version__} (the book's engine used scikit-learn 1.9)")
print(f"closure ahead: {100 * PI1:.2f}% of the book's paths, {100 * FRESH.ahead.mean():.2f}% of the fresh ones; "
      f"the law: 1 - e^-0.3 = {100 * P_CLOSE:.2f}%")
_, l5, ph5 = NONE["clear"]
print(f"no warning, Volume I's package: line {l5:.0f}, delay {ph5:.4f}, cost {C5:.2f}; chance of a breach "
      f"{100 * SPLIT.chance(PI1, NONE['clear']):.2f}% on the book's paths, {100 * NONE_FRESH[0]:.2f}% on the fresh paths")
print("the perfect warning (Chapter 9):")
for name, g, pi in (("all clear, no closure ahead", PERFECT["clear"], 0.0), ("warned, a closure ahead", PERFECT["warn"], 1.0)):
    print(f"  {name:28s} line {g[1]:3.0f}, delay {g[2]:.3f}, cost {g[0]:5.2f}, chance of a breach "
          f"{100 * SPLIT.chance(pi, g):.2f}%")
print(f"  expected cost {PERFECT['cost']:.2f}: worth {C5:.2f} - {PERFECT['cost']:.2f} = {C5 - PERFECT['cost']:.2f}; on the "
      f"fresh paths a chance of a breach of {100 * PERFECT_FRESH[0]:.2f}% and a cost of {PERFECT_FRESH[1]:.2f}")
scikit-learn 1.9.1 (the book's engine used scikit-learn 1.9)
closure ahead: 25.91% of the book's paths, 25.96% of the fresh ones; the law: 1 - e^-0.3 = 25.92%
no warning, Volume I's package: line 170, delay 0.6676, cost 10.44; chance of a breach 5.00% on the book's paths, 4.91% on the fresh paths
the perfect warning (Chapter 9):
  all clear, no closure ahead  line   0, delay 0.608, cost  4.87, chance of a breach 4.75%
  warned, a closure ahead      line 215, delay 0.504, cost 10.49, chance of a breach 5.71%
  expected cost 6.32: worth 10.44 - 6.32 = 4.12; on the fresh paths a chance of a breach of 4.91% and a cost of 6.33

Panel 1 — Read the record (Theorem 13.2(iii); Worked Example 13.1, Step 1; Figure 13.1(b); Online Appendix Example 13.B.2: 268 closures in 1,000 windows; the law's coefficients −0.323, −0.159, −0.352, 0.769, 1.974, separation 2.029, AUC 0.924; the ceiling, $1.44m, warning 58.86% of programs; $0.63m at an AUC of 0.86, $2.24m at 0.96)

AXIOM Lab 2.13, step 1. Compare the readings of windows with and without a closure, and compute the ceiling: the most any warning on these readings can be worth (Theorem 13.2(iii)). Given closure or not, the readings have the same covariance, Σ=λλ⊤+diag(1−λ2)\Sigma = \boldsymbol\lambda\boldsymbol\lambda^\top + \mathrm{diag}(1 - \boldsymbol\lambda^2), so the log-odds of a closure ahead is linear in them, with coefficients β=Σ−1s\boldsymbol\beta = \Sigma^{-1}\mathbf s. The log-likelihood ratio z=β⊤x−Δ2/2z = \boldsymbol\beta^\top x - \Delta^2/2 is normal with mean Δ2/2\Delta^2/2 for a program with a closure ahead and −Δ2/2-\Delta^2/2 for the others, and variance Δ2\Delta^2, where Δ=(s⊤Σ−1s)1/2\Delta = (\mathbf s^\top\Sigma^{-1}\mathbf s)^{1/2} is the readings' separation; the readings rank a program with a closure ahead above one without with chance AUC=Φ(Δ/2)\mathrm{AUC} = \Phi(\Delta/\sqrt2). The best warning reads zz: a threshold splits programs into warned and all clear, the law gives each group's size and closure share exactly, and each of 241 thresholds from −6Δ-6\Delta to 6Δ6\Delta gets its two packages; the cheapest pair prices the ceiling, which is then checked on the fresh paths. Scaling all five shifts traces the most a warning is worth against its readings' AUC (Figure 13.1(b)).

Python Code
YR, XR = record()
print(f"the desk's record: {len(YR):,} windows, {YR.sum()} with a closure ({100 * YR.mean():.1f}%)")
print("mean readings (standard units)  " + "".join(f"{s:>12s}" for s in SHORT))
print("  windows with a closure        " + "".join(f"{x:12.2f}" for x in XR[YR].mean(0)))
print("  windows without               " + "".join(f"{x:12.2f}" for x in XR[~YR].mean(0)))

def posterior_coef(shift=SHIFT, idx=None):
    # the declared law's log-odds coefficients beta = Sigma^-1 s and separation Delta, for all readings or a subset
    idx = list(range(5)) if idx is None else list(idx)
    b = np.linalg.solve(SIGMA[np.ix_(idx, idx)], shift[idx])
    return b, float(math.sqrt(shift[idx] @ b))

def auc(Delta):
    return float(stats.norm.cdf(Delta / math.sqrt(2.0)))

def posterior_split(Delta, t, pi1):
    # a threshold t on the log-likelihood ratio z ~ N(+/- Delta^2/2, Delta^2): the share warned, each group's closure share
    if Delta <= 0:
        return (0.0, pi1, pi1)
    a1 = 1.0 - stats.norm.cdf((t - Delta ** 2 / 2.0) / Delta)     # P(warned | closure ahead)
    a0 = 1.0 - stats.norm.cdf((t + Delta ** 2 / 2.0) / Delta)     # P(warned | none)
    sw = pi1 * a1 + (1 - pi1) * a0
    pw = pi1 * a1 / sw if sw > 0 else 0.0
    pc = pi1 * (1 - a1) / (1 - sw) if sw < 1 else 0.0
    return sw, pw, pc

def ceiling(Delta, grid=np.linspace(-6.0, 6.0, 241)):
    # the best warning the readings allow, read by the declared law (Theorem 13.2(iii))
    best = None
    for t in grid * max(Delta, 0.3):
        sw, pw, pc = posterior_split(Delta, t, PI1)
        d = SPLIT.two(sw, pw, pc)
        if d is not None and (best is None or d["cost"] < best["design"]["cost"]):
            best = dict(t=float(t), shares=(sw, pw, pc), design=d)
    best["value"], best["auc"], best["Delta"] = C5 - best["design"]["cost"], auc(Delta), Delta
    return best

BETA_LAW, DELTA = posterior_coef()
print("\nthe declared law: beta = Sigma^-1 s = (" + ", ".join(f"{b:.3f}" for b in BETA_LAW) + f"); separation "
      f"Delta = {DELTA:.3f}; AUC = Phi(Delta / sqrt 2) = {auc(DELTA):.3f}")
CEIL = ceiling(DELTA)
Z_FRESH = FRESH_X @ BETA_LAW - DELTA ** 2 / 2.0         # each fresh program's log-likelihood ratio of a closure ahead
CEIL_W = Z_FRESH >= CEIL["t"]
CEIL_FRESH = check(CEIL_W, CEIL["design"])
sw, pw, pc = CEIL["shares"]
cd = CEIL["design"]
print(f"the ceiling: warn when z >= {CEIL['t']:.3f}: {100 * sw:.2f}% of programs warned; closures ahead in {100 * pw:.2f}% "
      f"of the warned and {100 * pc:.2f}% of the all clear")
print(f"  all clear: line {cd['clear'][1]:.0f}, delay {cd['clear'][2]:.3f}, {cd['clear'][0]:.2f}; warned: line "
      f"{cd['warn'][1]:.0f}, delay {cd['warn'][2]:.3f}, {cd['warn'][0]:.2f}; expected {cd['cost']:.2f}; worth "
      f"{C5:.2f} - {cd['cost']:.2f} = {CEIL['value']:.2f}, against {C5 - PERFECT['cost']:.2f} for a perfect warning")
print(f"  on the fresh paths: {100 * CEIL_W.mean():.2f}% warned, chance of a breach {100 * CEIL_FRESH[0]:.2f}%, worth "
      f"{C5 - CEIL_FRESH[1]:.2f}")

SCALES = (0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 2.0, 2.5, 3.0, 4.0)
CURVE = [ceiling(DELTA * c) for c in SCALES]
print("\nthe most a warning is worth against its readings' separation (all five shifts scaled):")
print("  shifts x    " + "".join(f"{c:7g}" for c in SCALES))
print("  AUC         " + "".join(f"{x['auc']:7.3f}" for x in CURVE))
print("  ceiling     " + "".join(f"{x['value']:7.2f}" for x in CURVE))
the desk's record: 1,000 windows, 268 with a closure (26.8%)
mean readings (standard units)          bond  high yield  volatility     lending     funding
  windows with a closure                0.18        0.29        0.17        0.73        2.00
  windows without                      -0.02        0.01       -0.02       -0.06       -0.02

the declared law: beta = Sigma^-1 s = (-0.323, -0.159, -0.352, 0.769, 1.974); separation Delta = 2.029; AUC = Phi(Delta / sqrt 2) = 0.924
the ceiling: warn when z >= -1.826: 58.86% of programs warned; closures ahead in 42.80% of the warned and 1.75% of the all clear
  all clear: line 0, delay 0.671, 5.37; warned: line 190, delay 0.731, 11.55; expected 9.00; worth 10.44 - 9.00 = 1.44, against 4.12 for a perfect warning
  on the fresh paths: 58.94% warned, chance of a breach 4.90%, worth 1.43

the most a warning is worth against its readings' separation (all five shifts scaled):
  shifts x       0.25    0.5   0.75      1   1.25    1.5      2    2.5      3      4
  AUC           0.640  0.763  0.859  0.924  0.964  0.984  0.998  1.000  1.000  1.000
  ceiling        0.05   0.22   0.63   1.44   2.24   2.88   3.69   4.00   4.09   4.12

Panel 2 — Train and validate (Theorem 13.1; Proposition 13.2; Worked Example 13.1, Steps 2–3; Figure 13.2(a): the logistic model's coefficients −0.428, −0.264, −0.183, 0.968, 2.180 and AUC 0.942 on the record, 0.940 held out, 0.924 fresh; boosting's AUC 1.000 on the record, 0.922 held out, 0.888 fresh, its fresh AUC peaking at 0.910 after 24 stages)

AXIOM Lab 2.13, step 2. Fit a logistic model to the record by unpenalized maximum likelihood, and gradient boosting with 500 stages of depth-4 trees and learning rate 0.1, both with scikit-learn as in the book's engine. Score each on the record it learned, on held-out folds (five folds, stratified and shuffled with the record's seed: each window is scored by the model fitted on the other four folds), and on the 100,000 fresh programs, each with its own readings. The logistic class has six parameters; the boosted model can fit every window of the record, so by Theorem 13.1(iii) its record score says nothing about its fresh error, and by Proposition 13.2(iv) boosting lowers the record's loss at every stage whatever the fresh error does. The stage curve scores the boosted model after each stage on the record and on 20,000 of the fresh programs, drawn with the book's seed 26214 (Remark 13.B.1). Last, the Online Appendix's majority vote that is worse than every member (Proposition 13.2(iii)).

Python Code
def fit_logistic(X, y):
    return LogisticRegression(C=1e10, max_iter=5000).fit(X, y)        # unpenalized maximum likelihood, as the engine

def fit_boost(X, y, stages=500):
    return GradientBoostingClassifier(n_estimators=stages, max_depth=4, learning_rate=0.1,
                                      random_state=RECORD_SEED).fit(X, y)

def oof_scores(make, X, y, seed=RECORD_SEED, folds=5):
    # held-out scores: each window scored by the model fitted on the other folds (stratified, shuffled with the seed)
    s = np.empty(len(y))
    for tr, te in StratifiedKFold(folds, shuffle=True, random_state=seed).split(X, y):
        s[te] = make(X[tr], y[tr]).predict_proba(X[te])[:, 1]
    return s

LR = fit_logistic(XR, YR)
F_LR = lambda X: LR.predict_proba(X)[:, 1]
S_IN, S_OOF, S_FRESH = F_LR(XR), oof_scores(fit_logistic, XR, YR), F_LR(FRESH_X)
AUC_LR = dict(record=roc_auc_score(YR, S_IN), oof=roc_auc_score(YR, S_OOF), fresh=roc_auc_score(FRESH.ahead, S_FRESH))
ACC_LR = dict(record=float(np.mean((S_IN >= 0.5) == YR)), fresh=float(np.mean((S_FRESH >= 0.5) == FRESH.ahead)))
NEVER = float(1.0 - FRESH.ahead.mean())                  # never warning is right whenever no closure is ahead
print("the logistic model: log-odds = " + f"{LR.intercept_[0]:.3f} " + " ".join(
    f"{b:+.3f} x {s}" for b, s in zip(LR.coef_[0], SHORT)))
print(f"  AUC: {AUC_LR['record']:.3f} on the record, {AUC_LR['oof']:.3f} held out, {AUC_LR['fresh']:.3f} fresh; "
      f"right at 0.5: {100 * ACC_LR['record']:.1f}% of the record, {100 * ACC_LR['fresh']:.1f}% of fresh programs; "
      f"never warning: {100 * NEVER:.1f}%")

GB = fit_boost(XR, YR)
F_GB = lambda X: GB.predict_proba(X)[:, 1]
G_IN, G_OOF, G_FRESH = F_GB(XR), oof_scores(fit_boost, XR, YR), F_GB(FRESH_X)
AUC_GB = dict(record=roc_auc_score(YR, G_IN), oof=roc_auc_score(YR, G_OOF), fresh=roc_auc_score(FRESH.ahead, G_FRESH))
ACC_GB = dict(record=float(np.mean((G_IN >= 0.5) == YR)), fresh=float(np.mean((G_FRESH >= 0.5) == FRESH.ahead)))
print(f"boosting, 500 stages of depth-4 trees: AUC {AUC_GB['record']:.3f} on the record, {AUC_GB['oof']:.3f} held out, "
      f"{AUC_GB['fresh']:.3f} fresh; right at 0.5: {100 * ACC_GB['record']:.1f}% of the record, "
      f"{100 * ACC_GB['fresh']:.1f}% of fresh programs")

STAGE_SEED = 26214            # the book's seed for the stage curve's 20,000 fresh programs (Remark 13.B.1)
SUB = np.random.default_rng(STAGE_SEED).choice(N, 20000, replace=False)
ST_REC = np.array([roc_auc_score(YR, p[:, 1]) for p in GB.staged_predict_proba(XR)])
ST_FR = np.array([roc_auc_score(FRESH.ahead[SUB], p[:, 1]) for p in GB.staged_predict_proba(FRESH_X[SUB])])
PEAK = int(np.argmax(ST_FR))
print(f"stage by stage: the fresh AUC peaks at {ST_FR[PEAK]:.3f} after {PEAK + 1} stages and ends at {ST_FR[-1]:.3f}; "
      f"the record's is {ST_REC[49]:.3f} after 50 stages and {ST_REC[-1]:.3f} after 500")

# Proposition 13.2(iii): three classifiers each right on 60% of cases, whose majority vote is right on 40%
VOTE = np.zeros((15, 3), bool)
VOTE[:6] = True                                           # 40% of cases: all three right
for j in range(3):
    VOTE[6 + 3 * j: 9 + 3 * j, j] = True                  # 60%: exactly one right, each classifier a third of them
MEMBERS = VOTE.mean(axis=0)
MAJORITY = float((VOTE.sum(axis=1) >= 2).mean())
print("\nProposition 13.2(iii): members right on " + ", ".join(f"{100 * m:.0f}%" for m in MEMBERS)
      + f" of cases; their majority vote on {100 * MAJORITY:.0f}%")

fig, ax = plt.subplots(figsize=(7.6, 3.6))
k_ = np.arange(1, len(ST_REC) + 1)
ax.plot(k_, ST_REC, color="#E65100", lw=1.8, label="on the record")
ax.plot(k_, ST_FR, color="#1F4E79", lw=1.8, label="on 20,000 fresh programs")
ax.axhline(AUC_LR["fresh"], color="#999999", lw=0.9, ls=":")
ax.text(1.1, AUC_LR["fresh"] + 0.004, f"logistic model, fresh {AUC_LR['fresh']:.3f}", fontsize=8, color="#616161")
ax.plot([PEAK + 1], [ST_FR[PEAK]], "o", color="#1F4E79", ms=5)
ax.text(PEAK + 1, ST_FR[PEAK] - 0.008, f"peak {ST_FR[PEAK]:.3f} after {PEAK + 1} stages", fontsize=8, color="#1F4E79",
        ha="center", va="top")
ax.text(500, ST_FR[-1] - 0.008, f"{ST_FR[-1]:.3f}", fontsize=8, color="#1F4E79", ha="right", va="top")
ax.set_xscale("log")
ax.set_xticks([1, 10, 100, 500], ["1", "10", "100", "500"])
ax.set(xlim=(1, 500), ylim=(0.84, 1.01), xlabel="boosting stages (log scale)", ylabel="AUC (chance a closure ranks higher)",
       title="Boosting's record AUC reaches 1.00 as its fresh AUC falls (seed 26213)")
ax.legend(frameon=False, fontsize=8, loc="center right"); plt.tight_layout(); plt.show()
the logistic model: log-odds = -3.435 -0.428 x bond -0.264 x high yield -0.183 x volatility +0.968 x lending +2.180 x funding
  AUC: 0.942 on the record, 0.940 held out, 0.924 fresh; right at 0.5: 89.3% of the record, 87.3% of fresh programs; never warning: 74.0%
boosting, 500 stages of depth-4 trees: AUC 1.000 on the record, 0.922 held out, 0.888 fresh; right at 0.5: 100.0% of the record, 85.6% of fresh programs
stage by stage: the fresh AUC peaks at 0.910 after 24 stages and ends at 0.888; the record's is 0.992 after 50 stages and 1.000 after 500

Proposition 13.2(iii): members right on 60%, 60%, 60% of cases; their majority vote on 40%
Boosting's record AUC reaches 1.00 as its fresh AUC falls (seed 26213)

Panel 3 — Price the warning (Theorem 13.2(ii); Worked Example 13.1, Step 2; Figure 13.1: at accuracy's threshold, 0.5, the logistic warning warns 23.0% of programs, is right 87.3% of the time and is worth $0.53m; where the packages are cheapest, at a 4.09% chance, it warns 59.0%, is right 65.9% of the time and is worth $1.44m; never warning is right 74.0% of the time)

AXIOM Lab 2.13, step 3. Sweep the logistic warning's threshold and price each threshold by its two packages. Here each group's closure share is the law's: the book's 100,000 paths get their own readings (seed 1309), the fitted model scores them, and each group's share of paths with a closure ahead is read off the paths (Figure 13.1(a)). Accuracy counts a false alarm and a miss as equal; the packages do not. By Theorem 13.2(ii), the best rule warns when the chance of a closure exceeds a threshold set by the costs, with the appetite priced in; the cell finds it among 400 quantile steps of the scores. Accuracy and value are read on the fresh paths, each program with its own readings. Last, Theorem 13.2(iv)'s example: a prediction with the smaller squared error that leads to the costlier decision.

Python Code
def shares(score, y, tau):
    # warn at or above tau: the share warned and each group's closure share
    w = score >= tau
    return float(w.mean()), (float(y[w].mean()) if w.any() else 0.0), (float(y[~w].mean()) if (~w).any() else 0.0), w

_CP = {}
def cp_upper(k, m, conf=0.95):
    # Theorem 13.3(i): the one-sided Clopper-Pearson upper bound for a chance, k events in m cases
    if (k, m, conf) not in _CP:
        _CP[k, m, conf] = 1.0 if k >= m else float(stats.beta.ppf(conf, k + 1, m - k))
    return _CP[k, m, conf]

def thresholds(score, n_steps=100):
    return np.unique(np.concatenate([np.quantile(score, np.linspace(0.0, 1.0, n_steps + 1)), [np.inf]]))

def best_split(score, y, certify=False, n_steps=100):
    # the threshold and packages with the least expected cost, each group's closure share estimated from (score, y);
    # certify: each share replaced by its one-sided 95% upper bound
    best = None
    for tau in thresholds(score, n_steps):
        sw, pw, pc, w = shares(score, y, tau)
        if certify:
            m_w, m_c = int(w.sum()), int((~w).sum())
            pw = cp_upper(int(y[w].sum()), m_w) if m_w else 0.0
            pc = cp_upper(int(y[~w].sum()), m_c) if m_c else 0.0
        d = SPLIT.two(sw, pw, pc)
        if d is not None and (best is None or d["cost"] < best["design"]["cost"] - 1e-12):
            best = dict(tau=float(tau), shares=(sw, pw, pc), design=d)
    return best

def evaluate(score_fn, tau, design):
    # the design on the fresh paths, each program warned by its own readings
    warned = score_fn(FRESH_X) >= tau
    breach, cst = check(warned, design)
    return dict(breach=breach, cost=cst, value=C5 - cst, fresh_warned=float(warned.mean()),
                accuracy=float(np.mean(warned == FRESH.ahead)), believed=C5 - design["cost"], warned_mask=warned)

def show(name, x, value=True):
    sw, pw, pc = x["shares"]
    d = x["design"]
    print(f"{name}: warn at a chance of {100 * x['tau']:.2f}%: {100 * sw:.2f}% warned; closures ahead in {100 * pw:.2f}% of "
          f"the warned and {100 * pc:.2f}% of the all clear")
    print(f"  all clear: line {d['clear'][1]:.0f}, delay {d['clear'][2]:.3f}, {d['clear'][0]:.2f}; warned: line "
          f"{d['warn'][1]:.0f}, delay {d['warn'][2]:.3f}, {d['warn'][0]:.2f}; promises {x['believed']:.2f}")
    print(f"  fresh: {100 * x['fresh_warned']:.1f}% warned, right {100 * x['accuracy']:.1f}% of the time, chance of a "
          f"breach {100 * x['breach']:.2f}%, " + (f"worth {x['value']:.2f}" if value else "the appetite broken: no value"))

X_BOOK = readings(BOOK.ahead.astype(float), np.random.default_rng(BOOK_READ_SEED))
S_BOOK = F_LR(X_BOOK)
SWEEP = []
for share in np.linspace(0.0, 1.0, 51):
    tau = np.inf if share <= 0 else float(np.quantile(S_BOOK, 1.0 - share))
    sw, pw, pc, _ = shares(S_BOOK, BOOK.ahead, tau)
    d = SPLIT.two(sw, pw, pc)
    ev = evaluate(F_LR, tau, d)
    SWEEP.append(dict(share=sw, accuracy=ev["accuracy"], value=ev["value"], breach=ev["breach"], design_value=C5 - d["cost"]))
LAW05 = dict(tau=0.5)
LAW05["shares"] = shares(S_BOOK, BOOK.ahead, 0.5)[:3]
LAW05["design"] = SPLIT.two(*LAW05["shares"])
LAW05.update(evaluate(F_LR, 0.5, LAW05["design"]))
LAWBEST = best_split(S_BOOK, BOOK.ahead, n_steps=400)
LAWBEST.update(evaluate(F_LR, LAWBEST["tau"], LAWBEST["design"]))
print("each group's closure share as the law gives it:")
show("at accuracy's threshold", LAW05)
show("where the packages are cheapest", LAWBEST)
print(f"never warning is right {100 * NEVER:.1f}% of the time: the valuable warning is less accurate than no warning")
print("the sweep, by share of programs warned (%): " + ", ".join(
    f"{100 * x['share']:.0f}: {100 * x['accuracy']:.1f}% right, {x['design_value']:.2f}" for x in SWEEP[5:40:5]))

# Theorem 13.2(iv): order u against an outcome Y of 0 or 10 with equal chances; over-ordering costs 1 a unit, under 9
def order_cost(u):
    return 0.5 * (1.0 * max(u - 0.0, 0.0) + 9.0 * max(0.0 - u, 0.0)) + 0.5 * (1.0 * max(u - 10.0, 0.0) + 9.0 * max(10.0 - u, 0.0))
NV = {u: (0.5 * (u - 0.0) ** 2 + 0.5 * (u - 10.0) ** 2, order_cost(u)) for u in (5.0, 10.0)}
print("\nTheorem 13.2(iv): " + "; ".join(f"predict {u:g}: squared error {se:g}, its decision costs {c:g}" for u, (se, c) in NV.items()))

fig, (ax, bx) = plt.subplots(1, 2, figsize=(7.6, 3.6), gridspec_kw={"width_ratios": [1.15, 1]})
sh = [100 * x["share"] for x in SWEEP]
ax.plot(sh, [x["design_value"] for x in SWEEP], color="#1F4E79", lw=1.8, label="value of its packages")
ax2 = ax.twinx()
ax2.plot(sh, [100 * x["accuracy"] for x in SWEEP], color="#E65100", lw=1.6, ls="--", label="accuracy, fresh paths")
ax2.axhline(100 * NEVER, color="#999999", lw=0.8, ls=":")
ax2.text(3, 100 * NEVER - 1.5, f"never warn: {100 * NEVER:.1f}%", fontsize=8, color="#616161", va="top")
for x_ in (LAW05, LAWBEST):
    ax.plot([100 * x_["shares"][0]], [x_["believed"]], "o", color="#1F4E79", ms=5, zorder=5)
    ax2.plot([100 * x_["shares"][0]], [100 * x_["accuracy"]], "o", color="#E65100", ms=5, zorder=5)
ax.text(100 * LAWBEST["shares"][0] + 3, LAWBEST["believed"] + 0.08,
        f"\\${LAWBEST['believed']:.2f}m,\n{100 * LAWBEST['shares'][0]:.0f}% warned", fontsize=8, color="#1F4E79")
ax.text(100 * LAW05["shares"][0] + 3, LAW05["believed"] - 0.05, f"\\${LAW05['believed']:.2f}m", fontsize=8,
        color="#1F4E79", va="top")
ax2.text(100 * LAW05["shares"][0] - 2, 100 * LAW05["accuracy"] + 2.5, f"threshold 0.5: {100 * LAW05['accuracy']:.1f}%",
         fontsize=8, color="#E65100")
ax.set_zorder(ax2.get_zorder() + 1); ax.patch.set_visible(False)
ax.set(xlim=(0, 100), ylim=(0, 2.0), xlabel="programs warned (%)", ylabel="value of the warning ($ million)")
ax2.set(ylim=(20, 100)); ax2.set_ylabel("accuracy on fresh paths (%)", color="#E65100")
ax2.spines["top"].set_visible(False)
ax.set_title("(a) the logistic warning, every threshold", fontsize=9)
bx.plot([x["auc"] for x in CURVE], [x["value"] for x in CURVE], "o-", color="#1F4E79", lw=1.6, ms=3.5,
        label="all five shifts scaled")
for txt, a_, v_, xy in ((f"perfect warning: \\${C5 - PERFECT['cost']:.2f}m", 1.0, C5 - PERFECT["cost"], (0.56, 3.9)),
                        (f"the desk's five readings:\nAUC {CEIL['auc']:.3f}, \\${CEIL['value']:.2f}m", CEIL["auc"], CEIL["value"],
                         (0.56, 2.1))):
    bx.plot([a_], [v_], "o", color="#E65100", ms=6, zorder=5)
    bx.annotate(txt, xy=(a_, v_), xytext=xy, fontsize=8, va="center", arrowprops=dict(arrowstyle="-", lw=0.6, color="#333333"))
bx.set(xlim=(0.5, 1.01), ylim=(0, 4.5), xlabel="AUC of the readings", ylabel="most a warning is worth ($ million)")
bx.set_title("(b) the ceiling, by the readings' AUC", fontsize=9)
fig.suptitle("A \\$1.44m warning is right less often than no warning (seed 26213)", fontsize=10.5)
plt.tight_layout(); plt.show()
each group's closure share as the law gives it:
at accuracy's threshold: warn at a chance of 50.00%: 23.01% warned; closures ahead in 78.37% of the warned and 10.23% of the all clear
  all clear: line 125, delay 0.712, 9.45; warned: line 205, delay 0.665, 11.47; promises 0.53
  fresh: 22.9% warned, right 87.3% of the time, chance of a breach 4.90%, worth 0.53
where the packages are cheapest: warn at a chance of 4.09%: 59.00% warned; closures ahead in 42.71% of the warned and 1.73% of the all clear
  all clear: line 0, delay 0.669, 5.35; warned: line 190, delay 0.730, 11.54; promises 1.44
  fresh: 58.6% warned, right 65.9% of the time, chance of a breach 4.93%, worth 1.46
never warning is right 74.0% of the time: the valuable warning is less accurate than no warning
the sweep, by share of programs warned (%): 10: 82.7% right, 0.19, 20: 87.1% right, 0.45, 30: 85.9% right, 0.71, 40: 80.8% right, 0.95, 50: 73.5% right, 1.22, 60: 65.0% right, 1.43, 70: 55.9% right, 1.26

Theorem 13.2(iv): predict 5: squared error 25, its decision costs 25; predict 10: squared error 50, its decision costs 5
A $1.44m warning is right less often than no warning (seed 26213)

Panel 4 — A perfect record and a broken appetite (Theorem 13.1(iii); Proposition 13.2(iv); Worked Example 13.1, Step 3; Table 13.3: trusted, boosting warns 27.0% of the record with closure shares 99.26% and 0%, promises $4.04m and breaches 7.68% on the fresh paths; held out, it warns 58.0% with shares 44.48% and 2.38%, promises $1.16m, breaches 4.96% and is worth $0.59m)

AXIOM Lab 2.13, step 4, first part. From here on each group's closure share is estimated from the desk's record, as a design must be. The trusted design takes the shares from the boosted model's scores on the record it learned; the held-out design takes them from its out-of-fold scores. Each design's threshold is chosen among 100 quantile steps of the scores it is designed from, and each design is checked on the fresh paths.

Python Code
TRUSTED = best_split(G_IN, YR)
TRUSTED.update(evaluate(F_GB, TRUSTED["tau"], TRUSTED["design"]))
GB_HELD = best_split(G_OOF, YR)
GB_HELD.update(evaluate(F_GB, GB_HELD["tau"], GB_HELD["design"]))
show("boosting, trusted", TRUSTED, value=False)
show("boosting, held out", GB_HELD)
print(f"\ntrusted, the boosted warning promises {TRUSTED['believed']:.2f}, against {C5 - PERFECT['cost']:.2f} for a "
      f"perfect warning, and breaks the appetite: {100 * TRUSTED['breach']:.2f}% on the fresh paths")
boosting, trusted: warn at a chance of 2.51%: 27.00% warned; closures ahead in 99.26% of the warned and 0.00% of the all clear
  all clear: line 0, delay 0.608, 4.87; warned: line 215, delay 0.513, 10.56; promises 4.04
  fresh: 35.7% warned, right 81.0% of the time, chance of a breach 7.68%, the appetite broken: no value
boosting, held out: warn at a chance of 0.09%: 58.00% warned; closures ahead in 44.48% of the warned and 2.38% of the all clear
  all clear: line 0, delay 0.757, 6.06; warned: line 190, delay 0.739, 11.61; promises 1.16
  fresh: 68.3% warned, right 55.5% of the time, chance of a breach 4.96%, worth 0.59

trusted, the boosted warning promises 4.04, against 4.12 for a perfect warning, and breaks the appetite: 7.68% on the fresh paths

Panel 5 — Certify before paying (Theorem 13.3(i); Worked Example 13.1, Step 4; Table 13.3: designed from held-out shares, 44.75% and 0.98%, the logistic warning promises $1.73m and breaches 5.02% on the fresh paths; designed for their one-sided 95% upper bounds, 48.21% and 2.22%, it promises $1.17m, breaches 4.73% and is worth $1.06m)

AXIOM Lab 2.13, step 4. The audit committee's rule: a model is scored on cases held out from its fitting, and a design is certified on bounds, not point estimates. The held-out design takes each group's closure share from the logistic model's out-of-fold scores; the certified design replaces each by its one-sided 95 percent Clopper–Pearson upper bound, the 0.95 quantile of the beta law with parameters (k+1,m−k)(k + 1, m - k) for kk closures among mm windows (Theorem 13.3(i)). Both are checked on the fresh paths. Last, the logistic warning at accuracy's threshold with held-out shares (Online Appendix Example 13.B.2).

Python Code
HELD = best_split(S_OOF, YR)
HELD.update(evaluate(F_LR, HELD["tau"], HELD["design"]))
CERT = best_split(S_OOF, YR, certify=True)
CERT.update(evaluate(F_LR, CERT["tau"], CERT["design"]))
OOF05 = dict(tau=0.5)
OOF05["shares"] = shares(S_OOF, YR, 0.5)[:3]
OOF05["design"] = SPLIT.two(*OOF05["shares"])
OOF05.update(evaluate(F_LR, 0.5, OOF05["design"]))
show("logistic, held out", HELD)
w_ = S_OOF >= CERT["tau"]
KM = dict(k_w=int(YR[w_].sum()), m_w=int(w_.sum()), k_c=int(YR[~w_].sum()), m_c=int((~w_).sum()))
print(f"\nat the same threshold the record holds {KM['k_w']} closures in {KM['m_w']} warned windows and {KM['k_c']} in "
      f"{KM['m_c']} all clear: point estimates {100 * KM['k_w'] / KM['m_w']:.2f}% and {100 * KM['k_c'] / KM['m_c']:.2f}%, "
      f"one-sided 95% upper bounds {100 * cp_upper(KM['k_w'], KM['m_w']):.2f}% and {100 * cp_upper(KM['k_c'], KM['m_c']):.2f}%")
show("logistic, certified", CERT)
print()
show("logistic at 0.5, held-out shares", OOF05)
logistic, held out: warn at a chance of 3.48%: 59.00% warned; closures ahead in 44.75% of the warned and 0.98% of the all clear
  all clear: line 0, delay 0.683, 5.46; warned: line 185, delay 0.677, 10.96; promises 1.73
  fresh: 60.9% warned, right 63.9% of the time, chance of a breach 5.02%, worth 1.63

at the same threshold the record holds 264 closures in 590 warned windows and 4 in 410 all clear: point estimates 44.75% and 0.98%, one-sided 95% upper bounds 48.21% and 2.22%
logistic, certified: warn at a chance of 3.48%: 59.00% warned; closures ahead in 48.21% of the warned and 2.22% of the all clear
  all clear: line 0, delay 0.734, 5.87; warned: line 190, delay 0.742, 11.64; promises 1.17
  fresh: 60.9% warned, right 63.9% of the time, chance of a breach 4.73%, worth 1.06

logistic at 0.5, held-out shares: warn at a chance of 50.00%: 25.20% warned; closures ahead in 81.75% of the warned and 8.29% of the all clear
  all clear: line 115, delay 0.718, 9.19; warned: line 205, delay 0.670, 11.51; promises 0.67
  fresh: 22.9% warned, right 87.3% of the time, chance of a breach 5.03%, worth 0.72

Panel 6 — Repeat on 200 records (Theorem 13.3(i); Worked Example 13.1, Step 4; Figure 13.2(b): designs from held-out shares keep the appetite in 47.5% of records, with a mean true chance of 5.02%, a 90th percentile of 5.21% and a mean value of $1.53m; certified designs in 98.0%, 4.75%, 4.89% and $0.95m)

AXIOM Lab 2.13, step 4, last part. The desk's record is one of many it could have been. Draw 200 records of 1,000 windows from the declared law (seeds 913000–913199); on each, fit the logistic model, score it out of fold (folds shuffled with the record's own seed), design the held-out and the certified warnings (thresholds among 50 quantile steps of the scores), and check both on the fresh paths. This cell takes about a minute: 400 designs, each searching 51 thresholds.

Python Code
STUDY = []
for r in range(200):
    y_, X_ = record(seed=STUDY_SEED + r)
    m_ = fit_logistic(X_, y_)
    so_ = oof_scores(fit_logistic, X_, y_, seed=STUDY_SEED + r)
    f_ = (lambda mm: (lambda X: mm.predict_proba(X)[:, 1]))(m_)
    row = dict(seed=STUDY_SEED + r)
    for key, cert in (("held", False), ("cert", True)):
        b = best_split(so_, y_, certify=cert, n_steps=50)
        e = evaluate(f_, b["tau"], b["design"])
        row[key] = dict(breach=e["breach"], value=e["value"], believed=e["believed"])
    STUDY.append(row)
SUMMARY = {}
for key, name in (("held", "held-out shares"), ("cert", "certified")):
    b_ = np.array([x[key]["breach"] for x in STUDY]); v_ = np.array([x[key]["value"] for x in STUDY])
    SUMMARY[key] = dict(meet=float(np.mean(b_ <= ALPHA)), mean=float(b_.mean()), q90=float(np.quantile(b_, 0.9)),
                        value=float(v_.mean()), believed=float(np.mean([x[key]["believed"] for x in STUDY])))
    s = SUMMARY[key]
    print(f"{name:16s}: keep 5% in {100 * s['meet']:.1f}% of 200 records; true chance {100 * s['mean']:.2f}% on average, "
          f"{100 * s['q90']:.2f}% at the 90th percentile; worth {s['value']:.2f} on average (promised {s['believed']:.2f})")

fig, ax = plt.subplots(figsize=(7.6, 3.6))
for key, col, name in (("held", "#E65100", "held-out shares"), ("cert", "#1F4E79", "certified")):
    ax.plot([100 * x[key]["breach"] for x in STUDY], [x[key]["value"] for x in STUDY], "o", ms=3.2, color=col, alpha=0.75,
            label=f"{name}: {100 * SUMMARY[key]['meet']:.1f}% keep 5%")
ax.axvline(5.0, color="#999999", lw=0.9, ls=":")
ax.text(5.02, 0.08, "the appetite", fontsize=8, color="#616161")
ax.set(xlim=(4.4, 5.6), ylim=(0, 2.8), xlabel="chance of a breach on the fresh paths (%)",
       ylabel="value on the fresh paths ($ million)", title="Certified designs keep the appetite in 98.0% of records (seed 26213)")
ax.legend(frameon=False, fontsize=8, loc="upper left"); plt.tight_layout(); plt.show()
held-out shares : keep 5% in 47.5% of 200 records; true chance 5.02% on average, 5.21% at the 90th percentile; worth 1.53 on average (promised 1.53)
certified       : keep 5% in 98.0% of 200 records; true chance 4.75% on average, 4.89% at the 90th percentile; worth 0.95 on average (promised 0.96)
Certified designs keep the appetite in 98.0% of records (seed 26213)

Panel 7 — Compress and explain (Proposition 13.3; Worked Example 13.1, Step 5; Figure 13.3: the first principal component keeps 56.0% of the variance and allows at most $0.14m; the three market readings hold 51.1% and allow $0.02m, the funding spread alone 30.1% and $1.21m; attributions of the reading (1, 1, 1, 1, 1.5), interventional −0.413, −0.242, −0.176, 0.820, 2.131, conditional 0.048, 0.098, 0.043, 0.462, 1.469)

AXIOM Lab 2.13, step 5. Components. The record's principal components are the eigenvectors of its covariance, in order of their eigenvalues (Definition 13.3). A warning on the top rr components reads W⊤xW^\top x, for WW the first rr eigenvectors; its separation under the declared law is Δr=(s⊤W(W⊤ΣW)−1W⊤s)1/2\Delta_r = (\mathbf s^\top W(W^\top\Sigma W)^{-1}W^\top\mathbf s)^{1/2}, and it is priced like the ceiling. Subsets of the readings are priced the same way. Attributions. The fitted model's log-odds is linear. Under the interventional game, readings outside a coalition are set independently at their record averages, and the Shapley value of reading jj is βj(xj−xˉj)\beta_j(x_j - \bar x_j) (Proposition 13.3(iii)); under the conditional game they are set at their averages given the coalition, from a Gaussian fit to the record, and the Shapley values are computed exactly from the game's values on all 32 coalitions. The reading is one standard unit high on every indicator and 1.5 on the funding spread. Last, Proposition 13.3's two small examples.

Python Code
COV = np.cov(XR.T)
EIG, V = np.linalg.eigh(COV)
order = np.argsort(EIG)[::-1]
EIG, V = EIG[order], V[:, order]
if V[:, 0].sum() < 0:
    V[:, 0] *= -1
KEPT = np.cumsum(EIG) / EIG.sum()
PCS = []
for r in range(1, 6):
    Vr = V[:, :r]
    D_r = float(math.sqrt(SHIFT @ Vr @ np.linalg.solve(Vr.T @ SIGMA @ Vr, Vr.T @ SHIFT)))
    PCS.append(dict(r=r, kept=float(KEPT[r - 1]), Delta=D_r, auc=auc(D_r), value=ceiling(D_r)["value"]))
print("the record's principal components: eigenvalues " + ", ".join(f"{x:.3f}" for x in EIG))
print("  the first's loadings " + ", ".join(f"{x:.2f}" for x in V[:, 0]) + ": an equal-weighted financial-conditions index")
print("  top r   variance kept   separation    AUC    the most a warning on them is worth")
for x in PCS:
    print(f"  {x['r']:5d}   {100 * x['kept']:12.1f}%   {x['Delta']:10.3f}   {x['auc']:.3f}   {x['value']:6.2f}")

VAR = np.var(XR, axis=0, ddof=1)               # the record's variance of each reading
TR = float(np.trace(COV))
SUBSETS = {}
for name, idx in (("the three market readings", [0, 1, 2]), ("lending and funding", [3, 4]), ("funding alone", [4]),
                  ("all but funding", [0, 1, 2, 3])):
    b_, D_ = posterior_coef(idx=idx)
    SUBSETS[name] = dict(idx=idx, Delta=D_, auc=auc(D_), value=ceiling(D_)["value"],
                         var=float(np.var(XR[:, idx], axis=0, ddof=1).sum() / TR))
print("\nsubsets of the readings      variance held    AUC    the most a warning on them is worth")
for name, s in SUBSETS.items():
    print(f"  {name:26s} {100 * s['var']:12.1f}%   {s['auc']:.3f}   {s['value']:6.2f}")

def shapley_linear(beta, x, mean, cov, game="interventional"):
    # exact Shapley values of f(x) = beta . x: readings outside a coalition at their means (interventional) or at their
    # Gaussian conditional means given the coalition (conditional)
    k = len(x)
    if game == "interventional":
        return beta * (x - mean)
    def v(S):
        S = list(S)
        if not S:
            return float(beta @ mean)
        T = [j for j in range(k) if j not in S]
        ex = np.array(x, dtype=float).copy()
        if T:
            ex[T] = mean[T] + cov[np.ix_(T, S)] @ np.linalg.solve(cov[np.ix_(S, S)], x[S] - mean[S])
        return float(beta @ ex)
    phi = np.zeros(k)
    for j in range(k):
        others = [i for i in range(k) if i != j]
        for r in range(k):
            for S in itertools.combinations(others, r):
                wgt = math.factorial(r) * math.factorial(k - r - 1) / math.factorial(k)
                phi[j] += wgt * (v(S + (j,)) - v(S))
    return phi

X0 = np.array([1.0, 1.0, 1.0, 1.0, 1.5])
MU = XR.mean(0)
LOGODDS0 = float(LR.coef_[0] @ X0 + LR.intercept_[0])
PROB0 = float(LR.predict_proba(X0[None, :])[0, 1])
SH_I = shapley_linear(LR.coef_[0], X0, MU, COV, "interventional")
SH_C = shapley_linear(LR.coef_[0], X0, MU, COV, "conditional")
print(f"\nthe reading (1, 1, 1, 1, 1.5): log-odds {LOGODDS0:.3f}, a chance of {100 * PROB0:.1f}%: warned at the best "
      f"threshold ({100 * LAWBEST['tau']:.2f}%): {PROB0 >= LAWBEST['tau']}; at 0.5: {PROB0 >= 0.5}")
print("  record averages      " + "".join(f"{x:9.2f}" for x in MU))
print("  interventional       " + "".join(f"{x:9.3f}" for x in SH_I) + f"   sum {SH_I.sum():.3f}")
print("  conditional          " + "".join(f"{x:9.3f}" for x in SH_C) + f"   sum {SH_C.sum():.3f}")
print(f"  the three market readings: {SH_I[:3].sum():.2f} interventional, {SH_C[:3].sum():.2f} conditional; "
      f"f(x) - f(mean) = {LOGODDS0 - float(LR.coef_[0] @ MU + LR.intercept_[0]):.3f}")

# Proposition 13.3(ii): independent X1, X2 with variances 4 and 1, and Y = X2
PCA_EX = 4.0 / (4.0 + 1.0)
# Proposition 13.3(iv): f(x) = x1 and X2 = X1 with mean mu; the conditional game credits x2 with (x1 - mu) / 2
mu_, x1_ = 0.0, 2.0
v_ = {(): mu_, (1,): x1_, (2,): x1_, (1, 2): x1_}       # v(S) = E[f(X) | X_S = x_S]
PHI2 = 0.5 * (v_[(2,)] - v_[()]) + 0.5 * (v_[(1, 2)] - v_[(1,)])
print(f"\nProposition 13.3(ii): the first component keeps {100 * PCA_EX:.0f}% of the variance and none of Y")
print(f"Proposition 13.3(iv): f ignores x2, yet the conditional game credits it with {PHI2:g} = (x1 - mu) / 2 at x1 = {x1_:g}")

fig, (ax, bx) = plt.subplots(1, 2, figsize=(7.6, 3.6), gridspec_kw={"width_ratios": [1.1, 1]})
ax.plot([100 * x["kept"] for x in PCS], [x["value"] for x in PCS], "o-", color="#1F4E79", lw=1.6, ms=4,
        label="top 1–5 principal components")
for x in PCS[:3]:
    ax.text(100 * x["kept"] + 1.5, x["value"] - 0.05, f"{x['r']}", fontsize=8, color="#1F4E79", va="top")
for name, xy in (("the three market readings", (6, 0.42)), ("funding alone", (4, 0.9)), ("lending and funding", (4, 1.65))):
    s = SUBSETS[name]
    ax.plot([100 * s["var"]], [s["value"]], "s", color="#E65100", ms=5, zorder=5)
    ax.annotate(name, xy=(100 * s["var"], s["value"]), xytext=xy, fontsize=8,
                arrowprops=dict(arrowstyle="-", lw=0.6, color="#333333"))
ax.set(xlim=(0, 105), ylim=(0, 2.0), xlabel="variance of the readings kept (%)", ylabel="most a warning is worth ($ million)")
ax.legend(frameon=False, fontsize=8, loc="upper right")
ax.set_title("(a) variance kept is not value", fontsize=9)
yy = np.arange(5)[::-1]
bx.barh(yy + 0.18, SH_I, height=0.36, color="#1F4E79", label="interventional")
bx.barh(yy - 0.18, SH_C, height=0.36, color="#E65100", label="conditional")
bx.axvline(0, color="#333333", lw=0.7)
bx.set_yticks(yy, SHORT)
bx.set(xlim=(-0.6, 2.3), xlabel="contribution to the log-odds")
bx.legend(frameon=False, fontsize=8, loc="upper right")
bx.set_title("(b) the reading (1, 1, 1, 1, 1.5), two games", fontsize=9)
fig.suptitle("The index keeps 56% of the variance, \\$0.14m of value (seed 26213)", fontsize=10.5)
plt.tight_layout(); plt.show()
the record's principal components: eigenvalues 3.255, 1.427, 0.587, 0.321, 0.228
  the first's loadings 0.47, 0.46, 0.43, 0.43, 0.44: an equal-weighted financial-conditions index
  top r   variance kept   separation    AUC    the most a warning on them is worth
      1           56.0%        0.843   0.724     0.14
      2           80.5%        1.975   0.919     1.34
      3           90.6%        2.028   0.924     1.44
      4           96.1%        2.028   0.924     1.44
      5          100.0%        2.029   0.924     1.44

subsets of the readings      variance held    AUC    the most a warning on them is worth
  the three market readings          51.1%   0.577     0.02
  lending and funding                48.9%   0.914     1.27
  funding alone                      30.1%   0.910     1.21
  all but funding                    69.9%   0.698     0.10

the reading (1, 1, 1, 1, 1.5): log-odds -0.071, a chance of 48.2%: warned at the best threshold (4.09%): True; at 0.5: False
  record averages           0.04     0.08     0.03     0.15     0.52
  interventional          -0.413   -0.242   -0.176    0.820    2.131   sum 2.120
  conditional              0.048    0.098    0.043    0.462    1.469   sum 2.120
  the three market readings: -0.83 interventional, 0.19 conditional; f(x) - f(mean) = 2.120

Proposition 13.3(ii): the first component keeps 80% of the variance and none of Y
Proposition 13.3(iv): f ignores x2, yet the conditional game credits it with 1 = (x1 - mu) / 2 at x1 = 2
The index keeps 56% of the variance, $0.14m of value (seed 26213)

Panel 8 — Every warning side by side (Worked Example 13.1; Table 13.3; Table 13.2)

Worked Example 13.1's warnings, their packages and their checks on Chapter 9's fresh paths, in Table 13.3's layout. Warned: the share of the 100,000 fresh programs warned; packages: line ($ million) and delay (quarters); promised value: Volume I's $10.44 million less the expected cost the design expects; fresh: the true chance of a breach and the value on the fresh paths, each program with its own readings. The two logistic rows marked law take each group's closure share from the law; the other logistic and boosting rows estimate it from the desk's record. A fresh value is shown only where the fresh breach is near the appetite. Table 13.2's last column reads the desk's warning by each measure.

Python Code
def row(name, warned, d, fresh, show_value=True):
    cl, wa = d["clear"], d["warn"]
    return dict(name=name, warned=float(warned), clear=None if cl is None else (cl[1], cl[2]),
                warn=None if wa is None else (wa[1], wa[2]), promised=C5 - d["cost"], breach=fresh[0],
                value=(C5 - fresh[1]) if show_value else None)

TABLE = [
    row("None (Volume I)", 0.0, NONE, NONE_FRESH),
    row("Perfect (Chapter 9)", FRESH.ahead.mean(), PERFECT, PERFECT_FRESH),
    row("Declared law (ceiling)", CEIL_W.mean(), CEIL["design"], CEIL_FRESH),
    row("Logistic, at 0.5 (law)", LAW05["fresh_warned"], LAW05["design"], (LAW05["breach"], LAW05["cost"])),
    row("Logistic, cheapest (law)", LAWBEST["fresh_warned"], LAWBEST["design"], (LAWBEST["breach"], LAWBEST["cost"])),
    row("Logistic, held-out", HELD["fresh_warned"], HELD["design"], (HELD["breach"], HELD["cost"])),
    row("Logistic, certified", CERT["fresh_warned"], CERT["design"], (CERT["breach"], CERT["cost"])),
    row("Boosting, trusted", TRUSTED["fresh_warned"], TRUSTED["design"], (TRUSTED["breach"], TRUSTED["cost"]), False),
    row("Boosting, held-out", GB_HELD["fresh_warned"], GB_HELD["design"], (GB_HELD["breach"], GB_HELD["cost"])),
]
pk = lambda g: "--" if g is None else f"{g[0]:.0f}, {g[1]:.2f}"
print(f"{'Warning':26s} {'Warned':>7s}  {'All clear':>11s}  {'Warned':>11s}  {'Promised':>8s}  {'Fresh':>7s}  {'Fresh':>6s}")
print(f"{'':26s} {'(%)':>7s}  {'line, delay':>11s}  {'line, delay':>11s}  {'value':>8s}  {'breach':>7s}  {'value':>6s}")
for r in TABLE:
    print(f"{r['name']:26s} {100 * r['warned']:7.1f}  {pk(r['clear']):>11s}  {pk(r['warn']):>11s}  {r['promised']:8.2f}  "
          f"{100 * r['breach']:7.2f}  {'--' if r['value'] is None else format(r['value'], '6.2f'):>6s}")

print("\nTable 13.2, the desk's warning by each measure (fresh paths):")
print(f"  accuracy        {100 * LAW05['accuracy']:.1f}% at 0.5, {100 * LAWBEST['accuracy']:.1f}% where cheapest")
print(f"  AUC             {AUC_LR['fresh']:.3f}")
print(f"  decision value  ${CEIL['value']:.2f}m at most (the ceiling)")
print(f"  fresh breach    {100 * CERT['breach']:.2f}% certified")
Warning                     Warned    All clear       Warned  Promised    Fresh   Fresh
                               (%)  line, delay  line, delay     value   breach   value
None (Volume I)                0.0    170, 0.67           --      0.00     4.91    0.00
Perfect (Chapter 9)           26.0      0, 0.61    215, 0.50      4.12     4.91    4.12
Declared law (ceiling)        58.9      0, 0.67    190, 0.73      1.44     4.90    1.43
Logistic, at 0.5 (law)        22.9    125, 0.71    205, 0.67      0.53     4.90    0.53
Logistic, cheapest (law)      58.6      0, 0.67    190, 0.73      1.44     4.93    1.46
Logistic, held-out            60.9      0, 0.68    185, 0.68      1.73     5.02    1.63
Logistic, certified           60.9      0, 0.73    190, 0.74      1.17     4.73    1.06
Boosting, trusted             35.7      0, 0.61    215, 0.51      4.04     7.68      --
Boosting, held-out            68.3      0, 0.76    190, 0.74      1.16     4.96    0.59

Table 13.2, the desk's warning by each measure (fresh paths):
  accuracy        87.3% at 0.5, 65.9% where cheapest
  AUC             0.924
  decision value  $1.44m at most (the ceiling)
  fresh breach    4.73% certified

Validation — the book's numbers, and agreement with DCT_V2_Ch13_Lab.xlsx

Each checkpoint is a number printed in Chapter 13 (the abstract, Sections 13.2–13.4, Worked Example 13.1, Tables 13.2–13.3, Figures 13.1–13.3, the Key Takeaways, the LOS Guidance) or in Online Appendix Remark 13.B.1 and Example 13.B.2, with a tolerance of half a unit in its last printed digit; a few check what a theorem or proposition promises (1 = holds). Chances and shares are in percent. The workbook recomputes the checkpoints marked workbook with spreadsheet formulas and shows PASS against the same engine values on its Checks sheet.

Python Code
def bk(s):
    # a number as the book prints it: its value and half a unit in its last printed digit
    t = s.replace(",", "").replace("−", "-")
    dp = len(t.split(".")[1]) if "." in t else 0
    return float(t), 0.5 * 10.0 ** -dp + 1e-9

def c_(name, got, printed):
    v, tol = bk(printed)
    return (name, float(got), v, tol)

def flag(name, ok):
    return (name, float(bool(ok)), 1.0, 0.0)

def count(name, got, n):
    return (name, float(got), float(n), 0.0)

pc = lambda x: 100.0 * x

def design_checks(label, x, printed, wb=True):
    # printed: (warned %, closure share warned %, closure share clear %, clear line, clear delay, clear cost,
    #           warned line, warned delay, warned cost, promised); None where the book prints nothing
    keys = ("warned", "share warned", "share all clear", "all clear line", "all clear delay", "all clear cost",
            "warned line", "warned delay", "warned cost", "promised value")
    d = x["design"]
    got = (pc(x["shares"][0]), pc(x["shares"][1]), pc(x["shares"][2]), d["clear"][1], d["clear"][2], d["clear"][0],
           d["warn"][1], d["warn"][2], d["warn"][0], x["believed"] if "believed" in x else C5 - d["cost"])
    out = []
    for k_, g_, p_ in zip(keys, got, printed):
        if p_ is None:
            continue
        mark = " (workbook)" if wb and k_ not in ("all clear line", "all clear delay", "warned line", "warned delay") else ""
        out.append(c_(f"{label}: {k_}{mark}", g_, p_) if not k_.endswith("line") else count(f"{label}: {k_}", g_, int(p_)))
    return out

cd, pf = CEIL["design"], PERFECT
checks = [
    # Remark 13.B.1; the declarations
    c_("closure ahead: book paths, % (Remark 13.B.1)", pc(PI1), "25.91"),
    c_("closure ahead: fresh paths, %", pc(FRESH.ahead.mean()), "25.96"),
    c_("closure ahead: the law, 1 - e^-0.3, % (workbook)", pc(P_CLOSE), "25.92"),
    count("Volume I's package: line", NONE["clear"][1], 170),
    c_("Volume I's package: delay", NONE["clear"][2], "0.668"),
    c_("Volume I's package: cost (workbook)", C5, "10.44"),
    c_("Volume I's package: chance on the book paths, % (workbook)", pc(SPLIT.chance(PI1, NONE["clear"])), "5.00"),
    c_("Volume I's package: chance on the fresh paths, % (workbook)", pc(NONE_FRESH[0]), "4.91"),
    # the perfect warning (WE 13.1, the situation; Example 13.B.2)
    count("perfect warning: all clear line", pf["clear"][1], 0),
    c_("perfect warning: all clear delay", pf["clear"][2], "0.608"),
    c_("perfect warning: all clear delay, the situation", pf["clear"][2], "0.61"),
    c_("perfect warning: all clear cost (workbook)", pf["clear"][0], "4.87"),
    c_("perfect warning: all clear chance, % (workbook)", pc(SPLIT.chance(0.0, pf["clear"])), "4.75"),
    count("perfect warning: warned line", pf["warn"][1], 215),
    c_("perfect warning: warned delay", pf["warn"][2], "0.504"),
    c_("perfect warning: warned delay, half a quarter", pf["warn"][2], "0.5"),
    c_("perfect warning: warned cost (workbook)", pf["warn"][0], "10.49"),
    c_("perfect warning: warned chance, % (workbook)", pc(SPLIT.chance(1.0, pf["warn"])), "5.71"),
    c_("perfect warning: expected cost (workbook)", pf["cost"], "6.32"),
    c_("perfect warning: value (workbook)", C5 - pf["cost"], "4.12"),
    c_("perfect warning: fresh breach, %", pc(PERFECT_FRESH[0]), "4.91"),
    c_("perfect warning: fresh cost (workbook)", PERFECT_FRESH[1], "6.33"),
    # Panel 1: the record and the ceiling (Step 1; Remark 13.B.1; Example 13.B.2)
    count("the record: windows", len(YR), 1000),
    count("the record: windows with a closure", YR.sum(), 268),
] + [c_(f"the record: mean {s} reading with a closure", m, p) for s, m, p in
     zip(SHORT, XR[YR].mean(0), ("0.18", "0.29", "0.17", "0.73", "2.00"))] + [
    c_(f"the record: mean {s} reading without", m, p) for s, m, p in
    zip(SHORT, XR[~YR].mean(0), ("-0.02", "0.01", "-0.02", "-0.06", "-0.02"))] + [
    c_(f"the declared law: beta, {s} (workbook)", b, p) for s, b, p in
    zip(SHORT, BETA_LAW, ("-0.323", "-0.159", "-0.352", "0.769", "1.974"))] + [
    c_("the declared law: separation Delta (workbook)", DELTA, "2.029"),
    c_("the declared law: AUC (workbook)", auc(DELTA), "0.924"),
    c_("ceiling: threshold on the log-likelihood ratio", CEIL["t"], "-1.826"),
    c_("ceiling: warned, % (workbook)", pc(CEIL["shares"][0]), "58.86"),
    c_("ceiling: closure share, warned, % (workbook)", pc(CEIL["shares"][1]), "42.80"),
    c_("ceiling: closure share, all clear, % (workbook)", pc(CEIL["shares"][2]), "1.75"),
    count("ceiling: all clear line", cd["clear"][1], 0),
    c_("ceiling: all clear delay", cd["clear"][2], "0.671"),
    c_("ceiling: all clear cost (workbook)", cd["clear"][0], "5.37"),
    count("ceiling: warned line", cd["warn"][1], 190),
    c_("ceiling: warned delay", cd["warn"][2], "0.731"),
    c_("ceiling: warned cost (workbook)", cd["warn"][0], "11.55"),
    c_("ceiling: expected cost (workbook)", cd["cost"], "9.00"),
    c_("ceiling: value (workbook)", CEIL["value"], "1.44"),
    c_("ceiling: fresh warned, % (workbook)", pc(CEIL_W.mean()), "58.94"),
    c_("ceiling: fresh breach, % (workbook)", pc(CEIL_FRESH[0]), "4.90"),
    c_("ceiling: fresh value (workbook)", C5 - CEIL_FRESH[1], "1.43"),
] + [c_(f"value against AUC: AUC, shifts x {c:g} (workbook)", x["auc"], p) for c, x, p in
     zip(SCALES, CURVE, ("0.640", "0.763", "0.859", "0.924", "0.964", "0.984", "0.998", "1.000", "1.000", "1.000"))] + [
    c_(f"value against AUC: ceiling, shifts x {c:g}", x["value"], p) for c, x, p in
    zip(SCALES, CURVE, ("0.05", "0.22", "0.63", "1.44", "2.24", "2.88", "3.69", "4.00", "4.09", "4.12"))] + [
    c_("Step 1: $0.63m at an AUC of 0.86 (workbook)", CURVE[2]["auc"], "0.86"),
    c_("Step 1: $2.24m at an AUC of 0.96 (workbook)", CURVE[4]["auc"], "0.96"),
    # Panel 2: the learners (Step 2-3; Section 13.2; Example 13.B.2)
] + [c_(f"logistic: coefficient, {s}", b, p) for s, b, p in
     zip(SHORT, LR.coef_[0], ("-0.428", "-0.264", "-0.183", "0.968", "2.180"))] + [
    c_("logistic: intercept", LR.intercept_[0], "-3.435"),
    c_("Step 5: the funding spread's weight", LR.coef_[0][4], "2.18"),
    c_("Step 5: lending standards' weight", LR.coef_[0][3], "0.97"),
    flag("Step 5: the three market readings weigh negatively", np.all(LR.coef_[0][:3] < 0)),
    c_("logistic: AUC on the record", AUC_LR["record"], "0.942"),
    c_("logistic: AUC held out", AUC_LR["oof"], "0.940"),
    c_("logistic: AUC fresh", AUC_LR["fresh"], "0.924"),
    c_("Step 2: AUC on the record", AUC_LR["record"], "0.94"),
    c_("Step 2: AUC held out", AUC_LR["oof"], "0.94"),
    c_("Step 2: AUC fresh", AUC_LR["fresh"], "0.92"),
    c_("logistic: right at 0.5, the record, % (workbook)", pc(ACC_LR["record"]), "89.3"),
    c_("logistic: right at 0.5, fresh, % (workbook)", pc(ACC_LR["fresh"]), "87.3"),
    c_("never warning: right on fresh programs, % (workbook)", pc(NEVER), "74.0"),
    c_("boosting: AUC on the record", AUC_GB["record"], "1.000"),
    c_("boosting: right on the record, % (workbook)", pc(ACC_GB["record"]), "100"),
    c_("boosting: AUC held out", AUC_GB["oof"], "0.922"),
    c_("boosting: AUC fresh", AUC_GB["fresh"], "0.888"),
    c_("boosting: right at 0.5, fresh, % (workbook)", pc(ACC_GB["fresh"]), "85.6"),
    c_("Step 3: boosting's AUC held out", AUC_GB["oof"], "0.92"),
    c_("Step 3: boosting's AUC fresh", AUC_GB["fresh"], "0.89"),
    flag("Step 3: boosting's fresh AUC is below the logistic model's", AUC_GB["fresh"] < AUC_LR["fresh"]),
    c_("boosting: fresh AUC at its peak (workbook)", ST_FR[PEAK], "0.910"),
    count("boosting: the peak's stage (workbook)", PEAK + 1, 24),
    c_("boosting: fresh AUC after 500 stages (workbook)", ST_FR[-1], "0.888"),
    c_("boosting: record AUC after 500 stages (workbook)", ST_REC[-1], "1.000"),
    flag("Proposition 13.2(iv): after the fresh peak the record's AUC rises and the fresh AUC falls",
         ST_REC[-1] > ST_REC[PEAK] and ST_FR[-1] < ST_FR[PEAK]),
    c_("Proposition 13.2(iii): each member right on, % (workbook)", pc(MEMBERS.min()), "60"),
    c_("Proposition 13.2(iii): the majority vote right on, % (workbook)", pc(MAJORITY), "40"),
    # Panel 3: price the warning (Step 2; Section 13.3; Figure 13.1(a); Example 13.B.2)
] + design_checks("logistic at 0.5, law's shares", LAW05,
                  ("23.01", "78.37", "10.23", "125", "0.712", "9.45", "205", "0.665", "11.47", "0.53")) + [
    c_("logistic at 0.5, law's shares: fresh warned, % (workbook)", pc(LAW05["fresh_warned"]), "22.9"),
    c_("logistic at 0.5, law's shares: right on fresh programs, % (workbook)", pc(LAW05["accuracy"]), "87.3"),
    c_("logistic at 0.5, law's shares: fresh breach, % (workbook)", pc(LAW05["breach"]), "4.90"),
    c_("logistic at 0.5, law's shares: fresh value (workbook)", LAW05["value"], "0.53"),
    c_("Step 2: warns at 0.5, %", pc(LAW05["shares"][0]), "23"),
    c_("Step 2: the all-clear group's closures at 0.5, %", pc(LAW05["shares"][2]), "10.2"),
    c_("logistic, cheapest: the threshold's chance, %", pc(LAWBEST["tau"]), "4.09"),
    c_("Step 2: the threshold's chance, %", pc(LAWBEST["tau"]), "4.1"),
] + design_checks("logistic, cheapest, law's shares", LAWBEST,
                  ("59.0", "42.71", "1.73", "0", "0.669", "5.35", "190", "0.730", "11.54", "1.44")) + [
    c_("logistic, cheapest: fresh warned, % (workbook)", pc(LAWBEST["fresh_warned"]), "58.6"),
    c_("logistic, cheapest: right on fresh programs, % (workbook)", pc(LAWBEST["accuracy"]), "65.9"),
    c_("logistic, cheapest: fresh breach, % (workbook)", pc(LAWBEST["breach"]), "4.93"),
    c_("logistic, cheapest: fresh value (workbook)", LAWBEST["value"], "1.46"),
    c_("Step 2: warns where cheapest, %", pc(LAWBEST["shares"][0]), "59"),
    flag("Step 2: the cheapest threshold is worth the ceiling, to the cent", round(LAWBEST["believed"], 2) == round(CEIL["value"], 2)),
    flag("Theorem 13.2(ii): the valuable warning is less accurate than never warning", LAWBEST["accuracy"] < NEVER),
    c_("Theorem 13.2(iv): squared error of the prediction 5 (workbook)", NV[5.0][0], "25"),
    c_("Theorem 13.2(iv): cost of the decision on 5 (workbook)", NV[5.0][1], "25"),
    c_("Theorem 13.2(iv): squared error of the prediction 10 (workbook)", NV[10.0][0], "50"),
    c_("Theorem 13.2(iv): cost of the decision on 10 (workbook)", NV[10.0][1], "5"),
    # Panel 4: boosting (Step 3; Example 13.B.2)
] + design_checks("boosting, trusted", TRUSTED,
                  ("27.0", "99.26", "0", "0", "0.608", "4.87", "215", "0.513", "10.56", "4.04")) + [
    c_("boosting, trusted: fresh warned, % (workbook)", pc(TRUSTED["fresh_warned"]), "35.7"),
    c_("boosting, trusted: fresh breach, % (workbook)", pc(TRUSTED["breach"]), "7.68"),
] + design_checks("boosting, held-out", GB_HELD,
                  ("58.0", "44.48", "2.38", "0", "0.757", "6.06", "190", "0.739", "11.61", "1.16")) + [
    c_("boosting, held-out: fresh warned, % (workbook)", pc(GB_HELD["fresh_warned"]), "68.3"),
    c_("boosting, held-out: fresh breach, % (workbook)", pc(GB_HELD["breach"]), "4.96"),
    c_("boosting, held-out: fresh value (workbook)", GB_HELD["value"], "0.59"),
    # Panel 5: certify (Step 4; Section 13.3; Example 13.B.2)
    c_("logistic, held-out: the threshold's chance, %", pc(HELD["tau"]), "3.48"),
] + design_checks("logistic, held-out", HELD,
                  ("59.0", "44.75", "0.98", "0", "0.683", "5.46", "185", "0.677", "10.96", "1.73")) + [
    c_("logistic, held-out: fresh warned, % (workbook)", pc(HELD["fresh_warned"]), "60.9"),
    c_("logistic, held-out: right on fresh programs, % (workbook)", pc(HELD["accuracy"]), "63.9"),
    c_("logistic, held-out: fresh breach, % (workbook)", pc(HELD["breach"]), "5.02"),
    c_("logistic, held-out: fresh value (workbook)", HELD["value"], "1.63"),
    c_("Step 4: held-out share, warned, %", pc(HELD["shares"][1]), "44.7"),
    c_("Step 4: held-out share, all clear, %", pc(HELD["shares"][2]), "1.0"),
    flag("certified: the same threshold as the held-out design", CERT["tau"] == HELD["tau"]),
] + design_checks("logistic, certified", CERT,
                  (None, "48.21", "2.22", "0", "0.734", "5.87", "190", "0.742", "11.64", "1.17")) + [
    c_("logistic, certified: fresh breach, % (workbook)", pc(CERT["breach"]), "4.73"),
    c_("logistic, certified: fresh value (workbook)", CERT["value"], "1.06"),
    c_("Step 4: certified bound, warned, %", pc(CERT["shares"][1]), "48.2"),
    c_("Step 4: certified bound, all clear, %", pc(CERT["shares"][2]), "2.2"),
] + design_checks("logistic at 0.5, held-out shares", OOF05,
                  ("25.2", "81.75", "8.29", None, None, None, None, None, None, "0.67")) + [
    c_("logistic at 0.5, held-out shares: fresh breach, % (workbook)", pc(OOF05["breach"]), "5.03"),
    # Panel 6: 200 records (Step 4; Figure 13.2(b); Example 13.B.2)
    count("the study: records", len(STUDY), 200),
    c_("200 records, held-out: keep 5%, % (workbook)", pc(SUMMARY["held"]["meet"]), "47.5"),
    c_("200 records, held-out: mean true chance, % (workbook)", pc(SUMMARY["held"]["mean"]), "5.02"),
    c_("200 records, held-out: 90th percentile, % (workbook)", pc(SUMMARY["held"]["q90"]), "5.21"),
    c_("200 records, held-out: mean value (workbook)", SUMMARY["held"]["value"], "1.53"),
    c_("200 records, certified: keep 5%, % (workbook)", pc(SUMMARY["cert"]["meet"]), "98.0"),
    c_("200 records, certified: mean true chance, % (workbook)", pc(SUMMARY["cert"]["mean"]), "4.75"),
    c_("200 records, certified: 90th percentile, % (workbook)", pc(SUMMARY["cert"]["q90"]), "4.89"),
    c_("200 records, certified: mean value (workbook)", SUMMARY["cert"]["value"], "0.95"),
    c_("Section 13.3: certified designs keep it in, % of records", pc(SUMMARY["cert"]["meet"]), "98"),
    flag("Section 13.3: held-out designs keep the appetite only about half the time", 0.4 <= SUMMARY["held"]["meet"] <= 0.6),
    # Panel 7: components and attributions (Step 5; Figure 13.3; Example 13.B.2)
] + [c_(f"principal components: eigenvalue {k + 1}", e, p) for k, (e, p) in
     enumerate(zip(EIG, ("3.255", "1.427", "0.587", "0.321", "0.228")))] + [
    c_(f"principal components: variance kept by the top {k + 1}, % (workbook)", pc(x), p) for k, (x, p) in
    enumerate(zip(KEPT, ("56.0", "80.5", "90.6", "96.1", "100")))] + [
    c_(f"the first component's loading, {s}", x, p) for s, x, p in zip(SHORT, V[:, 0], ("0.47", "0.46", "0.43", "0.43", "0.44"))] + [
    c_(f"top {x['r']} components: AUC (workbook)", x["auc"], p) for x, p in zip(PCS[:3], ("0.724", "0.919", "0.924"))] + [
    c_(f"top {x['r']} components: ceiling", x["value"], p) for x, p in zip(PCS, ("0.14", "1.34", "1.44", "1.44", "1.44"))] + [
    c_("Step 5: the first component keeps, %", pc(KEPT[0]), "56"),
]
for name, (a_, v_, s_), short in (("the three market readings", ("0.577", "0.02", "51.1"), "51"),
                                  ("lending and funding", ("0.914", "1.27", "48.9"), None),
                                  ("funding alone", ("0.910", "1.21", "30.1"), "30"),
                                  ("all but funding", ("0.698", "0.10", "69.9"), None)):
    s = SUBSETS[name]
    checks += [c_(f"subset, {name}: AUC (workbook)", s["auc"], a_), c_(f"subset, {name}: ceiling", s["value"], v_),
               c_(f"subset, {name}: variance held, % (workbook)", pc(s["var"]), s_)]
    if short:
        checks.append(c_(f"Step 5: {name} hold, %", pc(s["var"]), short))
checks += [
    c_("the reading (1, 1, 1, 1, 1.5): log-odds (workbook)", LOGODDS0, "-0.071"),
    c_("the reading: chance of a closure, % (workbook)", pc(PROB0), "48.2"),
    flag("the reading: warned at the best threshold (workbook)", PROB0 >= LAWBEST["tau"]),
    flag("the reading: not warned at 0.5 (workbook)", PROB0 < 0.5),
] + [c_(f"the record's mean {s} reading", m, p) for s, m, p in zip(SHORT, MU, ("0.04", "0.08", "0.03", "0.15", "0.52"))] + [
    c_(f"interventional attribution, {s} (workbook)", x, p) for s, x, p in
    zip(SHORT, SH_I, ("-0.413", "-0.242", "-0.176", "0.820", "2.131"))] + [
    c_(f"conditional attribution, {s}", x, p) for s, x, p in zip(SHORT, SH_C, ("0.048", "0.098", "0.043", "0.462", "1.469"))] + [
    c_("interventional attributions: sum (workbook)", SH_I.sum(), "2.120"),
    c_("conditional attributions: sum (workbook)", SH_C.sum(), "2.120"),
    c_("Step 5: the funding spread's interventional credit (workbook)", SH_I[4], "2.13"),
    c_("Step 5: the market readings' interventional credit (workbook)", SH_I[:3].sum(), "-0.83"),
    c_("Step 5: the funding spread's conditional credit", SH_C[4], "1.47"),
    flag("Step 5: the conditional game gives every reading positive credit", np.all(SH_C > 0)),
    c_("Proposition 13.3(ii): the first component keeps, % (workbook)", pc(PCA_EX), "80"),
    c_("Proposition 13.3(iv): the conditional credit of an ignored feature (workbook)", PHI2, "1"),
]
# Panel 8: Table 13.3 and Table 13.2
T133 = {"None (Volume I)": ("0", None, None, None, None, "0", "4.91", "0"),
        "Perfect (Chapter 9)": ("26.0", ("0", "0.61"), ("215", "0.50"), "4.12", "4.91", "4.12"),
        "Declared law (ceiling)": ("58.9", ("0", "0.67"), ("190", "0.73"), "1.44", "4.90", "1.43"),
        "Logistic, at 0.5 (law)": ("22.9", ("125", "0.71"), ("205", "0.67"), "0.53", "4.90", "0.53"),
        "Logistic, cheapest (law)": ("58.6", ("0", "0.67"), ("190", "0.73"), "1.44", "4.93", "1.46"),
        "Logistic, held-out": ("60.9", ("0", "0.68"), ("185", "0.68"), "1.73", "5.02", "1.63"),
        "Logistic, certified": ("60.9", ("0", "0.73"), ("190", "0.74"), "1.17", "4.73", "1.06"),
        "Boosting, trusted": ("35.7", ("0", "0.61"), ("215", "0.51"), "4.04", "7.68", None),
        "Boosting, held-out": ("68.3", ("0", "0.76"), ("190", "0.74"), "1.16", "4.96", "0.59")}
for r in TABLE:
    b = T133[r["name"]]
    if r["name"] == "None (Volume I)":
        checks += [c_("Table 13.3, None: warned, %", pc(r["warned"]), "0"),
                   count("Table 13.3, None: all clear line", r["clear"][0], 170),
                   c_("Table 13.3, None: all clear delay", r["clear"][1], "0.67"),
                   c_("Table 13.3, None: promised value (workbook)", r["promised"], "0"),
                   c_("Table 13.3, None: fresh breach, % (workbook)", pc(r["breach"]), "4.91"),
                   c_("Table 13.3, None: fresh value (workbook)", r["value"], "0")]
        continue
    w_, (cl_l, cl_d), (wa_l, wa_d), pr_, br_, fv_ = b
    nm = f"Table 13.3, {r['name']}"
    checks += [c_(f"{nm}: warned, % (workbook)", pc(r["warned"]), w_),
               count(f"{nm}: all clear line", r["clear"][0], int(cl_l)), c_(f"{nm}: all clear delay", r["clear"][1], cl_d),
               count(f"{nm}: warned line", r["warn"][0], int(wa_l)), c_(f"{nm}: warned delay", r["warn"][1], wa_d),
               c_(f"{nm}: promised value (workbook)", r["promised"], pr_), c_(f"{nm}: fresh breach, % (workbook)", pc(r["breach"]), br_)]
    if fv_ is not None:
        checks.append(c_(f"{nm}: fresh value (workbook)", r["value"], fv_))
checks += [
    c_("Table 13.2: accuracy at 0.5, %", pc(LAW05["accuracy"]), "87.3"),
    c_("Table 13.2: accuracy where cheapest, %", pc(LAWBEST["accuracy"]), "65.9"),
    c_("Table 13.2: AUC", AUC_LR["fresh"], "0.924"),
    c_("Table 13.2: decision value at most", CEIL["value"], "1.44"),
    c_("Table 13.2: fresh breach, certified, %", pc(CERT["breach"]), "4.73"),
    # the abstract, the board's take-away, Key Takeaways, LOS Guidance
    c_("abstract: a perfect warning", C5 - pf["cost"], "4.12"),
    c_("abstract: the five readings allow at most", CEIL["value"], "1.44"),
    c_("abstract: the certified warning is worth", CERT["value"], "1.06"),
    flag("the board's take-away: the ceiling is about a third of a perfect warning", 0.30 <= CEIL["value"] / (C5 - pf["cost"]) <= 0.37),
    c_("the board's take-away: trusted boosting promises nearly", TRUSTED["believed"], "4"),
    c_("the board's take-away and Key Takeaways: certified, about", CERT["value"], "1"),
    c_("LOS 13.2: boosting on the record", AUC_GB["record"], "1.00"),
    c_("LOS 13.2: boosting fresh", AUC_GB["fresh"], "0.89"),
    c_("LOS 13.3: at accuracy's threshold", LAW05["believed"], "0.53"),
    c_("LOS 13.3: at a 4.1% threshold", LAWBEST["believed"], "1.44"),
    c_("LOS 13.4: held-out designs keep 5%, % of records", pc(SUMMARY["held"]["meet"]), "47.5"),
    c_("LOS 13.4: certified designs keep 5%, % of records", pc(SUMMARY["cert"]["meet"]), "98.0"),
    c_("LOS 13.4: at an average value of", SUMMARY["cert"]["value"], "0.95"),
    c_("LOS 13.5: the first component's value", PCS[0]["value"], "0.14"),
    c_("LOS 13.5: the funding spread alone keeps, %", pc(SUBSETS["funding alone"]["var"]), "30"),
]
ok = 0
for name, got, want, tol in checks:
    status = "PASS" if abs(got - want) <= tol else "FAIL"
    ok += status == "PASS"
    print(f"{name:84s} {got:11.4f}   book {want:10.4f}   {status}")
print(f"\n{ok} of {len(checks)} checkpoints PASS")
closure ahead: book paths, % (Remark 13.B.1)                                             25.9120   book    25.9100   PASS
closure ahead: fresh paths, %                                                            25.9600   book    25.9600   PASS
closure ahead: the law, 1 - e^-0.3, % (workbook)                                         25.9182   book    25.9200   PASS
Volume I's package: line                                                                170.0000   book   170.0000   PASS
Volume I's package: delay                                                                 0.6676   book     0.6680   PASS
Volume I's package: cost (workbook)                                                      10.4406   book    10.4400   PASS
Volume I's package: chance on the book paths, % (workbook)                                5.0000   book     5.0000   PASS
Volume I's package: chance on the fresh paths, % (workbook)                               4.9070   book     4.9100   PASS
perfect warning: all clear line                                                           0.0000   book     0.0000   PASS
perfect warning: all clear delay                                                          0.6084   book     0.6080   PASS
perfect warning: all clear delay, the situation                                           0.6084   book     0.6100   PASS
perfect warning: all clear cost (workbook)                                                4.8668   book     4.8700   PASS
perfect warning: all clear chance, % (workbook)                                           4.7498   book     4.7500   PASS
perfect warning: warned line                                                            215.0000   book   215.0000   PASS
perfect warning: warned delay                                                             0.5044   book     0.5040   PASS
perfect warning: warned delay, half a quarter                                             0.5044   book     0.5000   PASS
perfect warning: warned cost (workbook)                                                  10.4851   book    10.4900   PASS
perfect warning: warned chance, % (workbook)                                              5.7116   book     5.7100   PASS
perfect warning: expected cost (workbook)                                                 6.3226   book     6.3200   PASS
perfect warning: value (workbook)                                                         4.1180   book     4.1200   PASS
perfect warning: fresh breach, %                                                          4.9130   book     4.9100   PASS
perfect warning: fresh cost (workbook)                                                    6.3253   book     6.3300   PASS
the record: windows                                                                    1000.0000   book  1000.0000   PASS
the record: windows with a closure                                                      268.0000   book   268.0000   PASS
the record: mean bond reading with a closure                                              0.1790   book     0.1800   PASS
the record: mean high yield reading with a closure                                        0.2920   book     0.2900   PASS
the record: mean volatility reading with a closure                                        0.1717   book     0.1700   PASS
the record: mean lending reading with a closure                                           0.7348   book     0.7300   PASS
the record: mean funding reading with a closure                                           1.9953   book     2.0000   PASS
the record: mean bond reading without                                                    -0.0171   book    -0.0200   PASS
the record: mean high yield reading without                                               0.0079   book     0.0100   PASS
the record: mean volatility reading without                                              -0.0163   book    -0.0200   PASS
the record: mean lending reading without                                                 -0.0605   book    -0.0600   PASS
the record: mean funding reading without                                                 -0.0167   book    -0.0200   PASS
the declared law: beta, bond (workbook)                                                  -0.3232   book    -0.3230   PASS
the declared law: beta, high yield (workbook)                                            -0.1589   book    -0.1590   PASS
the declared law: beta, volatility (workbook)                                            -0.3522   book    -0.3520   PASS
the declared law: beta, lending (workbook)                                                0.7694   book     0.7690   PASS
the declared law: beta, funding (workbook)                                                1.9738   book     1.9740   PASS
the declared law: separation Delta (workbook)                                             2.0287   book     2.0290   PASS
the declared law: AUC (workbook)                                                          0.9243   book     0.9240   PASS
ceiling: threshold on the log-likelihood ratio                                           -1.8258   book    -1.8260   PASS
ceiling: warned, % (workbook)                                                            58.8639   book    58.8600   PASS
ceiling: closure share, warned, % (workbook)                                             42.7969   book    42.8000   PASS
ceiling: closure share, all clear, % (workbook)                                           1.7504   book     1.7500   PASS
ceiling: all clear line                                                                   0.0000   book     0.0000   PASS
ceiling: all clear delay                                                                  0.6708   book     0.6710   PASS
ceiling: all clear cost (workbook)                                                        5.3661   book     5.3700   PASS
ceiling: warned line                                                                    190.0000   book   190.0000   PASS
ceiling: warned delay                                                                     0.7307   book     0.7310   PASS
ceiling: warned cost (workbook)                                                          11.5458   book    11.5500   PASS
ceiling: expected cost (workbook)                                                         9.0037   book     9.0000   PASS
ceiling: value (workbook)                                                                 1.4369   book     1.4400   PASS
ceiling: fresh warned, % (workbook)                                                      58.9400   book    58.9400   PASS
ceiling: fresh breach, % (workbook)                                                       4.8980   book     4.9000   PASS
ceiling: fresh value (workbook)                                                           1.4322   book     1.4300   PASS
value against AUC: AUC, shifts x 0.25 (workbook)                                          0.6401   book     0.6400   PASS
value against AUC: AUC, shifts x 0.5 (workbook)                                           0.7634   book     0.7630   PASS
value against AUC: AUC, shifts x 0.75 (workbook)                                          0.8590   book     0.8590   PASS
value against AUC: AUC, shifts x 1 (workbook)                                             0.9243   book     0.9240   PASS
value against AUC: AUC, shifts x 1.25 (workbook)                                          0.9635   book     0.9640   PASS
value against AUC: AUC, shifts x 1.5 (workbook)                                           0.9843   book     0.9840   PASS
value against AUC: AUC, shifts x 2 (workbook)                                             0.9979   book     0.9980   PASS
value against AUC: AUC, shifts x 2.5 (workbook)                                           0.9998   book     1.0000   PASS
value against AUC: AUC, shifts x 3 (workbook)                                             1.0000   book     1.0000   PASS
value against AUC: AUC, shifts x 4 (workbook)                                             1.0000   book     1.0000   PASS
value against AUC: ceiling, shifts x 0.25                                                 0.0487   book     0.0500   PASS
value against AUC: ceiling, shifts x 0.5                                                  0.2207   book     0.2200   PASS
value against AUC: ceiling, shifts x 0.75                                                 0.6277   book     0.6300   PASS
value against AUC: ceiling, shifts x 1                                                    1.4369   book     1.4400   PASS
value against AUC: ceiling, shifts x 1.25                                                 2.2374   book     2.2400   PASS
value against AUC: ceiling, shifts x 1.5                                                  2.8796   book     2.8800   PASS
value against AUC: ceiling, shifts x 2                                                    3.6935   book     3.6900   PASS
value against AUC: ceiling, shifts x 2.5                                                  3.9987   book     4.0000   PASS
value against AUC: ceiling, shifts x 3                                                    4.0930   book     4.0900   PASS
value against AUC: ceiling, shifts x 4                                                    4.1178   book     4.1200   PASS
Step 1: $0.63m at an AUC of 0.86 (workbook)                                               0.8590   book     0.8600   PASS
Step 1: $2.24m at an AUC of 0.96 (workbook)                                               0.9635   book     0.9600   PASS
logistic: coefficient, bond                                                              -0.4280   book    -0.4280   PASS
logistic: coefficient, high yield                                                        -0.2641   book    -0.2640   PASS
logistic: coefficient, volatility                                                        -0.1825   book    -0.1830   PASS
logistic: coefficient, lending                                                            0.9675   book     0.9680   PASS
logistic: coefficient, funding                                                            2.1804   book     2.1800   PASS
logistic: intercept                                                                      -3.4346   book    -3.4350   PASS
Step 5: the funding spread's weight                                                       2.1804   book     2.1800   PASS
Step 5: lending standards' weight                                                         0.9675   book     0.9700   PASS
Step 5: the three market readings weigh negatively                                        1.0000   book     1.0000   PASS
logistic: AUC on the record                                                               0.9419   book     0.9420   PASS
logistic: AUC held out                                                                    0.9401   book     0.9400   PASS
logistic: AUC fresh                                                                       0.9240   book     0.9240   PASS
Step 2: AUC on the record                                                                 0.9419   book     0.9400   PASS
Step 2: AUC held out                                                                      0.9401   book     0.9400   PASS
Step 2: AUC fresh                                                                         0.9240   book     0.9200   PASS
logistic: right at 0.5, the record, % (workbook)                                         89.3000   book    89.3000   PASS
logistic: right at 0.5, fresh, % (workbook)                                              87.3240   book    87.3000   PASS
never warning: right on fresh programs, % (workbook)                                     74.0400   book    74.0000   PASS
boosting: AUC on the record                                                               1.0000   book     1.0000   PASS
boosting: right on the record, % (workbook)                                             100.0000   book   100.0000   PASS
boosting: AUC held out                                                                    0.9223   book     0.9220   PASS
boosting: AUC fresh                                                                       0.8883   book     0.8880   PASS
boosting: right at 0.5, fresh, % (workbook)                                              85.6440   book    85.6000   PASS
Step 3: boosting's AUC held out                                                           0.9223   book     0.9200   PASS
Step 3: boosting's AUC fresh                                                              0.8883   book     0.8900   PASS
Step 3: boosting's fresh AUC is below the logistic model's                                1.0000   book     1.0000   PASS
boosting: fresh AUC at its peak (workbook)                                                0.9103   book     0.9100   PASS
boosting: the peak's stage (workbook)                                                    24.0000   book    24.0000   PASS
boosting: fresh AUC after 500 stages (workbook)                                           0.8885   book     0.8880   PASS
boosting: record AUC after 500 stages (workbook)                                          1.0000   book     1.0000   PASS
Proposition 13.2(iv): after the fresh peak the record's AUC rises and the fresh AUC falls      1.0000   book     1.0000   PASS
Proposition 13.2(iii): each member right on, % (workbook)                                60.0000   book    60.0000   PASS
Proposition 13.2(iii): the majority vote right on, % (workbook)                          40.0000   book    40.0000   PASS
logistic at 0.5, law's shares: warned (workbook)                                         23.0120   book    23.0100   PASS
logistic at 0.5, law's shares: share warned (workbook)                                   78.3678   book    78.3700   PASS
logistic at 0.5, law's shares: share all clear (workbook)                                10.2328   book    10.2300   PASS
logistic at 0.5, law's shares: all clear line                                           125.0000   book   125.0000   PASS
logistic at 0.5, law's shares: all clear delay                                            0.7121   book     0.7120   PASS
logistic at 0.5, law's shares: all clear cost (workbook)                                  9.4469   book     9.4500   PASS
logistic at 0.5, law's shares: warned line                                              205.0000   book   205.0000   PASS
logistic at 0.5, law's shares: warned delay                                               0.6654   book     0.6650   PASS
logistic at 0.5, law's shares: warned cost (workbook)                                    11.4735   book    11.4700   PASS
logistic at 0.5, law's shares: promised value (workbook)                                  0.5273   book     0.5300   PASS
logistic at 0.5, law's shares: fresh warned, % (workbook)                                22.9240   book    22.9000   PASS
logistic at 0.5, law's shares: right on fresh programs, % (workbook)                     87.3240   book    87.3000   PASS
logistic at 0.5, law's shares: fresh breach, % (workbook)                                 4.8980   book     4.9000   PASS
logistic at 0.5, law's shares: fresh value (workbook)                                     0.5291   book     0.5300   PASS
Step 2: warns at 0.5, %                                                                  23.0120   book    23.0000   PASS
Step 2: the all-clear group's closures at 0.5, %                                         10.2328   book    10.2000   PASS
logistic, cheapest: the threshold's chance, %                                             4.0935   book     4.0900   PASS
Step 2: the threshold's chance, %                                                         4.0935   book     4.1000   PASS
logistic, cheapest, law's shares: warned (workbook)                                      59.0000   book    59.0000   PASS
logistic, cheapest, law's shares: share warned (workbook)                                42.7136   book    42.7100   PASS
logistic, cheapest, law's shares: share all clear (workbook)                              1.7341   book     1.7300   PASS
logistic, cheapest, law's shares: all clear line                                          0.0000   book     0.0000   PASS
logistic, cheapest, law's shares: all clear delay                                         0.6687   book     0.6690   PASS
logistic, cheapest, law's shares: all clear cost (workbook)                               5.3496   book     5.3500   PASS
logistic, cheapest, law's shares: warned line                                           190.0000   book   190.0000   PASS
logistic, cheapest, law's shares: warned delay                                            0.7305   book     0.7300   PASS
logistic, cheapest, law's shares: warned cost (workbook)                                 11.5437   book    11.5400   PASS
logistic, cheapest, law's shares: promised value (workbook)                               1.4365   book     1.4400   PASS
logistic, cheapest: fresh warned, % (workbook)                                           58.6460   book    58.6000   PASS
logistic, cheapest: right on fresh programs, % (workbook)                                65.8620   book    65.9000   PASS
logistic, cheapest: fresh breach, % (workbook)                                            4.9280   book     4.9300   PASS
logistic, cheapest: fresh value (workbook)                                                1.4584   book     1.4600   PASS
Step 2: warns where cheapest, %                                                          59.0000   book    59.0000   PASS
Step 2: the cheapest threshold is worth the ceiling, to the cent                          1.0000   book     1.0000   PASS
Theorem 13.2(ii): the valuable warning is less accurate than never warning                1.0000   book     1.0000   PASS
Theorem 13.2(iv): squared error of the prediction 5 (workbook)                           25.0000   book    25.0000   PASS
Theorem 13.2(iv): cost of the decision on 5 (workbook)                                   25.0000   book    25.0000   PASS
Theorem 13.2(iv): squared error of the prediction 10 (workbook)                          50.0000   book    50.0000   PASS
Theorem 13.2(iv): cost of the decision on 10 (workbook)                                   5.0000   book     5.0000   PASS
boosting, trusted: warned (workbook)                                                     27.0000   book    27.0000   PASS
boosting, trusted: share warned (workbook)                                               99.2593   book    99.2600   PASS
boosting, trusted: share all clear (workbook)                                             0.0000   book     0.0000   PASS
boosting, trusted: all clear line                                                         0.0000   book     0.0000   PASS
boosting, trusted: all clear delay                                                        0.6084   book     0.6080   PASS
boosting, trusted: all clear cost (workbook)                                              4.8668   book     4.8700   PASS
boosting, trusted: warned line                                                          215.0000   book   215.0000   PASS
boosting, trusted: warned delay                                                           0.5132   book     0.5130   PASS
boosting, trusted: warned cost (workbook)                                                10.5559   book    10.5600   PASS
boosting, trusted: promised value (workbook)                                              4.0377   book     4.0400   PASS
boosting, trusted: fresh warned, % (workbook)                                            35.7200   book    35.7000   PASS
boosting, trusted: fresh breach, % (workbook)                                             7.6760   book     7.6800   PASS
boosting, held-out: warned (workbook)                                                    58.0000   book    58.0000   PASS
boosting, held-out: share warned (workbook)                                              44.4828   book    44.4800   PASS
boosting, held-out: share all clear (workbook)                                            2.3810   book     2.3800   PASS
boosting, held-out: all clear line                                                        0.0000   book     0.0000   PASS
boosting, held-out: all clear delay                                                       0.7574   book     0.7570   PASS
boosting, held-out: all clear cost (workbook)                                             6.0594   book     6.0600   PASS
boosting, held-out: warned line                                                         190.0000   book   190.0000   PASS
boosting, held-out: warned delay                                                          0.7385   book     0.7390   PASS
boosting, held-out: warned cost (workbook)                                               11.6083   book    11.6100   PASS
boosting, held-out: promised value (workbook)                                             1.1629   book     1.1600   PASS
boosting, held-out: fresh warned, % (workbook)                                           68.3090   book    68.3000   PASS
boosting, held-out: fresh breach, % (workbook)                                            4.9600   book     4.9600   PASS
boosting, held-out: fresh value (workbook)                                                0.5908   book     0.5900   PASS
logistic, held-out: the threshold's chance, %                                             3.4785   book     3.4800   PASS
logistic, held-out: warned (workbook)                                                    59.0000   book    59.0000   PASS
logistic, held-out: share warned (workbook)                                              44.7458   book    44.7500   PASS
logistic, held-out: share all clear (workbook)                                            0.9756   book     0.9800   PASS
logistic, held-out: all clear line                                                        0.0000   book     0.0000   PASS
logistic, held-out: all clear delay                                                       0.6830   book     0.6830   PASS
logistic, held-out: all clear cost (workbook)                                             5.4643   book     5.4600   PASS
logistic, held-out: warned line                                                         185.0000   book   185.0000   PASS
logistic, held-out: warned delay                                                          0.6768   book     0.6770   PASS
logistic, held-out: warned cost (workbook)                                               10.9640   book    10.9600   PASS
logistic, held-out: promised value (workbook)                                             1.7315   book     1.7300   PASS
logistic, held-out: fresh warned, % (workbook)                                           60.8910   book    60.9000   PASS
logistic, held-out: right on fresh programs, % (workbook)                                63.8570   book    63.9000   PASS
logistic, held-out: fresh breach, % (workbook)                                            5.0190   book     5.0200   PASS
logistic, held-out: fresh value (workbook)                                                1.6275   book     1.6300   PASS
Step 4: held-out share, warned, %                                                        44.7458   book    44.7000   PASS
Step 4: held-out share, all clear, %                                                      0.9756   book     1.0000   PASS
certified: the same threshold as the held-out design                                      1.0000   book     1.0000   PASS
logistic, certified: share warned (workbook)                                             48.2117   book    48.2100   PASS
logistic, certified: share all clear (workbook)                                           2.2186   book     2.2200   PASS
logistic, certified: all clear line                                                       0.0000   book     0.0000   PASS
logistic, certified: all clear delay                                                      0.7340   book     0.7340   PASS
logistic, certified: all clear cost (workbook)                                            5.8720   book     5.8700   PASS
logistic, certified: warned line                                                        190.0000   book   190.0000   PASS
logistic, certified: warned delay                                                         0.7419   book     0.7420   PASS
logistic, certified: warned cost (workbook)                                              11.6354   book    11.6400   PASS
logistic, certified: promised value (workbook)                                            1.1683   book     1.1700   PASS
logistic, certified: fresh breach, % (workbook)                                           4.7260   book     4.7300   PASS
logistic, certified: fresh value (workbook)                                               1.0593   book     1.0600   PASS
Step 4: certified bound, warned, %                                                       48.2117   book    48.2000   PASS
Step 4: certified bound, all clear, %                                                     2.2186   book     2.2000   PASS
logistic at 0.5, held-out shares: warned (workbook)                                      25.2000   book    25.2000   PASS
logistic at 0.5, held-out shares: share warned (workbook)                                81.7460   book    81.7500   PASS
logistic at 0.5, held-out shares: share all clear (workbook)                              8.2888   book     8.2900   PASS
logistic at 0.5, held-out shares: promised value (workbook)                               0.6661   book     0.6700   PASS
logistic at 0.5, held-out shares: fresh breach, % (workbook)                              5.0330   book     5.0300   PASS
the study: records                                                                      200.0000   book   200.0000   PASS
200 records, held-out: keep 5%, % (workbook)                                             47.5000   book    47.5000   PASS
200 records, held-out: mean true chance, % (workbook)                                     5.0197   book     5.0200   PASS
200 records, held-out: 90th percentile, % (workbook)                                      5.2131   book     5.2100   PASS
200 records, held-out: mean value (workbook)                                              1.5333   book     1.5300   PASS
200 records, certified: keep 5%, % (workbook)                                            98.0000   book    98.0000   PASS
200 records, certified: mean true chance, % (workbook)                                    4.7545   book     4.7500   PASS
200 records, certified: 90th percentile, % (workbook)                                     4.8911   book     4.8900   PASS
200 records, certified: mean value (workbook)                                             0.9502   book     0.9500   PASS
Section 13.3: certified designs keep it in, % of records                                 98.0000   book    98.0000   PASS
Section 13.3: held-out designs keep the appetite only about half the time                 1.0000   book     1.0000   PASS
principal components: eigenvalue 1                                                        3.2547   book     3.2550   PASS
principal components: eigenvalue 2                                                        1.4271   book     1.4270   PASS
principal components: eigenvalue 3                                                        0.5865   book     0.5870   PASS
principal components: eigenvalue 4                                                        0.3207   book     0.3210   PASS
principal components: eigenvalue 5                                                        0.2279   book     0.2280   PASS
principal components: variance kept by the top 1, % (workbook)                           55.9520   book    56.0000   PASS
principal components: variance kept by the top 2, % (workbook)                           80.4854   book    80.5000   PASS
principal components: variance kept by the top 3, % (workbook)                           90.5686   book    90.6000   PASS
principal components: variance kept by the top 4, % (workbook)                           96.0822   book    96.1000   PASS
principal components: variance kept by the top 5, % (workbook)                          100.0000   book   100.0000   PASS
the first component's loading, bond                                                       0.4734   book     0.4700   PASS
the first component's loading, high yield                                                 0.4599   book     0.4600   PASS
the first component's loading, volatility                                                 0.4311   book     0.4300   PASS
the first component's loading, lending                                                    0.4345   book     0.4300   PASS
the first component's loading, funding                                                    0.4355   book     0.4400   PASS
top 1 components: AUC (workbook)                                                          0.7245   book     0.7240   PASS
top 2 components: AUC (workbook)                                                          0.9188   book     0.9190   PASS
top 3 components: AUC (workbook)                                                          0.9242   book     0.9240   PASS
top 1 components: ceiling                                                                 0.1409   book     0.1400   PASS
top 2 components: ceiling                                                                 1.3390   book     1.3400   PASS
top 3 components: ceiling                                                                 1.4364   book     1.4400   PASS
top 4 components: ceiling                                                                 1.4365   book     1.4400   PASS
top 5 components: ceiling                                                                 1.4369   book     1.4400   PASS
Step 5: the first component keeps, %                                                     55.9520   book    56.0000   PASS
subset, the three market readings: AUC (workbook)                                         0.5773   book     0.5770   PASS
subset, the three market readings: ceiling                                                0.0240   book     0.0200   PASS
subset, the three market readings: variance held, % (workbook)                           51.0963   book    51.1000   PASS
Step 5: the three market readings hold, %                                                51.0963   book    51.0000   PASS
subset, lending and funding: AUC (workbook)                                               0.9143   book     0.9140   PASS
subset, lending and funding: ceiling                                                      1.2710   book     1.2700   PASS
subset, lending and funding: variance held, % (workbook)                                 48.9037   book    48.9000   PASS
subset, funding alone: AUC (workbook)                                                     0.9104   book     0.9100   PASS
subset, funding alone: ceiling                                                            1.2145   book     1.2100   PASS
subset, funding alone: variance held, % (workbook)                                       30.1408   book    30.1000   PASS
Step 5: funding alone hold, %                                                            30.1408   book    30.0000   PASS
subset, all but funding: AUC (workbook)                                                   0.6976   book     0.6980   PASS
subset, all but funding: ceiling                                                          0.0991   book     0.1000   PASS
subset, all but funding: variance held, % (workbook)                                     69.8592   book    69.9000   PASS
the reading (1, 1, 1, 1, 1.5): log-odds (workbook)                                       -0.0711   book    -0.0710   PASS
the reading: chance of a closure, % (workbook)                                           48.2221   book    48.2000   PASS
the reading: warned at the best threshold (workbook)                                      1.0000   book     1.0000   PASS
the reading: not warned at 0.5 (workbook)                                                 1.0000   book     1.0000   PASS
the record's mean bond reading                                                            0.0354   book     0.0400   PASS
the record's mean high yield reading                                                      0.0840   book     0.0800   PASS
the record's mean volatility reading                                                      0.0341   book     0.0300   PASS
the record's mean lending reading                                                         0.1527   book     0.1500   PASS
the record's mean funding reading                                                         0.5225   book     0.5200   PASS
interventional attribution, bond (workbook)                                              -0.4128   book    -0.4130   PASS
interventional attribution, high yield (workbook)                                        -0.2419   book    -0.2420   PASS
interventional attribution, volatility (workbook)                                        -0.1763   book    -0.1760   PASS
interventional attribution, lending (workbook)                                            0.8198   book     0.8200   PASS
interventional attribution, funding (workbook)                                            2.1313   book     2.1310   PASS
conditional attribution, bond                                                             0.0484   book     0.0480   PASS
conditional attribution, high yield                                                       0.0980   book     0.0980   PASS
conditional attribution, volatility                                                       0.0434   book     0.0430   PASS
conditional attribution, lending                                                          0.4615   book     0.4620   PASS
conditional attribution, funding                                                          1.4688   book     1.4690   PASS
interventional attributions: sum (workbook)                                               2.1201   book     2.1200   PASS
conditional attributions: sum (workbook)                                                  2.1201   book     2.1200   PASS
Step 5: the funding spread's interventional credit (workbook)                             2.1313   book     2.1300   PASS
Step 5: the market readings' interventional credit (workbook)                            -0.8310   book    -0.8300   PASS
Step 5: the funding spread's conditional credit                                           1.4688   book     1.4700   PASS
Step 5: the conditional game gives every reading positive credit                          1.0000   book     1.0000   PASS
Proposition 13.3(ii): the first component keeps, % (workbook)                            80.0000   book    80.0000   PASS
Proposition 13.3(iv): the conditional credit of an ignored feature (workbook)             1.0000   book     1.0000   PASS
Table 13.3, None: warned, %                                                               0.0000   book     0.0000   PASS
Table 13.3, None: all clear line                                                        170.0000   book   170.0000   PASS
Table 13.3, None: all clear delay                                                         0.6676   book     0.6700   PASS
Table 13.3, None: promised value (workbook)                                               0.0000   book     0.0000   PASS
Table 13.3, None: fresh breach, % (workbook)                                              4.9070   book     4.9100   PASS
Table 13.3, None: fresh value (workbook)                                                  0.0000   book     0.0000   PASS
Table 13.3, Perfect (Chapter 9): warned, % (workbook)                                    25.9600   book    26.0000   PASS
Table 13.3, Perfect (Chapter 9): all clear line                                           0.0000   book     0.0000   PASS
Table 13.3, Perfect (Chapter 9): all clear delay                                          0.6084   book     0.6100   PASS
Table 13.3, Perfect (Chapter 9): warned line                                            215.0000   book   215.0000   PASS
Table 13.3, Perfect (Chapter 9): warned delay                                             0.5044   book     0.5000   PASS
Table 13.3, Perfect (Chapter 9): promised value (workbook)                                4.1180   book     4.1200   PASS
Table 13.3, Perfect (Chapter 9): fresh breach, % (workbook)                               4.9130   book     4.9100   PASS
Table 13.3, Perfect (Chapter 9): fresh value (workbook)                                   4.1153   book     4.1200   PASS
Table 13.3, Declared law (ceiling): warned, % (workbook)                                 58.9400   book    58.9000   PASS
Table 13.3, Declared law (ceiling): all clear line                                        0.0000   book     0.0000   PASS
Table 13.3, Declared law (ceiling): all clear delay                                       0.6708   book     0.6700   PASS
Table 13.3, Declared law (ceiling): warned line                                         190.0000   book   190.0000   PASS
Table 13.3, Declared law (ceiling): warned delay                                          0.7307   book     0.7300   PASS
Table 13.3, Declared law (ceiling): promised value (workbook)                             1.4369   book     1.4400   PASS
Table 13.3, Declared law (ceiling): fresh breach, % (workbook)                            4.8980   book     4.9000   PASS
Table 13.3, Declared law (ceiling): fresh value (workbook)                                1.4322   book     1.4300   PASS
Table 13.3, Logistic, at 0.5 (law): warned, % (workbook)                                 22.9240   book    22.9000   PASS
Table 13.3, Logistic, at 0.5 (law): all clear line                                      125.0000   book   125.0000   PASS
Table 13.3, Logistic, at 0.5 (law): all clear delay                                       0.7121   book     0.7100   PASS
Table 13.3, Logistic, at 0.5 (law): warned line                                         205.0000   book   205.0000   PASS
Table 13.3, Logistic, at 0.5 (law): warned delay                                          0.6654   book     0.6700   PASS
Table 13.3, Logistic, at 0.5 (law): promised value (workbook)                             0.5273   book     0.5300   PASS
Table 13.3, Logistic, at 0.5 (law): fresh breach, % (workbook)                            4.8980   book     4.9000   PASS
Table 13.3, Logistic, at 0.5 (law): fresh value (workbook)                                0.5291   book     0.5300   PASS
Table 13.3, Logistic, cheapest (law): warned, % (workbook)                               58.6460   book    58.6000   PASS
Table 13.3, Logistic, cheapest (law): all clear line                                      0.0000   book     0.0000   PASS
Table 13.3, Logistic, cheapest (law): all clear delay                                     0.6687   book     0.6700   PASS
Table 13.3, Logistic, cheapest (law): warned line                                       190.0000   book   190.0000   PASS
Table 13.3, Logistic, cheapest (law): warned delay                                        0.7305   book     0.7300   PASS
Table 13.3, Logistic, cheapest (law): promised value (workbook)                           1.4365   book     1.4400   PASS
Table 13.3, Logistic, cheapest (law): fresh breach, % (workbook)                          4.9280   book     4.9300   PASS
Table 13.3, Logistic, cheapest (law): fresh value (workbook)                              1.4584   book     1.4600   PASS
Table 13.3, Logistic, held-out: warned, % (workbook)                                     60.8910   book    60.9000   PASS
Table 13.3, Logistic, held-out: all clear line                                            0.0000   book     0.0000   PASS
Table 13.3, Logistic, held-out: all clear delay                                           0.6830   book     0.6800   PASS
Table 13.3, Logistic, held-out: warned line                                             185.0000   book   185.0000   PASS
Table 13.3, Logistic, held-out: warned delay                                              0.6768   book     0.6800   PASS
Table 13.3, Logistic, held-out: promised value (workbook)                                 1.7315   book     1.7300   PASS
Table 13.3, Logistic, held-out: fresh breach, % (workbook)                                5.0190   book     5.0200   PASS
Table 13.3, Logistic, held-out: fresh value (workbook)                                    1.6275   book     1.6300   PASS
Table 13.3, Logistic, certified: warned, % (workbook)                                    60.8910   book    60.9000   PASS
Table 13.3, Logistic, certified: all clear line                                           0.0000   book     0.0000   PASS
Table 13.3, Logistic, certified: all clear delay                                          0.7340   book     0.7300   PASS
Table 13.3, Logistic, certified: warned line                                            190.0000   book   190.0000   PASS
Table 13.3, Logistic, certified: warned delay                                             0.7419   book     0.7400   PASS
Table 13.3, Logistic, certified: promised value (workbook)                                1.1683   book     1.1700   PASS
Table 13.3, Logistic, certified: fresh breach, % (workbook)                               4.7260   book     4.7300   PASS
Table 13.3, Logistic, certified: fresh value (workbook)                                   1.0593   book     1.0600   PASS
Table 13.3, Boosting, trusted: warned, % (workbook)                                      35.7200   book    35.7000   PASS
Table 13.3, Boosting, trusted: all clear line                                             0.0000   book     0.0000   PASS
Table 13.3, Boosting, trusted: all clear delay                                            0.6084   book     0.6100   PASS
Table 13.3, Boosting, trusted: warned line                                              215.0000   book   215.0000   PASS
Table 13.3, Boosting, trusted: warned delay                                               0.5132   book     0.5100   PASS
Table 13.3, Boosting, trusted: promised value (workbook)                                  4.0377   book     4.0400   PASS
Table 13.3, Boosting, trusted: fresh breach, % (workbook)                                 7.6760   book     7.6800   PASS
Table 13.3, Boosting, held-out: warned, % (workbook)                                     68.3090   book    68.3000   PASS
Table 13.3, Boosting, held-out: all clear line                                            0.0000   book     0.0000   PASS
Table 13.3, Boosting, held-out: all clear delay                                           0.7574   book     0.7600   PASS
Table 13.3, Boosting, held-out: warned line                                             190.0000   book   190.0000   PASS
Table 13.3, Boosting, held-out: warned delay                                              0.7385   book     0.7400   PASS
Table 13.3, Boosting, held-out: promised value (workbook)                                 1.1629   book     1.1600   PASS
Table 13.3, Boosting, held-out: fresh breach, % (workbook)                                4.9600   book     4.9600   PASS
Table 13.3, Boosting, held-out: fresh value (workbook)                                    0.5908   book     0.5900   PASS
Table 13.2: accuracy at 0.5, %                                                           87.3240   book    87.3000   PASS
Table 13.2: accuracy where cheapest, %                                                   65.8620   book    65.9000   PASS
Table 13.2: AUC                                                                           0.9240   book     0.9240   PASS
Table 13.2: decision value at most                                                        1.4369   book     1.4400   PASS
Table 13.2: fresh breach, certified, %                                                    4.7260   book     4.7300   PASS
abstract: a perfect warning                                                               4.1180   book     4.1200   PASS
abstract: the five readings allow at most                                                 1.4369   book     1.4400   PASS
abstract: the certified warning is worth                                                  1.0593   book     1.0600   PASS
the board's take-away: the ceiling is about a third of a perfect warning                  1.0000   book     1.0000   PASS
the board's take-away: trusted boosting promises nearly                                   4.0377   book     4.0000   PASS
the board's take-away and Key Takeaways: certified, about                                 1.0593   book     1.0000   PASS
LOS 13.2: boosting on the record                                                          1.0000   book     1.0000   PASS
LOS 13.2: boosting fresh                                                                  0.8883   book     0.8900   PASS
LOS 13.3: at accuracy's threshold                                                         0.5273   book     0.5300   PASS
LOS 13.3: at a 4.1% threshold                                                             1.4365   book     1.4400   PASS
LOS 13.4: held-out designs keep 5%, % of records                                         47.5000   book    47.5000   PASS
LOS 13.4: certified designs keep 5%, % of records                                        98.0000   book    98.0000   PASS
LOS 13.4: at an average value of                                                          0.9502   book     0.9500   PASS
LOS 13.5: the first component's value                                                     0.1409   book     0.1400   PASS
LOS 13.5: the funding spread alone keeps, %                                              30.1408   book    30.0000   PASS

376 of 376 checkpoints PASS

Next. This chapter let data supply one declared part of Meridian's model, a learned warning of funding-window closures, judged it on data it had not seen, and left the board's commitments to the optimizer. Chapter 14, Artificial Intelligence for Enterprise Optimization, continues Part IV. AXIOM-13 runs the same instance interactively (Lab 2.13); this notebook and the workbook are its reproducible record.

Dynamic Corporate Transformation — Volume I: Foundations and Mathematical Framework; Volume II: Enterprise Optimization & Applications
Jump to any chapter

Volume I: Foundations and Mathematical Framework

  1. Ch. 1Introduction to Dynamic Corporate Transformation
  2. Ch. 2Enterprise Systems and Transformation
  3. Ch. 3Mathematical Foundations
  4. Ch. 4Enterprise Modeling Principles
  5. Ch. 5Enterprise State Representation
  6. Ch. 6Enterprise Transformation Operators
  7. Ch. 7Dynamic Enterprise Systems
  8. Ch. 8Stochastic Enterprise Dynamics
  9. Ch. 9Enterprise State Architecture
  10. Ch. 10Enterprise Capital Architecture
  11. Ch. 11Enterprise Performance Architecture
  12. Ch. 12Enterprise Risk, Resilience, and Robustness Architecture
  13. Ch. 13Unified Enterprise Transformation Architecture
  14. Ch. 14Mathematical Analysis of the Unified Enterprise Transformation Architecture
  15. Ch. 15Mathematical Formulation of Enterprise Optimization
  16. Ch. 16Synthesis, Future Directions, and Transition to Volume II

Volume II: Enterprise Optimization & Applications

  1. Ch. 1Introduction to Enterprise Optimization
  2. Ch. 2The General Enterprise Optimization Problem Revisited
  3. Ch. 3Convex Enterprise Optimization
  4. Ch. 4Nonlinear and Mixed-Integer Enterprise Optimization
  5. Ch. 5Dynamic Enterprise Optimization
  6. Ch. 6Optimal Control of Enterprise Systems
  7. Ch. 7Dynamic Programming for Enterprise Systems
  8. Ch. 8Hamilton–Jacobi–Bellman Enterprise Framework
  9. Ch. 9Stochastic Enterprise Optimization
  10. Ch. 10Neuro-Fuzzy Robust Enterprise Optimization
  11. Ch. 11Distributionally Robust Enterprise Optimization
  12. Ch. 12Multi-Objective Enterprise Optimization
  13. Ch. 13Machine Learning for Enterprise Transformation
  14. Ch. 14Artificial Intelligence for Enterprise Optimization
  15. Ch. 15Enterprise Digital Twins and Autonomous Enterprise Transformation
  16. Ch. 16Enterprise Applications and Integrated Transformation Case Studies

Dynamic Corporate Transformation

Springer Nature · Management for Professionals · Forthcoming 2027

  • Volume I — Foundations and Mathematical Framework
  • Volume II — Enterprise Optimization & Applications
Buy Volume I: Dynamic Corporate Transformation: Foundations and Mathematical FrameworkBuy Volume II: Dynamic Corporate Transformation: Enterprise Optimization & Applications
Jump to any chapter

Volume I: Foundations and Mathematical Framework

  1. Ch. 1Introduction to Dynamic Corporate Transformation
  2. Ch. 2Enterprise Systems and Transformation
  3. Ch. 3Mathematical Foundations
  4. Ch. 4Enterprise Modeling Principles
  5. Ch. 5Enterprise State Representation
  6. Ch. 6Enterprise Transformation Operators
  7. Ch. 7Dynamic Enterprise Systems
  8. Ch. 8Stochastic Enterprise Dynamics
  9. Ch. 9Enterprise State Architecture
  10. Ch. 10Enterprise Capital Architecture
  11. Ch. 11Enterprise Performance Architecture
  12. Ch. 12Enterprise Risk, Resilience, and Robustness Architecture
  13. Ch. 13Unified Enterprise Transformation Architecture
  14. Ch. 14Mathematical Analysis of the Unified Enterprise Transformation Architecture
  15. Ch. 15Mathematical Formulation of Enterprise Optimization
  16. Ch. 16Synthesis, Future Directions, and Transition to Volume II

Volume II: Enterprise Optimization & Applications

  1. Ch. 1Introduction to Enterprise Optimization
  2. Ch. 2The General Enterprise Optimization Problem Revisited
  3. Ch. 3Convex Enterprise Optimization
  4. Ch. 4Nonlinear and Mixed-Integer Enterprise Optimization
  5. Ch. 5Dynamic Enterprise Optimization
  6. Ch. 6Optimal Control of Enterprise Systems
  7. Ch. 7Dynamic Programming for Enterprise Systems
  8. Ch. 8Hamilton–Jacobi–Bellman Enterprise Framework
  9. Ch. 9Stochastic Enterprise Optimization
  10. Ch. 10Neuro-Fuzzy Robust Enterprise Optimization
  11. Ch. 11Distributionally Robust Enterprise Optimization
  12. Ch. 12Multi-Objective Enterprise Optimization
  13. Ch. 13Machine Learning for Enterprise Transformation
  14. Ch. 14Artificial Intelligence for Enterprise Optimization
  15. Ch. 15Enterprise Digital Twins and Autonomous Enterprise Transformation
  16. Ch. 16Enterprise Applications and Integrated Transformation Case Studies