Every number on this page is computed by the Volume II Chapter 14 notebook's own code with the book's seeds (Online Appendix Remark 14.B.1); every learned rule is priced exactly under the declared law. Your notes stay on this page until you leave it; copy the memo before you go.
Digital Solutions' AI team proposes an agent that learns Meridian's hiring rule by acting. The chief human resources officer and the audit committee must decide whether, where, and within what limits it may learn. You support the team preparing the answer.
Use AXIOM to train the agent in a simulator, price what exploration costs, learn simulators from records, and test learning on the company with and without the board's guardrail.
Worked Example 14.1: the agent reads capability in quarter-points, from 30 to 80 in 201 levels, and hires 0 to 2.5 points a quarter in quarter-point steps. It knows the declared costs, $4 million a point hired and $1.6 million a point-quarter short of 55, and the discount, but not the law of the swings. It learns by trial, takes a random decision one time in ten, uses the step size 1/(1 + (1 − β)n) after n trials of a pair, and starts untried decisions at zero. The board's guardrail: never plan below 55 unless hiring at the cap.
In the model panel, solve the declared program by policy iteration; record the best rule, its cost from 58, and the decisions the guardrail keeps.
Bars: the long-run share of quarters the best rule spends at each capability, from 58. The guardrail lets the agent hire anything from its lower edge to the cap (and only the cap where planning 55 is out of reach).
In the learning bench, run Q-learning in the simulator to a million quarters with several step sizes, and price each learned rule exactly.
In the exploration bench, run greedy, randomized, and optimistic choice among the target rules; compare their regret and how often they end on the best target.
In the records panel, learn the swings from records of 12, 40, and 120 quarters, and fit the dynamics to a record kept under Volume I's rule; price each simulator's best rule.
In the live panel, run the agent for 40 quarters with and without the guardrail, and compare its cost and its quarters below the floor with the worst rule inside the guardrail.
The ten agents' rules after two million simulated quarters, priced exactly; the guardrail checked at every capability; and policy iteration under the declared model started from each.
Excess: cost from 58 above the best rule's $231.18 million ($ million), the mean (* median) and the 90th percentile († the largest of 20 learners); below floor: share of quarters that start below 55, in the long run (* median) or over the 40 quarters on the company.
Slow convergence, and a rule good where the company goes while its values are wrong elsewhere (Theorem 14.1); greedy learners that stop exploring (Theorem 14.2(ii)); regret that grows like a logarithm for the optimistic rule (Theorem 14.2(iii)); a cheap simulator from the swings and a misleading one from practice (Theorem 14.3); and the guardrail's bound (Proposition 14.2(i)).
Whether and where the agent may learn, the guardrail and what it costs, the certificate the audit committee requires, and the records the engineering office should keep.