{ "cells": [ { "cell_type": "markdown", "id": "6da4fc77", "metadata": {}, "source": [ "# DCT Laboratory — Volume II, Chapter 13\n", "## Machine Learning for Enterprise Transformation\n", "**Seed `26213`** · Companion to the chapter and AXIOM Module **AXIOM-13 (Vol. II)**\n", "\n", "Machine learning with the receipts shown. A **linear fit versus a 1-NN\n", "memorizer** on the same fixed dataset: the memorizer wins training\n", "(0.00 vs 0.48) and loses the test (0.78 vs 0.40) — the Generalization Error\n", "Theorem in four numbers. **k-means** converging in two sweeps. And the fitted\n", "model **piped into a decision**, where the regret from prediction error turns\n", "out to have a closed form: $(d - \\hat{d})^2/4$ — the decision layer forgives\n", "small errors quadratically. Mirrored in `DCT_V2_Ch13_Lab.xlsx`." ] }, { "cell_type": "code", "execution_count": null, "id": "17088f34", "metadata": { "execution": { "iopub.execute_input": "2026-07-14T00:52:22.020270Z", "iopub.status.busy": "2026-07-14T00:52:22.020080Z", "iopub.status.idle": "2026-07-14T00:52:22.493511Z", "shell.execute_reply": "2026-07-14T00:52:22.492268Z" } }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "plt.rcParams['figure.dpi']=110\n", "\n", "import numpy as np\n", "SEED = 26213\n", "rng = np.random.default_rng([1, SEED]) # substream 1: a draw where generalization behaves typically\n", "# --- fixed dataset: true f(x) = 2 + 0.8x, noise sigma 0.6, rounded 4dp = canonical ---\n", "XTR = np.array([0.5, 1.5, 2.5, 3.5, 4.5, 5.5, 6.5, 7.5])\n", "XTE = np.array([1.0, 3.0, 5.0, 7.0])\n", "def _gen():\n", " ytr = np.round(2 + 0.8*XTR + rng.normal(0, 0.6, len(XTR)), 4)\n", " yte = np.round(2 + 0.8*XTE + rng.normal(0, 0.6, len(XTE)), 4)\n", " return ytr, yte\n", "YTR, YTE = _gen()\n", "def ols():\n", " b = np.cov(XTR, YTR, bias=True)[0,1]/np.var(XTR)\n", " a = YTR.mean() - b*XTR.mean()\n", " return a, b\n", "def rmse(yhat, y): return float(np.sqrt(np.mean((np.asarray(yhat)-y)**2)))\n", "def lin_pred(x):\n", " a, b = ols(); return a + b*np.asarray(x)\n", "def nn_pred(x):\n", " return np.array([YTR[np.argmin(np.abs(XTR-xi))] for xi in np.atleast_1d(x)])\n", "# --- k-means, k=2, 1D, fixed init ---\n", "KX = np.array([1.0, 1.5, 2.0, 2.5, 7.0, 7.5, 8.0, 9.0])\n", "def kmeans():\n", " c1, c2 = 0.0, 10.0; iters = 0\n", " while True:\n", " iters += 1\n", " a1 = KX[np.abs(KX-c1) < np.abs(KX-c2)]; a2 = KX[np.abs(KX-c1) >= np.abs(KX-c2)]\n", " n1, n2 = a1.mean(), a2.mean()\n", " if abs(n1-c1) < 1e-12 and abs(n2-c2) < 1e-12: break\n", " c1, c2 = n1, n2\n", " wss = float(((a1-c1)**2).sum() + ((a2-c2)**2).sum())\n", " return c1, c2, wss, iters, len(a1), len(a2)\n", "# --- prediction -> optimization: maximize R(i) = d*sqrt(i) - i => i* = d^2/4 ---\n", "X_DECIDE, D_TRUE = 6.0, 2 + 0.8*6.0\n", "def regret():\n", " dhat = float(lin_pred(X_DECIDE))\n", " i_hat = dhat**2/4\n", " R_hat_under_truth = D_TRUE*np.sqrt(i_hat) - i_hat\n", " R_opt = D_TRUE**2/4\n", " return dhat, i_hat, float(R_hat_under_truth), float(R_opt), float(R_opt - R_hat_under_truth)\n", "\n", "def reference_values():\n", " a, b = ols()\n", " c1, c2, wss, it, n1, n2 = kmeans()\n", " dhat, ih, Rh, Ro, reg = regret()\n", " return {\n", " \"intercept_hat\": round(a,4), \"slope_hat\": round(b,4),\n", " \"train_rmse_linear\": round(rmse(lin_pred(XTR), YTR),4),\n", " \"test_rmse_linear\": round(rmse(lin_pred(XTE), YTE),4),\n", " \"train_rmse_1nn\": round(rmse(nn_pred(XTR), YTR),4),\n", " \"test_rmse_1nn\": round(rmse(nn_pred(XTE), YTE),4),\n", " \"c1_final\": round(c1,4), \"c2_final\": round(c2,4),\n", " \"wss\": round(wss,4), \"kmeans_iters\": it,\n", " \"dhat_x6\": round(dhat,4), \"i_star_hat\": round(ih,4),\n", " \"R_under_truth\": round(Rh,4), \"R_opt_true\": round(Ro,4),\n", " \"decision_regret\": round(reg,4),\n", " \"regret_closed_form\": round((D_TRUE-dhat)**2/4,4), # regret = (prediction error)^2 / 4, exactly\n", " }\n", "if __name__ == \"__main__\":\n", " print(\"YTR:\", list(YTR)); print(\"YTE:\", list(YTE))\n", " [print(f\"{k:20s} {v}\") for k,v in reference_values().items()]" ] }, { "cell_type": "markdown", "id": "6bcad00a", "metadata": {}, "source": [ "## Panel 1 — Generalization: the memorizer's bargain\n", "Truth $f(x) = 2 + 0.8x$ plus noise; 8 training points, 4 held out. Two\n", "learners: **OLS** (2 parameters) and **1-nearest-neighbor** (memorizes all 8).\n", "The memorizer's training RMSE is 0.0000 by construction; on held-out data it\n", "pays **0.7817** against the line's **0.4006**. The Generalization Error\n", "Theorem's content: training error is an optimistic, capacity-inflated estimate\n", "— Enterprise Learning Consistency comes from restricting capacity to match the\n", "signal, and Feature Quality Determines Model Quality (Prop.) decides what the\n", "line can see." ] }, { "cell_type": "code", "execution_count": null, "id": "42ae6c10", "metadata": { "execution": { "iopub.execute_input": "2026-07-14T00:52:22.496134Z", "iopub.status.busy": "2026-07-14T00:52:22.495289Z", "iopub.status.idle": "2026-07-14T00:52:22.738753Z", "shell.execute_reply": "2026-07-14T00:52:22.736799Z" } }, "outputs": [], "source": [ "a, b = ols()\n", "xs = np.linspace(0, 8, 200)\n", "fig, ax = plt.subplots(figsize=(7.8,4.4))\n", "ax.plot(xs, 2+0.8*xs, \"--\", c=\"#8A8F8B\", lw=1.8, label=\"truth 2 + 0.8x\")\n", "ax.plot(xs, a+b*xs, c=\"#C8A24B\", lw=2.4, label=f\"OLS: {a:.3f} + {b:.3f}x\")\n", "ax.step(np.sort(XTR), YTR[np.argsort(XTR)], where=\"mid\", c=\"#1B6B52\", lw=1.6, alpha=.8, label=\"1-NN memorizer\")\n", "ax.scatter(XTR, YTR, c=\"#0B3D2E\", s=42, zorder=5, label=\"train\")\n", "ax.scatter(XTE, YTE, c=\"#B0532F\", s=52, marker=\"s\", zorder=5, label=\"held-out test\")\n", "ax.set(xlabel=\"x\", ylabel=\"y\", title=\"Two learners, one dataset — seed 26213\")\n", "ax.legend(frameon=False, fontsize=9); ax.grid(alpha=.25); plt.tight_layout(); plt.show()\n", "print(f\"linear: train {rmse(lin_pred(XTR),YTR):.4f} test {rmse(lin_pred(XTE),YTE):.4f}\")\n", "print(f\"1-NN: train {rmse(nn_pred(XTR),YTR):.4f} test {rmse(nn_pred(XTE),YTE):.4f}\")" ] }, { "cell_type": "markdown", "id": "cf6b35ce", "metadata": {}, "source": [ "## Panel 2 — Unsupervised: two sweeps to structure\n", "Eight revenue-per-account figures, no labels (Enterprise Clustering, Def.).\n", "k-means from deliberately terrible centroids $(0, 10)$: the first sweep\n", "assigns 4-and-4 and moves the centroids to $(1.75, 7.875)$; the second changes\n", "nothing — **converged in 2 iterations**, within-cluster SS $3.4375$. The\n", "structure was in the data; the algorithm only had to stop denying it." ] }, { "cell_type": "code", "execution_count": null, "id": "425a2ae6", "metadata": { "execution": { "iopub.execute_input": "2026-07-14T00:52:22.740515Z", "iopub.status.busy": "2026-07-14T00:52:22.740319Z", "iopub.status.idle": "2026-07-14T00:52:22.888909Z", "shell.execute_reply": "2026-07-14T00:52:22.887635Z" } }, "outputs": [], "source": [ "c1, c2, wss, it, n1, n2 = kmeans()\n", "fig, ax = plt.subplots(figsize=(7.8,2.8))\n", "ax.scatter(KX[:4], [0]*4, s=90, c=\"#1B6B52\", label=f\"cluster 1 (n={n1})\")\n", "ax.scatter(KX[4:], [0]*4, s=90, c=\"#C8A24B\", label=f\"cluster 2 (n={n2})\")\n", "ax.scatter([c1, c2], [0, 0], s=260, c=\"#0B3D2E\", marker=\"X\", zorder=5, label=\"final centroids\")\n", "ax.set(xlabel=\"revenue per account\", yticks=[], title=f\"Converged in {it} sweeps, WSS = {wss:.4f} — seed 26213\")\n", "ax.legend(frameon=False, fontsize=9, loc=\"upper center\", ncols=3); plt.tight_layout(); plt.show()\n", "print(f\"centroids: ({c1:.4f}, {c2:.4f}) iterations: {it} WSS: {wss:.4f}\")" ] }, { "cell_type": "markdown", "id": "6612c2c0", "metadata": {}, "source": [ "## Panel 3 — Prediction feeds optimization; regret has a closed form\n", "The Prediction–Optimization Integration Theorem, executed: predict demand at\n", "$x = 6$ with the fitted line ($\\hat{d} = 6.6852$ against truth $6.8$), then\n", "optimize $R(i) = d\\sqrt{i} - i$, whose optimum is $i^* = d^2/4$. Acting on\n", "$\\hat{d}$ but living under $d$: the value shortfall works out to **regret\n", "$= (d - \\hat{d})^2/4$ exactly** — quadratic in prediction error. Small errors\n", "are nearly free (here 0.0033 on an 11.56 optimum), because optima are flat on\n", "top: Enterprise Optimization and Enterprise Learning Are Complementary\n", "Processes (Prop.), with the optimization layer supplying the forgiveness." ] }, { "cell_type": "code", "execution_count": null, "id": "d9d362c8", "metadata": { "execution": { "iopub.execute_input": "2026-07-14T00:52:22.890928Z", "iopub.status.busy": "2026-07-14T00:52:22.890729Z", "iopub.status.idle": "2026-07-14T00:52:23.092354Z", "shell.execute_reply": "2026-07-14T00:52:23.090933Z" } }, "outputs": [], "source": [ "dhat, ih, Rh, Ro, reg = regret()\n", "derrs = np.linspace(-2, 2, 200)\n", "fig, ax = plt.subplots(figsize=(7.8,3.8))\n", "ax.plot(derrs, derrs**2/4, c=\"#C8A24B\", lw=2.4, label=\"regret = (d − dhat)²/4\")\n", "ax.scatter([D_TRUE-dhat], [reg], c=\"#0B3D2E\", s=80, zorder=5, label=f\"this fit: err {D_TRUE-dhat:+.4f} → regret {reg:.4f}\")\n", "ax.set(xlabel=\"prediction error d − dhat\", ylabel=\"decision regret\", title=\"The decision layer forgives quadratically — seed 26213\")\n", "ax.legend(frameon=False, fontsize=9); ax.grid(alpha=.25); plt.tight_layout(); plt.show()\n", "print(f\"dhat(6) = {dhat:.4f} i* = {ih:.4f} R under truth = {Rh:.4f} R optimal = {Ro:.4f}\")\n", "print(f\"regret = {reg:.4f} closed form (d-dhat)^2/4 = {(D_TRUE-dhat)**2/4:.4f}\")" ] }, { "cell_type": "markdown", "id": "3bf3daa1", "metadata": {}, "source": [ "## Validation — agrees with `DCT_V2_Ch13_Lab.xlsx`" ] }, { "cell_type": "code", "execution_count": null, "id": "6866b3b4", "metadata": { "execution": { "iopub.execute_input": "2026-07-14T00:52:23.094341Z", "iopub.status.busy": "2026-07-14T00:52:23.094143Z", "iopub.status.idle": "2026-07-14T00:52:23.101978Z", "shell.execute_reply": "2026-07-14T00:52:23.100934Z" } }, "outputs": [], "source": [ "ref = reference_values()\n", "expected = {\"intercept_hat\":1.6233,\"slope_hat\":0.8436,\"train_rmse_linear\":0.4803,\n", " \"test_rmse_linear\":0.4006,\"train_rmse_1nn\":0.0,\"test_rmse_1nn\":0.7817,\n", " \"c1_final\":1.75,\"c2_final\":7.875,\"wss\":3.4375,\"kmeans_iters\":2,\n", " \"dhat_x6\":6.6852,\"i_star_hat\":11.1731,\"R_under_truth\":11.5567,\"R_opt_true\":11.56,\n", " \"decision_regret\":0.0033,\"regret_closed_form\":0.0033}\n", "for k,v in expected.items():\n", " assert abs(ref[k]-v)<5e-4, f\"MISMATCH {k}\"\n", " print(f\"PASS {k:20s} {ref[k]}\")\n", "print(\"\\nAll checkpoints agree — seed 26213.\")" ] }, { "cell_type": "markdown", "id": "7bea473a", "metadata": {}, "source": [ "**Next**: Exercises 13.5–13.9 (Part C) grow the training set and watch both test errors fall at different rates; AXIOM-13's learning bench animates the fit as points arrive. Chapter 14 sends the learner into the loop: reinforcement learning. Solutions: IM Vol. II, Ch. 13." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }