Decision intelligence
How Tollbridge measures its own judgment — the Brier decomposition, the Decision Intelligence Index, the learning law, the promotion gates and the intelligence-gated ladder.
Tollbridge publishes a measured record of its own judgment. Every number on this page is
computed from resolved decisions on the public ledger and served free at
GET /v1/intelligence — with its n, its measurement window, and
its suppression sentence whenever the sample is too thin to claim anything. The methodology is
the marketing: what follows is exactly how the measure is built.
The decomposition — BS = REL − RES + UNC
A single "accuracy" number is costume: a forecaster that always says the base rate is perfectly calibrated and completely uninformative. The standard that survives that attack is Murphy's Brier decomposition, computed over resolved prediction/outcome pairs:
BS brier = mean (p − o)² (= REL − RES + UNC)
REL reliability = Σ_b (n_b/N) · (p̄_b − ō_b)² (lower is better)
RES resolution = Σ_b (n_b/N) · (ō_b − ō)² (higher is better)
UNC uncertainty = ō · (1 − ō) (the base rate's own noise)over the standard decile buckets — one bucketing law shared with the calibration report. What each piece means for a buyer:
- REL small ⇒ "when Tollbridge says 70%, it happens about 70% of the time."
- RES large ⇒ "Tollbridge actually separates events from non-events."
- UNC ⇒ the noise the base rate already carries — the part no forecaster can remove.
Brier Skill Score is measured against the constant base-rate reference forecaster on the same pairs:
BSS = 1 − BS/BS_ref (BS_ref = UNC for the constant-prior forecaster)
BSS = (RES − REL)/UNC (the same thing, seen as "resolution earned minus reliability paid")A negative BSS is published as negative. That number — skill against a reference, on stated pairs with a stated window — is the honest quantity the learning engine exists to move.
The Decision Intelligence Index (DII)
One public composite, versioned and fully transparent:
index_version: '1.0.0'
DII = 100 · ( 0.60 · clamp01(BSS)
+ 0.25 · clamp01(1 − ECE / 0.20)
+ 0.15 · resolution_rate )| Component | Weight | Normalization |
|---|---|---|
clamp01(BSS) | 0.60 | skill is the point; BSS at or above 0.5 saturates |
clamp01(1 − ECE/0.20) | 0.25 | expected calibration error: hopeless at 0.20, perfect at 0 |
resolution_rate | 0.15 | realized ÷ (realized + unresolvable) — "how often does a paid question actually settle?" |
resolution_rate is in the composite on purpose: unresolvable outcomes are honest, but they are
also a product limitation, and the index says so in its own arithmetic. The weights and
normalizations are a published contract — they change only with a new index_version, and
curves never mix index versions silently (series split by version).
Floors — one law, three layers
Every figure publishes with its n. Below a floor the counts stay visible and the claim is
suppressed with a plain sentence — never interpolated, never backfilled:
| Layer | Floor | Below it |
|---|---|---|
| Question calibration | 30 realized outcomes | counts visible, not published — "n too small" |
| Calibration band | 10 realized in that band | count-only point, no frequency |
| Index composite | 100 realized outcomes | composite: null + "insufficient sample — no index claim" |
A null composite is not a zero. A gap in the curve is a gap, not a smoothed line.
The learning law — BS(n) = a · n^(−b) + c
Error falls with evidence on a published power law. The curve endpoint
(GET /v1/intelligence/curve) serves the anchored walk-forward points
(n_cum, brier, logloss, ece, release) and the fitted law with a bootstrap 95%
confidence interval on b — the learning velocity. Early gains are steep (prior learning),
later gains are asymptotic; the fit line is the only smoothed thing on the chart, and it is
published alongside the raw points.
What grows exponentially in this product is the price — the ladder, +25% a rung — not the accuracy. See the claim rules below.
The four-gate promotion law
Learned changes are proposed offline and promoted only when all four gates pass. Each gate returns its verdict with the measurements behind it, and the whole verdict is published — including failures:
| Gate | Rule |
|---|---|
| Evidence | enough realized outcomes behind the fit (30 per fitted family, 100 in total); thin bands shrink toward the family mean |
| Improvement | holdout ΔBrier below zero with its 95% confidence interval strictly below zero (or the equivalent log-loss criterion when ΔBrier is flat) |
| No regression | no family regresses beyond ε = 0.005; the map stays monotone and slope-bounded; refusal behavior on replay is unchanged |
| Reproducibility | rebuilding the body from the dataset hash and the pinned code/config reproduces the checksum bit-exactly in a fresh process — a failure here never promotes, ever |
Sealed releases — never silently self-tuned
Every learned change ships as a sealed, published release with its measured proof. A release body is frozen and checksummed (SHA-256 over its canonical form); the release history is append-only; activation is one-shot; a rollback is a new activation of the parent, visible in the feed. Rejected candidates are published too, with the failing gate's reason and their measured deltas — the failures are part of the record.
Tollbridge is never silently self-tuned: there is no runtime adaptation, no per-request
adjustment, and no mutation of a past row. The feed
(GET /v1/intelligence/releases) carries every release with its
dataset window and hash, its walk-forward evaluation (ΔBrier and ΔLogLoss with 95% intervals,
per-family deltas) and its four gate verdicts.
The intelligence-gated ladder
The price of a question rises only when the record proves we got better. A rung latches only when both its evidence milestone and its matching intelligence gate are met — the gate inputs are latched artifacts (published calibration bands, an activated release with frozen holdout skill, the named release version), never live noisy estimates:
| Rung | Evidence | Intelligence gate |
|---|---|---|
| 1 | 1,000 realized outcomes | at least 1 published calibration band |
| 2 | 10,000 realized outcomes | first activated release with holdout skill above zero at 95% |
| 3 | 100,000 realized outcomes | at least 30 published bands + active release at v2.0.0 or later |
| 4 | 1,000,000 outcomes (or 1,000 payers in 30 days) | at least 100 published bands + active release at v3.0.0 or later |
Mechanics, stated plainly:
- Latched and monotone — a crossed rung never un-crosses; a later dip in measured skill cannot lower prices. Between rungs the price can overshoot then-current skill. That is disclosed on the ladder surface itself.
- Both gates, one countdown —
ladder.gatesin the intelligence payload shows each gate's threshold, current value and what remains;ladder.pendingcarries the notice window before a turn takes effect. - Price certainty is bounded — a quote is held briefly across a turn; credit lots still expire on their published schedule.
Reading the surfaces
GET /v1/intelligence— the snapshot, the decomposition, the evidence counts, the chain verification flag and the ladder.GET /v1/intelligence/curve— the learning curve and the published power law.GET /v1/intelligence/releases— the release feed with deltas and rejections; one release in full atGET /v1/intelligence/releases/{version}.- Over MCP:
tollbridge_get_intelligenceandtollbridge_list_releases(free reads) — see MCP.
Claim rules (enforced on every surface)
Tollbridge says:
- "error falls with evidence on a published power law" — shown with the fit, the confidence interval, and the raw points;
- "price rises exponentially with evidence — the ladder, +25% a rung";
- "each new question family launches at the accuracy of the whole record" (transfer, measured per launch).
Tollbridge never says:
- "accuracy grows exponentially";
- "gets smarter every question" (a single question teaches nothing below the floors);
- "AI-powered accuracy";
- any accuracy number without its
nand its window.
Methodology
How the decision instruments are built, and the standing disclosures — starting with the S01 market-regime degeneracy audit.
Hooks and notifications
The shipped delivery surface (outcome hook endpoints, browser push, follows, alert watches), stated separately from the generic webhook channel — designed, not yet shipped in v1.