Track record

We audited our own record.

We published 15 sealed predictions and checked all 15 against primary reporting. Eight were unusable, and we are voiding them in public. Four are misses. Then we found a fault in our own audit, and corrected it.

Last updated

Corrected on 23 September 2026: the first version of this audit judged some criteria on their first clause only, and every affected prediction has been re-judged against its full text. The corrected record of 15 sealed predictions: 8 voided, 4 misses, 1 weak hit, 2 still open. Voiding 8 of 15 and publishing 4 misses is worth more than a hit rate nobody can check.

Corrected on 23 September 2026

The first version of this audit, published on 16 September, was run from truncated criteria: our research brief passed only the first clause of several multi-clause criteria, so some were judged on their first clause alone. On 23 September every affected prediction was re-judged against its full text, clause by clause, with the sources opened. Five dispositions changed.

  • T-Mobile force-migrates 8M+ — still a miss, now for a precise reason. Our production system auto-resolved it as a hit on 20 September, citing an FCC complaint. That was an informal complaint, not the formal regulatory complaint the criterion required, and no class action was filed in the window. The production verdict was wrong, and it is reversed.
  • USPS — miss confirmed. The criterion's third clause, a USPS statement addressing affordability, was checked: USPS's only affordability wording was its routine rate-filing line of 9 April, published before implementation and not in response to pushback.
  • Verizon 'Simplicity' — moved from voided to a miss. Neither clause occurred: T-Mobile's "Experience 2.0" added financing at unchanged prices and did not respond to Verizon.
  • Xbox console price increase — moved from voided to a miss. No reversal, no compensatory bundle (the only new console was an $899.99 collector's edition), and no Microsoft response to sales data in the window.
  • Flock Safety — removed from the voids. Its second condition, a municipality acting after the 13 August announcement, could not have been met when it was sealed. It is open until 15 October and already tracking as met: Hopkinton terminated its contract on 20 August, and Tempe, Eastpointe and Chesterton followed.

The corrected record: 15 sealed, 8 voided, 4 misses, 1 weak hit, 2 still open. Everything below states the corrected dispositions.

The verdict, and every prediction

We audited all 15 forward predictions on our public record against primary reporting: source URLs were opened, not inferred. Eight are voided with a published reason, four are published as misses, one stands as a weak hit, and two are still open.

Voided predictions stay on the record with their reason. They are excluded from any count of accuracy, but they are never hidden, because publishing them is the point. The reason column below is why the disposition was reached, not a summary of the prediction.

The 15 sealed predictions and their dispositions, as corrected on 23 September 2026.
PredictionSealedDispositionReason
Instagram Plus subscription launch11 June 2026VoidedMeta never breaks out per-app subscribers, so the criterion could never be settled from public sources.
YouTube Premium increase to $15.9911 June 2026VoidedGoogle publishes no per-tier subscriber count, so the criterion could never be settled from public sources.
Verizon 'Simplicity' flat-rate plan23 June 2026MissNeither clause occurred: T-Mobile's "Experience 2.0" added financing at unchanged prices and did not respond to Verizon. First judged a void from a truncated criterion; corrected on 23 September.
Apple price increases on select products23 June 2026Hit, weakApple announced on 25 June, two days after sealing — but the increase had already been reported as planned, so the criterion was near-certain when it was sealed.
Kroger loyalty rewards to grocery discounts29 June 2026VoidedKroger does not report Rewards metrics, so the criterion could never be settled from public sources.
T-Mobile force-migrates 8M+ customers29 June 2026MissThe concession the prediction expected was explicitly refused on the record, to both CNET and Fierce Network. Our production system auto-resolved it as a hit on 20 September from an informal FCC complaint, not the formal one the criterion required; that verdict is reversed.
Xbox console price increase, third in 14 months29 June 2026MissNo reversal, no compensatory bundle — the only new console was an $899.99 collector's edition — and no Microsoft response to sales data in the window. First judged a void from a truncated criterion; corrected on 23 September.
EA removes paid progression from College Football 2712 July 2026VoidedThe criterion turned on "player sentiment metrics" with no baseline and no threshold.
USPS raises Forever stamp and postal prices12 July 2026MissNo legislative or regulatory response occurred within the window, and USPS's only affordability statement was its routine rate-filing line of 9 April. The best-built criterion of the set, and it failed.
T-Mobile retires 1,100+ legacy plans12 July 2026VoidedSealed after the outcome was already public: the story broke on 28–29 June.
T-Mobile forced migration of grandfathered customers18 July 2026VoidedBacklash was reported on 16 July, two days before sealing. This had been published as a hit; it is retracted.
Crunchyroll store restricted to paid tiers18 July 2026VoidedNo August launch date was ever announced, so there was nothing to judge the criterion against.
Disney+ free ad-supported tier19 July 2026OpenJudged after 10 October 2026. Currently tracking towards not met.
AT&T raises prices on legacy unlimited plans6 August 2026VoidedThe criterion was already satisfied when sealed: CNET published switching advice tied to the increase on 31 July.
Flock Safety mandatory privacy guardrails18 August 2026OpenJudged after 15 October 2026. Currently tracking towards met: Hopkinton terminated its contract on 20 August, and Tempe, Eastpointe and Chesterton followed. First judged pre-satisfied from a truncated criterion; corrected on 23 September.

The structural fault

Every prediction was sealed after its headline event was already public — from one day in the EA case to roughly three months for USPS and AT&T. That is not the fault on its own: a decision being public while its consequences are not is the whole product. What is fatal is a criterion whose answer is already visible at the moment of sealing, or that no public source could ever settle. Four faults account for all 8 voids.

  • Criteria that asked for numbers nobody publishes. Three turned on figures the target does not report — per-app subscribers, per-tier subscribers, loyalty metrics. They could never have been settled, whatever happened.
  • A criterion already satisfied when sealed. The AT&T criterion was met by CNET's switching advice on 31 July 2026, six days before sealing.
  • No defined threshold or anchor. Two used measures with no baseline: what "player sentiment metrics" would have to show, or a launch date that was never announced.
  • Sealed after the outcome itself was reported, which makes a prediction untestable however it is worded. Two were, and one of those had been published as a hit.

The four misses

USPS: we predicted pushback would lead to a formal legislative inquiry, an oversight hearing, or a USPS statement addressing affordability. None occurred — USPS's only affordability wording was its routine rate-filing line of 9 April, published before implementation. It resolves on a documented absence rather than an event, and it is among the most credible entries in the record precisely because it failed: the criterion was specific enough to be wrong.

T-Mobile: we predicted the pressure would force a concession on the 8M+ forced migration. T-Mobile refused on the record, its CMO telling CNET that "absolutely nothing is required of the customer, and it just is going to happen". Our own production system then got it wrong: on 20 September it auto-resolved the prediction as a hit, citing an FCC complaint that was informal, not the formal regulatory complaint the criterion required. That verdict is reversed. The case also exposed a second judging hazard: searches surface a 2023 Ars Technica story in which T-Mobile did concede, and an automated judge matching that headline would record a false hit on a 2026 prediction.

Verizon: we predicted the 'Simplicity' plan would produce reported net subscriber additions or a direct pricing response from a competitor. Neither happened. T-Mobile's "Experience 2.0" added financing at unchanged prices and did not respond to Verizon.

Xbox: we predicted that after the third console price increase in 14 months Microsoft would reverse or moderate at least one price, announce a compensatory bundle, or respond publicly to a measurable drop in demand. None of the three happened; the only new console was an $899.99 collector's edition.

Verizon and Xbox were first published in this audit as voids, because their criteria had been judged on their first clause only. Judged on their full text, they are misses. All four stay on the record with the same prominence as the hit. A record that only shows what went right is not a record.

What we changed so it cannot recur

Four guards now sit in the pipeline, at seal time, so a prediction that fails them is never published.

  • Artifact-anchored criteria. A criterion must name the public artifact that would settle it — a filing, a pricing page, a published statement. One that depends on a figure the target does not publish is refused.
  • A seal-time observability check. Anything already observable when the prediction is written is refused, and the predicted direction must match the sealed verdict. This is the guard that would have stopped AT&T.
  • Pre-registered resolution URLs, archived at judgement, so the evidence cannot be chosen after the fact to suit the outcome.
  • A terminal unjudged state. A prediction that cannot be settled is closed and published as unjudged, with its reason. Nothing sits pending forever.

Caveats

  • Several sources we relied on render without machine-readable publication dates — Ars Technica, Fierce Network, Kotaku, Grocery Dive, the AP, GMA, the ACLU, Flock's own blog, IGN, tmo.report and Light Reading among them. In those cases the dating rests on URL paths and relative phrasing such as "Thursday". We say so rather than presenting it as firmer than it is. Content farms contaminate the search space around the Crunchyroll, USPS and T-Mobile topics; none were used as evidence.

    An earlier correction, made on 16 September to this audit's own first draft: we initially wrote that the EA prediction named the wrong game. It did not — the sealed title correctly says College Football 27. The error was in our research brief, not in the prediction, and that prediction is voided for its undefined sentiment threshold alone.

    Still to build, and not yet started: a stated probability and published base rate per prediction, scored with a Brier score instead of counted as hits; pre-registered community-signal metrics with raw counts archived, used only as secondary evidence and labelled as stated intent, never churn; and evidence bundles archived and hashed at the moment of judgement.