GET THE AUTOPSY ➔

FROM THE DESK · HOW ZHP ACTUALLY WORKS

We ran our own lie detector on our own website. It caught us.

Every video we publish goes through the Zero Hallucination Protocol: no number ships without a receipt. Our own website did not have to. We fixed that this week, caught a fabricated statistic on the page about honesty, and wrote down the exact rule, the confidence math, and the procedure below so you can run the same thing on your own writing.

ZHP is not a slogan. It is a script:scripts/verify-issue.py for the daily reels, checking every quote, every link, every number against a source before anything renders. It has been mandatory on video for months. Nobody asked whether the website obeyed the same law, so we checked, and it didn't. The homepage had one narrow check. Eight other pages, prose making real claims, had none.

"Publish a fact-checking site with unverified facts" is the exact failure we built this company to catch in other people. So this week we pointed the machine at ourselves, then wrote down the whole method in one place, because a rule you can't hand to someone else isn't a rule, it's a habit only you can keep.

THE RULE, IN ONE SENTENCE

Every claim that ships (a number, a date, a name, a quote, a causal or comparative statement) must trace, in its exact wording, to a source you can point at. Not "the vibe is right." Not "the tool said so." The specific words used have to survive someone reading the source and checking them against what you wrote.

The part people skip: a claim with no source doesn't get softened into ship-ready. It gets cut, or turned into an open question ("we couldn't confirm this"). Vagueness is not a compliance strategy.

THE VIOLATION NOBODY NOTICES: OVERSTATING A REAL FACT

Inventing a number from nothing is the obvious failure and, honestly, the rare one. The common one is quieter: taking a real, sourced fact and writing it one notch stronger than the source actually said. If a filing says a regulator "investigated" a company, and a draft says the regulator "flagged" it, that draft has failed ZHP even though the underlying fact and the source are both completely real. We've shipped that exact mistake once, on a social comment, before the review gate caught the pattern. The topic was right. The verb was wrong. That's still a violation.

The fix is mechanical, not moral: read the verb the source actually used, and keep that verb's strength. "Estimated" cannot become "confirmed." "Asked for" cannot become "fined." If your sentence reads more dramatic than your source, your sentence is wrong, not your source.

HOW THE CHECK ACTUALLY WORKS

scripts/site-provenance-check.py reads the raw source of every page, strips everything that isn't rendered prose (CSS, script tags, SVG markup, JSX attributes, the year in a byline), and scans what's left for any number two digits or longer. Every survivor either traces to a real source or the build does not ship.

1Strip CSS, script tags, SVG, JSX attributes and expressions, date literals.
2What survives is prose a reader actually sees.
3Flag every 2+ digit number not on the allow-list.
4Allow-listed only with a written reason: a scale value, a price, a step number. Never "trust me."
RESULTUnexplained number left over → the build fails. Not a warning. A hard stop, wired into the same gate that ships to production.
FIG. 1 · What the check actually does. No AI judgment call in the loop, just a strip-and-scan on the real page source, so it can't be talked out of flagging something.

Two categories get a pass. Pages that assert nothing (the login screen, the 404 page, the pricing page's set price) are exempt by name, eight of them, listed in the script itself. Everything else needs either a plain-language reason on an allow-list ("100" is our BS-index scale max, not a claim about the world) or a companion receipt file, one per page, that shows its work.

BEFORE · 25 AUG 2026

  • Gate covered 1 page: the homepage
  • 8 other prose pages: unchecked
  • A false number could ship and nothing would notice

AFTER · 25 AUG 2026

  • Gate covers 9 pages that make claims
  • 1 has its own receipt file (token-wars, 11 claims individually sourced)
  • Wired into the same gate that blocks a bad production deploy
FIG. 2 · The actual diff. src/pages/*.astrohas 17 files. 8 make no factual claim and are exempt by name. Of the remaining 9, 8 are checked live and 1 carries a standalone receipt. Zero pages ship unaudited.

That script catches unsourced numbers. It does not, by itself, decide whether a SOURCED number is strong enough to publish bare, or needs a hedge, or needs to be cut anyway. That's a separate, human judgment, and it needs its own rule so two people (or the same person on two different days) land on the same answer. Here's that rule.

THE CONFIDENCE RUBRIC

Every surviving claim gets a percentage, and the percentage decides how it's allowed to appear on the page. This is the actual table we use, word for word.

95–99%Primary record read directly, plus 2+ independent corroborationsShips as a bare statement. No hedge needed.
85–94%Standard sources agree; primary not read directly, or minor variance existsShips bare, variance noted internally.
70–84%One strong source, or a genuinely contested point that leans one wayMust be attributed in the copy: "per X", "Y reports". Never asserted bare.
Under 70%Can't clear the barDoes not ship as fact. Reframe as an open question, or cut it.
FIG. 3 · The confidence rubric. A confidence score with no named source behind it is not calibrated, it's a guess wearing a percentage. This table is what turns "I think this is right" into a number someone else can audit.

THE FOUR THINGS THAT CAN HAPPEN TO A CLAIM

When a claim gets challenged (by a reviewer, a reader, or you re-reading your own draft with fresh eyes), it resolves to exactly one of four outcomes. Nothing lands anywhere else.

WHAT IT FOUND, ONE HOUR IN

The first run scanned mission.astro, the page that lists the six rules we hold ourselves to. Rule five reads: "We are skeptics, not cynics. When a claim survives the evidence, we stamp that just as loudly." The sentence that followed it, written back in an earlier commit, said: "Munger scored 60 percent. We said so."

Nobody could say where 60 percent came from. Not the corpus of Charlie Munger claims we've actually checked, not the commit that introduced the line, not any note anywhere. So we went and counted the real number, from the files that hold every claim we've checked against Poor Charlie's Almanack.

Claimed on the page (no source)
60%
Actual: 38 of 52 Munger claims hold
73%
FIG. 4 · The number we were about to keep publishing.52 Munger claims checked in our corpus: 38 hold, 13 need context, 1 contested. The real figure was better than the fake one and we still didn't get to use a number we hadn't earned honestly.
A fact-checker that fact-checks itself only on the pages nobody reads is not a fact-checker.

Running the four outcomes against this claim: SOFTEN was on the table, honestly. "The majority of Munger's claims hold" would have been true at 73 percent and still defensible. We went further and chose DROP instead, because the whole point of that sentence was demonstrating we credit what holds with a specific number, and a vaguer specific number is still a number somebody could ask us to re-derive later. Fixing it for real meant removing the figure entirely, not softening it into something merely defensible.

We had two ways to make the gate go green. Add "60" to the allow-list with a made-up justification, which is exactly the move we call BS when a company does it to us. Or fix the sentence. We fixed the sentence: it now says "When Munger's reasoning checks out, we say so," which is true, provable, and needs no number it can't stand behind. The commit that shipped this (f13a51c) says exactly what we found and why, in the open, same as any correction we'd demand from someone else.

THE PROCEDURE, IF YOU WANT TO RUN THIS YOURSELF

None of the above needs our specific scripts. It's three roles, run in order, never collapsed into one person grading their own homework (motivated reasoning is the entire failure mode this guards against). Solo writers can still run it: consciously switch hats between the three passes instead of defending the draft the whole way through.

WRITERDraft the piece, and while drafting, log every factual claim into a table: the claim as written, the source it's based on (if any), status UNVERIFIED. A claim you can't be bothered to log is a claim you can't be bothered to defend, and it doesn't ship.
RED-TEAMA separate pass tries to break every row. Run at least two personas: the bad-faith commenter (nitpicks, dunks, attacks rounded numbers and superlatives), and therival expert (the claim's best good-faith defender, arguing the strongest honest case it's wrong). Add apedant when dates, units, translations, or attribution chains are in play. Open every link for real. A source that loads but doesn't say what you claimed is the same as no source.
JUDGEA third, genuinely separate pass rules each attack SURVIVES, SOFTEN, HEDGE, or DROP, using the confidence rubric above. Anything under 85 percent gets attributed phrasing, not a bare claim. Anything under 70 doesn't ship. Keep a short contradiction ledger: what got attacked, how it was answered or conceded. That ledger is what survives a reader trying the same attack after you publish.
FIG. 5 · The three-role pipeline. Writer extracts, red-team attacks, judge rules. Same three roles whether it's three people, three separate agent passes, or one person deliberately changing hats between drafts.

The checklist version, worth pinning above your desk or your repo's PR template:

[ ] every claim extracted into a table (nothing ships ungoverned)
[ ] every link actually opened, confirmed live and on-topic
[ ] red-team run by a genuinely separate pass, not the writer
[ ] every attack ruled SURVIVES / SOFTEN / HEDGE / DROP
[ ] confidence assigned per the rubric, not vibes
[ ] anything under 85% attributed in the copy
[ ] anything under 70% cut, not shipped
[ ] the exact verb matches the source's strength (no upgrades)
[ ] contradiction ledger kept, for reuse if challenged later

THE PART THAT SURPRISED US LESS

While widening the gate we also caught a false positive of our own making: the scanner briefly flagged "700" on the papers page as an unsourced number. It wasn't a claim. It was a CSSfont-weight:700 value sitting inside a template-literal style attribute our stripping regex hadn't accounted for. We fixed the regex, re-ran the gate, and it went quiet. Worth saying plainly: an automated check is not magic. It has bugs, and finding them is part of the job, not a sign the check doesn't work.

This is the same standard /token-warsalready runs under, receipt file and all, and it's why that page links a claim table with a source, a method, and a confidence score for all eleven numbers on it, red-teamed before it shipped. Now every standalone page on bskiller.com either passes that test live or carries the same paper trail.

THE CHECK

We publish verdicts on other people's claims for a living, and one of our own pages was making a claim it couldn't back. The gate that would have caught it earlier didn't exist until this week. It exists now, it already found a real problem on its first run, and the fix is in the public commit history, not quietly edited away.HOLDS: nine pages, zero unaudited numbers, one honest correction on the record, and a procedure above that costs nothing to copy.