Invariant checks: the comment is the check you never wrote.

Twelve days ago I built a check that reads my company standards and asserts that every enforcement mechanism they name actually exists as code. Pointed at the repository where those standards are written, it found three rules describing gates nobody had built. I repaired all three, wired the check into that repository's pipeline, and watched it go green.

This afternoon, while drafting this post, I pointed the same check at a different repository: one that consumes those standards, and where it had never run once.

A hundred and forty-one claims across fifty files. Three of them unbacked, and two were the same fabricated gates I'd repaired twelve days earlier, sitting untouched in a tree I ship to.

The check was fine. It was correct on the first day and it's correct now. What I want to write about is the class of assertion it belongs to, why I'd been writing that class without a name for it, and the specific mechanical reason a repair I genuinely made was still not true in the place it needed to be true.

The comment

Somewhere in your codebase there's probably a comment like this:

If you add a column here, remember to add it to the SET map below.

I want to be precise about what that comment is. It isn't documentation. It's a person who, at the moment of writing it, held in their head the complete specification of a check: the two lists, the direction of the relationship, and the failure. They had everything they needed. They wrote a sentence to a stranger instead, and the stranger is in a hurry.

That comment is the check you never wrote. It marks the exact coordinate of a hazard and does nothing about it, and the reason it feels responsible is that it genuinely is more responsible than saying nothing. It's a local maximum. You can sit in one for years.

Three properties, all required

I call the thing that comment should have been an invariant check. Three properties have to hold in the check at once, and missing any one of them means you've built something else.

First, though, the word "pairing," because the whole class turns on it. A pairing is two things that have to be added together and routinely aren't. In the comment above it's the column and its entry in the SET map. Add one without the other and nothing complains, because nothing is watching the relationship between them. That relationship is the pair. A behavior test asks whether one thing does what it should; an invariant check asks whether both halves are still there.

The check is mechanically derived. It computes both sides rather than restating one. A new column, enum member, or route joins its scope without anyone editing the check. The moment you hand-enumerate one of the two lists you've created a second artifact that can drift from the first, and now you've got two problems.

The check asserts a pairing. When it fails, it names the missing half. Not "this value is wrong" but "you added this and not that." The failure message is a coordinate, not a judgment.

The check belongs to no single feature. One piece of work creates the seam and a different piece of work, months later, violates it. That's why no feature's own tests cover it: at the moment the pairing is created there's nothing to catch, and at the moment it breaks, nobody's looking at the place it was created.

That third property is what makes these invisible. A unit test belongs to a feature. An invariant check belongs to the space between two features, which belongs to nobody, which is why it doesn't get written.

Four of them already existed

When I went looking, I found I'd written four of these already. None of them were called anything.

One asserts that every database table exported by the schema has at least one reader outside the test suite, which catches a table you built, wired into tests, and never actually used. One is a runtime throw at construction time: registering a channel without a client fails immediately and loudly, rather than at three in the morning on the first real message. One asserts that every rule file over a size threshold has a sibling document holding its detail. One asserts that every deprecation tagged with a retirement issue is owned by an issue that's still open.

Four instances of one idea, in two repositories, in three completely different mechanisms, every one of them written reactively after the corresponding gap had already bitten me.

The mechanism is not the definition, which took me a while to see. A construction-time throw is as much an invariant check as a scheduled scanner. Choosing between them is a question of where the pairing can be observed most cheaply, not a question of what kind of artifact you're building.

And the reason to name the class isn't tidiness. An unnamed pattern can't be asked for at planning time, can't be audited for at the level of a whole board, and can't be hunted for by anyone who has no word for it. Naming it is what turns four accidents into something a gate can require. So now the question gets asked when work is planned: does this create a pairing that nothing asserts? "Nothing pairs here" is a fine answer. Nobody asking is the failure.

The fifth one, written on purpose

Then I built one deliberately and pointed it at my own standards.

Those standards are rule files plus longer sibling documents holding the detail, and they're the first thing my agents read before doing anything. The invariant is that every enforcement mechanism a rule names resolves to code that exists. The pairing is a claim in the corpus and an implementation in the repository.

The first sweep found three rules that specified enforcement in implementation-level detail and had shipped none of it.

The first of the three described a gate that would refuse to promote work to ready-for-dev without a test design, complete with an escape-hatch flag called --allow-no-td and a matching environment override. The script it named contained neither string.

The second described a checklist item the planning agent was supposed to add when planning new work. That agent had never been told to add it. By the time I found this, a second rule described its own gate as sitting "parallel to the IaC checkbox," which is to say parallel to nothing.

The third told developers to scaffold their first failing test with npm run new-test. No repository in the portfolio had a new-test script in any manifest.

The detail is what makes these dangerous rather than merely aspirational. A vague aspiration reads as an aspiration and gets treated like one. A documented flag name reads as a flag, and the reader's next move is to type it. I'd written three specifications convincing enough that an agent following them would have believed it was working inside a system of gates, and would have been alone.

This is the failure mode where documentation asserts coverage that never existed. Prose can't catch it, because prose is where it lives.

What it cannot do, said plainly

The check reads three kinds of claim: a named npm script, a runnable path, and a command-line flag attributed to a command I own. It doesn't read prose for meaning.

So of the three defects that motivated it, it catches the first and the third. It doesn't catch the second. A checklist item described only in sentences names nothing runnable, so there's nothing to resolve and nothing to fail on. A rule can still promise a gate in English and walk past every check I have. What it can no longer do is name a runnable thing that isn't there.

I'm spending this many words on the boundary deliberately. Overstating the coverage would be precisely the defect the check exists to catch, and I'd rather be tedious about the limit than publish a post that commits its own subject.

Making it quiet enough to survive

The engineering that mattered was not detection. It was suppression. A check that cries wolf gets disabled, and a disabled check asserts exactly as much as that comment does, which is nothing.

Flags were the hard part. gh pr checks --watch and git ls-remote --heads are flags on other people's tools and I'm not the one who implements them. An early version attributed any flag to whatever script happened to be named in the same section, which produced eight confident reports about flags that were never mine. So attribution runs command-first: if the surrounding text invokes a script I own, the flag is checked against that script; if it invokes somebody else's tool, there's no claim at all. But it can't be command-only, because a bare flag written in a sentence has no command attached, and that's exactly the shape the real defect took. --allow-no-td sat four lines below the script that was supposed to own it, in prose. The section-level fallback is load-bearing.

Environment variables are off by default. Operator-setup sections mix mine with everyone else's and nothing in a variable name tells you whose it is. Gating on them produced eleven false reports, and eleven false reports is how you teach someone to ignore a check permanently. Nothing is lost, because every environment override in these designs has a paired flag and the flag is checked. The original fabricated variable got caught anyway, through its twin.

Paths are narrow: runnable files under three directories. An earlier draft included documentation directories and reported seventy-one unbacked paths, nearly all of them either stale cross-links between rule files or claims about a consumer's tree rather than the publisher's. Both are real problems. Neither is this one, and folding them in would bury the signal under a backlog nobody triages.

I re-ran the tuned version against the authoring repository while writing this. A hundred and twenty claims across forty-five files, zero unbacked.

Why the two were still there

So the publisher is clean and the consumer isn't. There are two reasons, and the second is the one worth your time.

The first is that the corpus is authored in one repository and mostly describes the others. npm run dev resolves in a product repository and nowhere near the publisher. So the check runs in two modes, and running the consumer mode at the publisher reports seven unbacked claims that are all false: dev, lint, typecheck. Real scripts, in other repositories.

I wrote that asymmetry down when I built it, along with its consequence, which I'm going to quote because I was right and it did me no good whatsoever: at the publisher, a fabricated npm run new-test is indistinguishable from a real npm run dev, and only a consumer can tell them apart.

I had the correct theory of where this check needed to run, and I ran it in the other place.

The second reason is worse and more interesting. A failing claim only ever has two honest resolutions: build the thing, or stop saying you did. Which one is right depends on whether the gate was worth having, and deleting the sentence is only the answer when the mechanism genuinely isn't coming. Delete prose to turn a check green and you haven't fixed anything, you've just moved the lie somewhere the check can't see it.

So I repaired the three claims three different ways. One gate was worth having and I built it, though not where I'd said it was: it lives in the planning agent's own procedure rather than in the script the rule had named, and the prose changed to describe where the gate actually is. One I resolved the other way. And where the scaffolding command used to be, the upstream file now carries an explicit correction saying in as many words that the command was specified and never built, that no manifest anywhere has it, and that it had been written up as though it shipped. The claim is gone and the record of it isn't, which is the only version of deletion I trust. That correction is sitting there today.

It's sitting in the file that replaced the file the claim was written in.

A while back I consolidated thirty-three rule documents down to eight. The consolidation created the new files and distributed them everywhere. It didn't delete the old ones. My distribution manifest lists fourteen sibling documents; the consumer repository has nineteen sitting in that folder. Five are pre-consolidation ancestors: unowned, undelivered, never updated again, and completely indistinguishable from the live ones to anything that reads the directory. Including my agents.

So the correction landed on the descendant. The ancestor kept the original claim, kept shipping it, and kept reading as authoritative. Upstream: fourteen files, no orphans, zero unbacked. One repository over: nineteen files, five of them ghosts, and inside one of the ghosts, verbatim, the fabricated flag and the fabricated command.

I checked both by hand before writing this. The script contains neither the flag nor its environment variable. No manifest in that tree has a new-test script.

The third unbacked claim, for completeness, is the check being wrong. A rule describes CI running npm run typecheck --if-present, and that repository has no typecheck script. But --if-present is the idiom that means absence is fine; the claim never asserted the script exists. My check can't read that qualifier, so it flagged a gate behaving exactly as designed. Two real, one false. I'd rather print three and explain than round down to the number that flatters me.

The step after noticing

The check was right. The design note about publishers and consumers was right. The repair was right, and thorough, and honest about itself.

What failed was distribution, which is the second Sunday running that I've written that sentence in a different costume. Last week it was a retrospective whose findings were real and went nowhere. The noticing has never been my weak link. The step after the noticing is where I have no gate, no counter, and apparently no attention left over, and it keeps being the part that costs me.

There's a version of this post where I fix the orphans tonight and write all of this in the past tense. I'm not writing that one. They're the evidence, and they'll be a line on my roadmap tomorrow morning instead of a quiet commit tonight.

The comment you never turned into a check is one failure. The check you wrote and never pointed anywhere is the same failure wearing better clothes.

What's next

The cost gate: infrastructure spend as a merge requirement rather than a monthly surprise, and what it's like to have a pull request blocked by a number instead of a test.

• #building-lionshead
• #ci-cd
• #process