The fleet that builds my backlog: two roles, one budget, and a rule against thinking

Last week I wrote about three safeguards in this system that turned out to be lying to me. Several people asked the reasonable follow-up question, which is: fine, but what is the system?

This is that post. It is the build-out, not the postmortem.

Fair warning about the genre. There are a great many posts describing somebody's agent setup, and most of them describe a machine running under no constraint: no budget ceiling, no deadline, no merge gates, and frequently no production. The architecture is the entire content, so it reads as a demo of an idea rather than a report from a system that has to survive contact with something.

The only reason I think mine is worth writing down is the constraints. It runs against a launch date that does not move. It spends a metered budget that runs out. It merges to a repo I ship from. Every interesting decision in here is downstream of one of those three facts, and none of them would have surfaced in a system with nothing at stake.

The one idea

Two roles, and they are deliberately unequal. One is expensive and allowed to think. The other is cheap and forbidden from it.

Almost everything else follows from that split, so it is worth being precise about why it exists.

The expensive failure mode of an agent fleet is not a bad implementation. Bad implementations get caught: they fail tests, they fail review, they look wrong. The expensive failure is a cheap component quietly making a judgment call it was not equipped to make, being wrong in a plausible way, and having that wrongness inherited by ten downstream stories before anybody notices. Rework across ten merged cards is not a bug fix. It is a re-architecture.

So the split is not really about cost. It is about confining judgment to one place where I can audit it, and making everywhere else structurally incapable of improvising.

The planner

The planner is the only component that holds the whole graph. What is in the backlog, what the written strategy says matters, what depends on what, and which stories produce something another story needs.

It does three jobs.

It selects the work. It promotes stories out of the backlog into the ready queue itself, ranked against strategy I have already written down: the milestone I am currently shipping, the board priority field, the roadmap order, the product's own planning docs. It does not wait for me to hand-pick the next card.

That last part is the whole point, and it is the piece most orchestration setups skip. If a human has to curate the queue every cycle, the system has not removed the toil, it has relocated it to the one part of the loop that cannot be scaled. I would still be sitting down every evening deciding what to work on, which is exactly the ritual I was trying to stop performing.

It shapes the backlog. This is the part I did not anticipate needing. A queue-runner assumes a queue full of well-formed work, and real backlogs are not like that. So when the roadmap needs a story that does not exist, the planner writes a stub for it. When a story is too thin to build, it scopes it. When a story no longer fits the roadmap, it proposes killing it. When only I can unblock something, it asks me a specific question instead of guessing.

It is acting as a scrum master, not a filter. The difference matters: a filter takes a backlog as given and finds the buildable subset, which means a badly maintained backlog silently starves the whole machine. A scrum master's job is to make the roadmap buildable.

It stops at scope. The planner writes stubs. It does not write full specifications. A stub is a paragraph of intent: why, what, and a sketch of scope, enough for the spec-writing step to draft the real thing later, just in time, for the one story about to be built.

That boundary is load-bearing and I got it wrong on the first pass. Pulling full spec-writing into the planner turns every planning run into an expensive pass over the entire near-term graph, most of which will be re-planned before anybody builds it. Scoping is cheap and belongs to the whole graph. Speccing is expensive and belongs to the single story about to start.

The executor

The executor carries a story from ready to merged. For each one it runs three phases in order: write the spec, implement it, review it. It repeats that across a batch, several stories in flight at once.

This is where the time actually came from, and it is unglamorous. Before, those three phases were three sessions I opened by hand, dozens of times a week. The cost was never the typing. It was that each one required me to stop, decide it was time, and start something. The executor instantiates all three the moment they are due and I read outcomes instead.

The design rule that matters most is one line at the top of its instructions: cheap but humble, never cheap but guessing.

Concretely, it does not re-derive dependencies, does not resolve ambiguity, and does not adjudicate a borderline review verdict. Every one of those is an escalation, either back to the planner or to me. When it does not know, its job is to say so, and that instruction is repeated in its instructions more often than any other because it is the one that keeps the whole arrangement safe.

A cheap component that escalates is useful. A cheap component that improvises is a liability, and the liability compounds.

Model tiers, chosen per step

The fleet picks the model per phase rather than per run. Spec-writing and the mechanical coordination work go to a cheaper tier. Implementation goes to a stronger one.

The code reviewer is pinned to the top tier and nothing is permitted to cheapen it. Not when the budget is tight, not when the queue is deep, not ever.

That is the single hardest constraint in the system and it is worth being explicit about why. The reason the fleet is allowed to merge anything without me reading it first is that a separate, capable instance reviewed it, and one that did not write the code. Every other safeguard in the system is a nice-to-have compared to that one. Cheapening the reviewer to save tokens would be saving money by removing the only thing standing between an unattended agent and my main branch. There are places to economize. That is not one of them.

The corollary is a rule about review itself: the reviewer is never the author. Review is a separate instance, spawned fresh, that did not write the thing it is looking at. Continuing the implementing session into a review phase is cheaper and it is worthless, because you are asking something to find fault with reasoning it just produced and is still holding.

The budget, which is the part I would steal

Most agent orchestration I have read treats concurrency as a constant somebody picked. Mine treats it as the output of a control loop.

My Claude Code usage exports OpenTelemetry metrics to Grafana. The fleet queries that back, compares this week's consumption against a target line for the weekly window, and gets a band: on pace, squeezed, or no data.

When it is running hot it applies a ladder, cheapest lever first:

1. Cut parallelism. Lower the number of stories in flight. This is always the first response.
2. Slow down. Serialize entirely, then pause new pulls and let in-flight work finish.
3. Downgrade a model tier, and only in extremis. Spec-writing and implementation only. Never the reviewer.

Two details in there took me a while to get right, and both are about failure rather than success.

The order is not arbitrary. Cutting parallelism costs wall-clock and nothing else. Downgrading a model costs quality, and quality is the thing I cannot see from a dashboard. Spend latency before you spend correctness, always, because you will notice the latency.

And when telemetry is unavailable, the band is unknown, which is not the same as fine. An unconfigured export, an unreachable Grafana, a network blip: any of those and the fleet has no idea what it has spent. It runs conservatively and tells me the data is missing. Treating "I cannot see the gauge" as "the tank is full" is how you find out on a Friday that the week is gone.

What it may finish without me

Selection rules decide what the fleet starts. A separate set decides what it is allowed to finish, and those are the ones I would actually lose sleep over.

A never-a-back-off lane. Anything touching authentication, billing, money movement, schema migrations, or the merge gates themselves gets fully built, reviewed, and driven to green automatically, and then parked. Not merged. It sits there, green and out of draft, waiting for me to click.

The lane exists because the blast radius of those five categories is categorically different. A bad UI story is an embarrassing commit. A bad migration is a data-loss incident, and no review confidence I have earned in ten days justifies letting one through unattended.

One mechanical detail that matters more than it sounds: a parked story releases its slot immediately. It is done being built and only a human click remains, so it must not hold capacity. Getting that backwards means a couple of stories waiting on me can silently starve the entire machine of throughput, and the symptom looks like the fleet being slow rather than the fleet being blocked.

Confirmed green, never assumed green. Before any merge, the fleet reads the actual check conclusions from the actual pull request. It does not infer that CI passed from the fact that CI is configured, which is a distinction I would not have bothered writing down before I audited my checks and found several that had never run.

Explicit verdicts, never inferred ones. The absence of a review verdict is not a pass. This is the rule I paid for, in the post before this one: exactly two artifacts satisfy a gate, a recorded pass with evidence or my written waiver, and silence is neither. A gate that fails open is not a gate. It is a formality that clears whenever it breaks.

The gaps between the stories

The last piece is the one that is actually novel, and also the one that has already failed once.

Every story declares a small contract: what it produces, and what it consumes. The planner uses those to wire real dependency links, and the executor will not start a story until everything it consumes has landed.

This is a different question from "is this story ready." It is "does this story's assumptions still hold." Individually correct stories can sum to a broken system: a consumer with no producer, a feature complete except for the seam nobody owned, an event with listeners wired and nothing emitting. None of that is visible from inside any single story, which is exactly why no single story's review can catch it.

Here is the honest limit. Checking contracts catches the case where a story needs something that does not exist. It does not catch the case where a merge makes a claim in another in-flight story's design false without touching any dependency at all. Design documents are full of statements about the world (nothing reads this yet, this path is inert) that were true when written and can quietly stop being true while the story resting on them is still being built. That happened, a human caught it by reading, and the fix is a grep rather than a graph. I wrote that up last week.

What I have not run

The design calls for a retrospective pass after each batch: specialist lenses that look at ten merged stories at once and catch the drift no individual review is at the right altitude to see. Three stories that each invented a slightly different pattern for one job and each passed on its own terms. A now-complete feature that shipped undocumented because no story owned the seam.

It is designed, it is accepted, and as of this morning it is code that has never run. The lenses, the trigger arithmetic, and the rules for what a finding may become all shipped into my standards a few hours before this post did. Not one of them has met a real batch. So the compounding half of this idea, where what batch ten teaches raises the floor for batch eleven, is still a claim rather than an observation, while the throughput half runs every day.

I would rather tell you that than present a diagram of a system half of which is aspirational, because a diagram cannot tell you which boxes are real, and that turns out to be the only question that matters.

Would I recommend this

With one qualifier, which is that none of the above is the hard part.

The hard part is that this arrangement only works if the written strategy it reads is actually current. The planner ranks against my milestone, my board priorities, my roadmap. Feed it a stale roadmap and it will efficiently, tirelessly, and at considerable expense build the wrong quarter's work, and every individual story will pass review on the way.

Which is a strange thing to discover about automation. I set out to remove the work of deciding what to do next, and what I actually built is a machine that reads my decisions much faster than I used to be able to act on them. The bottleneck moved. It did not leave.

What's next

The retrospective pass, and whether specialist lenses reading ten merged stories at once find the drift the design promises, or whether that is one more thing I believed because I wrote it down.

• #building-lionshead
• #ai-agents
• #process