Controls for an autonomous coding pipeline

I’ve been working on an autonomous development pipeline I’m calling Daedalus.

Daedalus is a collection of skills and tools able to pick up an issue, write the code, run the tests, and commit, with nobody driving in between. This is great for throughput, but many things can go wrong when no human is in the loop: unreviewed code, token costs, an agent that edits files it was never asked to touch, a run that loops forever instead of stopping.

I built the Daedalus pipeline as a plugin for Claude Code and ran tasks through it end to end. Writing code end to end was reasonably easy early on, assuming your source issue had a solid specification. The interesting part was the controls: what makes an end-to-end pipeline reviewable, honest about its scope, bounded in cost, and able to stop itself.

I will use my pipeline (I call it Daedalus) as the concrete example throughout, but the point is the controls, not the tool. If you are building your own end-to-end pipeline, or evaluating whether to, these are the questions worth working through.

What “end to end” means here

The unit of work is one tracked issue from a backlog. The pipeline claims it, plans the change, implements it test-first, reviews the diff, simplifies, runs the quality gates, and commits with the issue reference, with no human turn between claiming the issue and the commit. Multiple coordinated agents do the work: an orchestrator that coordinates and never edits code, an implementer that writes the change, and a review gate that judges it. Around them sits a set of gates enforced by hooks, which is where most of the control actually lives.

Reviewability

The first thing you give up when you remove the human from the loop is the over-the-shoulder review. You get it back by making the pipeline produce evidence a human can audit after the fact, and by making one agent’s whole job to review.

In Daedalus the review gate is a separate agent with no edit tools. It did not write the code, nor does it have the context of the development sessions, so it judges it cold. It runs the project’s configured test and lint commands itself rather than trusting the implementer’s output, reads the issue body, and confirms the diff satisfies each acceptance criterion. It returns one of two verdicts: APPROVE or REJECT, with findings tied to file and line.

The orchestrator prints the evidence to the run’s transcript: test output, lint output, the verdict, the diff status, and every tracker change. When you come back to a finished run, you are reading what it ran and what the reviewer concluded. An autonomous pipeline is only as reviewable as the artifacts it leaves behind.

Decide up front what evidence a human will need to trust a run, and make the pipeline emit it every time.

Honesty

The failure mode that worried me most is the agent that makes the tests pass by changing the wrong thing: editing a file contrary to the plan, weakening a test, touching config it was told to leave alone. An agent optimizing for “tests green” will “cheat” if you let it.

The defense is to make them hard failures in the tool calls (via hooks) and review gates. The review gate treats any file changed outside the plan’s stated scope as an automatic REJECT, with no discussion. Separate path-protection hooks deny edits to environment files, lockfiles, the .git directory, and CI config (agents will mutate CI to get a bad build past the quality checks), and block force pushes and history resets. The commit itself is gated: a hook refuses git commit until the test and lint gates have passed recently, and another refuses to let the turn end on failing tests or a dirty tree. None of these depend on the agent choosing to behave.

Name the actions you do not want to happen and enforce them mechanically. A prompt that asks an agent to stay in scope is a suggestion; a reviewer that auto-rejects from its own context is a control.

How many agents should you use?

The obvious way to make a pipeline faster is to run more agents at once. This is where the published data changed how I think, and why I kept the pipeline more serial than instinct wanted.

Anthropic’s engineering write-up How we built our multi-agent research system reports a large gain from a multi-agent pattern on research work: a lead agent with parallel subagents “outperformed single-agent Claude Opus 4 by 90.2% on our internal research eval.” But, the gain is concentrated on a specific type of task. They found multi-agent systems “excel especially for breadth-first queries that involve pursuing multiple independent directions simultaneously.” And they are direct about where the pattern does not fit:

most coding tasks involve fewer truly parallelizable tasks than research

some domains that require all agents to share the same context or involve many dependencies between agents are not a good fit for multi-agent systems today

A single code change is exactly that kind of domain. Its parts depend on each other: a function signature and its callers, a schema and its migrations, an interface and its implementations. Getting one right means holding the others in the same context. Splitting that across agents that, in their words, “are not yet great at coordinating and delegating to other agents in real time” buys you coordination overhead and no parallel speedup. So implementation stays in one agent that holds the whole change.

The Daedalus pipeline does fan out at one point, and only where the work is genuinely independent: reviewing a high-risk change. There it runs three reviewers at once, each with a separate goal (security, regressions, test adequacy), all judging the same finished diff, and all three must approve. That is the breadth-first, independent-direction shape the data rewards. Independent judgments of a finished artifact need no coordination.

Decide where to parallelize by dependency structure, not by how many agents you can spawn. Independent directions fan out well. Dependent, shared-context work belongs in one agent.

Cost

Agents “use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats.” In Anthropic’s BrowseComp evaluation, token usage by itself explains 80% of the variance in performance, and they draw the obvious economic condition: the pattern is worth running only on tasks valuable enough to pay for it.

For a pipeline that runs unattended, that means bounding the spend instead of leaving it open. Each agent in Daedalus has a turn cap: the implementer runs at most 25 turns, the reviewer at most 12. The orchestrator works inside a roughly 40-turn budget. The expensive fan-out (three parallel reviewers) happens only on changes flagged high-risk, where three independent passes are worth the tokens. I place ceilings on how many agents run and for how long, which is the cost that 15× figure reflects.

An autonomous pipeline’s cost is mostly token count, and token count is mostly agent count times turns. Cap both, and reserve the costly multi-agent passes for the work that pays for them.

A blocked task is a valid outcome

The last worry is the run that neither finishes nor stops: an agent re-trying a broken approach, looping on a test it cannot pass, burning budget with no result.

The control is to make stopping a successful first-class outcome. When the reviewer rejects, the orchestrator re-spawns the implementer with the findings, but only up to three cycles. After that, or on repeated identical test failures, a spec that contradicts the code, a downed dependency, or a nearly exhausted budget, it marks the issue blocked with a written reason and stops. A blocked task with a recorded reason is a legitimate terminal state. An explicit human-approval boundary always wins: when the pipeline is configured to commit locally and wait for review, it does not push or open a pull request no matter what would be convenient.

Give the pipeline a way to give up cleanly. Bounded retries and a blocked-with-reason state are what keep “autonomous” from meaning “unbounded.”

Takeaways

If you are building an end-to-end pipeline, the design questions that matter are not about whether the agent can code. They are:

• What evidence does a run leave behind, so a human can review it after the fact?
• Which unwanted actions are enforced mechanically, rather than pleaded for in a prompt?
• Where does the context split into independent directions (parallelize) versus depend on shared context (keep in one agent)?
• What caps the token cost, and which expensive passes are reserved for high-value work?
• How does the pipeline stop itself when it cannot finish?
• Can the system verify its own work? Can it load a browser, start the dev server, and exercise its changeset?

References

• How we built our multi-agent research system, Anthropic Engineering. Source for the 90.2% breadth-first improvement, the breadth-first excellence statement, the coding and coordination quotes, the shared-context / dependencies boundary, the 4× and 15× token multipliers, the 80%-of-variance figure from BrowseComp, and the economic-viability condition.