Your agent fleet forgets everything

I ran 18 agents across a codebase last month to find everything wrong with one subsystem. It worked. It took about 40 minutes, cost more than I'd like to write down, and produced a genuinely good report — 12 real findings, ranked, with file references that held up.

I fixed 4 of them that week. Then I moved on.

Three weeks later I needed to know something about the same subsystem, and I had nothing. Not the report — I could find that. I had nothing that made the next run cheaper than the first. I was going to have to do the whole thing again.

The bit that impressed me, and the bit that should have

The impressive part was the fan-out. One instruction, 18 agents, each taking a different angle, converging into one ranked answer. That's a real capability and it wasn't possible for me two years ago.

The part I should have been watching was what happened afterwards, which was: nothing.

The 12 findings existed in a report. The report existed in a chat window. Nothing about how I'd got them — which angles were worth taking, which two agents kept colliding, which parts of the subsystem turned out to be a swamp — existed anywhere at all. The run had produced an output and no guidance.

That's the distinction the whole method turns on. A codebase is what runs. A contextbase is what guides. I'd generated a lot of the first kind of value and none of the second.

Twelve findings, twelve forgotten

Here's what actually evaporated.

Four findings got fixed, and the reason each one was a problem went into a commit message, which is the worst searchable place I own. The other 8 were real and I didn't get to them — so they're gone, and the next run will rediscover them at full price. This is the agent relearning problem at fleet scale, which is the same problem costing 18 times as much.

Worse, the structure went too. During the run I'd worked out that splitting by directory was useless and splitting by concern — auth, data access, error handling — produced far better coverage. That took me two bad attempts to learn. It survived in my head for about a week.

Anthropic's own writeup of building a multi-agent research system is candid about how much of this work is orchestration design rather than model capability — which angles, which splits, how the results come back together. That design is the expensive part. Mine went in the bin.

Stuart Leo

A fleet without a contextbase produces 12 findings and forgets all 12. The findings were never the asset. The shape that found them was.

What survived, and what didn't

Sorting it afterwards was clarifying, because the pattern is stark:

Everything in that table that had a file survived. Everything that didn't, didn't. That's not a subtle lesson.

The graph is an artefact, and it should be in git

Here's the claim I've landed on, and it's the one thing in this note I'd defend hardest.

A saved orchestration graph is not code. It's context. It doesn't ship to users. It doesn't run in production. It directs execution — which is precisely what a prompt brief does, and what a Router does, and what a gotcha does. By the definition C² already uses, that puts it in the contextbase, not the codebase.

And it's the densest form of context I've come across. A topology that worked once, version-controlled, re-runnable by name by anyone who clones the repo. Not a description of how to review a subsystem. The actual review, ready to fire again.

What I do now

Three changes, all cheap, none clever:

1. Every node writes what it learned to 03-knowledge/, not just its finding. The finding goes in the report. The gotcha goes in git, where the next session reads it before touching that code.
2. The merge step deduplicates against what's already known, rather than re-discovering it. That alone made the second run of the same job noticeably cheaper, because 3 of the 12 were things I'd already written down.
3. The orchestration itself gets saved and named. Whatever your agent calls its workflow directory, it belongs alongside docs/ and it belongs in the Router — the file your agent reads first. If a shape worked, it's an asset. Treat it like one.

The honest cost is that all 3 are discipline at the exact moment a run finishes and you feel done. That's the worst moment to rely on discipline, which is why the write-down has to be part of the run rather than a good intention afterwards.

The findings were never the asset

An expensive run that produces a good answer and nothing else is a purchase, not an investment. You bought an answer. You'll buy it again next month.

The version worth running is the one that leaves something behind — the gotchas in git, the dedupe that gets cheaper each time, the shape saved under a name. That's the difference between running a graph and compounding one.

Start here: see what a contextbase is, how to stop your agent forgetting, or read the method.