We built the agent layer twice. The second time told us if we had designed it.

the port that was supposed to take a week

We had an agent layer running on a publishing platform. Editors could ask it to do things — find a story, schedule it, fix a section, pull the numbers on last month's traffic — and it would go and do them against the platform's own operations.

Then we needed the same thing on an e-commerce backend. Different domain, different data, different everything. Catalogue, inventory, pricing, orders, returns. Nothing in common with a newsroom except that both are large systems with a lot of operations and a human who would rather describe an outcome than click through six screens to produce it.

I estimated a week to port it. I was wrong in an interesting way rather than an embarrassing one: the parts I thought would be hard moved in an afternoon, and two things I'd never thought about at all turned out to be welded to the first product.

This is what survived, what didn't, and what I now think the second implementation is actually for.

the second one is the test

Building something once tells you it works. It tells you nothing about whether you designed it, because on the first pass every decision has exactly one caller and every abstraction is fitted to a single set of facts. You cannot tell the difference between a general solution and a specific one when n = 1. Nobody can.

The second implementation is where you find out. Not because the second product is harder — it usually isn't — but because it's the first time anything pulls in a direction the original didn't anticipate. Every seam that was really a coincidence comes apart. Every abstraction you invented because it felt tidy, rather than because something needed it, turns out to cost more than it saves.

I've started treating this as a rule. If a piece of architecture has one caller, I don't describe it as architecture yet. It's a shape that has so far not been contradicted.

what a tool actually is

Before the port, the useful part: what we'd learned about giving a model tools in the first place.

A tool is not an API endpoint with a description attached. It's a small contract negotiated with something that has never read your documentation, has no idea what your product is called internally, and will confidently pick the wrong one if two of them look similar.

The description is a prompt, not documentation. This took embarrassingly long to internalise. I wrote descriptions the way I write API docs — precise, complete, slightly dry. The model kept choosing badly between two tools that were, to a human reader, obviously distinct. The descriptions were accurate and useless. What worked was writing them the way you'd brief a new colleague on their first day: what this is for, when you'd reach for it, and pointedly, when you wouldn't. The "when you wouldn't" line fixed more misfires than anything else I did.

Name them for the outcome, not the mechanism. A tool called updateEntityStatus is named after what the code does. A tool called schedule_for_publication is named after what somebody wants. The model is matching a user's intent against your names, so names that describe your internals force it to translate twice — once from intent to your domain model, and again from your domain model to your API. It gets that translation wrong in exactly the cases where the user's phrasing was unusual, which is to say the cases where you most needed help. Renaming a handful of tools from mechanism to outcome measurably reduced misfires, and cost nothing.

Fewer, wider tools beat many narrow ones. A surface with sixty small tools is a surface where the model spends its budget on selection instead of work, and where two tools inevitably overlap enough to be confusable. Collapsing families of related operations into one tool with a mode, and letting the model pass a richer argument, made selection dramatically more reliable. It also made the surface something a human could hold in their head, which turned out to matter for entirely separate reasons — mostly debugging.

Errors are instructions. The single biggest improvement we made was rewriting what a failing tool returns. A stack trace is noise. "Invalid input" is noise. What a model can act on is a sentence that says which field was wrong, what shape was expected, and what it might do instead. Get this right and a whole category of failure stops being failure: the model reads the error, corrects itself, and proceeds. Get it wrong and the model either gives up or — worse — invents a plausible reason it failed and tells the user that instead.

Expose what exists. The strongest decision we made, and the one I'd defend hardest, is that tools call the operations the system already has. Not a parallel path built for the AI. Not a convenience layer that skips the validation a human's request goes through. The same operations, with the same rules, taking the same route.

This is less about elegance than about drift. The moment you build an AI-specific path, you have two implementations of every rule, and they will diverge — not immediately, but on some Thursday eight months from now when somebody fixes a bug in one of them. A parallel path is a promise to maintain two of everything forever, made by someone who won't be the one keeping it.

what moved unchanged

The port went well in the places I'd been nervous about.

The loop itself — the thing that decides what to do next, does it, looks at the result, and decides again — moved without modification. That was a relief, because it's the piece with the most subtle behaviour in it. The protocol layer moved unchanged too, which was less of a surprise; it's a spec, and specs port.

The error-shaping conventions moved. So did the transcript recording, the budget accounting, and the general skeleton of how a tool declares itself. In aggregate, most of the mechanism was portable. If I'd stopped measuring there I'd have concluded the design was sound and gone home pleased with myself.

what didn't

Two things had to be rebuilt, and neither was where I'd been looking.

The vocabulary was baked in. Not in an obvious place like a constants file — I'd have caught that. It was in the tool descriptions, the argument names, the examples, the phrasing of the errors. The whole surface quietly assumed a world of documents, authors, sections and publication states. Moving it to a world of products, variants, stock and fulfilment wasn't a rename. Whole tools didn't have an equivalent, and several concepts in the new domain had no shape to move into. I ended up rewriting the entire descriptive surface, which is most of the design work, while the code underneath barely moved.

I don't think this was avoidable, exactly. But I'd stopped noticing that the descriptions are the product, and treated them as documentation of the code. They aren't. The code is the cheap part.

Retrieval was fitted to one shape of content. Finding the right operation when the user's words don't match anything in your system is its own problem, separate from executing it. Our approach to that had been tuned, without anyone deciding to, against long-form text with a title and a body. Catalogue data is short, repetitive, heavily attributed and full of near-duplicates — three variants of the same product differing by one field. What worked well on articles worked poorly on that, and the failure was quiet. It didn't error. It returned confident, plausible, wrong matches, which is the expensive kind of wrong.

That one cost the most time, and I'd have found it faster if I'd been suspicious of the parts that seemed fine rather than the parts that seemed hard.

testing a tool without a model

The most practical thing to come out of the second build: we can exercise the entire tool surface with no model involved and no API key anywhere.

A tool call is, underneath, a function taking structured input and returning structured output. Everything model-shaped about it — the description, the selection, the natural language — sits above that line. Below the line it's ordinary code and can be tested like ordinary code. Every tool gets called with valid input, invalid input, input the caller isn't permitted to use, and input that's valid but refers to something that doesn't exist. Four cases, no model, runs in milliseconds.

This is worth more than it sounds. Evaluating agent behaviour end-to-end is slow, expensive and stochastic; you need it, but you can't run it on every commit. Underneath it, the layer that actually touches your data is deterministic, and treating it as such means most regressions get caught by a normal test suite in the normal way. The stochastic tests are then free to test the stochastic part instead of re-testing your validation logic.

what I'd do differently

Write the second product's tool descriptions first. Even hypothetically, even for a product that doesn't exist. Ten minutes of drafting descriptions for an imaginary second domain would have exposed the vocabulary coupling before it was load-bearing. It's the cheapest possible test of whether you've built a mechanism or a costume.

Treat retrieval as domain-specific from the start. I'd assumed it was infrastructure. It behaves more like a feature: it needs to know what your content is shaped like, and pretending otherwise just means it's tuned for whichever domain you happened to build first.

Stop calling it architecture until something has contradicted it. I said this above and I mean it as a working practice, not a slogan. There's a real temptation, when the first build goes well, to write it up as a pattern. I've done it. The write-up is always premature, because the thing that makes a design real is surviving a caller it wasn't designed for — and until that's happened, all you have is a shape that hasn't been tested yet.

The thing I keep coming back to: the port was worth it even where it failed. A week of finding out that a third of the design was a costume is cheaper than several years of believing it wasn't.