Native tool calling is a portability trap

the swap that took an afternoon

We moved the primary model on a Tuesday. Most of the forty minutes was CI. Nothing in the agent loop changed, because nothing in the agent loop knows what a tool call is.

That's the argument, so here it is plainly: if you're going to run more than one model provider, don't build your agent on native tool calling. Ask the model for a structured "next action" object against your own schema, dispatch it yourself, and keep provider-specific code confined to a thin completion adapter. You give up real ergonomics doing this. I still think it's right, and I'll make the case against it as well as I can before answering it.

the loop

Plan, act, observe. Each step assembles a context — the system contract, the scope the run was invoked with, the transcript so far, whatever retrieval returned — and asks for exactly one thing:

export const nextAction = z.discriminatedUnion('type', [
  z.object({
    type: z.literal('plan'),
    thought: z.string().max(400),
    steps: z.array(z.string()).min(1).max(8),
  }),
  z.object({
    type: z.literal('tool'),
    thought: z.string().max(400),
    tool: z.string(),
    args: z.record(z.unknown()),
  }),
  z.object({
    type: z.literal('finish'),
    thought: z.string().max(400),
    answer: z.string(),
  }),
]);

const action = await this.llm.completeStructured({
  model: run.model,
  schema: nextAction,
  system: this.contract(run),
  messages: this.transcript(run),
});

completeStructured is the only method the loop calls. Underneath it's responseSchema on Gemini, response_format with a JSON schema on OpenAI and DeepSeek, and a schema-constrained call on Anthropic. Four adapters, one signature. The loop itself hasn't been edited once when a provider was added.

Everything after that is ordinary code. Validate the action. Check the named tool is inside the run's allowed scope. If the tool is marked destructive, stop the step and require confirmation rather than running it. Append the observation to the transcript. Decrement the step budget. Repeat until finish, or until the step or token budget runs out.

what this buys

The obvious answer is provider portability, and that's real — DeepSeek was about 140 lines and an afternoon — but it isn't the answer I'd lead with.

I'd lead with the transcript. If tool invocation lives inside the provider's message loop, your conversation state is that provider's conversation state: OpenAI's tool_calls with stringified arguments, Anthropic's tool_use and tool_result content blocks threaded together by id, Gemini's functionCall and functionResponse parts nested inside contents. Those aren't three names for one structure. They're three state machines with different rules about what may follow what. And that state is the thing you persist, replay while debugging, render in an audit view, and resume after a crash. Build on native tool calling and you've put a vendor into your database schema.

Second is governance, which is why I wouldn't switch back even running a single provider.

Scope gating, plan-and-confirm before any write, an always-stop on destructive tools, step and token budgets, a recorded transcript of every action with the arguments it actually ran with — all of it lives in one dispatcher. One code path, tested once, behaving identically regardless of which model is driving. The moment tool execution happens inside a provider SDK's own loop, every one of those controls has to be reimplemented per provider, and the one you get subtly wrong is the one that runs a delete against production because a retry replayed a step. I'd rather have a single place to be careful in.

Third, mundane, and it pays every month: model choice becomes a routing decision instead of a rewrite. Read-only runs plan on a cheaper model. A provider failing mid-run means retrying that step against a different one with the same transcript, because the transcript is mine.

the case against, taken seriously

Native tool calling is better. Not marginally, and not in a way I want to hand-wave past.

The models are trained on it. Tool schemas go in through a channel the model was post-trained to use, and argument generation is constrained at the sampling layer by the provider's own machinery rather than by me asking nicely in a system prompt. Parallel calls come free. Streaming partial arguments works. The round-trip bookkeeping is handled for you. Prompt caching behaves better, because a stable tool-definition block is a prefix the provider recognizes and caches, whereas my contract is a system prompt I keep editing, and every edit costs me the cache.

The accuracy gap is measurable, too. On the same 120-case fixture set I use for MCP tools, the strongest model I run picks the right tool 96% of the time through native tool calling and 91% through my structured-action contract. Five points. That is not a rounding error, and anyone who tells you the abstraction is free hasn't measured it.

Then the maintenance cost, which is the honest one. I'm not maintaining code here, I'm maintaining a prompt contract — a document that explains the action schema, gives worked examples, and says what to do when the model is unsure. Documents drift. When a tool's description changes, the contract's examples ought to change with it, and no compiler will tell me they didn't. I once shipped a contract edit that quietly stopped the model emitting plan actions for two days. Runs still completed. They were just worse, and the only reason I caught it was a step-count graph that had gone flat.

The strongest version of the argument is the one about time: you're building a compatibility layer for a problem that's evaporating. The shapes are converging. DeepSeek is already OpenAI-shaped for exactly this reason. Give it eighteen months and every provider speaks one dialect, and you'll have paid five points of accuracy and an unversioned prompt document for a difference that no longer exists.

I think that's half right, and it's the half that matters least.

Request shapes are converging. Behaviour isn't. How many tools a model will chain before it answers, what it does when a tool returns an error, whether it retries the identical call with identical arguments, how it handles parallel calls it shouldn't have made in parallel — those still differ sharply between providers on identical inputs, and those are what break agents in production. Field names have never broken anything. If I have to write per-provider handling for behaviour regardless, I'd rather that handling sit in my loop where I can read it than be scattered across four SDKs' internal control flow.

And the portability I actually care about isn't sideways, it's forwards. The loop I wrote runs today on models that didn't exist when I wrote it, because it asks for the one capability everyone converged on years ago: emit an object matching this schema. That bet also covers the case people forget about — a smaller open-weights model running inside a client's own network for data-residency reasons has mediocre native tool calling and perfectly adequate constrained JSON.

where the argument actually breaks

Providers are portable. Model tiers are not.

The five-point gap I quoted is for the strongest model I run. Put the same fixture set in front of the cheapest one and native tool calling holds up far better than my contract does — the weak model drops to around 68% on structured actions, because following a written protocol is precisely what weak models are bad at, and constrained decoding at the sampling layer doesn't care how much reasoning capacity is behind it.

So the rule I use: a model scoring below 85% on the fixture set doesn't get to drive the plan step. It can summarize and it can extract. It doesn't pick actions. That's a genuine limitation of this design, and I don't have a way around it beyond measuring every model before letting it near the loop.

I'd also break my own rule without much hesitation in one case. One product, one provider, no plausible reason to move, no compliance story that might force a self-hosted model later — use native tool calling and never think about it again. The trap doesn't close until the second provider arrives, and plenty of products never get a second provider.

The thing I still don't have a good answer for is streaming. Native tool calling lets you stream the assistant's prose while it's still deciding what to do. Asking for one structured object means waiting for the object. I paper over it by streaming the thought field as it arrives and rendering the plan incrementally, but there's a beat of nothing at the start of every step, and people notice it on the first step of every run. If someone has solved that cleanly without going back to native tool calls, I'd like to hear about it.