Human gates aren't there because the agent can't do the work

the wrong question, asked very seriously

The argument about agent autonomy has settled into a shape. On one side: models are getting better, the confirmation prompts are friction, and every gate you remove is a productivity gain. On the other: they still make mistakes, the mistakes are getting faster, and somebody needs to be watching.

Both sides are arguing about the same axis. Is it good enough to be trusted with this? The disagreement is only about where on that axis we currently are.

I think the axis is wrong, and I think you can see it is wrong by asking the same question about people.

Consider a payments engineer with fifteen years of experience, who has never once made a mistake that moved money incorrectly. Do they get to push a change to the settlement path without a second approver?

No. Obviously not. And nobody involved in that decision believes it is a judgement about their competence. The reviewer is frequently more junior than they are. The gate is not there because the engineer might get it wrong. The gate is there because moving money is the kind of action an organisation has to be able to account for afterwards — who did it, who agreed to it, under what authority, and what the reasoning was.

That is a different property from correctness, and no amount of demonstrated skill removes the need for it.

capability and accountability are different axes

Once you separate them, a lot of the autonomy debate resolves into a category error.

Capability is about whether the actor produces good outcomes. It improves with better models, better tools, better context. It is measurable, roughly, and it is genuinely getting better fast.

Accountability is about whether the organisation can answer for what happened. It does not improve with model quality at all. A perfect actor with no accountability trail is still a problem for anyone who has to explain a decision to a regulator, a customer, a court, or an angry finance director.

The confusion comes from the fact that in most engineering contexts, we have only ever used gates for the first reason. Code review exists mostly to catch mistakes. So when a tool starts making fewer mistakes, removing the gate feels like the obvious consequence.

But the categories of action where gates persist for humans — payments, deletion of records, access grants, anything with legal or contractual weight, anything that reaches a customer — are exactly the categories where the gate was never really about competence. We just did not have to notice, because the same review served both purposes.

Agents force the two apart. And the useful question stops being how good does it have to get and becomes what class of action is this, and who is answerable for it.

the design consequence

This is where the reframing stops being philosophy and starts changing what you build.

If a gate is a capability judgement, then the natural design is: give the agent an account, grant it the permissions it needs, and tune the confirmations down as it proves itself.

If a gate is an accountability boundary, that design is wrong from the first line. Because the moment the agent has its own account and its own permissions, you have created a second actor with a second set of rules — and the question who was answerable for this has no good answer. The agent did it. On whose authority? Its own, apparently.

The alternative is that an agent holds no permissions of its own.

It acts for a person. Its ceiling is whatever that person could do unaided — never more, and it does not get to accumulate a union of what everyone can do. If the person cannot delete a customer record, then neither can the agent while acting for them, and the agent's own notional capability is irrelevant.

Two properties follow, and they are the ones I would insist on:

The check happens at the moment of action, not at the start of the session. Anything decided up front is a snapshot, and snapshots go stale. Permissions get revoked. Someone leaves a team mid-task. An object changes owner between the plan and the execution. If authority is established once and then carried, you have built a system where the gap between decision and action is a window, and windows get exploited — usually not maliciously, usually by an ordinary race between a long-running task and someone updating a role in another tab.

The ceiling can only narrow, never widen. Whatever coarse envelope you compute at the start, the real check downstream re-derives authority from the same source a human request would. So the envelope is a cheap early rejection, not a grant. If the two ever disagree, the narrower one wins by construction, because the broad one was never the thing doing the authorising.

Put together, these mean the agent path and the human path cannot drift, because they are the same path. That is the actual prize. Not that the agent is constrained — that there is exactly one implementation of who-can-do-what, and it is the one that was already there and already tested.

the counterargument, which is good

The strongest objection is not that this is wrong. It is that it is expensive and slow.

Real version of the argument: if you gate every consequential action on a human, you have not built an agent, you have built an unusually chatty form. The value was supposed to be that it goes and does the work. An agent that stops eleven times to ask permission is worse than the six clicks it replaced, and users will abandon it — correctly.

I think this is largely true, and it is the failure mode I have watched happen. An agent that asks about everything trains people to approve without reading, which is worse than no gate at all, because now there is a signature on it. A confirmation that is always granted is not a control. It is a liability with a UI.

But the answer is not fewer gates. It is that the gate belongs to the class of action, not to the agent's confidence. Reads do not need one, ever, regardless of how uncertain the model is. Ordinary writes with a straightforward undo do not need one either — undo is your accountability mechanism, and it is a better one. What needs a gate is the small set of actions that are irreversible, that reach outside the system, that touch money, or that change who can do what. That set is usually much smaller than a nervous first implementation assumes, and it does not grow when the model has a bad day.

If your agent is asking eleven times, the problem is almost certainly that you classified by risk-of-being-wrong instead of by consequence-if-wrong. Those produce very different lists.

The second objection is more uncomfortable: that deriving everything from the acting human means an agent can never do the things a human is not permitted to do, and some genuinely useful automation lives exactly there — batch operations, cross-tenant maintenance, cleanup nobody has standing to perform individually.

That one I accept. My answer is that this work is not an agent acting for a person; it is a system job, and it should be built and governed like one — owned by a service identity, scheduled, logged, and reviewed as infrastructure. Dressing it up as an assistant does not make the accountability question go away, it just makes it harder to see.

what this is not

I am deliberately not claiming any of this makes anything safe. Safety is a property of a whole system under adversarial conditions, and I have not proved that about anything. What this design does is narrower and more defensible: it keeps the blast radius of a mistake bounded by what one person could already have done, and it keeps a single answer to the question of who was answerable.

Those are modest properties. They are also the two that come up first when something goes wrong and somebody senior asks what happened.

the shape of the argument, restated

The industry conversation is about a dial marked autonomy, and everyone is arguing about where to set it.

I think there are two dials. One is capability, and it is moving on its own without our input. The other is accountability, and it does not move at all — it is a property of what the action is and what your organisation has to be able to answer for.

Design against the second one and the first stops being a source of anxiety. Better models make the agent more useful inside a boundary that did not need to change. Design against the first, and every model release is a fresh argument about whether to loosen something you never had a principled reason for tightening.