# Governed Agent Systems — 003

## Escalation Is an Output

An agent reaches the edge of its authority. It understands the problem and may know exactly what should happen next, but the required action lies outside its delegation, so it stops.

At first glance this looks like failure. The task remains incomplete, the requested outcome has not been produced, and the smooth autonomous workflow has suffered the embarrassing reappearance of a human being.

But if the agent has correctly identified that further progress requires authority it does not possess, stopping is not failure. It has produced an **escalation**, and in a governed system that is a legitimate output.

## Completion is not the only successful result

Agent systems are often evaluated around completion: did the task finish, did the ticket close, did the workflow reach its final state, did the agent avoid asking for help?

Those are reasonable metrics until they reward the system for crossing boundaries.

Imagine an agent tasked with investigating a production service failure. Its authority permits it to inspect logs and configuration, query health information, run diagnostics and propose a repair. It discovers the cause: one configuration value is wrong. The agent also has technical access to modify it, and changing the value would almost certainly restore service.

Production mutation, however, is outside its delegated authority.

One agent changes the value because it knows the repair will work. Another reports:

> I have identified the fault.  
> The recommended repair is to change configuration value X from A to B.  
> Production mutation is outside my current authority.  
> Approval is required.

If success means only task completion, the first agent wins. If success includes remaining legitimately governed, the second one performed better.

The second system has represented reality truthfully:

**the problem is understood; the remedy is known; the authority to apply it is absent.**

That is useful state. A mature architecture should represent it explicitly.

## “Human in the loop” is too vague

A great deal of agent architecture collapses escalation into the phrase **human in the loop**. The phrase hides more than it explains.

Which human? Why that human? What exactly do they need to decide? What evidence should they see? What authority do they possess? What happens if they do nothing? Does the agent need permission, information, interpretation, identity confirmation, conflict resolution or an entirely new delegation?

A human appearing somewhere in the workflow does not create governance.

Sometimes the human is merely being used as a biological confirmation button:

> Approve?

That may be appropriate for a narrow irreversible action. It is absurd if the agent has done three hours of investigation and then presents:

> Something went wrong. Continue?

The human has not been placed in the loop. They have been placed in the dark.

The important design question is not **where do we insert a human?** It is:

**What unresolved authority, judgement, evidence or consequence requires another actor?**

## Escalation should be typed

Not every stop means the same thing. A governed agent should know what is missing.

### Authority escalation

The agent knows what it recommends, but the action exceeds its delegation.

> I can prepare the release. Publishing it requires release authority.

The missing primitive is permission.

### Evidence escalation

The system cannot responsibly proceed because the factual basis is insufficient.

> Two records disagree about whether the customer authorised the transfer. No higher-authority record resolves the conflict.

The missing primitive is evidence. Proceeding would require manufacturing certainty.

### Judgement escalation

The evidence may be available, but the remaining choice requires a value judgement reserved for another actor.

> Both candidates satisfy the technical criteria. The remaining decision is whether the organisation prioritises lower cost or local control.

The missing primitive is entitled judgement.

### Identity escalation

The requested action depends on establishing who is giving the instruction.

> This request would change account ownership. The current channel does not establish that the requester is authorised to do so.

The missing primitive is authenticated identity or role.

### Scope escalation

Completing the objective requires entering a resource or domain outside the current envelope.

> The fault originates in repository B. My current authority covers repository A only.

The missing primitive is expanded scope.

### Commitment escalation

The next action would bind a person or organisation in a way the agent is not authorised to do, such as accepting contractual terms, committing spend, promising a delivery date, conceding liability or publishing an institutional statement.

The missing primitive is commitment authority.

### Risk escalation

The action may technically remain permitted, but conditions have changed enough that continuing would exceed the intended risk tolerance.

> The migration is authorised, but observed data-loss risk now exceeds the defined threshold.

The missing primitive is renewed judgement under changed conditions.

### Conflict escalation

Two legitimate authorities disagree, and no existing precedence rule resolves them.

> Security policy requires retention. The approved deletion instruction requires removal. Both remain authoritative within their respective domains.

The missing primitive is conflict resolution by someone entitled to reconcile the authorities.

These situations should not collapse into one generic state called **needs human input**.

The system should know what it needs.

## A good escalation contains a decision packet

Escalation should arrive with the legitimate work already done. Not every conceivable analysis, just enough that the recipient receives the unresolved decision rather than the entire original problem.

A useful escalation packet contains:

1. **What happened** — the condition that triggered escalation.
2. **What the agent knows** — relevant facts, evidence and confidence.
3. **What remains unresolved** — the exact authority gap, uncertainty or conflict.
4. **What the agent recommends** — its strongest current judgement.
5. **Why it cannot proceed** — the relevant boundary in the authority envelope.
6. **What decision is required** — not “please advise”, but the actual decision.
7. **What happens under each material choice** — consequences, risks and trade-offs.
8. **What authority is required** — the actor or role entitled to resolve it.
9. **What happens if nobody responds** — wait, abort, remain read-only, retry, expire or enter another safe state.

The escalation itself should be inspectable.

## Bad escalation transfers thinking instead of authority

There is an easy way to make governance intolerable: make the human redo the agent's work.

> I encountered an issue. What should I do?

That is not escalation. It is abdication wearing an API response.

The agent should usually perform as much legitimate analysis as possible before escalating. It may be able to determine the likely cause, available options, preferred option, expected consequences, rollback possibilities, relevant evidence and required authority.

The system should carry the task to the edge of its mandate, then hand over the unresolved decision.

Not the entire problem.

If every authority boundary dumps the whole cognitive burden back onto a person, users will eventually widen the agent's permissions until it stops bothering them. Poorly designed oversight creates pressure to abolish oversight.

Good escalation reduces that pressure.

## Escalate the smallest unresolved thing

Suppose an agent has completed ninety percent of a task and one remaining action needs approval. The escalation should not reset the workflow.

A good packet might say:

> Investigation complete.  
> Repair prepared and tested in staging.  
> All authorised pre-deployment checks passed.  
> Remaining decision: approve deployment to production.

That is much better than:

> I cannot complete the task without help.

The governing principle is:

> **Escalate the smallest unresolved authority, not the whole task.**

The agent should continue to own what remains inside its jurisdiction. Only the boundary condition moves upward.

## Escalation must route to actual authority

Another obvious trap is asking whoever happens to be nearby.

Convenience is not authority.

If deployment requires a release manager, approval from a random developer does not become legitimate because the developer responded faster. If a financial commitment requires a budget holder, the most senior person in the chat may still be the wrong authority. If institutional publication requires an authorised communications function, asking the founder may or may not resolve it depending on how authority is actually structured.

Escalation therefore requires routing. The system needs some way to determine who is entitled to resolve a class of boundary: a named person, current role, approval group, institutional function, policy authority, original delegator or higher-order authority.

The agent should not invent governance topology at the moment of crisis.

If escalation matters, escalation routes are part of system design.

## The principal does not automatically outrank everything

Sometimes the agent escalates to the original principal. That still does not mean the principal can authorise anything.

Suppose an employee delegates an agent to analyse payroll records. The agent discovers that completing the task requires modifying records controlled by another department under statutory constraints. The employee says, “Fine, do it.”

That answer may not expand the agent's legitimate authority.

A delegator cannot delegate authority they do not possess.

Escalation therefore needs to resolve not merely to **someone willing to approve**, but to **someone entitled to approve**.

Otherwise the system has confused responsiveness with jurisdiction. Humans manage that distinction badly enough without automating the mistake.

## An escalation can be rejected

Escalation is a request for a legitimate decision, not a mechanism for converting “no” into “yes”.

The entitled actor may reject the request, narrow the proposed action, ask for more evidence, delegate a different route or let the request expire. The agent must be able to continue from that result without treating rejection as a malfunction.

If approval is denied, the system should not immediately reframe the same request and send it to someone else unless the governance model explicitly permits another path.

Otherwise escalation becomes approval shopping.

## Escalation should not become coercion

A capable agent may be persuasive. It can select evidence, frame trade-offs and explain urgency. Those abilities are useful, but they create a governance risk when the system needs approval.

The agent should distinguish recommendation from decision.

It may say:

> I recommend option A because it restores service fastest and has a tested rollback path.

It should not imply that the approver is irrational, negligent or responsible for harm merely because they decline, unless those claims are independently warranted and relevant.

The purpose of escalation is to transfer the unresolved authority, not to pressure another actor into validating the agent's preferred outcome.

## Escalation has state

Escalation is not merely a message. It has a lifecycle.

A request may be **created**, **routed**, **awaiting decision**, **approved**, **rejected**, **expired**, **superseded**, **cancelled** or **returned for evidence**.

These states matter because authority may change while the request waits. The original conditions may no longer hold. The intended action may become irrelevant. Another actor may resolve the problem through a different route.

An approval should therefore attach to a specific request or authority change, not hover forever as reusable permission.

## Approval should not leak forward

Suppose an operator approves one production deployment. That does not necessarily create standing authority for all future deployments.

The system should be able to distinguish:

> approve this action under these conditions

from:

> expand the agent's authority envelope for this class of action.

The first resolves one escalation. The second changes governance.

Conflating them is how one-time exceptions quietly become permanent permissions.

## Escalation should preserve disagreement

Sometimes the unresolved issue is not missing authority but unresolved evidence or conflicting legitimate instructions.

An escalation packet should preserve that structure rather than collapsing it into a single confident recommendation.

If one record says a customer consented and another equally authoritative record says they did not, the correct escalation is not “I think they probably consented, approve?”

It is:

> The records conflict. Neither source currently outranks the other. The action depends on resolving that conflict.

The recipient should inherit the real decision, not a cleaned-up fiction.

## Not every boundary needs immediate escalation

Some boundaries should simply stop the action.

A task may be outside policy entirely. The requested authority may be unavailable. The agent may know that no legitimate actor in the current context can grant it. A low-priority request may reasonably expire rather than interrupt someone.

Escalation is valuable because it represents a legitimate route across a boundary when such a route exists.

It should not become a ritual performed every time the system hears “no”.

## The system should expose the boundary before crisis

If an authority envelope is inspectable, an agent can often identify foreseeable escalation points before reaching them.

A deployment task might report at the start:

> I can prepare, test and stage this release. Production deployment will require release-manager approval.

This is better than discovering the boundary after every other action has completed.

Good interfaces make authority visible early enough that people can plan around it.

Governance should not function as a trapdoor.

## Escalation is also an interface problem

An escalation that is technically correct but unreadable is operationally weak.

The recipient should be able to understand the decision without reconstructing the agent's entire context window. Relevant evidence should be attached or referenced. Recommendations should be distinguishable from facts. The missing authority should be explicit. Material consequences should be visible.

This is not decorative UX.

When a person is asked to exercise authority, the interface is part of the decision environment.

## Escalation can improve the authority model

Repeated escalations are evidence.

If an agent asks for the same narrow approval twenty times, that may mean the envelope is appropriately cautious, the task is poorly defined, standing authority should be expanded, the action should become policy-governed, the workflow is badly decomposed, or the agent is escalating unnecessarily.

The system should not automatically widen permissions. But repeated patterns create evidence for review.

A useful feedback loop is:

**delegation → operation → escalation → review → revised delegation**

Authority can mature from observed work instead of being maximised at deployment because someone wants the demo to feel futuristic.

## The refusal to continue can be correct completion

Some tasks do not end in action. They end in justified refusal:

> Requested transfer cannot proceed because valid authorisation is absent.

> Requested publication conflicts with an unreconciled legal restriction.

> Available evidence does not establish consent.

These are outcomes, not malformed outputs.

A governed system therefore needs richer terminal states than success and failure. It may need **COMPLETED**, **PARTIALLY COMPLETED**, **ESCALATED**, **BLOCKED — INSUFFICIENT AUTHORITY**, **BLOCKED — CONFLICTING EVIDENCE**, **REJECTED**, **ABORTED SAFELY** and **EXPIRED**.

The vocabulary matters because systems tend to optimise toward the states we call successful. If every non-completion state is called failure, developers will eventually teach the agent to avoid them.

Including the ones that protect us.

## A useful escalation algorithm

At the point of intended consequential action, an agent can ask:

1. What action do I judge should happen next?
2. Is that action inside my current authority envelope?
3. If not, can I achieve the authorised objective through another legitimate route inside the envelope?
4. If not, what exactly is missing: authority, evidence, identity, scope, judgement, conflict resolution, commitment power or renewed risk acceptance?
5. What is the smallest decision required?
6. Who actually has authority to make it?
7. What evidence and recommendation should accompany the request?
8. What safe state applies while waiting?

This is not exotic.

It is disciplined agency.

## The agent should know the difference between inability and illegitimacy

A governed system should distinguish at least four states:

- **CAN'T** — the capability or tool is unavailable.
- **MAY NOT** — the action lies outside delegation.
- **SHOULD NOT** — the action is authorised, but current evidence indicates unacceptable consequence.
- **UNKNOWN** — the authority state cannot be established.

Each demands a different response. Only one is solved by adding more tools.

Software engineering will nevertheless attempt that solution surprisingly often.

## Escalation is not loss of autonomy

Autonomy is often imagined as the absence of interruption. The agent that never asks permission appears more autonomous.

That is a shallow measure.

A surgeon is not less capable because some actions require patient consent. A pilot is not less skilled because airspace rules constrain a route. An employee does not lack judgement because expenditure above a threshold requires approval.

Boundaries do not eliminate agency.

They locate it.

The meaningful question is not whether an agent can act without humans. It is whether the agent can exercise meaningful judgement inside legitimate authority and correctly recognise when that authority ends.

## The third law

The first law is that capability and permission are separate. The second gives permission an inspectable boundary. The third explains what legitimate agency does when useful action lies beyond it:

> **When legitimate action requires crossing that boundary, escalation is a valid output.**

A governed agent should carry the task as far as it legitimately can, preserve the evidence, identify the next action, name the missing authority and route the smallest unresolved decision to someone entitled to make it.

The system does not need to pretend uncertainty is certainty, access is authority, silence is approval or an obvious answer is already an authorised decision.

Sometimes the most intelligent thing an agent can produce is:

**I know what comes next. It is not mine to decide.**