What is a confidence threshold in an escalation flow?

A confidence threshold is a numeric boundary, set during design, below which the agent stops acting autonomously and routes the conversation to a human. The exact value depends on the consequence of a wrong answer: a refund dispute warrants a lower threshold than a shipping status query.

What is the difference between tier-1 human review and real-time handoff?

Tier-1 review happens after the agent completes a task, a human checks the output before it reaches the customer or takes effect. Real-time handoff interrupts the conversation mid-session and transfers context to a live agent. Use tier-1 review for low-urgency, high-stakes tasks; use real-time handoff when the customer is waiting and the stakes are immediate.

How do I know when my agent is hurting the business rather than helping it?

Watch repeat contact rate, customer effort score, and post-resolution NPS alongside cost-per-conversation. If cost falls while repeat contacts rise, the agent is resolving without solving. That gap is where revenue leaks before any dashboard turns red.

Does adding escalation logic slow down resolution times?

A well-designed escalation path adds seconds to a routed conversation but prevents the repeat contacts that cost three to five times more than first-contact resolution. The latency cost is real; the alternative is worse.

When Agents Hand Off: Designing Escalation Flows That Protect Revenue

What you will build, and why it matters

This article shows you how to design escalation flows that catch silent failures before they erase the savings your agents create. You will leave with a decision boundary model, a two-tier escalation choreography, and a metric set that signals agent harm before your dashboards turn red.

The Klarna story is now the canonical warning. The company’s AI customer service agent handled 2.3 million conversations in its first month after launching in February 2024 and saved $60 million by Q3 2025. It also raised repeat contacts by 25% and eroded customer NPS, because the agent provided generic answers on complex issues instead of escalating. By May 2025, Klarna was rebuilding human capacity it had cut. The savings were real. The damage was also real. Both happened at the same time, and cost savings were the only thing being measured.

Only 18% of organisations are measuring whether their AI investments are actually working; of those, 77% track only internal metrics like cost savings and employee usage, almost none measure client satisfaction or revenue impact. That measurement gap is what turns a confident agent into a quiet revenue problem.

Understanding how agents fail at boundaries is prerequisite knowledge here. If you need a foundation first, how AI agents work and agentic AI architecture cover the underlying mechanics.

Why agents fail at boundaries

An agent does not know what it does not know. A language model produces outputs with consistent fluency regardless of whether the underlying information is reliable, out of distribution, or simply absent from its context. The failure mode is not silence; it is confident wrongness delivered at scale.

Roughly 5% of enterprise AI agents ever reach production. The other 95% die in prototype, often because they were never stress-tested on edge cases that require escalation. The agents that do reach production frequently carry the same untested boundaries with them.

The boundary problem has three components:

Distributional shift: the agent encounters an input type it was not trained or prompted to handle.
Ambiguity accumulation: the customer’s intent is unclear, but the agent resolves ambiguity internally rather than surfacing it.
Stakes mismatch: the agent applies the same confidence to a $4 shipping query and a $4,000 disputed charge.

Designing for escalation means making these three failure modes explicit before deployment, not diagnosing them from NPS data six months later. Agent evaluation frameworks give you tooling to surface them in staging.

Decision boundary design

A decision boundary is a rule, or a set of rules, that determines whether the agent acts, waits, or hands off. The boundary is not a single number; it is a policy that varies by task type, customer context, and consequence of error.

flowchart TD
    A[Customer input received] --> B{Intent classification\nclear?}
    B -- No --> C[Request clarification\nonce]
    C --> D{Clarified?}
    D -- No --> E[Escalate: real-time handoff]
    D -- Yes --> F{Confidence above\ntask threshold?}
    B -- Yes --> F
    F -- No --> G{Low-urgency task?}
    G -- Yes --> H[Complete task,\nflag for tier-1 review]
    G -- No --> E
    F -- Yes --> I[Agent resolves autonomously]
    I --> J[Log outcome for\npost-session review]

The threshold values in the diagram are not universal. A billing dispute in financial services warrants a lower autonomous-action threshold than a product availability query in retail. Set thresholds by consequence class, not by agent capability.

Morgan Stanley’s approach to its DevGen.AI code translation tool is a useful structural reference. After reviewing 9 million lines of legacy code and saving 280,000 developer hours in five months, the firm deliberately kept a human verification layer for all legacy-to-modern code rewrites rather than allowing the agent to act autonomously on production systems. The boundary was not set by the agent’s capability; it was set by the consequence of a wrong output reaching a production codebase.

Escalation choreography: tier-1 review versus real-time handoff

Not every escalation requires a live human in the loop at the moment it occurs. The choice between tier-1 asynchronous review and real-time handoff depends on two variables: how long the customer can wait, and how much damage a wrong answer does before it is caught.

flowchart TD
    A[Escalation triggered] --> B{Customer\nstill in session?}
    B -- No --> C[Tier-1 review queue]
    C --> D[Human reviews output]
    D --> E{Approved?}
    E -- Yes --> F[Deliver to customer]
    E -- No --> G[Human corrects and delivers]
    B -- Yes --> H{High-stakes or\nambiguous intent?}
    H -- No --> I[Agent completes,\nlogs for async review]
    H -- Yes --> J[Real-time handoff\nwith full context transfer]
    J --> K[Human agent\ntakes conversation]

Thomson Reuters built this distinction into CoCounsel, its legal research agent. Agents handle multi-step document review autonomously, but all outputs require lawyer sign-off before client delivery. The agent does the work; the lawyer is the delivery gate. That is tier-1 review applied to a domain where a wrong answer has professional liability consequences.

Intercom’s Fin agent handles the real-time version well. With an 82% autonomous resolution rate across more than 40 million conversations, the architecture is designed so the agent recognises unclear intent and escalates before producing a wrong answer, not after. The incentive structure reinforces this: one outcome charge per conversation regardless of escalations, so the system is not penalised economically for handing off.

Context transfer is the part most teams under-specify. When a real-time handoff occurs, the receiving human agent needs the full conversation history, the agent’s last confidence signal, and the reason for escalation, in readable form, before they say a word to the customer. A handoff that forces the customer to repeat themselves is not a solved escalation; it is a second failure.

For teams building on specific frameworks, LangGraph’s architecture and CrewAI’s multi-agent patterns each have native mechanisms for passing state across handoff boundaries.

Metrics that signal harm before dashboards turn red

The standard cost-per-conversation metric tells you what automation costs to run. It does not tell you what automation costs to fix. You need a parallel set of customer-facing signals.

The metrics that catch escalation failures early:

Repeat contact rate by resolution type: if contacts resolved autonomously generate more repeat contacts than human-resolved contacts, the agent is closing without solving.
Customer effort score at session end: collected immediately, before the customer has processed the outcome. A low score on an autonomous resolution that later generates a repeat contact is a leading indicator.
Escalation refusal rate: how often the agent was in a low-confidence state but did not escalate. This requires logging the confidence signal alongside the action taken.
Post-handoff NPS delta: the difference in NPS between customers who were escalated cleanly versus those who had to request escalation. A large negative delta means context transfer is failing.
Time to escalation: if escalation consistently happens after three or more agent turns on ambiguous inputs, the agent is attempting resolution before recognising its boundary.

Implementing agent observability gives you the instrumentation layer that makes these signals available in near-real time. Without it, you are reading the post-mortem after the NPS drop, not the signal before it. Tools such as Prefactor sit in this observability category, surfacing confidence signals and handoff patterns across live agent traffic.

AI governance frameworks should specify which of these metrics trigger a human review of the agent’s decision boundary policy, and at what cadence. A monthly review cycle is too slow when repeat contact rate is climbing weekly.

Where to start

Before you build escalation logic, you need a clear picture of where your agent’s boundaries currently sit and which failure modes your measurement stack can actually detect. Take the agent readiness assessment at /ai-agent-readiness-assessment to identify the gaps before they surface in production data.