What is a zombie agent and how do I know if I have one?

A zombie agent is a deployed system that continues consuming compute, maintenance, and oversight resources without producing a tracked business outcome. If you cannot name the metric the agent is moving and show its current value, you likely have one.

How early should I define ROI metrics for an agent?

Before you write the first line of workflow logic. The metrics you care about determine which signals the agent needs to emit, which in turn shapes how you build the workflow. Bolting measurement on afterward usually means the data you need was never captured.

What is a realistic timeline to see ROI from an agent deployment?

According to a 2025 Google Cloud study, 74% of executives deploying agents in production report achieving ROI within the first year. That figure assumes the team was tracking the right metrics from launch, not from some later audit.

Do soft metrics like decision quality actually matter for a business case?

Yes, because they often predict hard metric movement before it shows up in revenue or cost figures. A rising escalation rate is an early signal that resolution costs will increase. Tracking it early gives you time to intervene before the cost impact lands.

Agent Deployments Don't Equal Returns: Building the Measurement Framework First

Agent deployments only prove their returns when measurement is designed in before launch. Write a job performance statement that sets the success metric, target and failure threshold, then instrument the workflow to emit three metric layers: hard operational metrics, soft quality metrics and revenue metrics. Review every agent at a 90-day gate: pass, remediate or decommission.

The gap between running and returning

According to McKinsey’s 2026 AI adoption data, 80% of organisations report measurable economic returns from AI agents, but only 23% say those agents have delivered significant ROI. That gap between “something is happening” and “we know what it is worth” is where most deployments stall.

Agents accumulate. Teams add a second agent before the first has proven its cost justification, then a third. After twelve months the organisation is running a portfolio of systems, some delivering, some coasting, and no one can tell them apart. Call those coasting systems zombie agents: live in production, consuming compute and oversight, but untethered from any outcome the business tracks.

The fix is not a better retrospective. It is designing measurement into the deployment from the start, the same way you design error handling into a workflow. You cannot instrument what you did not build hooks for.

Define the agent’s job before you define the agent

The first question is not “what should the agent do?” It is “what would count as this agent doing its job well?” Write that answer down in metric terms before the first workflow node exists.

Klarna did this cleanly. Their customer service agent had two measurable job definitions from deployment: handle volume (measured in the fraction of inquiries resolved without escalation) and handle it faster (measured in average resolution time). The results are documented: the agent now handles two-thirds of all customer inquiries, with average resolution time falling from 11 minutes to under 2 minutes, contributing to $60 million in documented savings by Q3 2025. Those numbers were trackable because the team knew what to track before launch.

For any agent you are planning, write a job performance statement in this form:

This agent succeeds when [metric A] reaches [target] within [timeframe], and fails review if [metric B] exceeds [threshold].

That statement becomes the acceptance criterion, the monitoring alert, and the decommission trigger.

The three-layer measurement stack

Layer one: hard operational metrics

These are the metrics with a direct line to cost or throughput. They are the easiest to defend in a budget review.

Cost per resolution. Total agent operating cost (compute, API calls, human-in-the-loop time) divided by resolved cases. Set a baseline using the pre-agent process cost and track the delta monthly.
Time saved. Measured in hours, not percentages. Morgan Stanley’s code review agent reviewed 9 million lines of legacy code and reclaimed approximately 280,000 developer hours, freeing 15,000 developers for product work. That figure is auditable. “Significant time savings” is not.
Throughput per unit cost. How many tasks does the agent complete per dollar spent? This metric exposes efficiency degradation over time as model costs or task complexity shifts.

Layer two: soft quality metrics

These metrics do not map directly to a line in the P&L, but they predict whether your hard metrics will hold.

Decision quality rate. For agents making judgment calls (routing, classification, drafting), sample outputs weekly and score them against a rubric. BNY Mellon’s Contract Review Assistant tracks legal review time as a proxy, reducing it from 4 hours to 1 hour across 3,000+ annual vendor agreements. The underlying quality signal is whether the agent’s output requires material correction.
Escalation rate. The fraction of tasks the agent hands to a human. A stable escalation rate means the agent’s capability envelope is well-matched to the task set. A rising rate means the task set is drifting, or the agent is degrading.
Capacity absorption. How much human capacity has been freed and then genuinely redeployed? If the team that used to handle a task is still the same size doing other things, the agent delivered capacity, not savings. Track what the freed capacity was used for.

Layer three: revenue metrics

Not every agent connects to revenue. But where one does, measure it directly rather than inferring it.

Conversion lift. For agents in sales or lead recovery workflows, measure conversion rate in agent-handled sessions against a control group. Cars24’s AI agents recovered 12% of previously lost seller leads across its automotive marketplace. That is a revenue metric, not a productivity metric.
Churn prevention. For customer-facing agents, track whether customers who interact with the agent renew or cancel at different rates than those who do not. Requires a clean cohort split from launch.

Designing workflows that emit measurable signals

An agent workflow that does not log its own decision points cannot be measured after the fact. Build the instrumentation into the workflow design, not the monitoring layer.

flowchart TD
    A[Task received] --> B[Agent attempts resolution]
    B --> C{Resolved?}
    C -- Yes --> D[Log: resolution time, cost, output]
    C -- No --> E{Escalated?}
    E -- Yes --> F[Log: escalation reason, handoff time]
    E -- No --> G[Log: failure mode, retry count]
    D --> H[Metric aggregation]
    F --> H
    G --> H
    H --> I{Review threshold crossed?}
    I -- Yes --> J[Alert: review agent performance]
    I -- No --> K[Continue operation]

Every branch in that flow is a data point. Resolution time, escalation reason, failure mode, retry count: these are the signals that tell you whether the agent is performing, degrading, or hitting an unknown category of task.

For AI agents operating in workflow automation contexts, the emit-on-branch pattern is the minimum viable instrumentation. Anything less and you are reading aggregate outputs, not agent behaviour.

The zombie agent decision gate

Run this check every 90 days for every agent in production:

flowchart TD
    A[Agent in production] --> B{Is there a current metric target?}
    B -- No --> C[Define target or decommission]
    B -- Yes --> D{Is the agent hitting it?}
    D -- No --> E{Has it missed for two cycles?}
    E -- Yes --> F[Remediate or decommission]
    E -- No --> G[Flag for review]
    D -- Yes --> H{Is the cost justified?}
    H -- No --> I[Optimise or decommission]
    H -- Yes --> J[Agent passes gate]

The gate has three exits: pass, remediate, or decommission. An agent that passes every gate is worth scaling. An agent that never reaches the gate is a zombie.

General Mills applied a version of this discipline at scale. Their logistics optimisation agents evaluate routing, carrier pricing, and load consolidation across 5,000+ daily shipments, producing $20 million in transportation cost savings since fiscal 2024, with $50 million in further waste reduction projected for 2026. The savings are auditable because the metric, transportation cost per shipment, was the design input, not the review output.

Connecting measurement to governance

Measurement is one input into a broader agent governance framework. The metrics you track should feed directly into whatever agent observability tooling your platform team runs. Tools like Prefactor sit in this category, giving teams a place to surface the signals each workflow emits and act on them without waiting for a quarterly review.

For teams building across multiple agent types, agent evaluation practices and AI governance best practices give you the structural context for where measurement sits in the broader delivery cycle. The workflow design choices you make early determine whether that evaluation is possible at all.

Where to start

Before your next agent goes into planning, write its job performance statement and identify the three metrics you will track from week one. If you are not sure which agents in your current portfolio are earning their keep, take the agent readiness assessment to map where your measurement gaps are and which deployments are most at risk of becoming zombies.