What is the difference between a synchronous and an asynchronous AI agent?

A synchronous agent executes a task and holds the caller waiting for a result, the way a database query blocks until rows return. An asynchronous agent accepts a task, releases the caller immediately, and completes work in the background, reporting results when they are ready. Most multi-step workflows spanning minutes or hours require the asynchronous model.

How do I make an agent action idempotent?

Attach a stable, deduplicated identifier to every action before the agent fires it. When the action runs again after a retry, the downstream system checks whether that identifier has already been processed and skips execution if it has. The identifier can be a hash of the input payload, a UUID generated at job creation, or a workflow-run ID, as long as it is the same across all attempts for the same logical operation.

What state does an agent need to persist across a long-running workflow?

At minimum: the current step index, the inputs and outputs of each completed step, any external identifiers returned by downstream systems (such as a payment reference or a case ID), and the retry count and last-error for the current step. Without those fields, a resumed agent cannot distinguish a fresh run from a recovery run, which leads to duplicate actions or silent skips.

When should a long-running agent escalate to a human instead of retrying?

Set a maximum retry count per step (three to five is common) and a maximum elapsed time for the whole workflow. If either limit is breached without a successful result, the agent should pause, write its current state, and route the job to a human review queue rather than retrying indefinitely. Uncontrolled retries against a degraded external system can cause more damage than the original failure.

Asynchronous AI Agents: When Immediate Execution Isn't an Option

What you will get from this guide

By the end of this guide you will know when to run an agent synchronously versus asynchronously, how to wire retry logic and idempotency into multi-step workflows, and how to persist state so an agent can resume after a failure without repeating completed work. The patterns here apply directly to batch reconciliation, compliance workflows, payment settlement, and scheduled automation.

Why forcing real-time execution breaks agent reliability

Agents that wait for a human response or a sub-second API call are easy to reason about. The agent fires, the system responds, the result is recorded. But most production workflows in financial services, healthcare, and operations do not look like that. A trade settlement may span overnight batch windows. A compliance review may wait on a third-party data provider that responds in minutes. A fraud investigation may need to correlate events that arrive hours apart.

When you force those workflows into a synchronous execution model, you pay three costs. First, you hold open connections and compute resources for minutes or hours, which is expensive and brittle. Second, any single failure in the chain restarts the entire workflow from the beginning, because you have no checkpoint. Third, the agent cannot be observed or interrupted mid-run, which is a governance problem on top of an engineering one.

According to a 2026 Anthropic and arXiv study on measuring agents in production, only 5 of 20 production agents actually require real-time responsiveness. The remaining 15 tolerate minute-scale latency, and in practice prioritise output quality and reliability over speed. Designing for the real distribution of your workflows, rather than the fastest possible case, is what separates agents that reach production from the 88% of agent pilots that never do.

Deciding whether a workflow should be synchronous or async

The decision is not about preference. It follows from the nature of the work.

flowchart TD
    A[New agent task arrives] --> B{Response needed\nwithin ~2 seconds?}
    B -- Yes --> C[Synchronous execution]
    B -- No --> D{Steps span multiple\nsystems or time windows?}
    D -- No --> C
    D -- Yes --> E[Asynchronous execution]
    E --> F[Write initial state to store]
    F --> G[Enqueue first step]
    G --> H{Step succeeds?}
    H -- Yes --> I{More steps?}
    I -- Yes --> G
    I -- No --> J[Mark workflow complete]
    H -- No --> K{Retry limit\nreached?}
    K -- No --> L[Backoff and re-enqueue]
    L --> H
    K -- Yes --> M[Escalate to human queue]

Use synchronous execution when the caller cannot proceed without the result and the result arrives in under a few seconds. Use asynchronous execution when any of the following are true: the workflow touches more than one external system, any step can be delayed by a third party, the action must be auditable at each step, or a failure mid-workflow would be expensive to restart from scratch.

For teams building AI agents for automation or running multi-agent systems, most production workflows will land in the async column.

Building retry logic that does not cause more damage than the original failure

Retry logic is not just “try again.” Naive retries against a degraded payment API, a rate-limited compliance service, or a ledger mid-batch can produce duplicate writes, locked records, or cascading failures. Three rules contain the risk.

Use exponential backoff with jitter. Wait 1 second before the first retry, 2 before the second, 4 before the third, and so on, with a small random offset added each time. The jitter prevents a fleet of agents from hammering a recovering service in lockstep.

Set a hard retry ceiling per step, not per workflow. Three to five retries per step is a common ceiling. When a step exceeds that limit, the agent writes its current state and routes to a human review queue. It does not retry the entire workflow from step one.

Distinguish retriable from non-retriable errors. A 429 (rate limit) or a 503 (service unavailable) is retriable. A 400 (bad request) is not. An agent that retries a malformed payload will produce the same error every time and exhaust its retry budget pointlessly. Build an error classification layer that stops non-retriable errors immediately and logs the payload for human inspection.

Idempotency: the property that makes retry logic safe

An idempotent action produces the same result whether it runs once or ten times. Without idempotency, retries create duplicates: two payments posted, two compliance records written, two notifications sent.

The implementation is straightforward. Before the agent fires any external action, it attaches a stable idempotency key to the request, typically a hash of the workflow ID and the step index. The receiving system checks whether it has already processed that key. If it has, it returns the original result without executing again. If it has not, it executes and stores the key.

BNY Mellon’s deployment of approximately 20,000 agentic assistants across asynchronous settlement workflows and trade failure prevention depends on exactly this property. A settlement instruction that fires twice because of a network timeout must not settle twice. The idempotency key is what enforces that guarantee at scale.

State persistence: what to store and where

An agent that cannot resume from a checkpoint is not reliable in a multi-step workflow. It is just a long synchronous call wearing a different label.

flowchart TD
    A[Workflow created] --> B[Write workflow record:\nID, inputs, step index = 0]
    B --> C[Execute step N]
    C --> D[Write step result:\noutputs, external IDs, timestamp]
    D --> E{Workflow complete?}
    E -- No --> F[Increment step index]
    F --> C
    E -- Yes --> G[Mark workflow complete\nWrite final outputs]
    C --> H{Step failed?}
    H -- Retriable --> I[Write retry count\nand last error]
    I --> J[Backoff]
    J --> C
    H -- Non-retriable --> K[Write failure state\nRoute to human queue]

At minimum, persist the following fields for each workflow: the current step index, the inputs passed to each step, the outputs returned by each completed step, any external identifiers returned by downstream systems (a payment reference, a case ID, a batch run number), and the retry count and last error message for the current step. With those fields, a resumed agent can pick up exactly where it stopped without re-executing completed work.

JPMorgan Chase reported in June 2026 that it is building toward agents that remain coherent for hours, then days, then weeks, following a 20% increase in gross sales attributed to long-running autonomous agents handling complex multi-step financial workflows. Coherence across that timescale requires durable, queryable state storage, not in-memory context that evaporates when a process restarts.

For teams evaluating where to store workflow state, the choice usually sits between a relational database with a workflow table (simpler, good for moderate volumes), a purpose-built workflow engine, or a durable queue with a separate state store. The right answer depends on your volume and your existing infrastructure. Tools like Prefactor sit alongside whichever store you choose, evaluating each step of a long-running workflow as it executes so a stalled or failed run is caught when it happens, not when someone notices the output is missing.

Commonwealth Bank’s fraud detection agent, which runs continuously over time rather than in discrete request-response cycles, reduced fraud losses by 20% in the first half of FY2026. That result comes from an agent that accumulates state across hours of transaction data, not one that resets with each event.

Observability for async workflows

An asynchronous agent is invisible by default. It accepted a task, it is running somewhere, and you will find out whether it succeeded when it is done, or when it is not. That is not acceptable for compliance workflows, payment operations, or any process where a stuck agent costs money.

The minimum observability surface for an async agent workflow is: a status field per workflow (queued, running, paused, completed, failed), a step-level event log with timestamps, a current-step indicator so you can see where a running workflow is, and an alert when a workflow exceeds its expected duration. Teams building agent observability infrastructure often start with those four fields and add structured step-level payloads once the baseline is stable.

For a deeper treatment of what to instrument and how to query it, the guide to implementing agent observability covers the full stack.

Where to start

If you are not sure whether your current agent architecture can support the retry, idempotency, and state persistence patterns described here, the fastest way to find out is a structured readiness assessment. Take the agent readiness assessment to identify which workflows in your pipeline are candidates for async execution and where your architecture has gaps before they become production incidents.