Evaluating AI Agents

Independent AI audits are coming: what evidence your agents need

Matt DoughtyMatt DoughtyCEO & Co-Founder, Prefactor
6 min read
Abstract illustration: Independent AI audits are coming: what evidence your agents need

An independent AI audit checks whether the risk management around your agents is actually in place, not merely documented in a policy. For most organisations deploying agents today, this question will arrive through a procurement questionnaire or enterprise buyer due diligence process well before any legislation applies to them.

What an independent AI audit is

The term borrows from financial auditing: an outside party reviews your processes and evidence to form an opinion on whether your controls are real. A financial audit checks whether the numbers reflect reality. An independent AI audit checks whether the risk-management process around a model or agent reflects reality, and whether you can prove it.

Illinois’ AI Safety Measures Act requires frontier AI developers to implement risk-management frameworks and undergo annual independent third-party audits. California has enacted measures establishing standards and infrastructure for independent AI auditing organisations. Both laws target frontier model developers, not the majority of companies deploying agents on top of third-party models. The Transparency Coalition’s 2026 State AI Legislation Report expects 2027 to bring certification and licensing standards for third-party auditors, procurement standards requiring vendors to meet higher testing benchmarks, and testing and monitoring requirements for advanced models.

That last category matters. Procurement standards reach every vendor in a supply chain, including teams building on top of frontier models. If your enterprise customers adopt those standards, you will answer audit-style questions whether or not a statute requires it.

Why procurement arrives before statute

According to a 2026 Grant Thornton survey, 78% of business executives said they could not confidently pass an independent AI governance audit within 90 days. That gap creates pressure at the contract level, because buyers do not want to inherit the risk.

Enterprise buyers in financial services, healthcare, and insurance already run vendor risk assessments that include AI behaviour. When Goldman Sachs deploys autonomous agents for transaction reconciliation and client onboarding, targeting a 30% reduction in client onboarding time, those agents touch compliance-sensitive workflows. The vendors whose tools sit inside that stack will be asked to show evidence of safe behaviour.

The useful question now is not whether the statute applies to you. It is what evidence an auditor or a careful enterprise buyer would ask for, and whether you have it.

What evidence an auditor would ask for

Documentation of intent, such as an AI use policy or a responsible use framework, does not hold up on its own. An auditor forms an opinion by looking at measured results, not stated goals. The evidence set that answers audit-style questions has five components.

Evaluation results over time. A single benchmark run at deployment tells an auditor nothing about drift. You need results from the same test set run at regular intervals, with dates, so the auditor can see whether behaviour has changed. See how to evaluate AI agents for the mechanics of building that cadence.

Test sets tied to known failure modes. Generic capability benchmarks do not satisfy an auditor asking about your specific risk surface. If your agent operates in a regulated domain, your test set should include cases that probe the failure modes that matter in that domain, and the results should show pass rates, not just pass/fail outcomes.

A change log with before-and-after results. Every model update, prompt change, or tool addition should be recorded with the evaluation results from immediately before and after the change. This is the evidence that your change-management process is real. Without it, you cannot show that a regression did not occur.

Production behaviour samples. A structured sample of actual agent outputs, collected on a defined schedule, gives an auditor confidence that evaluation results reflect what the agent does in practice, not just what it does in test conditions. Agent observability tooling makes this tractable at scale.

Incident history. A log of failures, including what happened, how it was detected, and how it was resolved, demonstrates that your monitoring catches problems. An empty incident log is a warning sign, not a mark of quality.

flowchart TD
    A[Agent change proposed] --> B[Run evaluation suite]
    B --> C{Results within threshold?}
    C -- No --> D[Block change, file incident]
    C -- Yes --> E[Log before/after results]
    E --> F[Deploy to production]
    F --> G[Sample production outputs]
    G --> H[Add to evidence store]
    D --> I[Remediate and re-evaluate]
    I --> B

How to structure your evidence store

The evidence set above is only useful if it is retrievable. An auditor on a time-limited engagement will not reconstruct your history from scattered logs. The structure that works is straightforward: a versioned record for each agent, containing evaluation results indexed by date and model version, the test set used and its coverage rationale, the change log, a production sample archive, and the incident register.

Some teams use a purpose-built AI governance framework to hold this. Others bolt it onto existing change-management tooling. The mechanism matters less than the habit: evidence must be recorded at the time of the event, not reconstructed before a review. Tools like Prefactor are built specifically to maintain this kind of agent audit trail.

flowchart TD
    A[Evidence store] --> B[Evaluation results\nby date and version]
    A --> C[Test set\nand coverage rationale]
    A --> D[Change log\nwith before/after]
    A --> E[Production\nbehaviour samples]
    A --> F[Incident register]
    B --> G[Auditor or\nenterprise buyer review]
    C --> G
    D --> G
    E --> G
    F --> G

The scale pressure behind this shift

As of April 2026, an estimated 40% of business applications will include AI agents by end of year, up from under 5% in 2025. Morgan Stanley’s code review agent reviewed more than 9 million lines of code and saved 280,000 developer hours. At that volume, a buyer cannot rely on spot checks or verbal assurances. They need systematic evidence.

The same logic applies to multi-agent systems, where behaviour emerges from the interaction of several components. Each agent in the chain needs its own evidence trail, and the system as a whole needs evaluation results that cover the integration points, not just the individual agents. AI agent governance at the system level is more complex than at the single-agent level, but the evidence categories are the same.

This is not a compliance exercise that sits apart from engineering. The teams building AI governance best practices into their development process now are building the evidence store as a side effect of doing the work carefully. The teams that treat it as a separate audit-preparation task will find the evidence gap much harder to close under time pressure.

For context on where audit requirements fit into the broader governance picture, the governance index covers the full landscape. The agent failures log documents real production failures and what evidence would have surfaced them earlier.

This article is not legal advice. The legal obligations described apply to frontier AI developers as defined by the relevant statutes, and the scope of those definitions may change as regulations develop.

Where to start

The gap between what most teams have documented and what an auditor would ask for is real, but it closes faster than most engineering leads expect once the evidence categories are clear. Take the agent readiness assessment to see which parts of your evidence store are in place and where the gaps are.

Matt DoughtyMatt DoughtyCEO & Co-Founder, Prefactor

Founder of Prefactor, writing on the operational reality of getting AI agents into production — evaluation, observability, governance, and the plumbing assistants never needed.

Frequently asked questions

What does an independent AI audit actually check?

An independent AI audit checks whether your risk-management process is real and documented, not just described in policy. Auditors look for evaluation results over time, records of model and prompt changes with before-and-after results, and incident history, because documentation without measured outcomes does not hold up.

Do AI agent deployers have to comply with Illinois' AI Safety Measures Act?

Illinois' AI Safety Measures Act targets frontier AI developers, not companies that deploy agents built on third-party models. Most organisations deploying agents are not in scope today, but enterprise buyers and procurement teams are already applying similar scrutiny through vendor due diligence.

What evidence should I collect now to prepare for an AI governance audit?

Collect evaluation results run at regular intervals, test sets tied to your known failure modes, a change log for every model or prompt update with before-and-after results, a sample of production outputs, and a record of incidents including how each was resolved. A policy document alone is not sufficient.

How do procurement requirements differ from statutory AI audit requirements?

Statutory requirements, where they exist, apply to specific developer categories and carry legal obligations. Procurement requirements come from enterprise buyers who want evidence of safe AI behaviour before signing contracts. Procurement pressure reaches most deployers earlier and applies regardless of which laws cover them.

Stay ahead of the curve

No spam. Unsubscribe anytime. A resource by Prefactor.

Almost there — check your inbox to confirm your subscription.