Security

How to govern agentic AI

Goal

Stand up governance for agentic AI that actually operates: decision rights on paper, a registry agents cannot skip, autonomy granted on evidence, and enforcement that lives in the runtime rather than in a review meeting.

Before you start

  • An agent inventory, or the willingness to build one as the first act of governance
  • A sponsor senior enough to make decision rights stick across teams
  • Agreement on which existing bodies — security review, change management, architecture — agentic decisions route through, so governance lands in one queue rather than four

Steps

  1. 1

    Put decision rights on paper before anything else

    Governance is, concretely, four questions with names attached: who may approve a new agent, who may widen an existing agent's autonomy or tool access, who may grant exceptions, and who may shut an agent down without a meeting. Write the answers as roles, give the kill decision the fewest approvals of the four, and publish it. Most agentic governance failures trace to one of these questions having no answer at the moment it was asked.

  2. 2

    Make the registry the spine of everything

    The rule that makes governance enforceable is no register, no run: an agent gets credentials when it gets a registry entry — owner, purpose, systems it reads and writes, autonomy level — and not before. Tie registration to something teams already need, credential issuance being the natural choke point, so the registry stays current as a side effect of operating rather than as a quarterly chore.

  3. 3

    Tier agents by blast radius and govern by tier

    Sort registered agents by what they can write to, and define two or three tiers with proportionate requirements: a read-only summariser should clear a checklist in a day; an agent that can move money or grant access earns the full review. Proportionality is what keeps governance honest — a process that treats every agent like the riskiest one will be routed around, and the routing around is how shadow agents are born.

  4. 4

    Define lifecycle gates with evidence at each

    An agent moves through proposal, pilot, production, and retirement, and each transition names its evidence: a risk profile to enter pilot; evaluation results, an audit trail, and a rehearsed revocation path to enter production; credential destruction and registry closure at retirement. Gates replace standing committees — the work is reviewing evidence against a published bar, not re-debating the agent each quarter. Retirement is the gate teams forget, and unretired agents are next year's incident inventory.

  5. 5

    Treat autonomy as a dial that evidence turns

    Define autonomy levels — every consequential action approved by a human, defined action classes pre-approved within limits, full autonomy inside a scope — and attach each level to the evidence that earns it, primarily the agent's evaluation record and incident history. Autonomy then widens one action class at a time, and narrows automatically when evaluations regress or an incident lands. The dial gives builders a path to earn freedom, which is what makes them participate rather than evade.

  6. 6

    Move enforcement into the runtime

    Approval-based governance decays the moment the agent starts running, because the agent's behaviour changes with every model version while the approval stays frozen on the original. The policies that matter — which tools, which limits, which actions need a human — must be enforced where actions happen, at runtime, with the registry as the source of truth for what each agent is allowed. Review decides; the runtime enforces; the gap between them is where agents drift.

  7. 7

    Run governance as an operating rhythm, not a document

    Set a recurring review — monthly is workable early — that looks at four feeds from the runtime: incidents and near-misses, evaluation drift, spend per agent, and exceptions granted. Retire what is unused, narrow what regressed, promote what earned it, and amend the tiers when reality disagrees with them. A governance framework that has not changed in two quarters is not mature; it is unread.

Common pitfalls

  • Adapting the content-AI policy and calling it done. Policies written for generative tools govern what people may produce with a model; agentic governance has to govern what software may do — identity, tools, autonomy — and the old policy is silent on all of it.
  • Governance by committee throughput. If every agent waits six weeks for a board, teams ship around the board, and the agents that most needed review are the ones that never got it.
  • A registry that decays after launch week, because registration was a request rather than the precondition for credentials.
  • Granting autonomy on demo performance. A demo is the agent's best day; the evaluation record is its average one, and autonomy priced on best days gets repaid in incidents.
  • Confusing governance with [security](/ai-security). Security bounds what an attacker can make an agent do; governance decides what the organisation lets it do on purpose. Each is unfinished without the other, and owning one does not discharge the other.

Frequently asked questions

How is agentic AI governance different from AI governance generally?

General AI governance grew up around models that produce content for humans, so it concentrates on data, bias, and acceptable use. Agentic AI adds actors: software that holds credentials and acts on systems. That forces governance to borrow from operational disciplines — identity, change management, incident response — and to put enforcement in the runtime, not just the policy. The old questions remain; a new class of questions about action arrives.

Do we need a separate framework for agentic AI governance?

Usually an extension, not a second framework. Keep the organisational frame you have — NIST AI RMF, ISO 42001, or your own — and add the agent-specific controls it lacks: per-agent identity, the registry, autonomy tiers, runtime enforcement. Two parallel governance regimes for one technology mostly produces two queues and one workaround.

Is your organisation ready for AI agents?

Take the assessment →