How to inventory the AI agents you already run
Build a first, honest inventory of every agent acting on your organisation's systems — including the ones nobody admits to — and turn it into a registry that stays current.
Before you start
- Access to cloud and SaaS billing data (LLM API spend is the strongest discovery signal)
- Read access to identity provider and service-account inventories
- A sponsor who can ask teams to self-report without it feeling like an audit
Steps
- 1
Follow the spend
Pull three months of billing for OpenAI, Anthropic, Azure OpenAI, Bedrock, Vertex, and any LLM gateway. Every cost line traces to something calling a model. Match each line to a team and ask what is calling. Expense-report API charges on corporate cards count — that is where pilots hide.
- 2
Sweep service accounts and tokens
List service accounts and API tokens created or modified in the last year, and flag those whose activity pattern looks automated but irregular — bursts at odd hours, novel API combinations. Cross-reference against known workloads; the unexplained remainder is your shadow-agent candidate list.
- 3
Ask, with amnesty
Send every engineering and operations team a short form: what agents or LLM-backed automations do you run, what do they touch, who owns them? State explicitly that the goal is inventory, not enforcement, and that nothing reported this round gets shut down without conversation. Amnesty is what makes the numbers honest.
- 4
Record each agent against a fixed schema
For every agent found, capture: name, owner, purpose in one sentence, systems and data it reads, systems it writes, credentials it uses, framework or platform, deployment location, and current status (pilot, production, abandoned). Resist recording more than this on the first pass — completeness beats depth.
- 5
Triage by blast radius
Sort the inventory by what each agent can write to. An agent that drafts internal summaries and an agent that can move money do not belong in the same review queue. The top of the sorted list is your governance roadmap for the next quarter.
- 6
Make the registry self-maintaining
A one-off inventory decays in weeks. Tie registration to something teams already need — credential issuance, deployment pipelines, or cost allocation — so an unregistered agent is also an agent that cannot ship. That single enforcement point is the difference between a registry and a stale spreadsheet.
Common pitfalls
- Treating the inventory as a compliance exercise — teams will report the minimum and the shadow agents stay hidden
- Recording only production agents; abandoned pilots still hold credentials
- Building a beautiful schema before finding the agents; find first, refine later
- Skipping no-code and SaaS-native automations because they don't look like engineering