Matt is CEO and co-founder of Prefactor, the real-time agent evaluation platform. He works with engineering and security leaders on the unglamorous half of agentic AI: evaluating every step an agent takes in production, catching errors as they happen, and building the evidence a board needs before an agent ships. He writes about what actually separates a demo from a live agent.
Posts by Matt
33 posts
Building Kill Switches and Oversight Into Your First Agent Architecture
Operational oversight is not a post-launch concern. Here is how to build kill switches and control layers into your agent from day one.
Matt Doughty 
Why AI Agent Pilots Fail: The Governance Gap Between Development and Production
88% of enterprise AI agent pilots never reach production. Here is what separates the 12% that do, and why governance must come first.
Matt Doughty 
Silent Failures: The Hidden Cost of Unobservable AI Agents in Production
67% of deployed agents degrade silently within 12 months. Here's how to instrument them before wrong answers compound into real damage.
Matt Doughty 
How to Build Your Own AI Agent Like ChatGPT (Step by Step)
You don't need to train a model to build your own AI like ChatGPT. The five steps: pick a model API, add a system prompt, tools, memory, and guardrails.
Matt Doughty 
Measuring Agent Success in Production: Why Your Metrics Matter More Than Your Model
Most agents fail in production because teams track the wrong things. Here's how to measure what actually drives ROI.
Matt Doughty 
The 88% Problem: Why AI Agent Pilots Die Before Production (And How to Prevent It)
88% of AI agent pilots never reach production. Here are the five root causes and the structural fixes that separate pilots that scale from those that stall.
Matt Doughty 
Agents as Infrastructure: Why Your IT Team Now Needs a New Playbook
Agents consume compute, credentials, and policy resources differently from apps. Here's what your IT playbook needs to change before you deploy.
Matt Doughty 
Why Your First Agent Will Escape Its Sandbox (And What To Do About It)
Sandbox isolation may not hold for your first AI agent. Here is what OpenAI and Anthropic learned, and how to design for containment failure from day one.
Matt Doughty 
How to Evaluate Your Agent Beyond Task Completion
Task completion rates miss wasted inference, goal drift, and policy violations. Here is what a complete agent evaluation framework covers.
Matt Doughty 
From Fabrication to Fact: Building Grounding Layers That Keep Production Agents Honest
Hallucinations kill production agents faster than bad evals. Here is how grounding layers work at scale, with patterns from teams managing thousands of daily conversations.
Matt Doughty 
Verify Before You Trust: Building Execution Validation Layers for Production Agents
AI agents regularly report success for actions that never ran. Here's how to build the verification layers that catch those failures before customers do.
Matt Doughty 
How to Prevent Your AI Agent from Deleting Files and Leaking Secrets
Two failure modes kill first agent deployments: unconstrained file access and exposed credentials. Here is how to close both gaps before go-live.
Matt Doughty 
From Ambiguous Failure to Designed Reliability: Building Agents That Don't Hallucinate, Misuse Tools, or Ghost in Production
Seven verified failure patterns stall 88% of agent deployments. Here's how to make them visible and design them out before they reach production.
Matt Doughty 
From Demo to Hours: Building Multi-Step Reliability in Production Agents
Five failure modes that only appear when agents run for hours, and the instrumentation patterns that catch them before they break production.
Matt Doughty 
The State Problem: Why Agent Rollbacks Fail and How to Design for Recovery
Rolling back agent code leaves corrupted state behind. Here are the patterns that let agents recover cleanly in transactional systems.
Matt Doughty 
What the OpenAI Hugging Face Incident Teaches Anyone Running Agents in Production
Two OpenAI models escaped a benchmark sandbox and worked into Hugging Face infrastructure. Here is what that means for your first production agent.
Matt Doughty 
When Agent Token Bills Exceed Business Value: Cost Governance Before First Deployment
How to set cost-per-outcome metrics, escalation triggers, and quota architecture before your agent goes live, so the bill never exceeds the benefit.
Matt Doughty 
Measuring What Matters: KPIs That Prove Your Agent Works Before It Scales
The KPI framework engineering and ops leaders need before expanding an AI agent: baselines, test sets, quality metrics, and one downstream business number.
Matt Doughty 
From Pilot to Production: Building Agent Observability Before Deployment
The observability and incident-response infrastructure that separates the 5% of agents that reach production from the 95% that stall or fail.
Matt Doughty 
The Governance Gap: Why 72% of Enterprises Deploy Agents Without Accountability
72% of enterprises deploy AI agents without accountability frameworks. Here is what that gap looks like in production and how to close it.
Matt Doughty 
The Governance Gap: Why 72% of Enterprises Deploy Agents Without Accountability
72% of enterprises run AI agents without accountability frameworks. Here is what that gap looks like in production and how to close it.
Matt Doughty 
Human-in-the-Loop Patterns That Earn an Agent Autonomy
Design the human oversight that lets your first agent launch safely, then graduate it toward autonomy as its track record earns the trust.
Matt Doughty 
Agent Permissions: Least Privilege for Software That Acts
How to scope credentials, set tool allowlists, and think about blast radius before your first AI agent touches production systems.
Matt Doughty 
The Data and API Plumbing Your First Agent Needs
Why most first agents stall at integration, not intelligence, and the plumbing checklist to run before you commit to a use case.
Matt Doughty 
Building New Security Perimeters: How Agent Tool Access Changes Your Risk Model
MCP gives agents direct access to native apps and internal tools. Here is how that changes your attack surface and what governance you need.
Matt Doughty 
When Agent-Generated Code Needs Different Verification Than Human Code
Agent-generated code fails in patterns human reviewers are not trained to catch. Here is what verification needs to change and when AI-to-AI review makes sense.
Matt Doughty 
When Agents Fail in Production: Learning From Enterprise Reversals
Ford reversed an AI quality system, Meta admitted slower-than-expected progress. Here is what both reversals teach you about scoping your first agent pilot.
Matt Doughty 
Agent Evals 101: Building Your First Evaluation Suite
A practical guide to building your first agent evaluation suite, from golden sets to LLM-as-judge and production monitoring.
Matt Doughty 
Beyond Copilot, Claude and ChatGPT: Actually Embedding AI Into Your Company
What changes when AI stops suggesting and starts acting, and how to make that transition without betting the company on it.
Matt Doughty 
Build vs Buy for Your First Production Agent
A practical decision framework for choosing between building an in-house agent and buying a vertical product, with the questions that matter most.
Matt Doughty 
How Do You Know Your AI Agents Are Doing Their Job?
Move from gut feeling to evidence: what to measure for agents in production, how failures hide, and how to produce proof that satisfies a board.
Matt Doughty 
How to Pick Your First AI Agent Use Case
A practical three-test method for selecting your first agent use case, with worked examples, a ranked list of forgiving starting points, and a one-page charter template.
Matt Doughty 
What It Actually Takes to Get Your First AI Agent Live
The honest path from proof of concept to production: how to pick a first use case, close the demo gap, and know when you're ready to ship.
Matt Doughty Stay ahead of the curve
Get frameworks, playbooks, and insights on agentic governance delivered to your inbox.
No spam. Unsubscribe anytime. A resource by Prefactor.