Adam Witanowski on Agentic Maturity Across the SDLC

Introduction

Adam: We often talk about the models. Maturity isn’t about a model, it’s about how much you trust the agent when no one’s watching it.

Matt: You’re listening to Agents After Dark, powered by Prefactor. My name is Matthew Doughty, CEO and co-founder of Prefactor.ai. As enterprises deploy more agents, one question becomes increasingly difficult to answer: how do you actually know what your agents are doing? Prefactor gives organizations a single place to see how their agents are behaving, whether they’re performing as intended, where risk is emerging, and what action needs to be taken.

I’m delighted to be joined today by Adam Witanowski. He’s an AI architect who has spent the past year building one of Australia’s most comprehensive end-to-end transformations from the inside: 11 integrated AI tools reshaping how engineering teams build software. He’s now a consulting Chief AI Officer and a keynote speaker on AI-powered engineering, with sharp, unfiltered views on productivity and governance.

From Chatbot to Shipping 18 Products Without Writing Code

Matt: I’m really interested in the journey you’ve gone on over the last couple of years with AI, and how you were able to embed some of that agentic learning into what was, and is, a massive corporate environment.

Adam: Like everyone else, this was thrust upon us. It started with something as simple as a chatbot, and that chatbot had an incredible amount of power. It was obvious very early that this was going to be interesting technology, and with each iteration it grew in capability. I started using it very early for coding, because I was a senior engineering manager at the time and coding wasn’t part of my daily workflow anymore. I used it as a reskilling tool, to check my own code. Over time its capability grew, and so did mine.

I haven’t written a line of code in probably about twelve months, maybe a little, but nothing of any real note. Yet I was able to ship 18 products to production, plus a similar number of personal projects. The last two years have really been about understanding the limits of the tools: new models come out constantly, and figuring out which ones solve which problems, what they cost, and what their inherent skill set is.

On the enterprise side, an organization is a team, not one person. The opportunity came from a visionary executive, Brendan Mills, who was CIO at NIB at the time. Between him and Matthew Finch, they’d already led NIB’s migration from on-prem to 100 percent cloud on AWS. Their next task was reimagining the company, and with that leadership support I was given a green field to reimagine the software development lifecycle into an AI-native SDLC. It started as a lab that I had to pitch to the business and then to all the engineers around how this would reshape the way we work. The twelve months before that were really just “what the hell does this do and how far can we drive it.”

Why Coding Copilots Alone Don’t Move the Needle

Matt: I think you’re underselling how hard it is to get AI into organizations. We speak to enterprises all the time and it’s rare to find one that has shipped something to production that isn’t just “copilot.” What actually worked to get traction, when you rolled out 15 or 20 different tools rather than just one or two?

Adam: Often in an enterprise you’ll hear, “we’ve introduced GitHub Copilot or Claude Code or Cursor, job done.” That’s fine, but if you give engineers something like Claude Code, which in my opinion is currently the best raw harness there is, or the newly released Sable 5, the ones who really adopt it will accelerate incredibly. We saw individual velocity increases from 120 to 400 percent plus. But studies show only 16 percent of an engineer’s time is spent actually writing code. So you’re optimizing for 16 percent of the pipeline. You bloat the middle of that pipeline and get no increase in overall flow. What I actually measured was somewhere between 0 and 40 percent end-to-end velocity increase, depending on the team.

Even 40 percent sounds incredible to most companies, but all that really tells you is there was deficiency in the flow to begin with. Most companies focus on this one small piece. I identified early that it had to be more than that: you need to increase velocity left and right of the coding step, or you’re not increasing anything meaningfully.

There’s also a trust drag. Most engineers and most companies won’t trust a new tool on day one. You’ll get your ten percent early adopters, but it takes six months or more for the rest to build confidence. You have to show engineers it won’t increase how often they get woken up at 3 a.m. because code fell over. Coding agents are incredible at generating large amounts of code, often better than what most engineers produce, but it will be generic, without context on how or why your organization works the way it does, its historical decisions, or its compliance rules. Even really good prompting is functionally weak steer for a specific organization. So you need better steering mechanisms all the way through the development lifecycle.

Introducing coding agents to the team was actually a small part of what I presented. I originally pitched twelve or fifteen tools and ended up delivering more, because more were needed as we discovered other gaps. The pitch to engineers was that we weren’t just optimizing their 16 percent of code-writing, we were building a meta-harness to protect the codebase and give them more time to think about the problems they’re solving. The pitch to the business was that we’d save money by delivering faster at the same quality, and their first question was always: prove it. So measurement becomes a critical conversation.

Engagement Up, Productivity Down: The Human Side

Matt: You mentioned engagement went up while productivity dipped. What accounted for that, once people moved through the change curve into everyone actually using the tools?

Adam: It’s never just one thing. NIB was going through a significant leadership change: a CEO who’d been there twenty years replaced, a whole new leadership team. As good as they might be individually, that creates unrest. Couple that with layoff stories in the broader market, and people get anxious about their jobs, which naturally decreases productivity.

When you introduce AI into an organization you need to make space for experimentation and for failure. We ran hackathons, weekly guilds, upskilling sessions, the whole gamut. But things quickly bottlenecked at the PR stage, and also at the front of the funnel, because engineers were getting upskilled while POs, PMs, and QAs, who should be shifting left too, weren’t. So what’s flowing into the funnel becomes too weak to support what the funnel can actually do, and everything downstream backs up. In a regulated business especially, cybersecurity gets slammed by sheer volume and legitimate anxiety about increased risk.

On top of that, while coding agents are mature, every other piece of tooling and infrastructure around them is still nascent. You either build it yourself or wait for the market. We decided to build, but in a modular way so we could pull pieces out individually and replace them with off-the-shelf tools as they matured.

Measuring What Matters, Not What’s Easy

Matt: A financial services company I spoke with recently wanted to compare Copilot output against GitHub activity using pretty blunt metrics. My instinct was that’s not a great measure. What’s your view on human review of agent-written code, and how involved should an engineer stay?

Adam: What you don’t measure, you can’t change, and you can only change things at the level of constraint. Early on it was important to measure across the whole SDLC, probably more thoroughly than the organization had ever measured before. Most organizations don’t measure like this. Places like Macquarie Bank can give you a metric on almost everything; most companies aren’t like that, and that’s partly a cultural choice.

Adoption metrics and token spend are horrible things to optimize for. Adoption means nothing unless there’s useful output. A better metric is rework: how many PRs from agent-initiated code need to be redone by a human engineer, rather than simply how many PRs an agent raises.

So I built a 58-point measurement harness pulling data from GitHub, AWS, PagerDuty, Jira ticket quality, essentially every point along the build-to-release path, into a master dashboard. I also built a PR system called Echo, a multi-agent system that scored the quality of code being delivered, and let engineers score the PRs too. Over time you can see whether there were more incidents, more security issues, more failovers in production, more rework, alongside your DORA metrics. Measuring the change across the entire product lifecycle matters more than just shipping code faster.

I also audited 10,000 previous PRs within the company and found roughly a third of them could most likely have been automated end to end. They had no significant auth impact and didn’t touch key security policies, yet we still had humans review them. If an organization isn’t ready to automate even a subset of its PRs, that’s a real maturity signal, and some organizations aren’t even raising agentic PRs yet, which is an even lower maturity level. If you still need a human governing every agent action, you’re not at the top level of agentic delivery.

Governor and Skynet: Building Trust Without Slowing Delivery

Matt: How do you actually move an organization’s maturity forward, given that “truly automating code” still feels scary to most companies?

Adam: Go to any experienced AI engineer today, and they’ve functionally moved to harness-and-loop engineering. They’re building dark factories for code, happily handing a problem to an agent that works for twelve hours straight and delivers something solid. That works at the individual level, but organizations are inherently multiplayer, not single-player, and we’ve built the whole ecosystem around single-player dynamics.

The trap to avoid is shoving human-in-the-loop review in too late. We feed agents a huge amount of context, a whole codebase, documents, plans, and then throw a human at the output with very limited context. It becomes a rubber-stamping exercise, similar to a lot of human PR review today if we’re honest.

What I built to shift this left is a Governor module. Think of vertical teams as slice owners: cyber, cloud, legal, risk, compliance. They have some say in what gets shipped. Then you have horizontal delivery teams with their own coding standards. In Governor, each team defines policy, such as “we always secure APIs this way” or “we always use this container type,” and that gets turned into policy as code, injected into the coding agent at invocation time, either as a hook event for tighter enforcement or via MCP for a lighter suggestion. It then gets reasserted again at CI. That policy layer, injected early and checked twice, means you can throw in as many rules as you want without engineers having to hold pages of Confluence documentation in their heads while under deadline pressure. An agent will happily obey injected policy every time; CI can assert and reject non-compliant work and feed back to the owning team how many PRs breached policy that week.

The other system I built is called Skynet. Coding agents are good at reasoning about a single codebase, but no enterprise is one codebase. We had 3,600 repos of varying ages and deployment states, full of dependencies: change something here and you might break the data team’s ingestion downstream, invisible to the agent making the change. Skynet ingested all 3,600 repos via static code analysis and vectorization into a knowledge graph, extracting code intelligence about how those codebases relate and evolve. PRs get attached to their Jira tickets and decisions get logged against the graph over time, so agents get a 360-degree view of downstream and upstream impact before making a change. That builds real trust for engineers: they know a request won’t quietly break another team’s system down the road. Giving agents that decision history, which humans accumulate over years at a company but agents don’t have natively, gives them a lot more freedom and frees engineers to think about bigger problems.

Bringing Non-Engineers Into the Journey

Matt: How do you apply everything you’ve learned in engineering, around trust, maturity, and confidence, to parts of the organization that have never worked in an engineering capacity but are now shaping decisions around agents?

Adam: We target engineering and call centers first because they’re the easiest to measure: engineering down to the ticket and point level with a rich git history as a source of truth, call centers down to the minute with a transcript of every word spoken to a customer. But trust still has to be earned regardless of how good the tooling is. Engineering teams in an organization range from “I don’t trust them” to “I don’t understand them” to “they’re the real heroes,” and every company approaches its engineers differently.

Often when engineering comes in and says “we can automate some of your work,” what people hear is “I’m going to lose my job.” That’s not always true. I’ve consulted with companies thrilled to have someone show them how to automate work, seeing a 40 percent productivity gain as more capacity to win more market rather than fewer jobs. That framing is a leadership decision, not any individual team’s choice. Leadership needs to back an environment where people are invited to explore and given assurance this isn’t purely a replacement exercise.

The other shift is that the market increasingly wants product engineers, not just people who write code, but people who sit with a team, observe them, and understand their friction. My old boss did this well: he had engineers sit with call center staff, watch them, and listen to their pain points. The best engineering work identifies pain and brings a remedy to it. That’s the low-hanging fruit for agentic automation in any organization: bringing empathy to a team’s actual problems, treating them as an internal customer, rather than treating automation as a purely technical exercise.

Change, Fear, and an Uncertain Future

Matt: Isn’t it inevitable that AI reduces the number of people doing today’s jobs, and that fear is therefore justified? How do you close that trust gap when the models keep getting more capable every few months?

Adam: Go back a couple of hundred years: ice farmers cut blocks from frozen lakes and shipped them to New York. Fifty years later someone builds an ice factory in Brooklyn, and the ice farmers are gone. Fifty years after that, Westinghouse invents the freezer, and the ice factory is gone too. None of the ice farmers became ice factory workers, and none of the ice factory workers built fridges. People lost jobs. That’s the reality of change, and software is eating the world faster than ever. There will absolutely be winners and losers.

I’m bullish on engineers, I think they should see themselves as apex predators ready to solve real business problems, but we don’t need to enjoy the disruption so much. These are people’s jobs and lives, and we should help them understand the tools available and how to increase their value.

I’m also a bit of an optimist. There’s a non-zero chance the world ends up in chaos with fighting robots in fifty years. There’s also a non-zero chance we land somewhere closer to a Star Trek utopia. The truth will land somewhere in the middle, and it’s not just AI pushing the boundary: quantum computing, world models, robotics, fusion energy are all advancing too. Fusion alone would drive the cost of almost everything toward zero. Combine that with quantum compute and general superintelligent AI, and nobody is really ready for that kind of transformed world.

We can feel threatened by that, or excited about what it makes possible. Anthropic just released a program for discovering treatments for underrepresented diseases. My background is bioinformatics, and research that used to take almost a year, I saw done in a day using AI. I can be sad about that, or I can celebrate what it means for people whose diseases were too rare to ever attract research funding. There will be negative externalities with every change, but there’s an equal weighting of potential positive ones. Even in the job market, we haven’t seen the downturn in employment that economists expected; for some people it’s become far more lucrative than they imagined. It’s genuinely a wait and see on how the negative externalities play out.

I’m a parent of two kids, and I have no idea what to tell them to do, because I don’t know what jobs exist in the pipeline anymore. You used to be able to say “study medicine” or “become an engineer.” Now you can probably only look twelve months ahead with any confidence, rather than five or ten years.

Closing

Matt: I think the change happening today is part of a much longer arc of technological change, but your point about being a parent really lands. My six-year-old told me he wants to be a football player, and I have no idea if that job exists in the way I picture it in twenty years. But I also started my career in sales fifteen years ago, and now I’m coding up internal demos I have no formal right to build. The opportunity seems to be there for anyone curious enough to get involved, whether or not you’re an engineer by training.

Thank you, Adam, this has been a real pleasure, and I hope to have you back before too long.

Adam: Let’s do it. I think you’re right that the complexity space just shifts over time, that’s probably going to keep happening generally. I’d love to come back.

Matt: This has been Matt Doughty at Prefactor with Adam Witanowski. Thanks so much.

Adam: Thanks a lot, mate.