Agents After Dark

Aaron Vanston on building trustworthy AI when a mistake can be life and death

Construction is an unforgiving place to put AI.

In this episode

  • Earning trust in AI when a wrong answer can mean a defect, a blown budget, or an injury on site
  • Showing every generative output to a human, with no hidden multi-step chains running behind the scenes
  • Graduating actions from always-ask to autonomous with a one-way versus two-way door test
  • Why evals are the real moat when everyone runs the same models

Key takeaways

  1. 2:47 AI gives context-rich but time-poor founders a way to ship again. Aaron moved from strategic founder back on the tools and is now coding more than he ever has, firing off ideas between meetings and letting the weak ones die cheaply.
  2. 3:50 To avoid getting under the engineering team's feet, he builds outside their stack rather than in the same repositories. The cost to spin up a standalone app, or to throw it away, is now close to zero, so prototyping new lines of business no longer disrupts the core.
  3. 8:22 Aaron reframes the SaaS is dead argument: the opportunity is building the system of record and the tools, MCP included, that customers build on top of, not another dashboard that tries to solve every need.
  4. 14:43 Buildpass started its AI journey on finite, encapsulated tasks like extracting context from a site image and digitising paperwork from a PDF, then moved to free-flowing chat and an MCP server only once tool use and context windows were reliable enough.
  5. 17:08 Human in the loop is non-negotiable in construction. There is no hidden multi-step AI chain by design: if an output is generative, it is shown to the user to scan and approve, because a wrong drawing can mean variation cost, defects, or an on-site accident.
  6. 21:22 Trust is built by not returning everything as 100% right. Buildpass surfaces confidence scores, sources, and links, and lets the user approve or reject, so people learn the system the way they learned to trust ChatGPT or Claude over time.
  7. 23:06 Risk gets classified with a one-way versus two-way door test. Reversible, low-impact actions can graduate to autonomous, while anything that manipulates data or touches safety keeps slower safeguards and an impact analysis behind it.
  8. 37:01 Evals are the moat. Since everyone runs the same state-of-the-art models, the differentiator is the internal tests that tell you whether a new model is actually better for your users, and an eval can be as informal as a vibe check.

Highlights

Construction safety is life and death, and it is not something we should be playing with willy-nilly.
Aaron Vanston
We don't want to remove humans. We just want to make their jobs a lot easier.
Aaron Vanston
The tests you have internally that determine if a model is good or bad, that becomes your moat as a company.
Aaron Vanston
Full transcript 39:34 · click to expand

From strategic founder back on the tools

Aaron: I think SaaS is dead from the point of view that maybe we should be building the system of record for these companies to build on, versus building out the exact SaaS layer.

Matt: Hello everybody. You are listening to Agents After Dark, powered by Prefactor. I am Matt Doughty, the CEO and co-founder of prefactor.ai. As enterprises deploy more AI agents, one question becomes increasingly difficult to answer: how do you actually know what your agents are doing? Prefactor gives organizations a single place to see how their agents are behaving, whether they are performing as intended, where risk is emerging, and where action needs to be taken.

I am very excited to be joined today by Aaron from Buildpass. Aaron is the CTO and co-founder of a construction tech platform built here in Australia, but growing rapidly, with a big go-to-market team over in North America where their CEO Matt is based in Austin. Thank you so much for being with me today, Aaron. I really appreciate your time.

Aaron: Thanks for having me. Keen.

Matt: Before the chat we were talking about AI and how it affects you and your business. One of the things you mentioned was that you have taken AI by the horns and are now moving from being more of a strategic founder to being a bit more on the tools as a result. I would be really keen to understand your journey with AI and what has brought you to the point you are at today, from a personal perspective, and then tell us a bit more about Buildpass.

Aaron: Yeah, one hundred percent. AI has come at a funny time for a lot of founders who are growing out a company. For context, Buildpass is about 65 people right now, and you get caught up building the company, stuck in meetings, finding yourself with a bit less time. Myself and the other co-founders and some of the early engineers were the ones who built everything from scratch at the start. That was all pre-AI, so it is all handwritten, artisanal code. Suddenly your craft becomes more about meetings, and AI hit me at a funny point. A few times, in between meetings, I would love to just fire off a prompt and get some stuff done, to scratch that itch, where I could build things, or at least try things out, or see an idea come to fruition.

It really compounded. It was like, oh, I actually can do a lot of the things that I am currently blocked on or do not have the time to do. I also have that technical expertise. I know the systems inside and out, I helped build them. That combination is something I think a lot of senior technical leaders, or even senior founders, are finding too. They might not have a lot of context and not a lot of time, and AI is a really good conduit to firing off a bunch of these ideas. You quickly work out that maybe not every idea is a good idea, and that is perfectly fine. They can live and die. But it is a really good way for that to happen, and I find myself more and more leaning in, really enjoying getting back on the tools, probably now doing more coding than I have ever done combined in the previous years before building Buildpass. I am absolutely loving it. For a founder who wants to build stuff and get out there, AI is a fantastic conduit to that.

Building without treading on the engineering team

Matt: When you talk about building, it almost feels like you could be getting under the feet of your engineering leadership team. How do you manage that, given that you have the vision, the ideas, and you speak to the customers? How do you ensure you are not just getting in the way?

Aaron: Genuinely a good question, and I have such an easy answer for it: do not build in the same stack as them. Actually force yourself out. That is ultimately what I ended up doing. To be fair, every developer loves starting a fresh new thing, and a founder is no different. I found that rhythm by not treading on the toes of the existing systems, and trying out new things, whether that is prototypes or building out new parts of the business. You can do that standalone now. The cost to spin up a whole app or a feature is coming down to literal zero in terms of getting it set up, and the cost to restart something, and also to throw it away, is quite minimal. So why not start again? Why not try some things out and explore more?

It is also about investing in the team. If the team builds in the same space and we are all on the same page, we can comfortably sit in the same repositories or applications and build on that too. Hopefully the team sees it as a value add, though I am sure there is a bit of a wrecking ball situation sometimes when it comes to some of the stuff that gets pushed in.

Matt: That is really interesting. Have you managed to build anything that has gone into production and is being used by customers today?

Aaron: Random bits and pieces, constantly. This idea of being able to convert a customer call, where you are hearing out pain points, into actually firing off things to fix them. There are a bunch of little paper cuts that I have had, that our customers have had, and I have worked with them as part of my journey as a founder without necessarily building day-to-day. I can do that because I am not prescribed to a road map. I am more free flowing.

More recently, not in production right this second but hopefully very soon, we have essentially afforded building out brand new lines of business using AI. For context, on the background of Buildpass: we started in site operations and safety for construction. That is really important in Australia in particular. There is a lot of paperwork, a lot of red tape to help cut through, and software is fantastic at that. We expanded into site operations, into quality, into scheduling. We never really touched some of the pre-construction and finance aspects, which are quite complicated and big pieces, very much the domain of the SaaS companies that do all of that themselves. What I am working on right now is building those out, using AI to help develop them in really short cycles. That is about to go into production, hopefully by the end of this month, hopefully in July. That is a huge piece that has been solely pushed through with AI.

Is SaaS dead in construction?

Matt: That is exciting. For everyone listening, for context, I know Aaron through my previous life working at Procore. If anyone has looked at the Procore stock price over the past few months, it has been painful reading. I would be curious to understand your perspective. You can build faster than ever before, with less resource than ever before. You can sit with customers, understand what they need, whether that is a new feature or a bug, and fix it. Procore has done a fantastic job over the last 25 years to build and often lead the way when it comes to technology. But the way you just described how fast you can build sounds almost like that archetypal example of, well, is SaaS dead? Are businesses like Buildpass going to eat Procore and Autodesk’s lunch? My theory is, why can’t Procore do the same thing?

Aaron: I think it is different. We can do the things we can do because of the size and the constraint that we are. We are also very capital constrained compared to, say, Procore. We cannot hire the incredible team they have. They have some incredible engineers, product, and design folks. So for me, I do not think it is black and white that SaaS is dead, as much as that might help in certain cases. There is going to be a mix of realities. We do play in a slightly different bucket to Procore. We are not necessarily in the big tier-one builders, and I am not sure a big tier-one builder would want to go to a quickly vibe-coded solution out there in the market. That is not to say we do that, we pride ourselves on quality.

But companies can build their own tools now. We have customers literally building tools on top of Buildertrend, instead of Buildertrend, in certain areas. That part is interesting to help support. SaaS is dead from the point of view that maybe we should be building the system of record for these companies to build on, versus building out the exact SaaS layer. Rather than trying to guess and build a dashboard that solves every customer’s needs, how about we just give them the tools, MCP or whatever, for them to build that on top of.

All the way through to the fact that there is a lot of SaaS I am still going to pay for. I am still going to pay for a HubSpot. I am still going to pay for Slack. There is still so much core SaaS in our lives. If we had Fable, we could probably fire off a prompt to build Slack, but why? What is the point? Where is your time and cost and value? Construction companies have to understand where their expertise lies, and what we can do as a software provider to help. Construction companies are going to be hiring AI operations people. That is going to be a new role coming to every business, and they are going to be looking at tools. So I look at it more as, how can we build those tools? Procore will do exactly the same thing. It is an up market for everyone involved who wants to lean in a bit, as long as you do not actively go against the builders and instead help support the builders. That is what is really important.

What customers are actually asking for

Matt: That is a really interesting point about building the tools that people can build on themselves. In a space like construction, I always used to get quoted the McKinsey line that construction was one of the least technologically developed industries in the world. I do not know whether I ever truly believed that stat. But what you are talking about is enabling people to build on top of your tools, or to use AI to do that. I would be keen to understand what you are seeing. You have a big go-to-market team out in the States, and my assumption is always that US-based companies tend to be slightly more advanced when it comes to technology than in Australia. Maybe that is the wrong assumption. What are you seeing? Are these organizations building with AI, or is it still too scary, push it into the future?

Aaron: We play in the mid market, which is a really hot, interesting space. There are companies that are forging ahead. They have AI operations people inside the business building on the latest stuff. It is a funny one, but if you have got people in construction asking you for an MCP server, as a tech company you kind of feel like you have lost the battle when it comes to winning this race. We are being pulled by some of our customers. We just relaunched our MCP server, and one customer said that is fantastic, we have actually already built one around your API for you.

So we have customers wanting us to train them up, more of a services piece, helping them transition their company and think about how they use AI internally. There is probably a business in there in itself, helping other companies, especially in construction. From my rough feel, Australia is just as interested in the AI and tech stuff as the US. I just think we have less saturation here. There are fewer tools, less availability, less stuff out there. So people get what they are given, and often that might be a powerful ChatGPT or a Claude subscription, which is still fantastic, but they do not have the ecosystem around that. It is crazy to have non-technical people in construction asking how this MCP can do this and that. You are left thinking we are some ways ahead and some ways behind on these pieces.

Matt: MCP is still super early in its journey. We were involved with it about six to twelve months ago, and I remember speaking to people who would go, what the hell is MCP? So that is an amazing story. Would this be an IT manager asking that question?

Aaron: No, this is a director. A non-technical director asking about this stuff. In this case, a non-technical director of a mid-size commercial builder. Typically, at our size of business, we work directly with the directors, founders, or the site team. We do not tend to work with a heap of IT focus. In some ways that is probably why it goes a bit faster, because the slightly bigger companies with an IT team have a bit more, not necessarily resistance, but more red tape, more unpacking. Which is fair in this day and age of AI, understanding what is actually safe and reasonable.

Now the directors out there probably feel the same pressure founders do: oh my god, this is new technology, I feel like I am not doing enough, I probably have FOMO in some regards. They are Googling what to do, or asking ChatGPT. So there is a real opportunity to connect directly with these people who want to pull themselves in this direction and actually understand or learn. A lot of them are just curious. They are so used to being able to do everything themselves, and they are thinking, okay, what is this AI thing, I need to learn it and jump in.

Building enough context in a hard environment

Matt: That is good to hear. Companies like Procore and Autodesk are investing heavily in this space, so they must see an opportunity. Taking a step back to your organization and how you are applying AI: construction is an unusually hard environment for AI and agents. You have data spread across so many different resources, whether that is drawings, site diaries, and so on. How do you create the right and enough context for the agents to operate within your platform and provide intelligent output?

Aaron: It is weirdly an area AI can thrive in, but it is harder to initially apply. There is so much context you need to unpack with construction: safety, quality, previously known items, and so much data. The way we initially started Buildpass’s journey on this was by picking very finite things, small encapsulated processes. Just extracting context from an image: what is in this image, so I can do better searching on it, show me all the ones where an elevator or a platform was on site, and unpack that. All the way through to paperwork. A big thing in construction is paperwork. Everything runs on paperwork. Buildpass in the early days built out a system to digitise that, and it was very manual, drag and drop stuff in. The best thing AI can build out is generative blocks that take in a PDF form and convert it. So we applied this in an encapsulated, rigid, almost deterministic flow of AI initially. That is very useful, because it just speeds things up.

Where AI becomes quite useful is when it is not so capped and constrained, when it is more free-flowing, the chat-like version of things. We are only just getting to that now. We launched our own version of a chat plus an MCP server that lets them connect to it. The reason it has taken us so long is that we wanted to get it right. Not perfect, but how do you make it useful? How do you make it actually work within the system, and what context do you pull in? Only recently did models, in terms of tool usage and context size, get to the point where you can do things effectively, repeatably, and deterministically. Especially when you are talking about safety and construction, you do not want to suddenly fib and hallucinate this stuff.

It has been a real eye-opener. AI immediately works well for that small 20% of use cases. But as you get broader, when it becomes less deterministic, which is the whole point of AI, it became harder and less straightforward to implement, and to explain to our users. A lot of these construction folk paid for X and want to get X. So how do you explain how AI works within a system where they want to ask, who is on site today, do they have the right qualifications, how does that work for them? Actually helping explain that, and getting them to that level of flexibility, has been a more recent push for us.

Why human in the loop is non-negotiable

Matt: One of the challenges of construction, we often talk about life and death. You have drawings that are designed to be exactly the way they are. How do you introduce non-deterministic AI into construction when ultimately you are handing off ownership of a decision, or a bit of information, to AI?

Aaron: This is genuinely one of the most important things to get right. The way we position ourselves is that human in the loop really does need to always be there, especially with these pieces. Construction safety is quite literally life and death, and it is not something we should be playing with willy-nilly. Even a construction drawing, if you get that wrong, could result in significant variation cost, defects, and quality issues. So we have always built in the idea that AI helps get the data pre-filled, or pulls things out, but a human is in the loop. We do not have a multi-step AI approach, on purpose. AI is not generating something and then passing it off behind the scenes. If it is generative, it is shown to the user. They can scan it and go through it. It then relies on that user to review it, but the way it is built, through approval steps and everything, the whole idea is that we do not want to remove humans. We just want to make their jobs a lot easier.

We see it as, how can we make that safety consultant or quality consultant more effective, able to pick up other things they would not have seen before, and actually decrease the likelihood of an on-site accident or a defect. So for us it is literally: do not make assumptions, do not combine results, and always show the user. Getting that human in the loop is so important with AI. We have stuck to that. It is interesting to see people do a very hands-off, away-from-keyboard approach to AI, but for us the risk is just too great to personally have that.

Trust, confidence scores, and the learning loop

Matt: It is really interesting. I was chatting with a news organization two days ago, and the challenge they were talking about was that their journalists want to review information faster. The question is not that you need fewer journalists, it is that you need more journalists, because now you can assess way more information and you still need a human to sign off on it. The way they were looking at it was a multi-step process, with a Python script running inside the agent so you could see each step, and every single step required a fact-checker who would then do non-agentic work, come back, and move forward to the next step. There is not a lot of talk around those environments. Journalism is an extremely important part of our day-to-day and our culture, and the assumption is that if everyone just used AI, why would you need journalists? But it is the same with construction, because you cannot replace 20 years of knowledge with AI, you can just make the job slightly easier. Really it comes down to trust. Whether you are dealing with an autonomous agent in construction or anywhere else, it comes down to: how do you trust that agent? If you are introducing any level of non-deterministic behaviour, trust is the number one problem. So how do you solve for the trust problem?

Aaron: It is a really important thing right now, and it comes with scaling risk factors. Not everything needs high trust. If we talked about extracting context from an image, you probably do not care as much, you push it out there. But when it comes to assessing drawings and pulling out potential issues, you would want a very skilled architect or someone to go through it and unpack it. The obvious answer for building trust is evaluations, evals, but those only go so far. The scorecards from the latest state-of-the-art models always use the latest models where possible, so have your own evals.

And build in confidence scores. Do not treat everything returned to the user as 100% right. It is about positioning. People are getting used to the idea of, I think this is it, I have a 50% confidence rating, versus, I am 100% certain, this is it, and here is why, here is the evidence. That is really important. We have all built trust with ChatGPT or Claude by using them heaps and seeing the value build up, learning the quirks. Our users are having to learn a separate system. So we try to provide as many key factors as we can: sources, links, and their own deterministic way to approve or reject, building that trust up over time.

Then there is the concept of reinforcement learning, the idea of getting a scorecard back on how our AI went with some of these pieces. That is where a lot of the moat will build up with these companies. The more we all use ChatGPT or Claude, we are all collecting very thin layers of reinforcement learning data that we can then use to optimise our execution, whether that is prompts or actually modifying some of the underlying models.

One-way versus two-way doors

Matt: I am going to come back to that, because it is a really interesting point. Going back to the way people trust ChatGPT and the journey they have gone through, how do you classify low-risk versus high-risk actions within the agents you have working?

Aaron: We have an internal mantra, not even just for AI, of a two-way or one-way door. When it comes to manipulating data, or burning trust, or anything to do with safety, it is very much a one-way door. You want to apply slower safeguards around that. It is literally asking yourself: if something was to go wrong here, what is the impact analysis, and unpacking that matrix of things. You do that on a micro feature, but you also bundle it up. You have to test it and use it yourself, know the limits, and be more of a leader in it internally before you start applying it out there and randomly stamping AI everywhere. If you do not personally know how to use it, that is a problem. A big learning for us internally has been upskilling our entire team to be better at using AI, so we know how to present it back and share those insights with our customers through tools. Then knowing what is a one-way door, what is a two-way door, what needs safety guards, and explaining those risks to the user in context.

Matt: Is there anything you would never allow an agent to do autonomously?

Aaron: Quite a few things. Off the top of my head, directly manipulating a database and customer data. Everything needs to be traceable. For compliance reasons, we need a system of record that is immutable and can be returned to, so they know the system was in a certain state: what forms or credentials a worker had when they came on site at a given time. We need that to be secure for our customers. It does not have to be in the AI. We put so much pressure on AI being perfect, but we are also imperfect humans. So I apply the same rules I would to, say, a junior engineer or a junior person on the team. I am not going to give that person direct production access to manipulate customer data either. But with high trust, with learning, with capabilities and safety nets and procedures, we will start to loosen the grips around the controls, in a way that we can observe.

The reason tools like these are so useful is that they do some pretty scary stuff. They are in your emails and so on, but there are safe ways to go about all of that. There were a bunch of breaches with people using these tools, not because the tool is innately insecure, but because people did not understand the whys and whats. Unfortunately, in this day and age, we still need to understand enough technical aspects of what we are building, and how we are using it, to protect ourselves, our customers, and our team. That will loosen up as AI gets better and the tools around us mature. Who would have thought that today we are all connecting our email and Slack to these agents. That would have seemed pretty scary a year ago, but now it is so useful, and there are good enough controls around it that we are pretty happy.

Delegating control without the accept-all trap

Matt: It is funny you should say that. I was chatting to a well-known Australian enterprise not long ago about their AI journey, and they had hired a head of AI risk and a head of AI compliance. They were using Copilot, everyone’s favourite AI. I asked, so what can your Copilot do, what can it access, what databases? They said, oh no, it cannot access anything internally. I said, well, so what are you actually risk managing? They said, oh, we are putting in place the systems so that in the future we can connect it up to different apps and emails. This was locked down to a score. There were no connectors, no SMTP servers. I thought, okay, that is an interesting approach to AI. We are at a point where we are happy connecting up to Google and Slack, and I suspect some of your customers would not be happy with that. Coming back to your product, in construction a confident mistake can really affect safety, compliance, cost, and schedule. What evidence do you look for before graduating an action in the agent from always ask to trusting the agent to do it?

Aaron: There are multi levels here. There are certain systems and tools where we put it out there, see failure rates, and collect data on how it went processing and what it was doing. For low-risk stuff, we have naturally graduated it. But not every tool should always get that graduation step, or at least it should provide controls for it. We are seeing that even with SMTP servers and general tools you connect in. Anthropic still very much allows granular control for the end user. We are moving to a world where we do that in a way that is smart, safe, and easy to use, not convoluted, so the end user can control it.

We have a wide range of customers. Some are a bit more willy-nilly with the data, they just want the ROI, throw everything at AI, paste it in, get the results back. Others are still on that journey of trusting and learning and growing, and they have different risk appetites. Allowing them to have that control is important. So we are moving the system towards not making that decision for them and instead letting them make it. Do you want that done autonomously? Do you want to auto-accept, or accept each time? It is the same treatment Claude Code and ChatGPT’s Codex give us, that fine-grained control. As long as it is done without a convoluted mess of controls everywhere that makes it harder and less compliant, because it is too hard and people just hit accept all. It is like how we started with Claude Code and everyone just hit dangerously accept permissions, because it was so annoying to constantly accept. Then both systems brought out auto mode, a much safer alternative. It is the same for us: how do we simplify this down for our end users without making assumptions about how they want to do it, and without being scary or over the top, so they actually understand.

Rollbacks and escape hatches

Matt: That is really interesting, because I have direct experience with just pressing yes. As a non-engineer, I do not understand most of the stuff I am saying yes to, which is why I am not in our product repos. What you are essentially doing is delegating the responsibility for trust to your users, in the same way Claude delegated responsibility to us to say yes to all these things. How do you judge the confidence level someone might have? I know I would just say yes. You have to assume people understand what it means when they are saying yes. How do you approach that?

Aaron: This is again an ever-evolving thing, because what yes means is very different to different people. Honestly, it is trial by fire. We have had customers where the agent said, do you want to remove this or delete this, and they said yes, and then it deleted it and they asked, why did it delete it? Because you said yes. So we have to be safer and take extra care in explaining that stuff, and doing things like rollbacks. We are working on a live system. If you think about Claude Code working in codebases, or in chat, the mistakes of it going wrong there can sometimes be disastrous if there are tools connected, but for the most part you can just undo what it did. You can undo that change or commit, or ignore that change to a piece of text. When you apply that generative framework to a live system, where you are actually deleting things, you do need to provide extra gates, explain things, and potentially allow a rollback.

We have had people where the whole idea is, we can upload documents and work out where you want to put them. Even when we explain what will be created, they might say, oh, actually I want that removed. So for us it is going the extra mile, deeply explaining the side effects, and always providing an escape hatch, always letting them go, actually, what that just did is pretty bad, so how do I undo it? To be fair, we learned that from making some initial mistakes. On internal data we were testing, we manipulated some data and it did exactly what it said it did. It deleted it, and then I wanted it back, and it is not as easy as that. So we are putting a lot of extra care around those tools. The same mistakes will be made by people using the software normally, but we can go beyond that and build extra mechanisms.

Quality loops and capturing intent versus outcome

Matt: I said I would come back to it. You talked earlier about learning loops and, ultimately, what we would describe as quality management. I would love to understand how you structure that. How do you go from agent runs, you learn the outcome, you have run evaluations, you have some qualitative mechanism where customers say yes, no, or maybe? How do you structure your quality loop and that continuous learning to improve, whether it is your prompt management or whatever it is within your agentic system?

Aaron: I would say it is ever-changing. I do not think we are necessarily nailing it, there is a lot to go. A lot of companies have something, and it is important, but given the month-on-month improvements with state-of-the-art models, what you are actually capturing with potential reinforcement learning data is very minimal compared to how much the underlying models and the harnesses around them are improving. Traditionally we had not done an amazing job of capturing that data, and probably did more of the eval stuff, and that has been perfectly fine.

But I think we are moving back towards the idea that the data you can collect is a bit of your moat. I do not mean I am going to go and train my own Opus or Fable model. I mean using that data to build a better prompt. Instead of you as a human building a prompt, you can use AI: here is the input, here is the eventual output of what they actually accepted in the system, let us do a bit of a loop and get to that point. We are early days, but we are starting to capture more of that data: what was the intent and what was the actual outcome, regardless of whether they even used AI. If they upload a construction form as a PDF and eventually have a digitised version, they might use AI to start it, but what were the tweaks and edits they made after? What was the hand-crafted work they did? That is an interesting data layer. We are not going to train a model with it, but we will tweak how things work, and it becomes a potential eval test in the future.

As a contrived example: with one model version, we always have to tweak the form a certain way, but then a new model like Fable comes out and it has fixed all of those problems, we do not have to re-tweak it, and we are seeing higher acceptance rates. That is more what I care about today, given our position as a company. If I had the money, I might reinforce-learn and train my own models off an existing provider, but you need a lot of data for that. Much more important is capturing the intent and the actual outcome, regardless of AI or not.

Catching agentic drift, and why evals are the moat

Matt: What I am really trying to understand, from everyone I speak to, is how you handle agentic drift, whether that is because models change or because underlying things change within the model. Coming back to what you do, construction safety is so important that any level of drift can really affect what is going on. How do you catch that before it happens?

Aaron: This happens even within your own development life cycle. If I am coding with an agent, I build up these paper cuts, these drifts of things it is not doing, and I might dump that into an agents.md file. It is a funny thing, because every time you build up this list of things it does not do, you almost have to forget that when the next model comes through, and reapply new tests each time. I do not have the golden answer, but for me it is always having your own evals. Sometimes those can just be vibes, but a lot of the time it is recording those frustrations, those deltas, those drifts, properly. That is the eval as moat idea: the tests you have internally that determine if a model is good or bad, that becomes your moat as a company.

That is important to expand on. If you suddenly have a model smashing it out of the park on some things, only you know about it, or only you can use it through the unique ways you manipulate it through prompts for your users. You should double down on that. So every time I have a frustrating moment internally as I am building with AI, I document it, either through tools or manually. I gave it this prompt, it did not do it. When the next model comes out, I try to see if it fixed that issue. If it did not, now I know I can steer it a different way, or I do not have to add that extra context, as these models get smarter. I do think the concept of evals, and they do not have to be as formal as people think, will become much more part of the day-to-day and the moat of a company.

Matt: That is really interesting. That is definitely a clip I am going to take: evals are a moat for your company. If everyone uses the same models, technically you get the same output, but it is the learnings you get from how the model interacts with the data you have, and the activities users do in your platform, that really make the difference. That is really insightful.

Aaron: Again, people think an eval has to be some sort of eval harness and testing loops with a specific output. It can literally be a vibe check. In some ways that is an eval, it is just how you weight it, which can be very different depending on the outcome. We test out new models all the time. Every time a new model comes out, part of your evals is having your engineering team, or your founders, or your directors jump on those new models, test them, and play with them. That is part of that personal eval phase. So it can be as strict or as simple as you want it to be.

Matt: Fair enough. This is the challenge we are trying to solve for, so it is a good one. Thank you so much, Aaron. I feel like that time has gone really fast.

Aaron: It did, yeah.

Matt: It has been great to hear about your journey and where you are at with AI. It sounds like you are full throttle and going really well. Great to hear from you, and great to chat.

Aaron: Yeah, thanks for having me. Appreciate it.

Frequently asked questions

How does Buildpass keep a human in the loop in construction?

Every generative output is shown to the user before it is used. There is no hidden multi-step chain where AI produces something and passes it on behind the scenes. Aaron's position is that construction safety is life and death, so a person always scans, reviews, and approves the result, and approval steps are built into the flow.

How does Aaron decide when an agent can act on its own?

He uses a one-way versus two-way door test. Reversible, low-impact actions like extracting context from an image can graduate to autonomous. Actions that manipulate data, touch safety, or cannot be undone stay behind confirmation, with confidence scores and sources shown so the user can approve or reject, and with rollbacks and escape hatches where a mistake would be costly.

Why does Aaron call evals a moat?

Everyone can run the same state-of-the-art models, so the differentiator is the internal tests that tell you whether a new model is actually better for your users. Buildpass captures intent versus outcome data, what a user asked for and what they accepted, and uses it to tune prompts and catch agentic drift when models change underneath the product. An eval can be as informal as a vibe check.

Is SaaS dead in construction?

Aaron does not see it as black and white. Plenty of core SaaS stays in daily use. His bet is that the larger opportunity is building the system of record and the tools, MCP included, that customers build on top of, rather than another dashboard that tries to solve every customer's needs.

Stay ahead of the curve

No spam. Unsubscribe anytime. A resource by Prefactor.

Almost there — check your inbox to confirm your subscription.