Your AI Agent Needs Containment

A long time I ago I found myself in a fighter jet in Dutch airspace. I wasn’t a fighter pilot, but I was granted control of the jet anyway. It was awesome.

Now there’s quite an interesting story as to how I got myself in that position, but that’s not what this blog is about. Today, I want to talk about AI for a bit. But before we do that, you may be wondering what idiot allowed me to fly a freakishly expensive fighter jet to begin with. While not being a pilot or Air Force staff. That ‘idiot’ was the qualified pilot sitting behind me in this two-seater. The ‘idiot’ whose only means of taking back control would be telling me ‘my aircraft’ or ‘I have control’ over the radio, and then hoping I’d let go of the controls. He couldn’t lock me out of doing anything stupid. We’ll call him Bob.

What does this have to do with AI?

I thought about this situation a lot recently, while reading people’s stories about their AI adventures — some glorious success stories, and some very expensive horror stories alike.

Bob took a very calculated risk, letting me fly. He had seen me fly the simulator before, so he knew I at least had some idea of how to fly the thing without going into a direct nose dive. He also worked with me in a different setting earlier, from which he knew that I actually would relinquish control of the jet if he would tell me to. Still, he had no ‘override switch’ to push when things would go sideways.

Using AI involves risk, and some form of assessing and/or mitigating the risk. The stories about both success and failure in using AI all have these same elements to them, whether the people writing these stories realize it or not: Controller Proficiency, Executor (or Agent) Proficiency, Executor Compliance, and Guardrails.

For me and Bob, this worked out to the following:

1. Controller Proficiency –> Bob was a qualified fighter pilot and instructor. He knew how to fly the jet himself, knew how to teach other people how to fly, and had vast experience sharing two-seaters with people who would make mistakes.
2. Executor Proficiency –> I had shown in the simulator that I could operate the jet without getting into accidents.
3. Executor Compliance –> I had shown earlier that I could follow guidelines, and wouldn’t go rogue.
4. Guardrails –> We didn’t have those.

Lack of Controller Proficiency

Quite often in the case of horror stories, the people using AI for things are a lot less proficient than Bob. They are using AI to do things they can’t actually do themselves. “I vibe coded my own SaaS without any SWE experience” is a real smash hit these days. Sometimes followed by the second, less successful single “My AI agent dropped the prod database during a code freeze”. It’s especially this group of users that tend to over-estimate the ability of AI, while simultaneously being the least qualified to identify risks.

What could possibly go wrong?

Lack of Executor Proficiency

Sometimes the AI is just not that great at doing what you want it to do. And sometimes, there’s no real harm in that. Ever generated images with text through ChatGPT? Yes, it messes up the text a lot. But usually the worst damage you can do with that is offend someone.

It gets a little worse when the AI starts writing bad code. Yes, it does that, and unless the Operator (you!) can actually assess the quality of said code, you’re going to ship garbage code. Hard coded secrets, lack of type safety or input sanitizing, lack of error handling. As long as the code compiles, AI succeeded at fulfilling your request, right?

Lack Executor Compliance

This is where things get even more interesting, because compliance can mean many things. If you ask your AI assistant to write a piece of code, it’ll write you code. But:

• If it hard-codes an API key to make your code compile, did it still do its job?
• If it finds your AWS credentials on disk and uses them because the tests fail without AWS credentials, did it still do its job?
• If it finds a means to get root permissions to install a dependency it thinks it needs, did it still do its job?
• If it spins up a bunch of cloud infra to mine crypto because you mentioned ‘getting rich’ in your prompt, did it still do its job?

There’s a burden on you, the operator to be extremely explicit about what ‘compliance’ means, and which rules should be followed. Which, again, requires you to be proficient yourself.

And then still, you should consider your AI an overzealous, overconfident, reckless, and gloriously resourceful junior dev on steroids. And you should act accordingly.

Guardrails

Not only do you need to tell your overzealous junior dev what it can and cannot do — you need to make sure it can’t forget/ignore the second part, or attempt to work around these limitations. Lately it seems the industry is starting to realize the risks here are very real, and both the tools available and the awareness have been too limited.

I played ‘the AI game’ on hard mode so far, by consistently ensuring that whatever tools I used (lately I’ve been on OpenCode), they were contained inside ephemeral DevContainer instances that I’d use for a single issue. This meant I couldn’t fully exploit the benefits of AI tools, but I found it more important to mitigate the risks that nobody was talking about, unless things really broke.

Tools like OpenShell from NVIDIA, and Ona’s Veto are steps in the right direction. Tools for containing AI, codifying policies, and adding guardrails the AI can’t reach.

So that you don’t have to fly on blind faith, whispering ‘my aircraft’ to the computer, and hoping it didn’t wipe out your hard drive or your credit card.