Engineering Notes · agent operations · policy as code
How I decide which of my AI agents’ rules stay as written instructions and which become programs that physically refuse. An AI agent will follow a written rule most of the time. “Most of the time” is fine for style preferences and fatal for invariants.
My AI agents run under a rulebook, and early on the whole rulebook was prose. Instructions in a CLAUDE.md: do not push to production branches, do not post raw ticket comments, do not debug with print statements. And the agents followed them. Mostly.
Mostly is the problem. A model reads its instructions at the start of a session and can drift 100,000 tokens later. A model swap or a harness update changes how well the same sentence lands. When a rule guards a style preference, the occasional miss costs nothing. When it guards an invariant, one miss is the incident. I learnt to draw the line hard: if it is unacceptable for the agent to do a thing, that thing cannot be a sentence the agent reads. It has to be a program that inspects the command and refuses.
So my setup splits every rule into one of two treatments. Judgment gets prose. Invariants get hooks: deterministic shell scripts that sit between the agent and the action, and block it with an explanation. The agent cannot talk its way past a bash script.
Every one of these exists because of a real incident or a real near-miss. None of them were written speculatively:
git stash in shared working trees. Born from a parallel agent running a mid-task stash and wiping sibling agents’ edits. The block message cites the incident.var_dump-style debugging into PHP. There is a real debugger wired in; guesswork printing is banned at the tool level.It explains itself. A good block message states what was blocked, why (citing the incident), and what to do instead. My agents read the message and follow the recovery path in-session. A bare “denied” just breeds retry loops.
It anticipates the workaround. An agent that hits a wall will helpfully try bash wrappers, env prefixes, sub-agents. The guards scan for those forms too, and the block message says explicitly: do not wrap, do not delegate, do not edit this hook. Belt and braces: the prose rulebook carries the same instruction.
The bypass belongs to the human. Every guard has an escape hatch, an environment variable exported before the session starts. The agent cannot set it mid-session. Deliberate override stays a one-liner for me; impossible for the tool.
Per-project opt-out is explicit. Not every project needs every guard. A small file in the repo lists disabled hooks by name. Divergence is visible and reviewable, not configured in someone’s head.
Learn the hook contract before trusting it. My hook system fires post-command hooks only for commands that exit zero, and the payload has no exit-code field. I found that out empirically, and it changed how guards record evidence. That contract is now itself asserted by my rule-eval suite every run, so a harness update that changes it gets caught the same day.
Guards can bite their own tooling. My eval suite tests the guards by replaying banned commands, and the guards scan command strings, including the eval script’s own. The suite had to learn to pass prompts via files so no banned literal ever appears in a command. In fact, while drafting this very article, the ticket-comment guard blocked the write, because the draft contained a banned command as an example. If your guards are good, they will catch you too.
Wire-up decays; verify it. The embarrassing one: a later audit found several projects with zero guard hooks actually wired, because a master-template edit did not propagate to the per-project configs. Intention decays. Now drift telemetry compares every project’s wiring against the template, and the rule evals prove the guards fire, not just exist on disk.
The industry name for half of this is policy-as-code, and I am not claiming to have invented it. The part that matters is where the line sits: anything whose failure I cannot accept is enforced by a program, and the programs themselves are tested. Everything else is allowed to be prose. Hope is not a control.
All from my claude-skills-central repo. Every guard ships with a .test.sh beside it.
git stash in shared trees; block message cites the incidentI use my tooling predominantly on Mage-OS (Adobe Commerce / Magento) e-commerce projects, and my own AI Booking Agent. If you have not swapped to Mage-OS yet, you are falling behind ;)