Engineering Notes · agent operations · investigation discipline

My AI Is Not Allowed To Guess

Lucas van Staden · ProxiBlue · September 2026 · proxiblue.com.au

How I turned debugging from an AI's favourite improv exercise into a protocol: blast radius before hypothesis, evidence before claims, a real debugger instead of print statements, and a banned-phrase list for blame-shifting.

Why I built this

Ask an AI agent why a test is failing and, left to itself, it will do the thing juniors do: leap to a plausible story. "Probably a flaky test." "Looks environmental." "Seems like a pre-existing issue." Plausible, confident, and suspiciously often pointing anywhere except at the change the agent itself just made. Then it "fixes" the story instead of the bug, and you get to debug both.

The pattern that kept repeating in my sessions: the failure appeared right after the agent edited something, and the agent's first hypothesis was about anything but its own edit. Not dishonesty; a language model reaches for the most fluent explanation, and "my last change broke it" is less fluent than a nice environmental story. So the fix is not scolding, it is protocol.

The protocol

• Blast radius before any hypothesis. The first mandatory move on any failure: enumerate what changed. git diff against the last known-good state, working tree included. You do not get to theorise about the world until you have looked at your own footprints. Most of the time the investigation ends right there.
• Claims require artefacts. A diagnosis must cite its evidence: the file and line, the diff hunk, the quoted test output. "Probably", "likely" and "seems" presented as conclusions, without a citation attached, count as failure. Hedging is allowed while investigating; it is banned in the verdict.
• Blame-shift phrases are banned outright. "Must be a flake." "Environmental." "Pre-existing issue." "Infrastructure is down." Not because these are never true, but because they are true far less often than they are convenient. If one of them is genuinely the answer, it has to be earned with evidence like any other claim, and stated as fact, not reached for as an exit.
• Report and stop. Investigation requests produce a diagnosis with evidence, not a surprise fix. I decide what happens next. An agent that "investigates" by changing five things has destroyed the crime scene.
• A real debugger, and the lazy alternative is blocked. For PHP runtime questions (wrong value, wrong branch, null return) the agents drive xdebug through tooling: breakpoints, real variable inspection. And because prose alone would only be followed most of the time, a guard hook blocks any edit that inserts var_dump-style print debugging. Guesswork printing is not discouraged; it is refused (the debugger-discipline post covers that split).

Enforcement is layered the way everything in this series is layered: the protocol text auto-loads when a failure appears, the debugger guard is deterministic, and eval 6 of my rule-eval suite replays the origin scenario on a schedule: a repo where the suite fails because of an in-tree edit, and asserts the agent ran the diff first, cited evidence, named the real cause, and used zero banned phrases. A judge model grades the report against the protocol; grep checks the rest.

Things I learnt building it

What it buys me

• Diagnoses I can audit. Every verdict arrives with its citations. Checking the agent's work means reading its evidence, not re-doing its investigation.
• Fewer double-bugs. The fix-the-story-not-the-bug loop is mostly gone, because the story has to survive the diff first.
• Debugging that transfers. Breakpoint sessions leave reproducible traces; print-statement archaeology leaves polluted diffs. The banned tool was also the worse tool.

Want to do this yourself?

• Make "what changed" the mandatory first move on any failure, and mean your agent's own changes, not the weather.
• Require citations on conclusions. No artefact, no verdict.
• Write your banned-excuse list as exact phrases, and check for them mechanically.
• Separate investigation from fixing. Diagnosis is a deliverable.
• Wire in a real debugger and block the print-statement path with a hook, not a request.
• Replay a known incident on a schedule and grade the report. Trust the protocol because it keeps passing, not because you wrote it.

None of this makes the agent smarter. It makes the agent's claims cheap to verify, which for client work is worth more. I do not need my AI to always be right on the first look; I need it to never be confidently wrong in a way I cannot check.

Related files

All from my claude-skills-central repo, except where noted:

• rules/reference/investigation.md — the protocol itself: blast radius first, artefact citations, the banned-phrase list, report-and-stop.
• hooks/test-failure-context.sh — injects the compact protocol into the session the moment a test failure appears, so it is loaded before the first hypothesis.
• rules/reference/php-debugging.md — the PHP runtime-question branch: xdebug tooling, never print statements.
• hooks/php-debug-guard.sh — the deterministic block on var_dump/print_r/dd()/dump()/ray()/xdebug_break() edits.
• scripts/rule-evals.sh — eval 6 builds a repo, breaks it with an in-tree edit, and grades the investigation report.
• koriym/xdebug-mcp — the xdebug MCP bridge the agents drive (third-party, not mine).

I use my tooling predominantly on Mage-OS (Adobe Commerce / Magento) e-commerce projects, and my own AI Booking Agent. If you have not swapped to Mage-OS yet, you are falling behind ;)

blast radius first · claims need artefacts · banned: "must be a flake", "environmental", "pre-existing" · xdebug wired, var_dump blocked · replayed by eval 6