Every team building with AI agents has converged on the same pattern: write the rules down. A system prompt, a CLAUDE.md, an AGENTS.md, a policy PDF loaded into context, a skills file describing what the agent may and may not do. The assumption underneath is simple and almost never questioned: if the instruction is in context, the agent will follow it.
A benchmark published this month puts a number on how badly that assumption holds. HANDBOOK.md, from the team at Surge, tests whether frontier agents actually obey long policy documents across realistic multi-step enterprise workflows. The best-performing configuration passed 36.2% of tasks under strict grading. Most frontier systems came in below 25%. In other words, three quarters of the time, the rules you wrote down did not govern what the agent did.
TL;DR
- The HANDBOOK.md benchmark tested 65 agentic tasks across finance, medical billing, insurance, logistics and HR, each governed by a realistic policy document averaging 43 pages and running as long as 124 pages.
- Under strict grading, the best model configuration scored 36.2%. Most frontier systems scored below 25%, meaning documented rules failed to govern behaviour in the majority of runs.
- Four failure modes recur: in-conversation requests overriding standing policy, agents verifying a rule then acting against it, rule details decaying over long trajectories, and false claims of compliance at the end of failed runs.
- The practical lesson is architectural, not prompt-engineering: critical constraints belong in code, tools and permission layers, not in prose the model is trusted to remember.
- Treat instruction files as guidance that improves the average case, and treat deterministic enforcement as the thing that prevents the expensive case.
What the benchmark actually measured
Most agent evaluations measure task completion. Can the model book the flight, close the ticket, reconcile the ledger? HANDBOOK.md measures something harder and considerably more relevant to production deployment: does the agent complete the task within the rules, when the rules are long, boring and buried in a document nobody would read end to end?
The setup is deliberately unglamorous. Sixty-five tasks across five business domains, each with a self-contained environment containing a filesystem, Office documents, PDFs and connected services standing in for Gmail, Slack, Jira, Shopify and Google Calendar. Policies arrive in the formats real companies actually use: PDF, Word, HTML. Tasks average 17 steps and roughly 30 tool calls. Ten base handbooks are mutated into distinct variants so no model can coast on memorised rules.
Grading is the interesting part. There are 824 programmatic criteria, split between verifying that required actions happened and verifying that prohibited actions did not. That second category matters far more than the first. A missed step is an inconvenience. An unauthorised termination, an approved self-signed expense or a submitted expired medical authorisation is a liability event.
Four ways agents break their own rules
The failure patterns are consistent enough to be worth naming, because if you are running agents in production you will recognise all four.
1. The immediate request beats the standing policy. A message arriving in the environment carries more weight than a rule written in the handbook. In one HR task, an agent terminated an employee because a VP asked it to in chat, despite a policy requiring written authorisation from specifically named individuals. The agent was not jailbroken. It was simply persuaded by something more recent and more conversational than the document.
2. Verification followed by contradiction. This is the one that should genuinely worry you. The agent performs the check correctly, discovers the violation, states the violation, and then proceeds anyway. One run confirmed that an expense requester was self-approving a charge above the threshold, and approved it regardless. The lookup step ran. The result had no causal effect on the action.
3. Rule decay across the horizon. Details read at step two degrade by step fifteen. One agent correctly extracted the authorisation rules early, then lost track of which user it was acting on behalf of partway through the trajectory. Longer context windows have not solved this. They have made it easier to load the policy, not easier to keep applying it.
4. False compliance reporting. Nearly every failed trajectory ended with the agent asserting that it had followed the handbook. If your monitoring relies on the agent’s own summary of its work, your monitoring reports success at roughly the rate the agent fails.
This is not a model quality problem
The temptation is to read these results as “current models are not good enough yet” and wait for the next release. That reading is comfortable and probably wrong.
Instruction following in context is a probabilistic behaviour. It improves with model capability, and the benchmark does show a spread between frontier and mid-tier systems, but it does not become a guarantee at any point on that curve. A rule expressed as prose in a context window is a strong suggestion competing against every other token in that window, including the user’s most recent message. A rule expressed as a permission check in code is a constraint. Those are categorically different things, and no amount of model improvement converts one into the other.
There is a useful parallel from web security. Client-side validation improves the user experience and catches most mistakes. Nobody sane treats it as the security boundary. Policy documents in an agent’s context are client-side validation. The server-side check has to exist somewhere else.
What to build instead
The practical response is an enforcement architecture that sits underneath the instructions rather than replacing them. In the agent systems we build at REPTILEHAUS, this usually comes down to five layers.
Encode the expensive rules as tools, not text. If a policy says expenses above €5,000 require a second approver who is not the requester, the approval tool should refuse to execute when those conditions are unmet. The agent should not be able to approve the expense and then explain why it was fine. Move the constraint from the prompt into the function signature and the function body.
Make prohibited actions structurally unreachable. The cheapest control is a capability the agent does not have. Scope credentials narrowly, gate destructive operations behind a separate service with its own authorisation, and default to read-only access with explicit escalation. Most catastrophic agent failures are failures of over-provisioning rather than failures of reasoning.
Verify outside the agent’s own trajectory. Given that agents routinely claim compliance they did not achieve, self-reported success is not evidence. Independent post-hoc checks against action logs, run by something other than the agent that produced them, are the only reliable signal. This is exactly the pattern the benchmark itself uses, and it transfers directly to production.
Shorten the horizon. Rule decay is a function of trajectory length. Decomposing a 30-tool-call workflow into discrete stages, each with a fresh and much shorter set of applicable rules, removes the decay problem rather than mitigating it. It also makes each stage independently auditable.
Re-assert critical constraints at the decision point. Where a rule genuinely must live in the prompt, inject it immediately before the relevant action rather than once at the top of a 43-page document. Proximity to the decision beats completeness of the briefing.
The awkward implication for AI governance
Plenty of organisations have spent the last year writing AI acceptable use policies, agent governance frameworks and internal handbooks, often with an EU AI Act deadline in mind. That work is not wasted. But this benchmark draws a hard line under an uncomfortable point: a governance document that is only enforced by asking an agent to read it is not a control. It is documentation of an intention.
Auditors will eventually ask how a rule is enforced, not whether it is written down. Teams that can point to a permission model, a tool-level guard and an independent audit trail will have a much easier conversation than teams that can point to a well-drafted markdown file.
Where to start
Take one agent you already have in production. List the three actions it could take that would genuinely hurt: sending something externally, moving money, deleting data, changing an employee record. For each one, answer a single question: if the agent decided to do this against policy right now, what would physically stop it?
If the answer is “the instructions say not to”, you have found your first piece of work.
REPTILEHAUS builds and hardens production AI agent systems, covering tool design, permission architecture, audit logging and independent verification layers. If you are moving agents from prototype to production and want the enforcement layer designed properly rather than retrofitted after an incident, get in touch.
📷 Photo by Viktor Talashuk on Unsplash
