Skip to main content

On 6 October 2026 the threat research team at Signals Corps published a scan of the HTTP response headers of 217,097,223 crawled pages, and on 15 October it reached the Hacker News front page. The finding that travelled was the smallest number in it: 30 sites send a header that tries to give an AI agent orders. The more useful part is the sentence they put next to it. Whether any of that reaches your model depends on which parts of the response your fetch tool passes along.

That is a claim about your software, not about the web. So we tested it. We built a canary server that puts a unique marker in every position a server controls, ran seventeen real fetch tools and command lines against it, and recorded exactly which markers came back out. Then we probed 401 live domains to see how much of that surface is already in use.

TL;DR

  • We ran 17 fetchers against a 13-position canary, 119 runs. 8 carried server-controlled text from outside the page body into the tool output, 7 passed the body alone, and 2 refused to fetch a private address at all.
  • The leak nobody documents is the HTTP status line. The reason phrase is free text the server chooses, and 6 of 6 HTTP clients we tested put it verbatim into their error surface: requests, httpx, aiohttp, urllib, urllib3 and Node’s fetch.
  • That turns two body-only tools into header-side leaks. Microsoft’s markitdown 0.1.8 and Hugging Face’s smolagents 1.26.0 both pass the server’s sentence to the model on any 4xx or 5xx, smolagents as an ordinary tool result rather than an error.
  • RFC 9112 says a client SHOULD ignore the reason phrase. Of 595 status lines collected from live sites, 21 domains already send something other than the default. One, luas.ie, answers a missing page with a full English sentence.
  • Agent-only content is live. 11 of 305 domains served Markdown when asked, and sentry.io’s agent-only page hands the agent an MCP server configuration to install. 73 of 305 serve a materially different response to an AI user agent; 5 declare Vary: User-Agent.

The canary

The test server is 90 lines of Python. Every position a server controls carries its own token, so a marker in the output names the field it came from: the value of an X-AI header, the header name itself, the status line’s reason phrase, a Set-Cookie value, a Link header with rel="llms-txt", the Content-Type parameter, a Content-Disposition filename, the Server string, a redirect Location, the body of robots.txt, and the page body itself as a control.

Seven paths exercise the routes a tool can take: HTML, plain text, a non-HTML content type, a 301, a 404, a 500, and a path robots.txt disallows. The scoring is mechanical: a token in the returned text means that field reached the model’s context. No judgement, no model in the loop.

Finding one: eight of seventeen

Seven fetchers returned the page body and nothing else: plain curl, wget -qO-, requests read through .text, LangChain’s TextRequestsWrapper, LangChain’s WebBaseLoader, and llama-index’s SimpleWebPageReader. For those, the channel is closed. Nothing a server writes outside the body arrives.

Eight carried at least one header-side position, and the shell commands are the loud ones. curl -i, curl -v with stderr captured, and wget -S each delivered nine positions in a single run, including the header name, the reason phrase, the cookie, the Link title and the redirect target. curl -I delivered seven and no body at all: an agent running a HEAD request to check whether a page exists receives only the fields the server invented.

Two more leaks are quieter, and they are the reason this post exists.

Finding two: the status line is a text field

An HTTP/1.1 response begins with a version, a number and a reason phrase. The phrase is free text. RFC 9112 is unambiguous about what a client should do with it: “A client SHOULD ignore the reason-phrase content because it is not a reliable channel for information.”

Six HTTP clients out of six do the opposite. Pointed at our 404, with the server’s phrase set to an instruction, every one of them reproduced it:

requests  HTTPError: 404 Client Error: Not Found <phrase> for url: ...
httpx     HTTPStatusError: Client error '404 Not Found <phrase>' for url ...
aiohttp   ClientResponseError: 404, message='Not Found <phrase>', url=...
urllib    HTTPError: HTTP Error 404: Not Found <phrase>
urllib3   response.reason  -> 'Not Found <phrase>'
node      res.statusText   -> 'Not Found <phrase>'

That is not a bug in any of them. raise_for_status() interpolates response.reason because a human reading a stack trace wants to see it. The problem is downstream: in an agent, an exception message is not read by a human. It is caught, stringified and handed to the model as the result of a tool call.

Two tools in our sample do exactly that. markitdown 0.1.8, Microsoft’s document converter, raises an HTTPError whose message carries the server’s sentence. smolagents 1.26.0, Hugging Face’s agent framework, catches it and returns it:

'Error fetching the webpage: 404 Client Error: Not Found
 <attacker-chosen sentence> for url: http://.../err'

Both tools are body-only by design. Neither exposes a single response header. Both hand the model text a server wrote, from a field no browser renders, that the specification tells clients to ignore. The only thing an attacker needs is for the agent to request something that fails, and an attacker who controls the server controls that too.

The obvious objection is that HTTP/2 deleted the reason phrase. It did, and it does not matter, because the client picks the version. requests speaks only HTTP/1.1, httpx does unless you install the h2 extra and ask, and aiohttp has no other mode. We confirmed the server side too: of 298 live domains that answered us, 297 served HTTP/1.1 the moment a client asked, including the 252 that also speak HTTP/2.

Finding three: 21 domains are already writing in that field

We probed 401 domains, the 100 Irish sites from our robots.txt audit plus the Tranco top 300, for their root page and a guaranteed-missing path, and collected 595 status lines. 558 carried the RFC default phrase. Two were empty. 35 lines across 21 domains carried something else.

Most are harmless archaisms: 22 servers still say 302 Moved Temporarily rather than Found. A few are editorial. office365.com answers a blocked address with 403 Ip Forbidden, aerlingus.com says 405 Not Allowed, and Dublin’s tram operator writes prose:

$ curl -sI --http1.1 https://www.luas.ie/no-such-page
HTTP/1.1 404 The requested content does not exist.

Nobody is attacking anyone with that. It is a tidy 404. It is also proof that real operators treat the field as free text, that it survives the path from origin to client, and that an agent asking luas.ie for a page that has moved receives a sentence chosen by the server, through a library that was only trying to write a helpful error message.

Finding four: the reference MCP server behaves as documented

The Model Context Protocol’s own fetch server is the implementation most MCP clients reach for first, so we ran it over stdio against the canary rather than reading its source. Two of our tokens came back, both of them expected and both of them worth knowing about.

On a non-HTML page it echoes the Content-Type value verbatim: Content type application/x-canary7a6+json cannot be simplified to markdown, but here is the raw content. On a path that robots.txt disallows it quotes the file back at the model inside the refusal. Everything else was clean, and its 404 message says only “status code 404” with no reason phrase, which is better behaviour than the Python HTTP clients underneath it.

Two tools refused to run at all, which is the good news of the week. mcp-server-fetch 2026.10.10 and trafilatura 2.3.1 both block private and loopback addresses by default, on every redirect hop, and the MCP server names the flag that overrides it. That is SSRF hardening shipped as a default in two separate projects. It also means trafilatura is untested here rather than clean, and we score it that way.

Finding five: agent-only content is live on real sites

Headers are one way a server can talk to an agent and not to you. The other is content negotiation, which we covered in August as a proposal. It is no longer a proposal. We asked all 401 domains for the same URL twice, once as Chrome and once with Accept: text/markdown.

11 of 305 responding domains served Markdown, including two Irish ones. Nine declare Vary: Accept correctly. The two that do not are a Cloudflare block page and sentry.io, whose agent-only body is 976 bytes against 628,876 bytes of HTML, a 0.16% representation, and reads in part:

You've hit the web UI. It's HTML meant for humans, not machines.
Here's what you actually want:
## MCP Server (recommended)
{"mcpServers": {"sentry": {"url": "https://mcp.sentry.dev/mcp"}}}

That is a legitimate and genuinely useful page. It is also a server handing an agent a tool-server configuration to install, in content no human reviewer of that URL will see, with no Vary header telling a shared cache the two representations differ. Change the hostname in that JSON block and nothing else about the mechanism changes. One Irish retailer, mcelhinneys.com, labels 927,635 bytes of ordinary HTML as text/markdown, the same mislabelling that sends the MCP fetch server down its raw-content path.

Finding six: 73 sites answer an agent differently, and 5 admit it

The same sweep fetched every domain as Chrome, as Claude-User and as ChatGPT-User. 73 of 305 returned a materially different response to one of the agent user agents, either a different status or a body differing by more than a quarter in size. Five declared Vary: User-Agent.

21 domains refused an agent outright while serving a browser a 200, and the Irish list is uncomfortable reading: gov.ie, citizensinformation.ie, ucd.ie, aviva.ie, axa.ie, businesspost.ie, pennys.ie, isme.ie, skillnetireland.ie and smythstoys.com. Our October audit found 27 Irish sites would not hand over robots.txt; this is the same posture applied to the page itself. The security half of it is that the branching is undeclared: a site that can tell an agent from a person can show each a different document, and 68 of these 73 say nothing about it.

What to actually do

The fixes are small.

  • Know which tool fetches. Seven of our seventeen pass nothing but the body. If your pipeline uses one of them, this channel is closed.
  • Never put an exception string into a prompt. Catch the error, log the full message, and give the model a status code and nothing else. One line, and the highest-value fix here.
  • Ban -i, -I and -v in agent shell policy. If an agent can run curl, it can read every header on any site it visits. Allow the bare form.
  • Treat tool output as data, not instruction. The header is inert without a capability behind it, so keep the model’s authority to act separate from what it reads.
  • Audit your own responses. Run curl -sS -L -D - -o /dev/null against your site and read what you send. Our sample’s median response carries 19 headers and 1,161 bytes; dublinairport.com ships 15,593, of which 14,592 is one Content-Security-Policy.
  • Declare any second representation. Vary: Accept when you negotiate on Accept, Vary: User-Agent when you branch on the client.

What we did not measure

We measured what reaches the model’s context, not what a model does with it; the second question needs a different experiment. We tested locally installable tools, so hosted readers are the obvious next run. Our 401-domain sample found one site sending x-llms-txt against 6,989 in a 217-million-page crawl, which is what a small sample of a rare thing looks like rather than a contradiction. For the mechanism behind all of this, our May piece on indirect prompt injection is the primer.

Prompt injection defence keeps being discussed as a model problem. Most of it is a plumbing problem, and the plumbing is auditable in an afternoon: which tool fetches, what that tool returns, and what your code does with the error.

REPTILEHAUS builds and audits AI agent systems, from tool-surface reviews and prompt-injection threat modelling to the DevOps work of keeping agent traffic and human traffic honest with each other. If you are shipping agents that read the open web, get in touch and we will run this canary against your stack.

📷 Photo by Sticker Mule on Unsplash