Serving Markdown to AI Agents: A Second Representation Is a Second Thing to Maintain

On 26 August 2026 a single-page proposal called acceptmarkdown.com reached the Hacker News front page with 122 points and 68 comments. Its argument is one sentence long: when a client sends Accept: text/markdown, serve it Markdown instead of HTML.

Nothing in the mechanism is new. HTTP content negotiation is specified in RFC 9110, the text/markdown media type has been registered since RFC 7763, and this is the machinery that has served language variants for decades. What is new is the client population: a measurable share of the requests hitting your origin now come from coding agents, and they do not want your navigation, your consent banner or your 400KB of JavaScript.

The proposal is technically correct. It is also, for most sites, a content operations decision dressed up as a header.

TL;DR

• Content negotiation via Accept: text/markdown keeps one canonical URL: Markdown to agents, HTML to browsers. It requires Vary: Accept and correct q-value handling.
• Payload reductions are genuine: Checkly measured 99.7% fewer tokens on its documentation, and Cloudflare claims up to 80% converting at the edge.
• Adoption is narrow and lopsided. Coding agents send the header (Claude Code, Cursor, OpenCode, Copilot CLI); consumer chatbots almost all still fetch HTML. That tells you which pages are worth converting.
• The three unbudgeted costs are cache fragmentation from a high-cardinality Vary, semantic loss in conversion, and owning a second representation that nothing tests.
• Measure your own access logs first. Most sites will find the traffic sits entirely on developer documentation.

What is actually being proposed

The convention is deliberately small. A client sends Accept: text/markdown. A server that can honour it returns Content-Type: text/markdown; charset=utf-8, adds Accept to Vary so caches keep the variants apart, and respects quality factors when the client expresses a preference rather than a demand. A server that cannot serves the HTML instead.

There is a second, quieter discovery path that matters more than it looks:

<link rel="alternate" type="text/markdown" href="/docs/tutorial.md">

That is the HTML4-era mechanism behind RSS auto-discovery, and it can also be sent as an HTTP Link header. It is not redundant with the Accept route: OpenAI’s Codex CLI sends no Markdown header at all, and instead fetches the HTML and looks for exactly that link element. Implement one and not the other and you will miss clients.

The numbers are real, and also flattering

Checkly’s February 2026 content negotiation test put concrete figures on it: serving Markdown for its documentation produced a 99.6% reduction in response size and 99.7% fewer tokens. Cloudflare, announcing its own edge conversion, quotes a more conservative “up to 80%”. Both are honest, and the gap is the useful part. A page rendered by a JavaScript application shell is close to a best case, because almost everything in the response is framework rather than content. A lean server-rendered article is a far less dramatic win.

Then ask who the saving accrues to. It goes to the agent operator, and publishers in the thread put the objection bluntly: why do extra work to hand content to AI companies and get nothing back? The economics only invert when you are the agent operator yourself, running a support assistant or an internal retrieval pipeline over your own content.

Who actually sends the header

Here is where enthusiasm meets the traffic log. Checkly tested seven agents and found three sending Markdown: Claude Code (text/markdown, text/html, */*, relying on header order rather than q-values), Cursor, and OpenCode (text/markdown;q=1.0, text/x-markdown;q=0.9, text/plain;q=0.8, text/html;q=0.7, the most standards-literate of the three). Codex, Gemini CLI, GitHub Copilot and Windsurf sent browser-style or wildcard values.

The acceptmarkdown status page, tracking the same question continuously, now lists seven supporting agents (adding Copilot Chat, Copilot CLI and Microsoft Copilot, verified June 2026), one partial, and fifteen with none, including ChatGPT, Claude.ai, Gemini, Grok, Perplexity, Devin and Zed. The two sources disagree about GitHub Copilot, and that disagreement is the finding: this matrix moves with point releases of tools you do not control, so re-check it quarterly.

The shape is consistent, though. Coding agents and IDE assistants send the header; consumer chatbots, the ones your marketing pages care about, do not. If your goal is visibility in AI answers, this is not the lever, and that remains a question of durable URLs and clean, crawlable HTML.

Three costs nobody budgets for

1. Cache fragmentation

Vary: Accept is mandatory for correctness and dangerous done naively. Accept headers are high-cardinality: compare Claude Code’s terse three-value list with OpenCode’s four-entry q-value string. A cache keyed on the raw header treats every distinct string as a separate object, and your hit ratio quietly collapses. Roy Fielding’s old objection to Accept-based negotiation, raised again in the thread, was precisely this trade-off.

The fix is normalisation: decide at the edge whether a request prefers Markdown, collapse the header to that single bit, and vary on the normalised value. Same discipline that keeps Vary: Cookie from destroying a CDN, new axis. Note too that Cloudflare’s converted responses drop ETag and Last-Modified, so that variant loses conditional revalidation entirely.

2. Semantic loss

Markdown cannot express everything HTML can. The example that stuck in the thread was chemistry: H₂O becomes H2O, because Markdown has no subscript. The same goes for MathML, complex tables and anything whose meaning lives in attributes. Cloudflare’s converter mitigates this by emitting YAML frontmatter from your meta tags and preserving JSON-LD in a fenced block, but mitigation is not equivalence. Where precision depends on markup Markdown cannot carry, you have introduced a class of error nobody notices until an agent quotes a wrong figure to a customer.

3. A second version of reality

This is the expensive one. When Cloudflare shipped its feature, SEO consultant Jono Alderson framed the risk well: the moment you publish a machine-only representation of a page, you have created a second candidate version of reality. Google’s John Mueller and Microsoft’s Fabrice Canel both pushed back for the same reason, Canel noting that non-user versions are often neglected and broken, and that crawlers will fetch the HTML anyway to check similarity.

Cloaking is the headline risk and the secondary one, because deliberate abuse is a policy problem with a policy answer. The likelier failure is neglect. A variant generated from canonical HTML at request time cannot drift; a separately authored artefact, or a static export refreshed by a job that fails silently in March, will drift worst on the pages you update most.

Two sane ways to ship it, and one to avoid

Edge conversion. Cloudflare’s Markdown for Agents, announced 12 February 2026, is a dashboard toggle on Pro, Business and Enterprise plans and for SSL for SaaS customers at no extra cost. It converts your origin’s HTML at the edge and returns frontmatter, body and JSON-LD, with x-markdown-tokens and x-original-tokens headers so the client can budget its context window. One source of truth, no template changes. The cost is inheriting someone else’s opinion about which parts of your page were navigation.

Markdown as the source. If your docs are authored in Markdown and compiled to HTML, you are being asked to serve a file you already have. Perfect fidelity, total control, and the implementation is a routing rule. For documentation this is almost always the right answer.

The anti-pattern is hand-maintaining a parallel Markdown corpus for a site authored in a CMS: that is a second publishing pipeline with no editorial workflow, and it will diverge inside two quarters.

What we would advise

Sort by content type, not by site. Developer documentation and API reference: ship it, because that is where the agents sending the header go and the Markdown usually already exists. Knowledge base content: ship it if you run your own assistant over it. Marketing pages: not yet, because the clients reading those do not send the header. Pricing and anything with legal weight: only with automated equivalence checks, since a variant quoting a superseded price is a commercial problem, not a technical one.

Then, in order:

1. Measure before you build. Log the Accept header for a fortnight and break the text/markdown requests down by user agent and path. Decide from your own numbers, not from a blog post’s.
2. Normalise, then vary. Collapse Accept to a single negotiated value before setting Vary, and check your hit ratio afterwards rather than assuming.
3. Pick one representation source. Convert from canonical HTML, or serve authored Markdown. Never operate both for the same content.
4. Add the alternate link in the head, or as a Link response header, to catch the clients that look rather than ask.
5. Test equivalence in CI. Fetch both variants of a sample of pages and assert headings, prices, figures and outbound links match. This is the one control that prevents the neglected-shadow-copy failure, and almost nobody implements it.
6. Fail honestly. If you cannot produce Markdown for a URL, serve the HTML. An agent parsing HTML is an inefficiency; an agent parsing a broken Markdown stub is a wrong answer.

The pattern underneath

Every few years the web grows a new class of consumer and the industry reaches for a parallel representation: RSS, mobile subdomains, AMP. AMP is the instructive one, because Google dropped its Top Stories requirement in 2021 and left a great many teams maintaining a second version of their site that no longer paid for itself.

Content negotiation is a better bet than any of those for one structural reason: it keeps a single canonical URL. No m. domain to redirect, no separate cache to invalidate, no link equity to split. It genuinely is the right architecture. What it does not change is the arithmetic of maintenance: one URL and two representations still means two things that can be wrong, and one of them that no human ever looks at. Ship it where the traffic justifies it, generate it from the canonical source, and test that the two agree.

At REPTILEHAUS we build and operate production web platforms, and increasingly that means designing for two audiences at once: people, and the agents acting on their behalf. If you are weighing up how much of your infrastructure should be agent-aware and how much of it is fashion, get in touch.

---

📷 Photo by Cody Moore on Unsplash