A developer spent last week moving his agent prompts off Anthropic and OpenAI and onto a self-hosted Ollama stack. The hardware was not the problem. The model size, he says explicitly, was not the problem either. The problem was that a 35kb system prompt which had been running cleanly against a frontier API for months fell apart within three minutes of arriving on his own machine. His write-up reached the Hacker News front page on 14 September. Most of the comment thread argued about his politics. The measurement underneath is the part worth keeping.
TL;DR
- A 35kb system prompt consumed roughly 14% of a 65k self-hosted context window before the agent did a single turn of useful work, and the agent immediately began thrashing: identical tool calls, re-reading files it had already read, rewriting finished work.
- The failure was capacity, not model quality. The same prompt is well-formed against a 200k window and broken against a 65k one without a single character changing.
- Large frontier context windows have been quietly subsidising imprecise prompts, because chain-of-thought has room to reconstruct what you meant, and providers only return summaries of it so you never see the repair happening.
- Context headroom is an undeclared dependency. Most teams can name their model and their monthly spend. Very few can say how much of the window each agent actually needs.
- Five context-exhaustion signals are loggable against your current provider today, with no migration and no new infrastructure.
Three minutes to empty
The setup is a 128GB AMD Ryzen AI MAX+ 395, with 32GB reserved for the host operating system and the remainder allocated to inference. That yields an effective context window of about 65,000 tokens. It is a serious piece of local hardware, not a hobby rig.
The 35kb preprompt lands in that window and takes about 14% of it immediately. The arithmetic is unremarkable and it checks out: 35kb of English is in the region of 9,000 tokens, and 9,000 against 65,000 is roughly one seventh of the budget gone before the user has typed anything. Add the session history that any real agentic task accumulates, and saturation arrives in a handful of exchanges.
What saturation looks like in practice is the useful part. The agent issued the same tool calls back to back. It read files it had already read. It rewrote work it had already finished. The author’s description is the one that will stay with people: with a limited window, the preprompt is briefing a man who is reincarnated every ninety seconds. He performs your most recent instruction with no awareness of the fifteen that preceded it.
Your abstraction layer does not cover this
Model-agnostic architecture, as most teams implement it, means an interface in front of the API call. We have argued for that pattern ourselves when writing about tooling consolidation and portable context.
It is necessary and it is not sufficient, because the prompt is also part of the coupling. A system prompt is a specification written against an implicit capacity budget that nobody wrote down. It was tuned by iteration against one provider’s window, and the window was never a stated requirement. It was just there.
We covered one half of this problem in August, when a detailed investigation showed that the serving stack is part of the model: quantisation method, KV cache precision and attention backend all change behaviour on tool calls even when the weights are byte-identical. This is the other half. Even with a perfect serving configuration, an agent whose instructions were sized for somebody else’s hardware will fail on yours, and it will fail in a way that looks like stupidity rather than capacity.
Big windows have been paying for small mistakes
The sharpest observation in the write-up is not about Ollama at all. It is that one of the most valuable things you get from a frontier provider is not the model, it is the window, and the window has been doing work you were not aware of.
Generous context gives chain-of-thought room to run. That reasoning space is frequently spent reconstructing what an underspecified prompt was trying to ask for. You see a good answer and conclude the prompt is good. Anthropic and OpenAI return summarised reasoning rather than the raw trace, so the repair work is invisible by design. With a fat window and enough reasoning slack, mediocre prompts produce excellent results, and you receive no signal that anything needed fixing.
Which means the quality of your prompt library is currently unmeasured. The measurement only arrives at the moment the slack disappears, and the moments when slack disappears are rarely convenient: a pricing change, a deprecated model, a data residency requirement, a client whose procurement team will not accept US inference, or a context window that gets quietly rebalanced at the tier you are paying for.
Five signals you can log this week
The genuinely portable contribution in the piece is a list of context-exhaustion symptoms. These are all observable in agent logs you are probably already producing, against the provider you are already using. Treat them as a slack meter.
- Identical tool calls back to back. The agent has lost the result of the previous call.
- Multiple reads of the same file in one session.
- The agent restating its own objective. Re-anchoring behaviour, usually just before it drifts.
- Tool-call parse failures. Worse than they look, because a failed parse tends to dump raw output into the window and accelerates the exhaustion that caused it.
- High turn count relative to files changed. The aggregate metric. Rising turns per unit of actual work is the signature of an agent spending its budget on remembering rather than doing.
If those numbers are already non-zero on a 200k window, you do not have a migration problem waiting for you. You have a prompt problem happening now, partially concealed by capacity you are renting.
What to do before you are forced to move
- Record prompt size as a percentage of window, per agent. Make it a number in the repository next to the prompt, and assert on it in CI. A prompt that has grown 40% since it was written should fail a check, not surprise somebody during an incident.
- Split monolithic preprompts into single-objective units. One problem, one resolution, one agent. This is the change the author found did the most work, and it improves results on frontier models too.
- Replace prohibitions with positive directives. “Only do Y” costs fewer tokens and fewer reasoning cycles than “do not do X”, and it degrades far better under pressure.
- Persist session state to disk and re-read only the slice you need. Frequent, deliberate handoffs beat one long session that slowly forgets its brief.
- Budget tool calls per agentic step. Every call returns tokens into a finite space. Unbounded tool use is unbounded context use.
- Run one agent against a deliberately small window as a standing regression test. This is the highest-value item on the list. A 32k ceiling in CI turns an invisible dependency into a failing build, and it costs you one test job rather than one migration.
Note what that list is not. It is not “self-host everything”, and it is not “stay where you are”. Every one of those six changes makes your agents cheaper, faster and more predictable on the provider you are using today. Portability is the side effect, which is the only kind of portability work that reliably gets done.
The part worth leaving behind
The original post is written in anger. It accuses frontier providers of stealing users’ ideas from session histories, prompted by a public dispute earlier this month over credit for a mathematical result. That accusation is not established. “Cannot rule it out” is a lawyer’s sentence, not evidence, and it is not a foundation to build an architecture on.
You do not need it. Capacity is a dependency whether or not anyone is reading your transcripts, and the case for knowing your own context requirements survives the removal of every grievance in the source material. Build for it because you should be able to name your dependencies, not because you are angry at a vendor.
Where this lands
The teams that will move providers smoothly are not the ones with the cleverest abstraction layer. They are the ones who can answer a simple question: how much window does this agent need, and what happens at half that? Almost nobody can answer it today, because the answer has never cost anything.
REPTILEHAUS builds and operates agentic systems, evaluation harnesses and the DevOps around them, including hybrid deployments where sensitive work runs on infrastructure clients control and the rest does not. If you are running agents in production and cannot say what their context budget is, that is a short engagement with a long payoff. Get in touch.
📷 Photo by Wesley Tingey on Unsplash
