Two small models crossed the Hacker News front page this month, and between them they settle an argument most engineering teams are still having in the abstract. One is a 4B-parameter model that produces Postgres query plans 1.81x faster than the database’s own planner, trained for roughly $1,200. The other is a 29 MB automation model that, when a few hundred developers poked at its public demo, turned the lights a warmer colour when asked to warm the house. Both are genuine pieces of work. The gap between them is not budget, talent or model quality. It is whether the task arrived with a number a machine could score.
TL;DR
- QORL, a 4B model trained for about $1,200, produced Postgres plans with a 1.81x geometric-mean speedup across 113 join-heavy benchmark queries and zero regressions.
- Cactus Needle 3 ships tool-calling models at 8 to 29 MB, but public testers found it unreliable on anything less literal than “turn all the lights off”.
- The difference is the reward signal. Query latency is measured with a stopwatch. “What did the user mean” needs a human.
- QORL’s zero regressions came from the harness, not the model: generate candidate plans, time them, keep the database default if nothing wins.
- Before funding a narrow model, test four things: an automatic reward, a bounded input distribution, enough repeat volume to amortise, and a safe fallback.
What $1,200 actually bought
QORL is a solo project by Rohan Bansal. The policy is a distilled 4B Qwen adapted with LoRA, about 21.2 million trainable parameters, roughly 42.5 MB of adapter weights. It was bootstrapped with supervised fine-tuning on 420 trajectories generated by a frontier model, then sharpened with agentic reinforcement learning against a live Postgres instance. Plans reach the database through pg_hint_plan, the model emitting structured hint comments prepended to the original SQL rather than replacing the planner outright.
Training ran on the Cardinality Estimation Benchmark, 13,646 queries across 16 templates. Evaluation ran on the Join Order Benchmark: 113 queries, 33 templates, against the 8.5 GB IMDb dataset. The result was a 1.81x geometric-mean speedup, a 44.7% reduction in summed workload latency, 68 queries faster than the Postgres default and no regressions. The bill was about $800 for roughly 95 hours on a two-H100 Lambda node, plus about $400 in frontier API fees for the bootstrap trajectories.
What 29 MB actually bought
Cactus Needle 3 attacks the other end of the size curve. It ships as 8 to 29 MB binaries, 25 to 121 million parameters at 2-bit quantisation, with each layer from 2 to 20 deployable as a standalone subnetwork so you can pick a size that fits a watch, a Raspberry Pi or an IP camera. It does not chat by design. It emits tool calls and structured JSON. On the vendor’s Mobile Actions benchmark the 20-layer model scores 86.0% against 82.4% and 76.0% for two considerably larger baselines, and a fine-tuned 4-layer version edges past DeepSeek V4 Flash on Android commands.
Then several hundred developers tried the public demo. “I need a wee” triggered music, because “wee” looked like a genre. “Turn the kitchen to 230°C” executed at 0.9015 confidence. “My car crashed I need help” returned a music query at confidence 1. One developer benchmarking tool-calling for a games database got 32.2% correct tool shapes from a fine-tuned Needle 3 against 90.9% from a fine-tuned FunctionGemma. Another found it degrades past about ten tool definitions.
The founder’s replies were candid and, read closely, prescriptive: cut the tool list, write better tool descriptions, constrain numeric ranges, add deterministic triggers, fine-tune for your specific task. Every one of those is an instruction to narrow the domain until the behaviour becomes checkable. That is the QORL recipe, arrived at from the opposite direction.
The reward signal is the whole story
Query planning is an unusually kind problem. There is one correct notion of better, latency, and the environment computes it for free every time you run the query. The model can be wrong ten thousand times during training and each wrong answer comes back with an exact score attached. That is what $1,200 of compute buys: a very large number of cheap, honest corrections.
“Did this tool call match what the human meant” has none of that. There is no environment that returns a number. Ground truth is a labelled dataset somebody has to build and keep building, which is why the strongest observation in that Hacker News thread came from a developer who had tried the same problem and concluded the dataset mattered more than the architecture. Needle 3 was trained on 360B tokens of structured data, and the failures testers found were not compression artefacts. They were the long tail of human phrasing, which no amount of parameter efficiency addresses.
So the question to ask about any narrow-model proposal is not “how small can we get it”. It is “who or what produces the score, and what does one score cost”.
Zero regressions came from the harness, not the model
The QORL headline that should interest a CTO is not 1.81x. It is zero regressions, and that number is an engineering artefact rather than a model property. The system generates several candidate plans per query, times them, and keeps the winner. The Postgres default is always in the candidate set, so the floor on performance is the behaviour you already had. The model can only improve on the baseline or be ignored.
Needle 3’s demo failures have no equivalent floor. The output goes straight to the actuator, and a wrong tool call at 0.9 confidence is indistinguishable from a right one until the oven is at 230°C. This is the same lesson as the serving stack being part of the model: the thing you deploy is the whole loop, not the weights.
If your task cannot be scored at inference time, you cannot build that floor, and you are relying on average-case model quality in production. Sometimes that is acceptable. It should be a decision, not a default.
Four questions before you fund one
- Is there an automatic reward? A number a machine computes with no human in the loop: latency, compile success, test pass, schema validation, a diff against a known-good output. If your answer involves annotators, price the annotation before the GPUs.
- Is the input distribution bounded and observable? QORL trained on one company’s shape of workload and did not need to generalise beyond it. Do you have logs of the real inputs, and are they narrower than “anything a customer might say”?
- Does the volume amortise? Training cost is fixed and inference cost is near zero, so the case strengthens with every repetition. Analytic queries that run daily qualify. A monthly report does not.
- Is there a safe fallback? Can you generate candidates, score them and keep the incumbent when nothing beats it? If not, you are shipping a model with no floor.
Score four out of four and a narrow model is one of the best returns available in AI engineering right now. Score two, and you are funding a research project with a product deadline attached.
The honest cost
The $1,200 is the compute, and quoting it alone is how these projects get mis-budgeted. The expensive part was the surrounding engineering: orchestrating four Postgres containers to keep cache contention from poisoning the measurements, tuning shared_buffers to 2 GB to drag timing noise from roughly 5% down to near zero, and building a reward loop trustworthy enough to train against. If your reward signal is noisy, reinforcement learning happily optimises the noise.
That is weeks of senior engineering time, and it is the line item that tends to be missing when a narrow-model proposal reaches a board paper. We have written before about the costs of AI in production that nobody budgets for, and measurement infrastructure is the most reliably underestimated of them.
Where this leaves your roadmap
Do not read this as a case against frontier models. Read it as a case for putting them in the right place: use one as a teacher and a fallback, not as a per-request dependency for a task you perform a million times a day with a measurable outcome. The same logic underpins the shift towards local inference, and the economics now favour it wherever the scoring problem is solved.
REPTILEHAUS builds AI systems for teams who have to run them afterwards, including the evaluation harnesses that make results like this repeatable rather than anecdotal. If you have a high-volume task with a measurable outcome and a growing API bill, that is exactly the shape worth examining. Get in touch and we will tell you honestly whether it scores four out of four.
📷 Photo by William Warby on Unsplash
