On 14 August 2026, Google published the case for something cryptographers have promised for fifteen years and almost nobody has shipped: AI inference that runs on data the server cannot read. The vehicle is HEIR, an open-source compiler that converts pre-trained models designed for ordinary data into models that operate directly on encrypted inputs.
The stated ambition is a one-click path for teams without a cryptographer on staff. That ambition is not met yet, and Google is reasonably candid about it. But it reframes a decision many companies are currently making badly: what to do with sensitive data you want to run a model over but cannot legally hand to a third party.
TL;DR
- Google open-sourced HEIR, an MLIR-based compiler (Apache 2.0) that converts pre-trained AI models into fully homomorphic encryption (FHE) models, so a server can run inference on ciphertext and return a result it never sees in the clear.
- The performance cost is still brutal. FHE runs 1,000x to 10,000x slower than plaintext, and ML inference can land five orders of magnitude behind. This is not a technology for your chat interface.
- It already ships in narrow, high-value places: Apple’s Live Caller ID Lookup, Apple Enhanced Visual Search, Microsoft Edge Password Monitor and encrypted balances on Ethereum.
- For most businesses in 2026, confidential computing is the pragmatic answer, not FHE. Hardware enclaves give near-native performance and solve most of the same commercial problem with a weaker but usually sufficient trust model.
- The driver is regulatory, not cryptographic enthusiasm. Data residency pressure, GDPR and Cyber Resilience Act reporting duties from 11 September 2026 are pushing “the vendor cannot see it” from nice-to-have to procurement requirement.
What HEIR actually is
Homomorphic encryption lets you compute on encrypted data without decrypting it. You encrypt an input, send the ciphertext to a server, the server does arithmetic on it, and you decrypt the result. The server learns nothing. It is not obfuscation or a trust promise; it is a mathematical property.
The catch has always been that expressing a useful program in the operations FHE supports is specialist work. Google’s own framing is blunt: converting existing programs by hand requires a team of cryptographers. HEIR attacks that bottleneck as a compiler problem rather than a library problem, which is the genuinely interesting design choice. Built on MLIR, it targets the major schemes across several backends: BGV, BFV and CKKS through OpenFHE and Lattigo, CGGI through tfhe-rs and Jaxite. Optimisation passes handle decisions that used to be research projects in their own right, such as automatically selecting a packing mode, which Google reports can accelerate a program by up to 145x.
Four workloads have been demonstrated: private content recommendations, credit card fraud detection, encrypted network anomaly detection, and a hotword detector that never hears your audio. Note the shape of that list. Every one is a small model doing a narrow, high-value classification on sensitive data. That is the most useful signal in the announcement.
The honest performance picture
We are not going to pretend this is ready for general use, because the numbers do not support it. Across schemes and workloads, FHE runs roughly 1,000x to 10,000x slower than the equivalent plaintext computation. Additions are cheap; multiplications and comparisons are where the cost concentrates, which is inconvenient given that comparisons are how most interesting decisions get made. For machine learning inference specifically, the overhead has been measured at around five orders of magnitude. GPU-accelerated research has run a GPT-2 class forward pass around 200x faster than CPU baselines, which is a real achievement and still nowhere near interactive.
The trajectory is better than the snapshot. Bootstrapping, the expensive noise-reduction step, now takes single-digit milliseconds on a modern CPU core and under a millisecond on H100-class GPUs. Intel demonstrated a fabricated FHE accelerator at ISSCC in February 2026 reporting speedups between 1,074x and 5,547x over a Xeon, though it remains a research prototype. Google’s hardware partners on HEIR (Belfort, Niobium, Cornami, Optalysys) are all working the same problem.
The correct read: small models on sensitive data are viable now, general-purpose encrypted LLM inference is not, and the gap is closing fast enough to be worth understanding rather than adopting.
What already runs in production
The scepticism this technology attracts is mostly deserved and slightly out of date. FHE has quietly crossed into production wherever the privacy requirement is absolute and the computation is small.
Apple’s Live Caller ID Lookup, shipping since iOS 18, uses a BFV-based private information retrieval scheme to check unknown numbers against a database without revealing the number being looked up, and Enhanced Visual Search matches landmarks in your photo library the same way. Microsoft Edge’s Password Monitor checks credentials against breach databases homomorphically. Zama’s protocol has run encrypted token balances on Ethereum mainnet since December 2025. Duality Technologies runs encrypted analytics with healthcare partners including Dana-Farber.
None of these are demos. They are also all narrow. That is the pattern to plan around.
The decision teams are actually facing
Strip away the cryptography and the business problem is familiar. You have data you cannot casually export: patient records, financial transactions, legal documents, anything covered by a data residency clause your largest client insisted on. You want to run a model over it. There are three options, and most teams seriously consider two.
Send it to a hosted provider is fastest to build and increasingly the option that gets vetoed in procurement, because “our vendor’s sub-processor” is a sentence that ends deals in regulated sectors. Self-host an open-weight model is now the right answer for many mid-sized companies, as we have written before: the data never leaves your infrastructure, and the cost is operational maturity. Compute on data nobody can read is the third, and the one HEIR is about. Here the distinction that matters commercially is not FHE versus everything else, but FHE versus confidential computing.
Why confidential computing is the 2026 answer for most of you
Trusted execution environments take a different route to a similar destination. Instead of making data mathematically unreadable, they make it unreadable to everything outside a hardware-isolated enclave: the host operating system, the hypervisor, the cloud provider’s staff. Remote attestation lets you cryptographically verify what code is running before you send it anything.
The trade is explicit. TEEs rest on a hardware trust assumption and have a real history of side-channel attacks, so the guarantee is weaker than FHE’s. In exchange you get near-native performance, and confidential GPU support means you can run a serious model inside one.
That trade is right for the overwhelming majority of commercial workloads. Industry forecasts put well over half of enterprises processing sensitive AI data on a path to requiring enclave-based isolation as a deployment condition, up from a small minority two years ago. Treat the percentage as vendor-adjacent optimism and the direction as real: it is already appearing in client security questionnaires.
Being opinionated about it: if you have a genuine confidentiality requirement today, evaluate confidential computing first. Keep FHE on the watch list for the narrow cases where a hardware trust assumption is not acceptable, typically because your counterparty is also your competitor, or because a regulator has views about who holds the keys.
The regulatory forcing function
The reason this is a board-level topic rather than a research curiosity is compliance. GDPR increasingly demands demonstrable protection of data in processing, not just at rest and in transit; cross-border transfer mechanisms remain contested; Cyber Resilience Act reporting obligations go live on 11 September 2026. Arguing the adequacy of contractual safeguards is a legal exercise with an uncertain outcome. Demonstrating that a processor is technically incapable of reading the data is an engineering artefact you can put in a data protection impact assessment. That is why procurement questions are arriving before the technology is fully ready.
Two criticisms worth taking seriously
The first is scope. FHE protects the inference and does nothing about the behavioural data collected everywhere else in a platform. Encrypted inference is not a privacy programme.
The second is the local alternative. Running a small open-weight model on the user’s own device or your own hardware gives you privacy by construction, with no cryptographic overhead and no trust assumption at all. If the workload fits in a few billion parameters, that is often simply the better answer, and it is available today.
What to do about it now
- Classify your inference workloads by confidentiality requirement. Most teams have never separated “we would prefer this stayed private” from “we are legally unable to expose this”. Only the second category justifies this complexity.
- Evaluate confidential computing for anything in that second category. Every major cloud offers confidential VMs and confidential GPU instances. This is a procurement and architecture exercise, not a research project.
- Check whether the workload is small enough to be a candidate. Fraud scoring, classification, matching, private lookup and anomaly detection are the FHE-shaped problems. If your use case resembles one of Google’s four demos, it is worth a prototype.
- Keep the privacy boundary explicit and the implementation swappable, exactly as you would with an LLM provider. Do not build a permanent abstraction around an immature primitive.
- Put it in the roadmap conversation, not the current sprint. The right posture for most companies is an informed watch: know what it would take, know your library and accelerator options, revisit in twelve months when the hardware lands.
Capabilities that lived in cryptography papers acquire tooling, then compilers, then defaults. Post-quantum migration went through exactly this cycle, and a compiler landing is how these things usually start moving.
At REPTILEHAUS we build AI systems, secure architectures and the DevOps to run them for entrepreneurs and management teams with real data protection constraints rather than theoretical ones. If you are working out whether a confidentiality requirement rules out the AI feature you want to build, get in touch.
📷 Photo by Towfiqu barbhuiya on Unsplash
