Google is making private AI practical with homomorphic encryption

AI
Security
Privacy
Open Source
Infrastructure

Google’s post introduces HEIR, an open source compiler and optimization stack for homomorphic encryption, aimed at making private AI inference less painful to build. The promise is simple to state and hard to achieve: a user encrypts data locally, the server runs a model on ciphertext it cannot read, and the user decrypts the result. That is a stronger privacy story than ordinary encryption in transit or at rest, because the provider never needs plaintext during inference.

The discussion landed on one blunt point. “Practical” still means niche. People with direct experience in privacy-preserving machine learning said the overhead remains enormous, and the concrete numbers shared in the comments back that up. Small demos can run in around a second. Larger ones still take minutes. Benchmarks for related FHE operations are measured in tens of milliseconds for simple integer ops and seconds for harder ones. That is a huge improvement over the old reputation of FHE as pure theory, but it is still nowhere close to ordinary cloud inference or even many local deployments. The more useful framing was not “does this replace local models” but “what threat model justifies the cost.” FHE looks interesting where plaintext exposure is the blocker, not raw compute price. Examples people kept returning to were medical queries, biometric matching, leak checking, private database lookups, and other cases where the input is sensitive enough that a 10x to 1000x penalty might still be acceptable. For mainstream chatbots or hot-path product traffic, many concluded local inference or on-prem hardware wins on cost, latency, and autonomy. There was also a clear distinction between FHE and secure enclaves. Enclaves can get you much closer to normal performance, but the trust anchor is hardware vendors and the operator’s software stack. FHE’s appeal is that the guarantee comes from the cryptography itself. That difference matters if you care about adversarial providers, subpoenas, or future hardware bugs more than you care about speed. The mood around Google specifically was skeptical. A lot of people simply do not trust Google to be the messenger for privacy tech. Still, the stronger technical comments separated that corporate distrust from the cryptographic claim. If the system really is FHE and the code is open, the privacy guarantee does not depend on goodwill. The real caveat is elsewhere. FHE can hide inputs and outputs, but it does not by itself prove the server ran the exact model you expected, nor does it solve availability, account lockout, or the broader question of whether cloud dependence is acceptable in the first place.

Treat this as infrastructure for regulated or unusually sensitive workloads, not a general answer to private AI. If you need privacy now, compare FHE against local inference and secure enclaves by threat model, latency budget, and who you are actually trying to avoid trusting.

August 14, 2026
blog.google
Discuss on HN

Key insights

FHE and secure enclaves solve different trust problems

Secure enclaves can keep plaintext hidden from the host machine, but they still require trust in the hardware vendor, the enclave implementation, and the code the provider chooses to run inside it. FHE changes the trust boundary. The provider never gets plaintext at all, so even a malicious operator cannot simply add logging, exfiltration paths, or rely on future enclave breaks like the ones cataloged at sgx.fail. That makes FHE a worse performance trade and a better adversarial-provider story.

Do not compare FHE to enclaves on speed alone. Pick based on who you need protection from, including the cloud operator, hardware bugs, and compelled access.

Attribution:

siddthesquid #1 #2
llleeeoooh #1 #2
j2kun #1

Control flow is the real performance killer

The painful part is not just “encryption is slow.” FHE handles fixed arithmetic circuits much better than computations with data-dependent branches. Sorting, hash maps, ReLU-style conditionals, and lookups are nasty because the server cannot peek at plaintext to choose a path, so it has to simulate the logic in a branch-free way. That is why some ML workloads are less hopeless than general-purpose computing, and why benchmark choice can wildly distort how viable FHE looks.

When evaluating FHE, inspect the model and preprocessing pipeline for branching and lookup-heavy stages. Arithmetic-heavy models may be candidates, while anything with irregular control flow probably is not.

Attribution:

mswphd #1 #2
odo1242 #1
hamper653 #1

Encrypted inference does not prove the model was honest

Hiding the data is only half the problem. FHE does not automatically attest that the provider ran the exact model, weights, or approximation regime you expected. Even an honest implementation can return garbage if inputs fall outside the bounds assumed by polynomial approximations. So confidentiality can be strong while correctness and integrity remain weaker than many people assume from the word “encrypted.”

If model fidelity matters, ask for an attestation story on top of FHE. Privacy alone is not enough for high-stakes workflows like medical or financial decisions.

Attribution:

noman-land #1
LoganDark #1
mswphd #1

The near-term market is narrow but real

The credible use cases were not “private ChatGPT for everything.” They were specific jobs where plaintext handling is the barrier to adoption or legality, such as leaked-credential checking, biometric authentication, private medical queries, dot-product style matching, and other small cryptographic subroutines that are painful to custom-build each time. A general-purpose tool matters because it lowers the cost of applying FHE outside teams that can hire cryptographers to hand-roll protocols.

Look for workflows where the sensitive input is small, the computation is structured, and compliance friction is high. That is where FHE can move from research novelty to purchasable product.

Attribution:

u1hcw9nx #1 #2
j2kun #1

The server can compute without understanding the plaintext

Several comments cleared up a recurring confusion about how encrypted computation can work at all. The server is not “reading enough of the ciphertext” to infer the prompt. It applies a fixed circuit to every ciphertext, much like linear algebra on encoded values, and the result only becomes meaningful after the client decrypts it. Private Information Retrieval was offered as the simplest mental model for this, and it helps explain why FHE is mathematically coherent even if it feels counterintuitive.

If your team is new to FHE, teach it with fixed-circuit examples like Private Information Retrieval before jumping to AI. That will save a lot of bad objections and bad architecture decisions.

Attribution:

drdeca #1
mswphd #1
pluto_modadic #1
Ar-Curunir #1

Against the grain

High overhead can still beat mandated backdoors

A less common view was that a 1000x slowdown is not automatically disqualifying if the alternative is regulated plaintext scanning or weakened end-to-end encryption. For abuse detection, law-enforcement filtering, or other politically charged categories, expensive cryptography could be the price of preserving a stronger default privacy guarantee while still satisfying pressure to inspect content.

Watch regulated communications products closely. Some of the first serious FHE deployments may emerge where legal pressure makes expensive privacy-preserving inspection preferable to explicit backdoors.

Attribution:

hellohello2 #1

A thousandfold penalty is survivable over time

One commenter pushed back on the doom around a 10^3 slowdown, arguing that fixed overheads that look absurd today can become tolerable as hardware and systems improve. Most people did not buy that for mainstream inference, but it is a useful reminder that “commercially impossible” and “not yet worth deploying” are not the same claim.

Avoid making roadmap decisions from today’s benchmark gap alone. If a privacy-sensitive workflow has strategic value, keep an eye on hardware acceleration and compiler progress rather than dismissing the category outright.

Attribution:

hellohello2 #1 #2

In plain english

ciphertext ↩

Data after it has been encrypted into a form that should be unreadable without a key.

FHE ↩

Fully Homomorphic Encryption, a kind of homomorphic encryption that supports arbitrary computations on encrypted data.

HEIR ↩

Google’s open source toolkit for compiling and optimizing computations to run with homomorphic encryption.

homomorphic encryption ↩

A form of encryption that lets a server perform computations on encrypted data without decrypting it first.

plaintext ↩

The original readable data before encryption, or after decryption.

privacy-preserving machine learning ↩

Methods for training or running machine learning systems while limiting what data other parties can see.

Private Information Retrieval ↩

A cryptographic technique that lets someone query a database without revealing which item they asked for.

ReLU ↩

Rectified Linear Unit, a common neural network operation that outputs zero for negative inputs and the input itself for positive ones.

sgx.fail ↩

A website collecting attacks and weaknesses against Intel Software Guard Extensions enclaves.

Reference links

Benchmarks and performance references

Oblivious Sorting under Fully Homomorphic Encryption: A Comprehensive Survey and Performance Analysis
Shared as a concrete source for current FHE performance numbers on sorting and related operations.
TFHE-rs benchmarks
Referenced for primitive encrypted integer operation timings such as equality, addition, and division.
HE-LRM paper
Used to cite inference latencies on UCI and Criteo datasets for homomorphic-encrypted logistic models.
FHE benchmarking suite
Pointed to as a better overview of practical FHE performance than a single repo folder name suggests.
SOFAR CIFAR demo
Given as an example of a recent GPU-accelerated FHE image inference demo around 200 milliseconds.

Google project resources

HEIR benchmark code
Linked as the benchmark code for Google’s HEIR project.
Google fully homomorphic encryption demos
Suggested as a clearer place to inspect runnable Google FHE demos.
HEIR project site
Referenced as the project page claiming fully homomorphic encryption support.
HEIR intro talk from 2023
Shared as background on the project’s goals and design.

Background explainers and learning resources

Programmable Cryptography 1
Recommended as an accessible overview of FHE and related programmable cryptography ideas.
FHE Textbook
Suggested as a learning resource for understanding how homomorphic encryption works.
Jeremy Kun FHE overview
Used to explain ciphertext expansion and key size overhead in modern FHE schemes.

Alternative private inference approaches

Tinfoil
Mentioned as a secure-enclave based provider offering private inference at much lower overhead than FHE.
Private Mode
Listed as another secure-enclave style alternative to FHE-backed private AI.
sgx.fail
Shared as evidence that secure enclaves have a long history of practical attacks.

Other companies and ecosystems

Zama
Named as another active company building tools in the homomorphic encryption space.

Google is making private AI practical with homomorphic encryption

Discussion mood

Key insights

Against the grain

In plain english

Reference links

Benchmarks and performance references

Google project resources

Background explainers and learning resources

Alternative private inference approaches

Other companies and ecosystems