VaultGemma is Google Research + DeepMind’s newest language model: a 1-billion-parameter open-weight LLM trained from scratch with rigorous differential privacy. The research includes new “scaling laws for differentially private language models,” showing how performance scales with privacy budget, compute, and data.
Compared to its non-private counterpart (Gemma-3 1B) and older models like GPT-2, VaultGemma loses some ground on benchmarks but is roughly comparable to models from about five years ago.
It comes with formal privacy guarantees (ε ≤ 2.0, δ ≤ 1.1e-10 at sequence level) and strong empirical and theoretical protections. Model weights have been made public via Hugging Face and Kaggle.
