How much do language models really remember? Meta’s new framework defines the bit-level model capacity

Introduction: Memory Challenges in Language Models

Modern language models face increasing scrutiny on their memory behavior. Through models like the 8 billion parameter transformer trained in 15 trillion tokens, the researchers questioned whether these models remember their training data in a meaningful way. General technologies, including data extraction and member reasoning, are lacking because memory and generalization are often inaccurate.

Limitations of existing methods

Previous frameworks (such as extraction-based approaches or differential privacy) run at the dataset level rather than considering specific memory. Language modeling through compression and verbal models that evaluate capabilities through factual memory such as RNN and quantized transformers provide partial insights, but lack scalability and precision, especially for deep transformer architectures.

A novel way to measure memory

Researchers from Fair, Meta, Google DeepMind, Cornell University and Nvidia, have proposed a novel approach to estimate specific data points that a model “knows” to measure the capabilities of modern language models. They divide memory into two components: unexpected memory, which represents the model containing information and generalizations of the dataset, which captures information about the process of real data generation. They compute the total memory by eliminating generalizations to provide an accurate estimate of model capacity, which suggests that the GPT family model’s capabilities are approximate to 3.6 bits per parameter. The researchers also formulated a series of expansion laws that correlate model capacity and data size with member inference by training hundreds of transformer language models.

Experimental framework and training method

Using the GPT-2 architecture, the team trained hundreds of models, ranging from 100K to 20M parameters, with different depths (1-8 layers) and hidden sizes (32-512). Training involved:

• 10^6 steps
• Batch size: 2048
• Accuracy: Bfloat16
• Hardware: Single A100 GPU

These models are trained for synthetic sequences and have 64 token text sequences trained from the FineWeb dataset. These experiments ensure minimal interference from generalization to careful dataset construction.

Model capability insights and key findings

• bits for each parameter: Cross-configuration, the model is always stored in 3.5 and 3.6 bits/parameters.
• Double drop: As the training dataset size approaches the model’s ability to have a training dataset size, the test loss initially decreases (overfitting), and then improves again as the model begins to generalize.
• Effects of accuracy: Training in Float32 slightly increases storage capacity (~3.83 BPP) compared to BFLOAT16 (~3.51 BPP).

Unlock memory and generalization

The team observed the conversion from synthetic data to real-time datasets, please observe:

• Sample-level unexpected memory increases with parameter count.
• Memory decreases as training setting size increases.
• Accurate estimates of model memory require deduplication and reference to the Oracle model with baseline compression.

Member reasoning and scaling law

The researchers modeled the success rate (F1 score) of loss-based member reasoning, a function of the ratio between model capacity and dataset size. Key observations:

• As the dataset grows, member reasoning becomes unreliable.
• For models up to 1.5B parameters, the predictive scaling law is accurate within 1-2%.

Conclusion: Better understanding of model behavior

This work establishes a framework in principle for measuring memory in language models. By introducing quantifiable metrics and scalable experiments, it deepens our understanding of how the transformer model encodes training data and proposes clear boundaries between memory and generalization. The resulting insights can guide future developments in model evaluation, privacy and explanatory.

View paper. All credits for this study are to the researchers on the project. Also, please feel free to follow us twitter And don’t forget to join us 99K+ ml reddit And subscribe Our newsletter.

▶ Want to promote your product/website/service to 1 million+ AI Engineer/Developer/Data Scientist/Architects/CTOS/CIO? Let your partner…