Memory Models: Towards Agents That Learn

This blog post by Letta argues that true AI agents must be able to learn continuously from experience, and the key to that is memory models -- specialized models that generate and curate token-space memories. Current LLMs are great at using context but terrible at creating durable context; memory models optimize for cross-task, cross-model, and cross-session learning through a meta-RL training loop. The post envisions a future where agents self-improve by scaling compute on context (e.g., "sleep-time compute"), with memory becoming a more valuable asset than any single frontier model.

---

1. The Memory & Learning Problem for Agents

Agents should develop expertise over time based on their experiences and available context. Without learning from experience, they face clear limitations in:

• Adapting to proprietary datasets and workflows – while techniques like self-study and skill learning can turn private data into reusable contexts, existing methods are too expensive and suffer from overgeneralization and memory rot.
• Embedding human expertise for digital cloning – agents today cannot fully absorb human expertise, values, or natural-language feedback the way a human can.
• Deep personalization – despite plenty of interaction data, agents struggle to recall relevant context and generate new memories, limiting how personalized they can become.

Solving memory and learning unlocks a new scaling axis: scaling compute on context. The more time an agent spends refining its memory (via self-study or human feedback), the more expertise it develops.

"Without the capability to learn from experience, agents have clear limitations in use cases such as adapting to proprietary datasets and workflows, embedding human expertise for digital cloning, and deep personalization."

---

2. What Will Continual Learning for Agents Look Like?

The post argues that expertise from lifelong learning will be encoded as token-space representations that many different downstream models can consume. The most valuable agents of the future will be multi-model – able to carry memory across model generations and even architectures. Agents will be compound systems with multiple underlying models (coding models, real-time interaction models, etc.) working together. Token-space representations serve as the "connective tissue" for continual learning.

Multi-Model Learning

Agents must learn across model generations to maintain frontier performance. The "best" model varies by task (e.g., GPT-5.5 for backend engineering, GLM 5.2 for frontend design), and frontier model releases are accelerating while the time horizon of tasks agents can complete is rapidly lengthening. The post shows a projection of a crossover in late 2026 between model release cadence and agent task horizon, meaning agents will soon be capable of tasks lasting weeks.

"As agents become capable of continual learning, their memory will quickly become a more valuable component of the agent than any individual underlying model."

While general-purpose models are replaceable and rapidly outdated, an agent's memory is costly to lose and compounds in value over time.

Continual Learning Through In-Context Learning (ICL)

Token-space learning grows more powerful as ICL capabilities improve. Models are a depreciating asset, but learning represented in token space is an appreciating asset: as models improve, memories become more effectively leveraged. The post highlights the bizarre power of ICL – even models trained only on pre-1930 data can learn Python (created in 1991) through ICL. Recent research suggests transformers actually learn to approximate gradient descent during inference.

"We are only just beginning to understand how to optimize adaptation of models through token-space learning."

Each new generation of frontier models uses context more effectively than the last – this is called the bitter lesson of ICL.

---

3. Learning in Token Space

Despite the advantages of token-space learning, existing techniques for creating these representations are rudimentary.

Limitations of General-Purpose Models Today

Current frontier models are not built for continual learning. Most data is task-scoped; even highly adaptable models hit limitations over time (shown in Recovery-Bench). Letta's research highlights:

• Memories become generic and lossy after repeated refinements.
• Memories are overly specific rather than generalizable.
• Memories fail to adapt behavior consistently across sessions or across different models.

These limitations exist because current paradigms optimize for task completion, not for cross-task, cross-session, or cross-model learning.

Optimizing for the Present vs. the Future

There's a fundamental tension between doing the task and learning across tasks. LLMs today are trained in stateless RL environments focused on task completion. Techniques like long context windows (1M–10M tokens), self-compaction, context folding, and Recursive Language Models help extend task horizons but are not mechanisms for true continual learning.

Meta-Learning: Learning to Learn

This tension has been formalized as meta-learning (roots in the 1980s/90s). In meta-learning, the objective is the act of learning itself – training a machine to learn its own learning algorithm to adapt to new environments at test time. Meta-RL trains agents in a nested loop where the inner loop adapts to tasks and the outer loop optimizes the learning process itself, rather than pure task reward.

---

4. Memory Models: Learning to Learn in Token-Space

The post proposes that continual learning will be solved by memory models – models specialized in generating and curating context. These are optimized end-to-end through meta-RL to create token-space memories that enable agents to learn across tasks.

Standard RL vs. Memory-Native RL

In standard RL, an agent is trained to maximize task reward directly (weight-space learning). In memory-native RL, the agent instead produces token-space memories that, when conditioned on, maximize reward on future tasks – a form of meta-learning in token space.

The post illustrates this with a diagram:

• Standard RL: task → agent harness (model params) → experiences → outcome + reward → weight-space learning.
• Memory-native RL: task → memory-native harness conditioned on token-space memory and memory model params → a sequence of experiences that update memory across open-ended tasks → outcome + reward → both weight-space and token-space learning.

"Memory-native RL trains an agent to produce token-space memories that, when conditioned on, maximize reward on future tasks — meta-learning in token space."

Distillation of Memory into the Agent Harness

The harness itself is a learning surface. Memory models can directly modify harness configuration (toolsets, hooks, permissions, even executing arbitrary code at control points). Letta's harnesses (e.g., Letta Code, Pi) support self-modification via mods and extensions. Recent work on "meta-harnesses" shows this can unlock significant performance gains.

Distillation of Memory into Model Weights

Token-space memory can be easily integrated with closed APIs where model weights are unavailable. But if weights are accessible, token-space memories can be distilled into model weights, combining continual learning in both token space and weight space for additional gains in personalization and efficiency.

---

5. Scaling Compute on Context

The post ties to their previous work on sleep-time compute – allowing models to think offline about contexts before queries. Memory models are trained to generate memories during sleep-time (agent dreaming) that improve test-time performance. The more powerful the memory models become, the greater the gains from scaling compute on context.

"Eventually, we expect the vast majority of the world's inference load will be driven by sleep-time compute."

---

6. Building Machines That Learn

Letta's mission is to build agents capable of meaningful learning and adaptation over infinite time horizons, sharing context across model generations and architectures. They believe scaling inference for context learning is the next major scaling direction, and memory models are the key to performant, economical continual learning at scale.

"The only way to unlock true recursive self-improvement is to make our agents capable of remembering and learning through persistent representations that live outside any model's weights."

The post ends with an invitation for researchers and engineers interested in agent memory and lifelong learning to join them.

---

Conclusion

This post makes a compelling case that the future of capable AI agents lies not in ever-larger models, but in systems that can learn continuously by managing token-space memory. Memory models – trained via meta-RL to generate durable, transferable contexts – are presented as the missing piece. By scaling compute on context (especially offline during sleep-time), agents can compound their expertise over time, making memory an asset that grows more valuable with each model generation. The vision is ambitious, but the technical arguments and real-world examples (Letta Code, skill learning, Recovery-Bench benchmarks) ground it in concrete progress.