AI Agent Memory Management: It’s Not Just About the Context Limit
Emerging Memory Architectures for AI Agents
Large Language Models (LLMs) based agents excel at diverse tasks, yet they suffer from brittle procedural memory that is manually engineered or entangled in static parameters. In this work, we investigate strategies to endow agents with a learnable, updatable, and lifelong procedural memory. We propose Memp that distills past agent trajectories into both fine-grained, step-by-step instructions and higher-level, script-like abstractions, and explore the impact of different strategies for Build, Retrieval, and Update of procedural memory. Coupled with a dynamic regimen that continuously updates, corrects, and deprecates its contents, this repository evolves in lockstep with new experience. Empirical evaluation on TravelPlanner and ALFWorld shows that as the memory repository is refined, agents achieve steadily higher success rates and greater efficiency on analogous tasks. Moreover, procedural memory built from a stronger model retains its value: migrating the procedural memory to a weaker model yields substantial performance gains.
Procedural memory exhibits transferability from strong models to weaker ones.
For a procedural memory constructed from a strong model in an offline memory library, we aim to verify whether this form of procedural memory can be effectively transferred to other models, or even weaker models. This exploration underscores the significance of memory transfer, as it could potentially enhance the adaptability and efficiency of various models by leveraging the knowledge and experience encapsulated within the strong model’s memory structure. As shown in Figure 4 (b), procedural memory generated by GPT-4o was employed by Qwen2.5–14B. On the Travel Plan benchmark, the 14 billion parameter model raised its task completion rate by 5% and cut the average number of steps by 1.6. Similar gains, both in success rate and trajectory length, appeared on ALFWorld. These outcomes confirm that procedural knowledge from a stronger model can be distilled into a reusable memory bank and transferred to a smaller model with minimal overhead, giving that smaller model a clear boost in task solving ability. Moreover, by leveraging procedural memory transfer, we can rapidly migrate the experiential knowledge that one model has acquired to another, which is highly beneficial for agents as they adapt to new tasks with greater efficiency and robustness.
Scaling Memory Retrieval Improves Agent Performance.
While our main experiment has already demonstrated that procedural memory improves an agent’s task accuracy and reduces the number of steps required, vector-based storage and retrieval confer an advantage over human procedural memory: they can be scaled both in total capacity and in the number of memories retrieved. To investigate whether an agent’s performance continues to rise as the procedural-memory store and the number of retrieved memories increase, we designed a set of follow-up experiments. As showned in Figure 4 (b), as the number of retrieved procedural memories increases, the agent’s performance also improves steadily, exhibiting an upward trend followed by a plateau. However, retrieving too many memories can lead to a decline in the agent’s performance. This is because excessive retrieval can affect the context length and also introduce less accurate procedural memories, which can interfere with the overall effectiveness.
—
Conclusion and Future Work
We introduce Memp, a task-agnostic framework that elevates procedural memory to a core optimization target in LLM-based agents. By systematically studying strategies for memory construction, retrieval, and updating, Memp enables agents to distill, reuse, and refine their own past experiences across diverse, long-horizon tasks. Empirical results on housework automation and information-seeking benchmarks show that leveraging procedural memory significantly boosts task success rates and efficiency. Beyond improving individual episodes, Memp supports continual learning and robust generalization, marking a step toward self-improving, resilient agents.
In our experiments, Memp has achieved promising results in both construction and retrieval. Moving forward, we plan to enhance this work in several ways. Firstly, we will develop more diverse retrieval strategies. The current approach involves constructing different keys for vector-based retrieval. However, traditional methods like BM25 could also be explored to retrieve precise memories more effectively. Secondly, in Memp, we currently rely on the standard reward signals provided by the benchmark. However, in real-world scenarios, many tasks do not have clear reward signals, making it difficult for the agent to determine whether a task has been completed successfully. In such cases, using a large language model (LLM) as a judge to assess task completion could be a viable solution. This would transform the agent’s lifecycle into a continuous loop of executing tasks, self-assessing completion, building memories, and then proceeding to new tasks.
Large Language Models (LLMs) have demonstrated remarkable prowess in generating contextually coherent responses, yet their fixed context windows pose fundamental challenges for maintaining consistency over prolonged multi-session dialogues. We introduce Mem0, a scalable memory-centric architecture that addresses this issue by dynamically extracting, consolidating, and retrieving salient information from ongoing conversations. Building on this foundation, we further propose an enhanced variant that leverages graph-based memory representations to capture complex relational structures among conversational elements. Through comprehensive evaluations on the LOCOMO benchmark, we systematically compare our approaches against six baseline categories: (i) established memory-augmented systems, (ii) retrieval-augmented generation (RAG) with varying chunk sizes and k-values, (iii) a full-context approach that processes the entire conversation history, (iv) an open-source memory solution, (v) a proprietary model system, and (vi) a dedicated memory management platform. Empirical results demonstrate that our methods consistently outperform all existing memory systems across four question categories: single-hop, temporal, multi-hop, and open-domain. Notably, Mem0 achieves 26% relative improvements in the LLM-as-a-Judge metric over OpenAI, while Mem0 with graph memory achieves around 2% higher overall score than the base Mem0 configuration. Beyond accuracy gains, we also markedly reduce computational overhead compared to the full-context approach. In particular, Mem0 attains a 91% lower p95 latency and saves more than 90% token cost, thereby offering a compelling balance between advanced reasoning capabilities and practical deployment constraints. Our findings highlight the critical role of structured, persistent memory mechanisms for long-term conversational coherence, paving the way for more reliable and efficient LLM-driven AI agents.
Mem0’s pipeline consists of two phases — Extraction and Update — ensuring only the most relevant facts are stored and retrieved, minimizing tokens and latency .
In the Extraction Phase, the system ingests three context sources — the latest exchange, a rolling summary, and the m most recent messages — and uses an LLM to extract a concise set of candidate memories. A background module refreshes the long-term summary asynchronously, so inference never stalls .
In the Update Phase, each new fact is compared to the top s similar entries in the vector database. The LLM then chooses one of four operations:
ADD new memories
UPDATE existing entries
DELETE contradictions
NOOP if no change is needed
These steps keep the memory store coherent, non-redundant, and instantly ready for the next query .
Mem0ᵍ enhances Mem0 by storing memories as a directed, labeled graph. In the Extraction Phase, incoming messages feed into an Entity Extractor to identify entities as nodes and a Relations Generator to infer labeled edges — transforming text into a structured graph .
During the Update Phase, a Conflict Detector flags overlapping or contradictory nodes/edges, and an LLM-powered Update Resolver decides whether to add, merge, invalidate, or skip graph elements. The resulting knowledge graph enables efficient subgraph retrieval and semantic triplet matching for complex multi-hop, temporal, and open-domain reasoning .
Multi-agent systems built on Large Language Models (LLMs) show exceptional promise for complex collaborative problem-solving, yet they face fundamental challenges stemming from context window limitations that impair memory consistency, role adherence, and procedural integrity. This paper introduces Intrinsic Memory Agents, a novel framework that addresses these limitations through structured agent-specific memories that evolve intrinsically with agent outputs. Specifically, our method maintains role-aligned memory templates that preserve specialized perspectives while focusing on task-relevant information. We benchmark our approach on the PDDL dataset, comparing its performance to existing state-of-the-art multi-agentic memory approaches and showing an improvement of 38.6% with the highest token efficiency. An additional evaluation is performed on a complex data pipeline design task, we demonstrate that our approach produces higher quality designs when comparing 5 metrics: scalability, reliability, usability, cost-effectiveness and documentation with additional qualitative evidence of the improvements. Our findings suggest that addressing memory limitations through structured, intrinsic approaches can improve the capabilities of multi-agent LLM systems on structured planning tasks.
Conclusion
This paper introduces Intrinsic Memory Agents, a novel multi-agent LLM framework that constructs agent-specific heterogeneous structured memories to enhance multi-agent collaboration in discussion and planning tasks. Evaluation on the PDDL dataset, and on a practical data pipeline design problem demonstrates our framework’s improved performance on structured planning tasks, with a 38% increase over the next best memory architecture, and maintaining the best token efficiency despite the increased token usage.
Results on the data pipeline case study further show the Intrinsic Memory Agents’ enhanced ability to collaborate on complex tasks. The Intrinsic Memory Agent system outperforms the baseline system across all quality measures of scalability, reliability, usability, cost-effectiveness, and documentation, as well as an ability to more closely follow the task specification, providing more actionable recommendations by suggesting specific tools and frameworks, as well as trade-off details of each component in the pipeline.
Large Language Models (LLMs) demonstrate remarkable capabilities, yet their inability to maintain persistent memory in long contexts limits their effectiveness as autonomous agents in long-term interactions. While existing memory systems have made progress, their reliance on arbitrary granularity for defining the basic memory unit and passive, rule-based mechanisms for knowledge extraction limits their capacity for genuine learning and evolution. To address these foundational limitations, we present Nemori, a novel self-organizing memory architecture inspired by human cognitive principles. Nemori’s core innovation is twofold: First, its Two-Step Alignment Principle, inspired by Event Segmentation Theory, provides a principled, top-down method for autonomously organizing the raw conversational stream into semantically coherent episodes, solving the critical issue of memory granularity. Second, its Predict-Calibrate Principle, inspired by the Free-energy Principle, enables the agent to proactively learn from prediction gaps, moving beyond pre-defined heuristics to achieve adaptive knowledge evolution. This offers a viable path toward handling the long-term, dynamic workflows of autonomous agents. Extensive experiments on the LoCoMo and LongMemEvalS benchmarks demonstrate that Nemori significantly outperforms prior state-of-the-art systems, with its advantage being particularly pronounced in longer contexts.
Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of NLP tasks, but they remain fundamentally stateless, constrained by limited context windows that hinder long-horizon reasoning. Recent efforts to address this limitation often augment LLMs with an external memory bank, yet most existing pipelines are static and heuristic-driven, lacking any learned mechanism for deciding what to store, update, or retrieve. We present Memory-R1, a reinforcement learning (RL) framework that equips LLMs with the ability to actively manage and utilize external memory through two specialized agents: a Memory Manager that learns to perform structured memory operations, including adding, updating, deleting, or taking no operation on memory entries; and an Answer Agent that selects the most relevant entries and reasons over them to produce an answer. Both agents are fine-tuned with outcome-driven RL (PPO and GRPO), enabling adaptive memory management and utilization with minimal supervision. With as few as 152 question-answer pairs and a corresponding temporal memory bank for training, Memory-R1 outperforms the strongest existing baseline and demonstrates strong generalization across diverse question types and LLM backbones. Beyond presenting an effective approach, this work provides insights into how RL can unlock more agentic, memory-aware behavior in LLMs, pointing toward richer, more persistent reasoning systems.
Conclusion
We introduced Memory-R1, a reinforcement learning (RL) framework that enables large language model agents to manage and utilize external memory effectively. By training two specialized agents, one for memory operations and another for selective reasoning over retrieved memories, our approach overcomes the limitations of static, heuristic-driven memory pipelines. On the LOCOMO benchmark, Memory-R1 outperforms strong baselines across F1, BLEU, and LLM-as-a-Judge scores, achieving these gains with as few as 152 training examples. Ablation studies further confirm that RL enhances every component of the system. Memory-R1 opens new directions for research on compositional memory architectures, long-term knowledge retention, and more agentic LLM behavior, underscoring RL as a path to adaptive memory capabilities.
