ReadPaper Blog
AUTOMEM: Automated Learning of Memory as a Cognitive Skill
AUTOMEM argues that memory management in LLM agents should be treated as a trainable cognitive skill rather than a fixed storage component. The paper addresses long-horizon tasks that exceed context windows by giving agents file-system memory actions and then automatically improving both the memory scaffold and the model’s proficiency at using it. Its results on Crafter, MiniHack, and NetHack suggest that optimizing memory alone can produce large performance gains and narrow the gap between open-weight and frontier systems.
Source: AUTOMEM: Automated Learning of Memory as a Cognitive Skill
Memory Is a Skill, Not Just a Box
AUTOMEM frames long-horizon agent failure as a problem of metamemory: the learned ability to decide what to encode, when to retrieve it, and how to organize knowledge for later use. The paper argues that an LLM’s context window functions like bounded working memory, which becomes inadequate when tasks run for thousands or even tens of thousands of steps. Rather than treating external memory as a fixed retrieval database, scratchpad, or summary buffer, the authors promote file-system operations such as read, write, search, append, and create to first-class actions in the agent’s action space. This design lets the same model that acts in the environment also decide how to manage files that store maps, inventories, encounter records, rules, strategies, and action histories. The key implication is that memory is made observable and optimizable: every storage or retrieval choice becomes part of the trajectory that can later be reviewed and improved.
AUTOMEM Splits Memory Into Two Targets
The paper decomposes memory skill into two targets: structure and proficiency. Structure refers to the prompts, file schemas, validation logic, and action vocabulary that shape how the agent can use external memory, while proficiency refers to the model’s learned ability to choose useful memory actions within that structure. AUTOMEM automates both targets through two sequential outer loops around a shared inner-loop agent that uses a file system as memory. The inner-loop agent alternates between a LOG routine, which decides what recent information is worth recording, and a PLAN routine, which decides what to search or read before committing the next world action. This separation lets the framework improve memory support through scaffold revision first and then improve the model’s own memory decisions through training. The paper’s broader claim is that long-horizon performance can be improved by targeting memory as an independent skill rather than by modifying the agent’s task-action behavior directly.
Loop 1: A Meta-LLM Rewrites the Scaffold
AUTOMEM’s first outer loop uses a strong meta-LLM to inspect complete episode trajectories and revise the scaffold that governs memory use. The motivation is practical: in environments where an episode may span 10^4 to 10^5 steps, a single poor memory decision may only reveal its cost much later, making human review of full traces unrealistic. The meta-LLM reviews these long trajectories, diagnoses failure patterns in the agent’s file use, and iteratively changes code, prompts, memory file schemas, and related scaffolding. Because memory actions are explicit file-system operations, the reviewer can identify whether the agent failed to record an important event, searched the wrong file, organized records poorly, or relied on stale information. The paper presents this loop as scaffold optimization: it pushes performance as far as structural revisions can take it before changing any model weights. A notable implication is that better memory structure can also improve task behavior indirectly, because clearer records reduce redundant exploration and unfocused action.
Loop 2: Train Memory Proficiency
AUTOMEM’s second outer loop trains memory proficiency by selecting useful memory-operation traces from the agent’s own experience and turning them into supervised finetuning data. A meta-LLM acts as a training engine that sets data standards and composition, filters good memory decisions from a larger trace pool, and chooses LoRA finetuning configurations. The paper deliberately finetunes only a dedicated memory specialist, while the task model that commits world actions remains frozen. This design isolates memory learning from task-action competence, reducing the risk that improving file use will degrade the base model’s ability to act in the environment. The training target is not a generic instruction-following behavior but the narrower skill of deciding when to append, create, search, read, or rewrite memory files in long-horizon episodes. By learning from the agent’s own successful memory decisions, AUTOMEM treats memory management as a behavioral policy that can be sharpened with experience.
Big Gains on Hard, Stochastic Worlds
The paper evaluates AUTOMEM on three procedurally generated long-horizon game environments: Crafter, MiniHack, and NetHack. These settings are chosen because episodes are stochastic, pretraining knowledge is less likely to solve the task, and success depends on maintaining external records such as maps, inventories, encounter logs, and strategy notes. Using Qwen2.5-32B-Instruct as the base model, the authors report that optimizing memory alone improves progression by roughly 2x to 4x over the base agent without modifying task-action weights. The optimized 32B agent outperforms Qwen2.5-72B-Instruct across the three games, which the paper interprets as evidence that well-managed memory can be more valuable than simply increasing model scale for these tasks. The same optimized agent reaches performance competitive with frontier proprietary systems including Claude Opus 4.5 and Gemini 3.1 Pro Thinking. The central implication is that memory management is a high-leverage objective for long-horizon LLM agents and can substantially close performance gaps without retraining the whole agent for task actions.
