Dashboard

How AI Agent Memory Actually Works

The model is stateless between requests, so agent memory is a store your application owns plus a convention about when to read and write it. That changes how you debug it.

Carlo Zuercher
Carlo Zuercher
Staff Engineer, Platform
21 October 20261 min read

When a product says an AI agent "remembers" you, it almost never means the model retained anything. The model is stateless between requests. What changed is that somebody built a store, wrote facts into it, and arranged for those facts to be read back into the next request. Agent memory is a file your application owns, plus a convention about when to read and write it. Knowing that one fact changes how you debug it, how you price it, and what you promise a client about it.

The three mechanisms people call memory

Three separate things get the same label, and they fail in different ways.

The context window. Everything in the current request: system instructions, conversation so far, attached files, tool results. It is working memory and it is gone when the request ends. Its only limit is size, which we covered in what a context window is.

Summarisation of the current session. When a long conversation approaches the window limit, something has to give. Server-side compaction summarises the older turns so the session can continue. This keeps a conversation alive, but it is lossy by construction, and the thing it drops is usually detail rather than topic.

A persistent store. Files or rows that outlive the session, written during one conversation and read in the next. This is the only one of the three that is memory in the sense a user means.

What a real implementation looks like

Anthropic's memory tool is a useful specimen because the documentation is explicit about where the data lives. The tool "operates client-side: Claude requests file operations, and your application executes them." The model issues commands such as view, create, str_replace, insert, delete and rename against paths under /memories, and your handler maps that prefix onto real storage, "such as a per-user directory or keys in a database."

So the architecture is:

  1. The model asks to read its memory directory before starting work.

  2. Your code answers with a directory listing.

  3. The model reads the files it thinks are relevant.

  4. The model works, and records what it learned by asking your code to write files.

  5. The next session repeats step one against the same store.

Nothing persists unless your handler persists it. The documentation says it directly: "Memory lives entirely in your application. A later conversation continues from the same memory when it sends the same tools entry and your handler serves the same store."

That is the line to read twice if you are deciding whether to build this or buy it. The hard part was never the model side.

A reasonable instinct is to store memories as embeddings and retrieve the closest matches. Plenty of systems do. The file-based approach wins on a different axis: an agent that writes project-decisions.md and reads it back gets the whole document, in order, with its own structure intact. Retrieval by similarity returns fragments, and fragments lose the reasoning that connected them.

The pattern Anthropic's docs call just-in-time retrieval sits between the two. Rather than loading everything up front, the agent checks a small index, then reads only the files it needs. The context stays focused on the current task, which is the actual constraint in long-running work.

The failure modes, in the order you will meet them

The agent writes nothing useful. Memory files fill with restatements of the task instead of conclusions. The fix is instruction, not infrastructure: tell it what is worth recording. The documentation suggests scoping it explicitly, for example "Only write down information relevant to one named topic in your memory system."

Memory files grow until they crowd out the task. Cap file size, and cap how many characters a read returns so the agent pages through long files instead of swallowing them whole.

Stale facts outlive their truth. A memory that says a client prefers email follow-ups is correct until it is not, and nothing in the mechanism expires it. Anthropic's own guidance is to "periodically delete memory files that haven't been accessed in a long time." Treat memory as a cache with a policy, not an archive.

Path traversal. This one is a security bug, not an annoyance. The docs carry an explicit warning: a path such as /memories/../../secrets.env "can reach files outside the /memories directory," so every command's path must be validated. Resolve to canonical form, confirm the result is still inside the memory root, and reject traversal sequences including URL-encoded ones such as %2e%2e%2f. An agent reading its own memory is expected behaviour. An agent reading your environment file because a filename came back from a model is an incident.

Secrets written into memory. Models usually decline to store credentials, and usually is not a control. Strip sensitive data in the handler before the write, rather than hoping. The same reasoning applies to environment variables and secrets in an AI-built app.

A pattern that actually holds up across sessions

For work that spans days, ad hoc note-taking does not survive. The structure that does:

  • An initialising session that creates the memory files before any real work: a progress log, a scope checklist, and a pointer to whatever script starts the project.

  • Each later session opens by reading those files, which restores state without re-exploring the codebase.

  • Each session closes by updating the progress log with what finished and what is next.

One rule makes or breaks it: mark something complete only after end-to-end verification, not when the code was written. A progress log that records intentions rather than outcomes is worse than no log, because the next session trusts it.

This is the same discipline that makes an AGENTS.md file work. Durable instructions live in a file the agent is guaranteed to read, not in a conversation it will not see again.

What to tell a client

Three sentences, and they are all defensible. The agent does not learn from your data, it reads notes it wrote earlier. Those notes live in storage you control, and you can read, edit or delete them. If the notes are wrong, the agent will be confidently wrong in the same direction until somebody fixes the file.

That last sentence is the one people need. Memory does not make an agent more correct. It makes it more consistent, which is valuable and is not the same thing. If you are still deciding what you are building, agent, assistant or copilot is the more basic question, and how AI models work covers the statelessness this whole mechanism exists to work around.

How did this land?

About the author

Carlo Zuercher
Carlo Zuercher

Staff Engineer, Platform

Carlo works on the platform that turns prompts into running apps. He writes the engineering deep dives and the changelog notes worth reading.

Share

Get the next post in your inbox

One email a month. Product updates, engineering posts, and the best of Built with Swarmz.

I agree to receive emails about AI building tips and Swarmz product news. Unsubscribe any time.