What is AI agent memory? How long-term memory for LLM agents works

Last updated October 31, 2026

A language model remembers nothing between calls: everything it knows about your task has to be in its context window on that turn. AI agent memory is what fills that gap. The agent writes what it learns to a store outside the context window, and reads back the parts that matter on later turns or in later sessions. Long-term memory can be plain files, a vector database searched by meaning, a knowledge graph, or a memory service the agent reaches through tools. The hard parts are deciding what to save, finding the right piece later, and keeping old facts from crowding out new ones.

Short-term and long-term memory

• Short-term memory is the context window: the instructions, the conversation so far and everything the agent has read on this task. It's fast and exact, but it's limited, it costs tokens on every turn, and it's gone when the session ends.
• Long-term memory lives outside the model. The agent, or a process around it, writes to it, and the agent pulls pieces back into its context when it needs them.

Anthropic's engineering guide on context calls this technique "structured note-taking, or agentic memory": the agent "regularly writes notes persisted to memory outside of the context window," and those notes "get pulled back into the context window at later times." The same guide treats context as "a finite resource with diminishing marginal returns," which is why an agent can't simply keep everything loaded.

Four ways to build it

These mix. Many memory services use vector search and a graph inside, and many agents use a short instruction file plus a memory service.

How memories get written

There are two broad approaches:

• Automatic capture. Hooks or a background process record what the agent did, and a model extracts facts or summaries from it. Nothing is missed, but the store fills with steps as well as decisions, and the extraction can be wrong without anyone noticing.
• Explicit saves. The agent calls a tool to save a memory when it learns something worth keeping, such as a decision and the reason for it. Memories stay short and readable, but the agent has to be told what's worth saving, and it can miss things.

How memories get read

Most agents read long-term memory in two moments:

1. At the start of a session, a short summary or index loads: what this memory is for and where the key notes are.
2. Before a step, the agent searches for what that step touches and reads only the best matches.

Loading everything up front is the most common mistake. It costs tokens on every turn and buries the note that matters.

What goes wrong

• Stale facts. A decision changes, the old note stays, and the agent follows it. Memory needs a way to replace or retire old notes, not only add new ones.
• Near copies. The same fact is saved five times with small differences. Check for an existing note before saving.
• No record of who changed what. When a note is wrong, you need to know which agent or person wrote it and what it said before.
• Silent overwrites. Two agents change the same note at once and one change is lost.
• Secrets. Passwords and API keys end up in memory. Save where a secret lives, never the secret.

Memory for coding agents

Coding agents such as Claude Code and Codex start each session with their instruction files and no memory of earlier sessions. Most teams start with CLAUDE.md or AGENTS.md, and add a memory server when the file gets too long or when several agents need the same knowledge. See Why your coding agent forgets between sessions and Keep CLAUDE.md short.

A worked example: Memlo

Memlo is a memory server for the agents you use. It takes the explicit-save approach:

• Agents connect over MCP at https://memlo.ai/mcp and sign in with your account; there's no API key.
• They save memories with create_memory, or with smart_add_memory, which first checks for notes that already say the same thing.
• They find memories with search_memories and read one with read_memory. get_prime_directive loads a short start-here page at the start of a session.
• Every change is a new version. memory_history lists the versions with when each was written and by whom, and restore_memory brings one back. edit_memory refuses a write when someone else changed the note first.
• One memory works across Claude Code, Codex, claude.ai, ChatGPT and other MCP apps, and you can share a space with people.

It doesn't capture sessions automatically, and it has no SDK for building it into your own app. For that, see the frameworks compared in Mem0, Letta, Zep and Cognee.

Frequently asked questions

Do LLMs have long-term memory?

Not on their own. A model only sees what is in its context window on each call. Long-term memory is a store outside the model that the agent, or the app around it, reads from and writes to.

What is the difference between RAG and agent memory?

RAG usually means searching a fixed set of documents to answer a question. Agent memory is written by the agent as it works, and changes over time, so it also needs ways to update, replace and retire what it holds.

Should an agent remember everything?

No. Saving every step makes search worse and costs more. Save decisions and their reasons, how things work, preferences and mistakes not to repeat. Don't save numbers that change every day, or secrets.

How do several agents share one memory?

Connect them all to the same memory service and the same account or shared space. See How to share one memory between AI agents.

Sources

Checked on October 31, 2026:

• Effective context engineering for AI agents, Anthropic
• Claude Code memory and CLAUDE.md
• Memlo: the setup page and its list of tools