Memory in AI Agents

Why an agent needs memory

A language model is stateless — it remembers nothing between calls. Everything it "remembers" is in the context window of that call. That's a problem: a support bot that doesn't remember what you said 3 messages ago, or a personal assistant that forgets your name every time — aren't useful.

Agent memory is the mechanism that lets it store and retrieve information over time — within a conversation and across conversations — to give a continuous, personal experience.

Types of memory

It's common to distinguish several types, inspired by human memory:

• Working / short-term: the current context — the active conversation history. Lives in the context window.
• Episodic: "what happened" in previous interactions — past conversations, actions taken.
• Semantic: stable facts and preferences about the user/world ("the customer prefers email," "the company is in finance").
• Procedural: "how to do" — procedures and patterns the agent learned.

In practice you mainly implement two: short-term (the context) and long-term (an external store retrieved as needed).

Short-term memory — managing the conversation

This is the active context. The challenge: a long conversation swells and hits the context-window limit (and context rot). Strategies:

• Rolling window: keep only the last N messages in full.
• Rolling summary: summarize the old messages into a paragraph, and keep only the latest in full. Preserves context without bloating.
• Fact extraction: during the conversation, extract important facts ("name: Dana," "issue: double charge") and store them separately — they also move to long-term memory.

Long-term memory — an external store

Information that needs to persist across conversations (preferences, history, facts) is stored outside the context — usually in a Vector DB or a regular DB — and retrieved only when relevant. This is exactly like RAG, but over the user's memory:

1. Writing: after an interaction, store facts/a summary as embeddings in the store, with a user id.
2. Retrieval: at the start of a new conversation, retrieve the most relevant memories for the current question and put them in the context.
3. Updating: if a fact changed ("I moved to another company") — update/replace, don't pile up contradictions.

The rule: don't retrieve all the memory on every call — only the relevant. Otherwise the context bloats and behavior suffers.

Common mistakes

• Retrieving everything. Streaming the user's whole history into the context bloats it and hurts quality. Retrieve only what's relevant.
• Contradictory memory. Storing old and new facts without updating → the agent gets confused. Manage updates.
• Storing noise. Not every message is worth remembering. Extract only the meaningful.
• Privacy. Memory about users = personal data. Secure it, encrypt it, and allow deletion (GDPR / privacy law).
• No user separation. One user's memory leaking to another is a serious failure. Always filter by id.