AI-103 Lab: Agent Memory and State, Short and Long Term

11. Agent Memory and State is a 17-minute hands-on lab from the free AI-103 Developing AI Apps and Agents on Azure course, in which you design memory for an AI agent so it remembers what matters across sessions without cluttering every conversation. It is for developers preparing for AI-103 and anyone whose agent forgets the user’s name and preferences the moment a session ends.

The lesson describes that problem as a goldfish: the user explains their role and project, the session ends, and tomorrow everything starts from zero. You fix it by separating the conversation from what should outlive it. The lab sits in the agent section of the AI-103 course page.

What this lab covers

The lab has two exercises, each with Python scripts and printed results.

Exercise 1: short-term versus long-term memory.

• What a thread keeps. Short-term memory is the current conversation: its messages and current work. It lives in the thread and ends with it. You run the same agent on two threads, one for a Monday session and one for a Friday session. Monday’s thread holds its four messages; Friday’s starts empty and holds only two. The thread is the boundary, and nothing carries over unless you store it somewhere that outlives the conversation.
• A long-term store. Long-term memory is what should still be true next month, such as a user’s preference for a unit or currency. The lab builds a namespaced key-value store with put, get, search and delete. Only statements that look worth keeping (marked by words such as “prefer” or “always”) are written; other chatter is ignored. In a new session the agent recalls the fact for that user, while another user’s search returns nothing.
• The security point. The lesson stresses that the namespace is the security boundary. If it is not scoped per user, one user’s preference can leak to another.
• The option people forget. Much of what looks like memory is data you already hold in your own systems.

Exercise 2: personalise without blowing the context window.

• Personalisation. The same question is answered differently for two users, using a handful of carefully chosen facts rather than everything you know about them. Stuffing the whole profile into every prompt wastes tokens and can make users feel watched rather than understood.
• When state gets too big. A script builds a 24-message conversation, calculates its token count and cost, and compares strategies in a table, including trimming against summarisation, so you see what each approach loses.

The lab ends with guidance on choosing a memory service based on what must survive.

What you will do

These are the steps the instructor takes in the video, in order, with the moment each one starts. Watch the video for the full walkthrough.

1. Let’s start by looking at the agent’s current memory problem. (0:27)
2. You will choose where each piece of state lies, write a fact once and recall it in a new session, and personalize responses without blowing the context window. (0:53)
3. Let’s start by opening step one and exploring what the thread remembers. (2:08)
4. We’ll create two threads, one for Monday’s session and one for Friday’s. (3:23)
5. We’ll run the same agent on both, and you’ll see that Monday’s conversation stays in Monday’s thread, while Friday’s thread starts completely empty. (3:28)
6. We import our agent and thread classes from the shared agent kit. (3:36)
7. We create a support agent with a simple system prompt. (3:39)
8. Then we create a brand new thread called session Friday, run a single message, and print its message count, which will be just two, the user message and the assistant reply. (3:52)
9. We’ll run a script that simulates a Monday conversation, then a Friday recall. (6:33)
10. We’ll start by taking the same question and giving two different users a personalized answer. Then we’ll look at what happens when state gets too big and compare trimming against summarization. (8:49)
11. We start by importing helpers for token counting and output formatting. (10:48)
12. We’ll run a script that builds a 24 message conversation, calculates its token count and cost, then shows a table comparing three strategies. (14:02)
13. Choose your memory service based on what must survive. (16:54)

Where it fits in the AI-103 course

This is module 11 of 23, in the middle of the agent section. Just before it, 10. Agent Tools and Function Calling makes tool calls reliable by shaping their schemas; this lab makes sure those calls and everything else the user says build up useful context. Before that, 09. Foundry Agent Service introduces agents themselves.

Next, 12. Multi-Agent Solutions coordinates several agents, and 13. Autonomy with Safeguards sets limits on what an autonomous agent may do. Clear memory boundaries matter even more once several agents share work.

How to study this lab

1. Watch the first two minutes to understand the “goldfish” problem and the plan.
2. Run the two-thread script. Confirm the message counts yourself and explain why Friday’s thread is empty.
3. Build the long-term store. Add your own marker word and check which lines get stored and which are ignored.
4. Test the namespace. Recall for a second user and confirm it returns nothing. Then imagine the bug if the namespace were shared.
5. Compare trimming and summarisation. Note what each one loses for the 24-message conversation.
6. Write a one-line rule for each piece of state in your own agent: thread, long-term store, or existing system data.
7. Check yourself. The course’s mock exam has an Agents area; the 10-question sample is free, and the full mock comes with a subscription.

Frequently asked questions

What is the difference between short-term and long-term agent memory?

In the lesson, short-term memory is the current conversation held in the thread, and it ends with the session. Long-term memory is information that should still be true later, such as a preference, kept in a separate store and recalled in new sessions.

How do I stop one user’s memory leaking to another?

The lesson scopes the long-term store by a namespace per user and calls that namespace the security boundary. A search under one user’s namespace returns nothing for anyone else.

What do I do when the conversation gets too long?

The lab compares strategies such as trimming and summarisation on a 24-message conversation, with token counts and cost, so you can see the trade-off before choosing.

Is this lab free?

Yes. It is part of the free AI-103 course on YouTube; follow all modules from the AI-103 course page.