AI-103 RAG Lab: Retrieval-Augmented Generation Explained

07. Retrieval-Augmented Generation is an 18-minute hands-on lesson from the free AI-103 Developing AI Apps and Agents on Azure course, in which you build a complete retrieval-augmented generation (RAG) loop in Python. It is for developers preparing for AI-103 and anyone who has seen a model confidently invent an answer and wants a reliable way to stop it.

The lesson’s starting point is simple: the fix for invented answers is not a better prompt, it is giving the model the right text at the right moment. By the end you have scripts that retrieve, ground and cite, and you have seen how one setting, the chunk size, changes the answers. The lesson is part of the course on the AI-103 course page.

What this lesson covers

The lesson has three hands-on parts.

1. The RAG loop, with and without grounding. You ask a question about a company’s own documents twice. Without retrieval, the model either invents an answer or, with a careful system prompt, refuses because it has no context. With retrieval, a hybrid search finds the most relevant passages, they are assembled into a context string tagged with document IDs, and a second assistant answers from that context and cites its sources. The lesson shows that three instructions in the system prompt make this work: answer only from the context, say you do not know if the answer is missing, and cite the document ID. It also summarises when to use RAG and when to use fine-tuning: RAG for changing private facts, fine-tuning for style or behaviour.

2. Chunking experiments. The lesson calls chunking the decision that most affects RAG quality. The same technical manual is split three ways, at 80, 220 and 600 characters with overlap, and the same query is run against each. Chunks that are too small cut a fact in half; chunks that are too large bury the right sentence in unrelated text and lower its relevance. For that query, the middle size kept the fact and its context together.

3. Citations and faithfulness. You build an answer that returns its sources alongside the text, with a table of document IDs, titles and sections. The lesson lists what a citation must carry (a stable ID, a readable title, a locator and the retrieved passage) and warns about the main failure mode: a model citing a document that does not support the claim. Showing the passage is what catches that. The lesson then looks at a claim with no supporting passage, and introduces groundedness as the most important RAG metric.

Key points from the lesson

• By the end of this module, you’ll be able to build a complete retrieval augmented generation loop and see firsthand how chunking decisions change the answers your system produces. (0:00)
• Retrieval augmented generation is only as good as the chunks you retrieve. (1:39)
• You will see the model refuse when it has no context, then give a sourced, accurate reply when it does. (3:20)
• In this step, you will see the exact same corpus split three ways, 80, 220, and 600 characters and watch how the top retrieve chunk for a real query changes. (5:42)
• Then for each size, it builds the index, runs a hybrid search for the firmware query, and prints the first 150 characters of the top result. (7:23)
• A technician asks, “What does fault F21 mean?” The system runs a hybrid search and pulls the top two chunks from the knowledge base. (10:09)
• The most important RAG metric is groundedness. Does every claim in the answer actually come from the retrieved context? (12:53)
• It then defines three answers, one grounded, one half-invented, and one entirely invented. (14:35)

Where it fits in the AI-103 course

This is module 07 of 23. Just before it, 06. Reasoning Pipelines and Orchestration has a model critique and revise its own output, and shows that simple deterministic checks can catch errors self-reflection misses. This lesson moves from that internal reasoning to external knowledge.

Next, 08. Azure AI Search for Grounding takes retrieval further with search indexes. Later, 09. Foundry Agent Service and the agent modules rely on grounding, and 14. Evaluation and Observability returns to groundedness as one of the scores you measure on a golden set.

How to study this lesson

1. Watch the first three minutes to see the problem and the plan.
2. Run the grounded and ungrounded versions yourself. Compare the two answers side by side and note which sentences in the system prompt make the difference.
3. Repeat the chunking experiment. Change the chunk size and overlap and watch the top result change. Try a size between the lesson’s three.
4. Inspect the citations. For each cited document, check that the passage really supports the claim.
5. Write your own rules of thumb for chunk size, overlap and citation fields.
6. Move straight on to module 08 while retrieval is fresh.
7. Check yourself. The course’s mock exam has an area on generative AI apps and retrieval; the 10-question sample is free, and the full mock comes with a subscription.

Frequently asked questions

What is retrieval-augmented generation in simple terms?

As the lesson explains it, RAG first searches your own documents for relevant passages, then tells the model to answer only from what it found. That lets the model answer about documents it has never seen, with sources you can check.

Which chunk size should I use?

The lesson does not give one universal number. It shows that very small chunks lose context and very large ones dilute relevance, and that for its sample manual and query, the middle size worked best. Test sizes on your own documents.

Should I use RAG or fine-tuning?

The lesson’s summary: use RAG when the model needs private facts that change, and fine-tuning when you want to change style or behaviour.

Is this lesson free?

Yes. It is part of the free AI-103 course on YouTube; follow all modules in order from the AI-103 course page.