AB-730: LLMs and Microsoft 365 Copilot Architecture

04 LLMs & Copilot Architecture is the core theory lesson of the free AB-730 Microsoft 365 Copilot course: 25 minutes on how generative AI works and how Microsoft 365 Copilot turns your prompt into a grounded answer. It is for anyone preparing for AB-730, and for business users and IT staff who want to understand what Copilot does with their data before they rely on it.

The lesson itself calls the Copilot architecture one of the most frequently tested topics in the AB-730 exam, so it is worth watching slowly and taking notes. It sits on the AB-730 course page right before the hands-on app labs.

What this lesson covers

The lesson builds the picture in layers, from the basic idea of generative AI to the full request flow inside Copilot.

Generative AI versus traditional AI. Traditional AI recognises patterns and classifies things, such as spotting spam. Generative AI creates new content. The lesson’s analogy: traditional AI is a librarian who finds the right book, generative AI is an author who writes a new one.

How an LLM works. A large language model is a neural network trained on huge amounts of text. It reads tokens (pieces of words), is built on the transformer architecture with its attention mechanism, and predicts the next most probable token. GPT, the model family behind Microsoft 365 Copilot, stands for generative pre-trained transformer.

The core vocabulary. Your input is a prompt; the model’s output is a completion. A hidden system prompt sets guard rails before every interaction, and the context window limits how many tokens the model can consider at once, including your prompt, history and any data added from Microsoft Graph.

The Copilot request flow. A request starts in a Microsoft 365 app, passes to the Copilot orchestrator, which calls Microsoft Graph and the semantic index for relevant organisational data. That data and your prompt go to Azure OpenAI Service, where the GPT model generates a response, and the orchestrator formats it and returns it to the app.

Permissions and privacy. Copilot reaches your data through Microsoft Graph, which respects existing Microsoft 365 permissions: Copilot only surfaces what the signed-in user can already access. The lesson also covers tenant isolation, data residency, encryption and the key exam point that your prompts and organisational data are not used to train the underlying models.

Semantic index, context and grounding. The semantic index uses vector embeddings to match on meaning, not just keywords. Copilot’s answers are shaped by app context, your work content and, when enabled, web data. Grounding through retrieval-augmented generation (RAG) anchors answers to real data, and numbered citations let you check the sources.

The lesson then maps what Copilot can do in each core app, such as Word, Excel and PowerPoint.

Key points from the lesson

• Start at the very foundation. What exactly is generative AI and why does it matter? Generative AI is a category of artificial intelligence that creates new content. That content could be text, images, code, music, video or any other … (0:00)
• Forward-looking tone. The text that copilot generates in response to your prompt is called a completion. (4:07)
• Orchestrator calls Microsoft Graph and the semantic index to retrieve relevant organizational data. The retrieve data and your original prompt are then combined and sent to Azure OpenAI service where the GPT model generates a … (6:24)
• Data privacy and security are the top concerns for any organization evaluating an AI tool. So, let me walk through exactly how Microsoft has designed Copilot to be enterprise safe. (9:30)
• The underlying technology that allows Copilot to understand genuinely natural language questions such as what did the team decide about the product launch last month and retrieve the relevant meeting notes or email thread even when … (12:51)
• The content will be entirely generic based on what sales reports typically look like. (16:01)
• Add individual slides on demand, and suggest layout redesigns. For example, create a 10 slide deck from this word report is a valid and powerful prompt. (18:21)
• An LLM or large language model is a neural network trained on vast amounts of text data that can generate humanlike text in response to prompts. (21:38)

Where it fits in the AB-730 course

This is the fourth video in the 20-video course. Before it come the course introduction, M365 Copilot Certification - AB 730 Exam Overview and Microsoft Stopped Free M365 Developer Tenants? Create One This Way!, which gets you an environment to practise in.

Straight after it come the app labs: LAB - Working with Copilot In MS Word, then Excel, Outlook and PowerPoint. Those labs show the “app context” idea from this lesson in action: the same Copilot behaves differently in each app. Later, Responsible AI in Microsoft Copilot builds on the privacy and grounding ideas introduced here.

How to study this lesson

1. Watch it in two halves. Stop around the 9-minute mark, after the request flow, and summarise it before continuing to privacy, the semantic index and grounding.
2. Draw the flow. App, orchestrator, Microsoft Graph and semantic index, Azure OpenAI Service, back to the app. If you can draw it from memory, you understand it.
3. Make a vocabulary list. Token, prompt, completion, system prompt, context window, grounding, RAG, semantic index, vector embedding.
4. Write down the exam points the lesson flags, such as Copilot accessing data through Microsoft Graph and respecting existing permissions.
5. Test it in the next lab. In the Word lab, look for citations and notice how the output reflects the app you are in.
6. Check yourself. The course’s mock exam includes a Copilot foundations area; its 10-question sample is free, and the full timed mock comes with a subscription.

Frequently asked questions

Do I need to understand the maths behind LLMs for AB-730?

No. The lesson says you do not need the underlying mathematics. What matters is that LLMs predict the next most probable token based on the context they are given.

Does Microsoft 365 Copilot use my organisation’s data to train the model?

According to the lesson, no. Your prompts and organisational data are not used to train the underlying LLMs, and the lesson presents this as one of the most frequently tested points.

Can Copilot show me a file I do not have permission to open?

No. The lesson explains that Microsoft Graph enforces your existing Microsoft 365 permissions, so Copilot can only surface data the signed-in user can already access.

Is this lesson free?

Yes. It is part of the free AB-730 course on YouTube; follow the whole series from the AB-730 course page.