atlas by clearpeople

Microsoft 365 Copilot and Business Chat Explained

16 March 2023

Microsoft 365 Copilot (Copilot) is developed by Microsoft and integrated into your Microsoft 365 tenant (subject to licensing and so forth), and was announced to the wider world on March 16, 2023.

When you use Copilot, you will experience it in two predominant ways.

  • First, Copilot is embedded within Microsoft 365 apps such as Outlook, Word, PowerPoint, Excel, Teams and more.
  • Second, it is surfaced through a new app called "Business Chat".

In this blog, I am going to give a high level introduction to Copilot and Business Chat. You might also want to head over to this one later for a more detailed breakdown: "Everything you need to know about Microsoft 365 Copilot" (including pricing). Also, to read about the 18 July 2023 Bing Chat Enterprise announcements, go here.

What is Microsoft 365 Copilot all about - in plain English

Business Chat allows you - the end user - to ask for information or data using your natural language (this is what is known as the natural language prompt). The format and structure of the answer will depend on how you ask (i.e. the prompt) and the context you are asking within. 

You might for example ask if anything happened yesterday with a specific customer, as shown in this image below.

Example showing use of Business Chat

This appears to be a pretty magical piece of technology - but rather than magic, I am fairly certain that it is the result of a lot of hard work, research and trial-and-error in the Microsoft development labs.

So let me try to break down what is happening. In plain English, the above example shows the result of these steps:

    1. You ask a question in natural language from within a Microsoft app or Business Chat.
    2. Your question (the prompt) is processed and the prompt is enriched with your data (from emails, meetings, files, etc.) using the "Microsoft Graph".
    3. This more enriched prompt (which now is a combination of what you asked and the additional data extracted from the Graph) is sent to the "Large Language Model" or LLM (which most of us know as ChatGPT).
    4. The LLM (or the AI if you prefer) formulates a response according to how it was trained.
    5. The response is then processed a bit further in order to structure it to suit the context of the original prompt. In other words, you would get different response structures and applications instructions depending on whether you asked from within Teams, Word or Excel.

Microsoft refers to the architecture behind the orchestration of the steps above as "The Copilot System".

The Copilot System

The first part of the process starts with the user prompt, which then gets "pre-processed" in what is referred to as "grounding". This improves the quality of the prompt to make sure that the responses are relevant to the context and also making sure that answers are actionable when they need to be (such as providing links to content).

Microsoft 365 Copilot processes and grounds your prompt to structure it and apply context

A very important part of this grounding process is to make a call to the Microsoft Graph (Graph). The Graph is essentially your organisation's index of "stuff going on inside Microsoft 365", and so will contain an index of people, conversations, messages, emails, files, etc. Think of the Graph as a super-charged search index. So the grounding process adds the call to the Graph, to get the business content that is relevant to the prompt you made. This is then combined into the prompt, and this modified prompt is now sent to the LLM. This is a crucial detail - and a cause for a great deal of questions. It is this automatic enrichment of the prompt in the grounding process, which "layers in" business data (you can read more about the use of your internal data in this other blog) that feeds LLM (such as the ChatGPT service). The LLM itself has been trained with data (and possibly your data through fine-tuning) but it does not otherwise hold your data or look for your data on its own.

Microsoft 365 Copilot combines the prompt with your data before sending the fully enhance prompt to LLM such as ChapGPT

The LLM now processes the prompt (i.e. the prompt that was modified to include the user's data) according to the model type, such as for instance ChatGPT, and sends the response back. To clarify, there are a number of different LLMs (Large Language Models) available, where ChatGPT is the most commonly known.

A big difference to what you may have experienced with ChatGPT, is that the response is not going straight back to the user, but it is instead sent back to Copilot for post-processing.

When the response comes back, it is processed by Microsoft 365 Copilot first, before being sent to the user in the originating app

The post-processing is a hugely complex task, which reviews the response ("responsible AI checks"), performs security, privacy and compliance reviews and generates application commands (i.e. the commands that will be required for the app where the user is active and waiting for a response). 

The end result is the response that goes back to the user, combines relevant data with application commands. Making the response relevant to the end-user, and also relevant and actionable within the app the user is working within.

Microsoft 365 Copilot ensure that relevant data is combined within the response, as well as instructions (commands) to the app.

Microsoft 365 Copilot can therefore be seen as the automated approach to gather your data in sensible ways, and send it for processing in the AI, combined with packaging the responses up in ways that are useful, within both the user's context and the relevant application(s). 

For example without Microsoft 365 Copilot, you might get tabular data back, with references to other data or formulas which you would have to manipulate and fix up for use within Excel. With Microsoft 365 Copilot, the responses are processed and prepared with the instructions that are relevant for the specific application, such as Excel.

More blogs on artificial intelligence:


Atlas: The most advanced Digital Workplace platform for Microsoft 365

Nothing else comes close to the power and flexibility of Atlas Digital Workplace.

  •  Automated governance and information architecture to minimize risks of “hallucinations, invalid or irrelevant content.
  • Aggregates scattered information and make this more accessible to Microsoft Copilot
  • Seamless integration to a broader ecosystem of apps and data sources.
  • Atlas elevates the user experience of Microsoft technology by providing a visually appealing and intuitive interface.  

Find out what Atlas can do. Book a demo now.

Book a demo

Author bio

Gabriel Karawani

Gabriel Karawani

Gabriel is Co-Founder of ClearPeople, responsible for the overall technical and Atlas Intelligent Knowledge Platform vision. He works closely with colleagues at Microsoft on roadmap alignment and innovative Content AI Services programs such as Microsoft Viva Topics and SharePoint Premium (previously known as Syntex). Gabriel was part of Microsoft's partner program for Project Cortex.

View all articles by this author View all articles by this author

Get our latest posts in your inbox