Microsoft 365 Copilot (Copilot) is developed by Microsoft and integrated into your Microsoft 365 tenant (subject to licensing and so forth), and was announced to the wider world on March 16, 2023.
When you use Copilot, you will experience it in two predominant ways.
In this blog, I am going to give a high level introduction to Copilot and Business Chat. You might also want to head over to this one later for a more detailed breakdown: "Everything you need to know about Microsoft 365 Copilot" (including pricing). Also, to read about the 18 July 2023 Bing Chat Enterprise announcements, go here.
Business Chat allows you - the end user - to ask for information or data using your natural language (this is what is known as the natural language prompt). The format and structure of the answer will depend on how you ask (i.e. the prompt) and the context you are asking within.
You might for example ask if anything happened yesterday with a specific customer, as shown in this image below.
This appears to be a pretty magical piece of technology - but rather than magic, I am fairly certain that it is the result of a lot of hard work, research and trial-and-error in the Microsoft development labs.
So let me try to break down what is happening. In plain English, the above example shows the result of these steps:
Microsoft refers to the architecture behind the orchestration of the steps above as "The Copilot System".
The first part of the process starts with the user prompt, which then gets "pre-processed" in what is referred to as "grounding". This improves the quality of the prompt to make sure that the responses are relevant to the context and also making sure that answers are actionable when they need to be (such as providing links to content).
A very important part of this grounding process is to make a call to the Microsoft Graph (Graph). The Graph is essentially your organisation's index of "stuff going on inside Microsoft 365", and so will contain an index of people, conversations, messages, emails, files, etc. Think of the Graph as a super-charged search index. So the grounding process adds the call to the Graph, to get the business content that is relevant to the prompt you made. This is then combined into the prompt, and this modified prompt is now sent to the LLM. This is a crucial detail - and a cause for a great deal of questions. It is this automatic enrichment of the prompt in the grounding process, which "layers in" business data (you can read more about the use of your internal data in this other blog) that feeds LLM (such as the ChatGPT service). The LLM itself has been trained with data (and possibly your data through fine-tuning) but it does not otherwise hold your data or look for your data on its own.
The LLM now processes the prompt (i.e. the prompt that was modified to include the user's data) according to the model type, such as for instance ChatGPT, and sends the response back. To clarify, there are a number of different LLMs (Large Language Models) available, where ChatGPT is the most commonly known.
A big difference to what you may have experienced with ChatGPT, is that the response is not going straight back to the user, but it is instead sent back to Copilot for post-processing.
The post-processing is a hugely complex task, which reviews the response ("responsible AI checks"), performs security, privacy and compliance reviews and generates application commands (i.e. the commands that will be required for the app where the user is active and waiting for a response).
The end result is the response that goes back to the user, combines relevant data with application commands. Making the response relevant to the end-user, and also relevant and actionable within the app the user is working within.
Microsoft 365 Copilot can therefore be seen as the automated approach to gather your data in sensible ways, and send it for processing in the AI, combined with packaging the responses up in ways that are useful, within both the user's context and the relevant application(s).
For example without Microsoft 365 Copilot, you might get tabular data back, with references to other data or formulas which you would have to manipulate and fix up for use within Excel. With Microsoft 365 Copilot, the responses are processed and prepared with the instructions that are relevant for the specific application, such as Excel.