Microsoft Copilot AI on a desktop.
Image credit: Microsoft

Microsoft is bringing chat AI to its 365 suite with a new large language model-powered tool called Copilot, the company said today.

Microsoft 365 Copilot combines large language models, integrated with a user’s own data in Microsoft Graph (which draws from context and content, such as emails, files and meetings) and the Microsoft 365 apps. Copilot is an AI writer; it’s able to draft email responses, write copy, plan and summarize meetings, and answer questions such as “Which product was most profitable this year?”

Jump to:

Copilot takes advantage of Microsoft’s 365 ecosystem

An interesting disambiguation here is that Copilot takes advantage of the Microsoft 365 ecosystem to pull in context and data. Microsoft brands this as a totally different way to interact with software. The more information you give the AI – or the more Microsoft 356 tools you use – the more powerful it is.

For example, Copilot can build a PowerPoint presentation based on a report composed in Word. Business users can ask it to base the text of a report on the format from a different assignment saved on their computer in Word. The text is based on the AI – and Microsoft emphasizes that humans should always check and polish what the AI spits out – but also on the user’s own typical behavior.

Outlook and other Microsoft 365 apps will show a Draft With Copilot option. Once the AI composes the message, the user can adjust for length and tone or ask the AI to try again. The user can also adjust text and images from there. The demonstration during Microsoft’s event today included a PowerPoint project where the user could instruct Copilot to easily add pictures and transitions to a presentation based on written instructions. Copilot can also summarize live meetings in Teams while they’re happening.

Copilot sits in a chat window at the side of a Word document, allowing users to polish, shorten or lengthen text right away based on naturally phrased instructions. The same chat panel in Excel can be used to analyze data or create new sheets. Copilot “post-processes” a large language model response through the Microsoft Graph.

SEE: Artificial Intelligence Ethics Policy (TechRepublic Premium)

New productivity assistant: Business Chat

Microsoft also announced Business Chat, a Copilot tool that appears in a tab alongside Teams (Figure A). Business Chat can summarize what users need to catch up on for work, scanning documents, chats, emails and meetings. This Copilot tool helps pull together disparate information, and can respond to requests for things like risks or quarterly revenue; it can also create a strengths vs. weaknesses analysis about your business. It’s still possible for data pulled in by the AI to be incorrect, so in the presentation today Microsoft made sure to frequently demonstrate the human user changing some of the text.

Figure A

A Business Chat dialogue prompt box offers suggestions.
A Business Chat dialogue prompt box offers suggestions. Image credit: Microsoft

Introducing Copilot for Viva Sales

Copilot was actually first released in some form as the Dynamics 365 copilot for CRM and sales. Now, it also integrates into the Viva Sales platform in a suite of capabilities known as Copilot for Viva Sales, which works with Microsoft Dynamics 365 or Salesforce.

Copilot can provide you a “catch up” to refresh yourself on what you already know about the customer. It can also provide meeting notes, and merge them into your calendar or email to help compose a follow-up message. The idea is to eliminate busywork like sending meeting invitations or reminders.

How does Microsoft approach Copilot’s security and privacy?

It’s at the Graph layer that Microsoft overlays its security. Microsoft stated that it has taken privacy concerns into consideration and monitors Copilot in real time, and has built it in accordance with internal responsible AI standards from the Aether Committee, the Office of Responsible AI and Responsible AI Strategy in Engineering. (Another AI ethics and society team was part of a large round of layoffs at the company this week.)

Microsoft’s Bing search continues to ride the AI trend. It has for the last five weeks been running on GPT-4, the new system from the much-lauded OpenAI.

The hardware behind the AI

Meanwhile, Microsoft Azure is looking to scale up supercomputing in response to the call for more artificial intelligence-driven applications and services. Microsoft has been upgrading in order to run AI for its customers and internal products like the ones in the 365 suite.

In a March 13 blog post, Microsoft introduced the ND H100 v5 VM, a virtual machine that offers the computing power of eight to thousands of NVIDIA H100 GPUs interconnected by NVIDIA Quantum-2 InfiniBand networking – however many a given customer needs.

AI models will benefit from a lengthy list of processing upgrades:

  • 8x NVIDIA H100 Tensor Core GPUs interconnected via next gen NVSwitch and NVLink 4.0
  • 400 Gb/s NVIDIA Quantum-2 CX7 InfiniBand per GPU with 3.2Tb/s per VM in a non-blocking fat-tree network
  • NVSwitch and NVLink 4.0 with 3.6TB/s bisectional bandwidth between 8 local GPUs within each VM
  • 4th Gen Intel Xeon Scalable processors
  • PCIE Gen5 host to GPU interconnect with 64GB/s bandwidth per GPU
  • 16 Channels of 4800MHz DDR5 DIMMs

Organizations like Inflection have used the combined Microsoft and NVIDIA infrastructure to power advanced AI models.

“Our focus on conversational AI requires us to develop and train some of the most complex large language models. Azure’s AI infrastructure provides us with the necessary performance to efficiently process these models reliably at a huge scale. We are thrilled about the new VMs on Azure and the increased performance they will bring to our AI development efforts,” Mustafa Suleyman, CEO of Inflection, said in a press release.

Where else is Microsoft using AI?

On March 6, Microsoft announced that its low-code application development tool Power Platform would include ChatGPT-fueled capabilities. Sales, customer service and marketing tool Dynamics 365 Copilot also now has AI features in place to automate data gathering, analysis or creating an email campaign. In particular, chatbot tool Power Virtual Agent and workflow automation tool AI Builder will gain ChatGPT’s text-generation functionality.