Satya Nadella, CEO and Chairman, Microsoft
At Ignite 2023, Microsoft unveiled a comprehensive approach to its end-to-end AI stack, showcasing innovations spanning from cloud infrastructure to AI-powered applications and security measures.
For Microsoft and its ecosystem, this year’s Ignite conference proved to be extraordinary and different. Traditionally, Ignite has been a conference that generally focuses on infrastructure and operations, while Microsoft’s flagship event, Build, typically caters to a developer audience. However, announcements about generic AI directed at developers and ML engineers took center stage at Ignite 2023. This was not limited to developers or IT professionals, rather it became a significant moment for the entire Microsoft ecosystem.
Microsoft wants to become a major force in the AI ecosystem and Microsoft CEO and Chairman Satya Nadella made this clear in his keynote speech. From developing its own AI accelerator chips to launching a marketplace for co-pilots, Microsoft has a long-term strategy.
Here’s a detailed analysis of how Microsoft is using AI to maintain its leadership and dominance in the industry:
Azure is the new AI operating system, and Copilots are the new apps
Microsoft has a highly successful track record of building platforms. The earliest version of the platform was built on Windows, where developers took advantage of OLE and COM to create applications through Visual Basic. Announced in the early 2000s, Microsoft.NET and Visual Studio created a new platform that rekindled interest among developers creating Web services. In the last decade, Microsoft had successfully launched another platform in the form of Azure.
When Microsoft creates a platform, it leads to a new ecosystem of independent software vendors and solution providers to help enterprises take advantage of it. This was evident in the success of Microsoft Windows, Office, Visual Studio and more recently Azure.
With AI, Microsoft wants to replicate the magic of building an entirely new platform that will result in a rich ecosystem of developers, ISVs, system integrators, enterprises and consumers.
This season, Azure becomes the operating system providing the runtime and platform services, while the apps are the AI assistants that Microsoft calls Copilot. So, Azure is the new Windows and Co-Pilot is the new applications. Foundation models, such as GPT-4, form the kernel of this new OS. Similar to Visual Studio, Microsoft has invested in a set of developer tools in the form of AI Studio and Copilot Studio. This stack is very similar to Windows, .NET, and Visual Studio, which ruled the developer landscape for decades.
Microsoft’s approach clearly reflects a sense of urgency. This is evident given the current market dynamics and the lessons learned from failed attempts to build an ecosystem around mobile platforms. Satya is incredibly committed to ensuring that Microsoft becomes a company that leads in artificial intelligence by bringing the capabilities of generic AI closer to its customers. He doesn’t want the company to miss out on the next big thing in technology, as they did with search and mobile.
In just a few months, the company has shipped a number of co-pilots for its products, ranging from the Bing search engine to Microsoft 365 to the Windows operating system. It also added various capabilities to the Edge browser, enhancing the user experience. The pace at which Microsoft has adopted generative AI in recent months has been astonishing, making it one of the leading AI platform companies.
Microsoft invests in developing its own CPU, GPU and DPU
For decades, the CPU set the rules for software architecture and shaped its evolution. Now, AI software is shaping the development of chips, giving rise to purpose-built processors.
Microsoft formally announced that it will begin manufacturing its own silicon and processors, including CPUs, AI accelerators, and data processing units.
Let’s start with the CPU. Azure Cobalt, Microsoft’s own CPU, is based on Arm architecture for optimal performance and watt efficiency, and it powers common Azure cloud workloads. The first generation of the series, Cobalt 100, is a 64-bit 128-core chip that improves performance by up to 40% compared to current generations of Azure Arm chips and powers services like Microsoft Teams and Azure SQL. Following the Neoverse N1, the first Arm-based CPU purchased from Ampere Computing, the Cobalt 100 becomes the second Arm-based processor available on Azure.
Then there’s Azure Maia, the first in a series of custom AI accelerators designed to run cloud-based training and inference for AI workloads like OpenAI models, Bing, GitHub Copilot, and ChatGPT. With 105 billion transistors, the Maia is the first generation of the 100 series and one of the largest chips on the 5nm process technology. It includes many innovations in the areas of silicon, software, networking, racks and cooling. The new AI accelerator becomes an alternative to GPUs by optimizing Azure AI’s end-to-end systems to run state-of-the-art Foundation models like GPT.
Finally, Azure Boost, Microsoft’s own DPU, became generally available. Microsoft acquired DPU company Fungible earlier this year to improve the efficiency of Azure data centers. Software functions such as virtualization, network management, storage management, and security are offloaded to dedicated hardware with Azure Boost, allowing the CPU to dedicate more cycles to workloads rather than system management. Because the heavy lifting is moved to a purpose-built processor, this offloading significantly improves the performance of cloud infrastructure.
In addition to bringing its own silicon into the mix, Microsoft has partnered with AMD, Intel, and NVIDIA to bring the latest CPUs and GPUs to Azure. It will have the latest NVIDIA H200 Tensor Core GPU by next year to run larger Foundation models with lower latency. AMD’s new MI300 accelerator will also become available on Azure early next year.
Less dependency on OpenAI with homegrown and open source foundation model
While Azure remains the platform of choice for enterprises to run inference on OpenAI-based models, Microsoft is investing in training its own foundation models that complement the existing models available in Azure OpenAI and Azure ML.
Phi-1-5 and Phi-2 are small language models that are lightweight and require fewer resources than traditional large language models. Phi-1-5 have 1.3 billion parameters, while Phi-2 has 2.7 billion parameters, making them much smaller than Llama 2, which starts at 7 billion parameters and goes up to 70 billion parameters. These SLMs are ideal for embedding within Windows to provide a local co-pilot experience without roundtripping to the cloud. Microsoft is releasing an extension for Visual Studio Code that allows developers to fine-tune these models in the cloud and deploy them locally for offline inference.
Microsoft Research has developed Florence, a foundation model that brings multimodal capabilities to Azure Cognitive Services. This model allows users to analyze and understand images, video, and language to provide customizable options for building computer vision applications. This model is already available in Azure.
Azure ML now supports additional open source foundation models, including Llama, Code Llama, Mistral 7B, Stable Diffusion, Whisper v3, BLIP, CLIP, Flacon, and NVIDIA Nemotron.
Azure ML, Microsoft’s ML PaaS provides models as a service to consume foundation models as APIs without the need to provision GPU infrastructure. This significantly simplifies the integration of AI with modern applications.
The combination of Azure OpenAI and Azure Model Catalog provides customers with the broadest and widest range of foundation models, becoming a key differentiating factor of Azure.
Microsoft Graph and Fabric at the core of the data platform
AI requires large amounts of data for pre-training, fine-tuning, and retrieval. Microsoft Fabric and Microsoft Graph are two major products that significantly contribute to Microsoft’s generative AI efforts.
Microsoft Fabric, which was announced at Microsoft Build 2023, is a significant addition to Microsoft’s data product line. Satya emphasized its importance by comparing it to the release of SQL Server, which signaled a fundamental shift in Microsoft’s data management and analytics strategy.
At Ignite 2023, Microsoft announced the general availability of Fabric. It includes a component called OneLake, which is a transformational data lakehouse platform. OneLake has been integrated into Azure Machine Learning and Azure AI Studio, representing a major enhancement to the data management capabilities of Azure Machine Learning. This platform is designed to handle large and diverse datasets in a unified and efficient manner, optimizing data storage and retrieval for AI applications. Its integration with the Azure AI platform is particularly important for scenarios that require high-volume data processing and complex computational tasks, which are common in advanced AI and machine learning projects. What’s interesting about OneLake is the concept of shortcuts that bring data from external sources, including Amazon S3 and Databricks, into the fabric.
Microsoft Graph, a powerful tool in Microsoft’s arsenal, plays an important role in the field of AI co-pilot. This has become key to developing AI Copilot, which offers unified APIs to access diverse data across Microsoft 365 services. Microsoft Graph enables co-pilots to provide personalized assistance by collecting data from emails, calendar events, and team interactions. This integrated approach ensures a contextual understanding of users’ professional environment, which is essential for making intelligent suggestions. Microsoft Graph supports real-time data access, which is critical for timely co-pilot responses. Its compliance with Microsoft 365 security standards ensures secure handling of sensitive data.
Microsoft Fabric and Microsoft Graph have become the foundation for building co-pilots based on real-time data available through APIs.
Overall, Microsoft’s strategy in Ignite 2023 has a clear focus on leading the AI revolution, leveraging its platform legacy, and innovating across hardware and software to maintain industry dominance.
