Microsoft is preparing to release a new generation of its Azure Maia custom AI accelerator to multi-gigawatt deployments beginning in 2027, positioning its in-house silicon as the primary inference platform for the company’s products and services.
Microsoft already has the Maia 200 available in Azure and has just announced the third generation of the custom chip, the Maia 300. The company expects the new system to deliver up to 40% more tokens per megawatt than leading GPU-based systems and as much as seven times the rack-scale token throughput of Maia 200.
The custom line underscores Microsoft’s strategy of designing its AI infrastructure around its own custom silicon with memory, networking and software optimizations to improve performance and control the cost of operating AI services at scale.
Maia 300 is designed for increasingly demanding inference workloads, including long-context reasoning, latency-sensitive conversational applications, and multi-step AI agents that perform tasks with limited human intervention. Microsoft is emphasizing system-level efficiency, particularly the movement of data between processing elements, memory and networking infrastructure.
The accelerator employs a multi-chiplet architecture that separates compute, memory-controller and input/output functions. This approach allows Microsoft to optimize individual components for their respective workloads and manufacturing requirements.
The architecture also incorporates a software-defined dataflow system and AI-assisted kernel optimization. These features are designed to improve data locality, reduce unnecessary transfers and keep compute resources more fully utilized.
Microsoft’s Maia 300 rack configuration combines 96 accelerators within a 167-kilowatt power envelope. The company specifies 27.6TB of aggregate HBM capacity and 1.7 PB/s of combined HBM bandwidth across the system’s high-speed scale-up domain.
The rack architecture integrates accelerator-to-accelerator interconnects with a custom AI transport built on Ethernet-based networking infrastructure. Microsoft intends to extend its dataflow approach across compute, memory and networking resources, enabling systems to scale beyond individual racks.
According to Microsoft, a fully configured Maia 300 rack can deliver up to seven times the rack-scale token throughput of Maia 200. The company also projects a 40% improvement in tokens per megawatt compared with leading GPU-based systems, although it did not identify the specific competing systems used for that comparison.
But the Maia 200 is no slouch. Microsoft said Maia 200 is already running production workloads around the world. Its applications include Microsoft Copilot, the company’s MAI models and models from OpenAI and other developers.
The company claims that Maia 200 delivers up to 1.7 times more tokens per watt for built-in AI capabilities in Teams, Copilot and Cowork experiences compared with leading GPU infrastructure.
Maia 300 is scheduled to begin production deployments in 2027, although exactly when it has not been disclosed. Microsoft’s longer-term objective is to make the Maia family its primary inference system across its own products and services, although the announcement does not mean that GPUs from Nvidia and other suppliers will disappear from Azure.
