Microsoft’s Maia 300 chip: 40% more tokens per megawatt

Microsoft has unveiled Maia 300, its third-generation custom artificial intelligence inference chip, and says the design will process about 40 percent more tokens per megawatt of power than leading GPU-based systems. The company positions Maia 300 as a purpose-built inference accelerator intended to become the primary inference infrastructure for Microsoft products and services.

Microsoft expects Maia 300 to enter production deployments in 2027. In a statement Scott Guthrie, executive vice president of cloud computing at Microsoft, said the improvements will let the company rely more on its own custom chips and reduce dependence on Nvidia and AMD. Guthrie added that Microsoft will begin developing the Azure Maia chip family at multi-gigawatt scale from 2027 so the chips can serve as the main inference infrastructure across the company.

Performance and role in Microsoft’s stack

In a full configuration with 96 accelerators, Maia 300 can raise rack-level token processing throughput up to seven times higher than Maia 200, the prior-generation chip Microsoft introduced only in January. Maia 200 is already deployed across a range of Microsoft workloads including Microsoft Copilot, GitHub Copilot, Microsoft Foundry, and the company’s proprietary AI models. Guthrie said Maia 200 remains the best compute infrastructure in Microsoft’s fleet by real AI service performance metrics, while Maia 300 is designed as a complete system for next-generation AI workloads.

Microsoft describes Maia 300 as optimized for reasoning workloads with long contexts, low-latency conversational experiences, and continual multi-step agent tasks. The company frames the chip as targeted at inference rather than training.

Chip architecture and manufacturing

Maia 300 uses a chiplet architecture built on TSMC’s N2P process. Microsoft says the design separates compute blocks, memory controller, and input-output functions so more die area can be allocated to processing and each block can deploy technology tailored to its role. The Azure hardware and infrastructure team says this approach yields higher efficiency and allows the chip to be tuned for specific inference functions.

Microsoft also signals that Maia 300 will play a significant role in the company’s own model development and deployment plans. Mustafa Suleyman, CEO of Microsoft AI, said the company must develop all layers of the technology stack, from models to silicon, and that at the scale Microsoft is planning a dedicated platform co-designed with its models is required. He added that running Microsoft’s proprietary models on Maia is a top priority for the team.