GPUs are unquestionably powerful, especially for AI tasks like model training and execution. However, they come with their own issues. For one, they’re energy-intensive and generate significant heat, requiring advanced cooling systems. “GPUs, despite revolutionising performance-intensive workloads like modelling, simulation, training, and inferencing in AI and machine learning (ML), are power hungry, and require additional cooling,” says Mario Morales, vice president analyst at IDC, as reported by Network World.

On top of technical challenges, the market is grappling with a supply crunch. Nvidia’s latest Blackwell GPUs, for instance, are already sold out for the next 12 months, according to analysts at Morgan Stanley. The supply constraint has pushed cloud providers to explore alternatives that not only bypass the need for GPUs but also offer better efficiency.

Custom accelerators are filling the gaps left by GPUs. Unlike general-purpose processors, these chips are designed for specific workloads, making them more efficient. “These accelerators are becoming increasingly important in cloud infrastructure due to their superior price-performance and price-efficiency ratios,” Morales explains. They allow hyperscalers to deliver faster computing at a lower cost, providing a better return on investment.

Microsoft’s journey into custom silicon began last year, with the introduction of Maia and Cobalt chips aimed at internal AI workloads. The chips helped the company improve the energy efficiency of its data centres. This year, Microsoft has doubled down by introducing two more chips designed to tackle specific challenges.

At Ignite 2024, Microsoft introduced the Azure Boost DPU, a chip intended to speed up data processing. The processor uses a hardware-software co-design to run a lightweight operating system, resulting in higher performance and lower power consumption than traditional systems. The corporation also announced the Azure Integrated HSM, a security-focused chip that keeps encryption keys private in hardware boundaries while maintaining performance.

Beyond chips, Microsoft is innovating its data centre infrastructure. It announced a new liquid-cooling rack for AI servers as well as a disaggregated power rack co-designed with Meta, which allows 35% more AI accelerators to be housed in a single server rack. The enhancements aim to make Azure a more robust and efficient platform for handling demanding workloads.

While Microsoft’s Azure Boost DPU is a step forward, industry analysts say the company is still catching up to competitors. Google, for example, co-developed its E2000 IPU with Intel, and AWS offers its Nitro system for DPU-oriented tasks. Nvidia Bluefield and AMD Pensando are also popular choices for cloud providers focusing on similar workloads.

Microsoft is making progress, but the DPU space is already competitive, according to Alvin Nguyen, senior analyst at Forrester. “Other cloud providers use Nvidia Bluefield and AMD Pensando for these applications,” he explains.

Custom chips aren’t just about performance – they’re also enhancing security. Microsoft’s Azure Integrated HSM chip is a case in point. By handling encryption tasks entirely in the hardware, it minimises vulnerabilities and reduces latency, making it easier to scale.

AWS and Google are also using custom silicon for security. AWS’s Nitro ensures that main system CPUs can’t modify firmware, providing an extra layer of protection. Google’s Titan chip establishes a hardware-based root of trust, ensuring the integrity of the system and enabling more secure operations.

Each provider has its own approach, Nguyen notes. “While Nitro provides the critical security function of ensuring that the main system CPUs cannot update firmware in bare metal mode, Titan provides a hardware-based root of trust that establishes the strong identity of a machine, with which we can make important security decisions and validate the health of the system.”

As cloud providers continue to innovate, custom silicon is poised to play a bigger role. Beyond addressing GPU shortages, custom chips are enabling hyperscalers to rethink how they approach performance, scalability, and cost-efficiency.

“Once you have the expensive capability to do custom chips, it’s logical to look at where you are sending the most margin to vendors, where that’s growing fastest, and apply it,” says Alexander Harrowell, principal analyst at Omdia.

The shift toward custom silicon is more than a response to immediate challenges; it is a long-term strategy. Cloud providers can deliver cutting-edge solutions to meet the growing demands of AI, security, and specialised workloads by designing chips that are tailored to specific needs.

With innovation ramping up, the cloud computing landscape is entering a new era–one when the chips that power the cloud are as unique as the tasks they perform.