And while Nvidia didn’t complain about supply on its most recent earnings call, apparently it does have a problem with it. Omdia says leading server OEMs like Dell, Lenovo, and HPE are not able to fulfil GPU server orders yet, due to a lack of GPU supply from Nvidia. OEMs indicated a lead time of 36 to 52 weeks for servers configured with Nvidia H100 GPUs.
That’s in part because a few players are taking all of the supply. Omdia noted that both Microsoft and Meta are on track to receive 150,000 of Nvidia’s H100 accelerators by the end of this year – which is three times as many as Nvidia’s other major customers, Google, Amazon and Oracle.
These high-powered servers are also driving demand for better power efficiency and management. Data center operators have to get more compute power out of the same power envelope due to constraints and power supply. Omdia said rack power distribution revenue in 1H23 was up 17% over last year, while UPS revenue growth for the first half of 2023 was 7% ahead of last year.
“With a ramp of professional services for generative AI enabling broad enterprise adoption in 2024 and beyond, the only thing that can curb the current rate of AI deployment is power availability,” Omdia said in its report.
AI is also driving demand for liquid cooling, since air cooling is simply no longer efficient for the very hot processors used in AI. Cooling vendors and server OEMs tell Omdia direct-to-chip liquid cooling is ramping in line with its forecast for 80% revenue growth within the year, and it noted that server vendor Super Micro recently said that it expects 20% of the servers it ships in 4Q23 will use direct-to-chip liquid cooling.
Up through 2027, Omdia expects continued growth in rack power density, server performance improvement, and server fleet consolidation. There will be a strong focus on computing performance to enable the commercialization of AI. AI models will continue to be a research project requiring a great deal of tuning, even libraries of pre-trained models.
