- Network implications: “Ethernet is currently the dominant underpinning for AI for the majority of use cases that require cost economics, scale and ease of support. According to the Dell’Oro Group, by 2027, as much as 20% of all data center switch ports will be allocated to AI servers. This highlights the growing significance of AI workloads in data center networking,” Lee wrote. “Furthermore, the challenge of integrating small form factor GPUs into data center infrastructure is a noteworthy concern from both a power and cooling perspective. It may require substantial modifications, such as the adoption of liquid cooling solutions and adjustments to power capacity.”
- Performance challenges: “The use of Graphics Processing Units (GPUs) is essential for AI/ML training and inference, but it can pose challenges for data center IT infrastructure from power and cooling perspectives. As AI workloads require increasingly powerful GPUs, data centers often struggle to keep up with the demand for high-performance computing resources. Data center managers and developers, therefore, benefit from strategic deployment of GPUs to optimize their use and energy efficiency,” Lee wrote.
- Power constraints: “AI/ML infrastructure is constrained primarily by compute and memory limits. The network plays a crucial role in connecting multiple processing elements, often sharding compute functions across various nodes. This places significant demands on power capacity and efficiency. Meeting stringent latency and throughput requirements while minimizing energy consumption is a complex task requiring innovative solutions.”
- Adopter strategies: “Early adopters of next-gen AI technologies have recognized that accommodating high-density AI workloads often necessitates the use of multisite or micro data centers,” Lee wrote. “These smaller-scale data centers are designed to handle the intensive computational demands of AI applications. However, this approach places additional pressure on the network infrastructure, which must be high-performing and resilient to support the distributed nature of these data center deployments.”
Cooling is another important consideration in data centers that are handling AI workloads. “Traditional air-cooling methods can be inadequate in AI/ML data center deployments, and they can also be environmentally burdensome. Liquid cooling solutions offer a more efficient alternative, but they require careful integration into data center infrastructure,” Lee wrote.
“The industry has to work on liquid cooling. We just simply cannot cool the chips that are coming,” she said.
Indeed, as server rack densities increase and temperatures rise, more data centers are finding ways to add liquid cooling to their facilities. The global data center liquid cooling market was estimated at $2 billion in 2022 and is expected to grow at a compound annual growth rate of 15% between 2023 and 2032, according to Global Market Insights.
Cisco pursues energy networking
Another interesting technology that could help sustainability efforts is a relatively new class of power system known as Class 4 Fault Managed Power (FMP). The National Electrical Code (NEC) in late 2022 adopted Class 4 FMP, which can handle substantial power levels, up to 2,000 watts, over longer distances compared to older circuit classes, the NEC stated.
Class 4 FMP systems are described as “sustainable” as they allow the use of smaller cable gauges, no conduit, and intelligent control over power delivery; these attributes can lead to reduced material usage and embodied carbon per project, contributing to more sustainable electrical infrastructure, according to NEC. They’re designed to provide power distribution in various applications, such as power over Ethernet, Internet of Things (IoT) devices, smart building systems, monitoring and control of electronics and appliances, security systems, and electronic components in large areas like buildings, stadiums, or campuses.
“Class 4 is ‘touch safe’ direct current so it’s easy to work with,” Lee noted. “We are fast working on this technology with others in the market for the education and the awareness and the adoption of it over time.”
