Energy Crunch Opportunities: Balancing AI Innovation and Data Center Demands

Jim Madden, CFA, Tony Tursich, CFA, and Beth Williamson

All online interactions depend on a foundation of data stored in distant servers. Those servers, stacked together in data centers around the world, require a lot of energy. Currently, data centers account for about 1.0%–1.5% of the world’s electricity use¹, and this figure is expected to increase to 3%–4% by the end of the decade², in large part due to the exploding boom in artificial intelligence (AI).

In our view, AI has emerged as a transformative force, changing how we process, analyze, and utilize data across all industries. AI is advancing so quickly that the World Economic Forum reports the computational power required to sustain AI’s rise doubles roughly every 100 days³.

And AI computations are far more energy-intensive than conventional internet queries. For example, ChatGPT queries are 6x to 10x more power hungry than traditional Google searches⁴.

Power consumption per query/search (Wh)

Source: Google, SemiAnalysis

According to a Schneider Electric white paper⁵, AI represents about 4.5 GW of power consumption today and is projected to grow at an annual rate of 25% to 33%, resulting in a total consumption of 14 GW to 18.7 GW by 2028.

Source: Schneider Electric

AI Workload Demands Are Daunting

As referenced in the above table, the AI workload is completed in two key stages (training and inference). Both stages impact the environment mainly through energy use and water consumption. At present, the environmental footprint of AI is split, with training responsible for approximately 20% and inference 80%⁶.

During the training phase, AI models learn patterns by digesting vast amounts of data, requiring significant amounts of energy. For example, the graphics processing units (GPUs) that trained GPT-3 (the precursor to ChatGPT) are estimated to have consumed 1,300 megawatt-hours of electricity, roughly equal to that used by 1,450 average U.S. households per month⁷. These models also require water for cooling (Scope 1) as well as water for power generation and manufacturing (Scope 2 and 3 accordingly).

According to JPMorgan’s ChatESG⁸, “Training GPT-3 in Microsoft’s state-of-the-art U.S. data centers evaporate 700,000 liters of clean freshwater.”

Once trained, AI models step into the inference phase to run live. While the inference phase requires less energy and water because fewer computations are involved, over time this phase is the largest contributor to emissions⁹.

New Tools Can Mitigate AI Workloads

Capping Power. To reduce energy consumption across AI workloads, manufacturers and developers alike are researching methods to limit the amount of power a GPU can draw and improve workload accuracy. “Capping power” is one technique that can be employed at data centers via software. It involves setting limits on the power consumption of hardware components such as GPUs or CPUs to manage energy usage. According to MIT’s Lincoln Laboratory¹⁰, power capping GPUs during AI model training can result in a 12%–15% reduction in energy use. The downside, capping power can increase the task time by approximately 3%. However, given a model’s training duration of days, weeks or months, this time increase is negligible.

Weeding Out Underperformance. Also, during training, AI developers can focus on improving accuracy. By analyzing the rate at which the model learns, developers can stop underperforming models early. Referencing Lincoln Laboratory’s research studies again, “… early stopping led to dramatic savings: an 80% reduction in the energy used for model training.”

Optimizing Hardware. To improve energy efficiency during the inference stage, an optimizer can be utilized to match an AI model with the most carbon-efficient mix of hardware for example, high-power GPUs for the computationally intense parts of inference and low-power central processing units (CPUs) for the less-demanding aspects can decrease energy use by 10% to 20% without compromising performance¹¹.

Market Application

Microsoft is one example of a company that has adopted tools to lessen AI workloads. According to the company, as of June 2023, Microsoft deployed its power capping system to millions of servers across the company’s data centers thereby freeing up hundreds of MWs of harvested power¹². This capping system also allowed Bing and Bing Ads to safely enhance performance by maximizing air intake, also known as a turbo boost, resulting in performance improvements of ~20%.

Microsoft has also developed custom data center chips like Azure Maia, also known as Maia 100, an AI-optimized GPU designed for running complex AI workloads. It’s built on a 5nm node and optimized for scalability and sustainability, with features like dynamic power optimization and liquid cooling.

Data Center Infrastructure Innovations May Be Vital to Success

The energy needs of data centers are driven by computing (40% of electricity demand) and cooling (40%). The remaining 20% is divided between power supply, storage, and communication equipment¹³. As such, data center infrastructure improvements can also play a significant role in reducing the environmental impacts of data centers.

Since cooling accounts for 40% of a data center’s energy needs, efficient cooling is a top priority. The shift from air cooling to liquid cooling is a potential infrastructure innovation many tout as transformational. In the direct-to-chip liquid cooling approach, “… a cooling fluid is circulated through the servers to absorb and dissipate heat, and is quickly gaining popularity as a more effective way to handle the concentrated heat generated by AI clusters.”¹⁴ Compared with air cooling, liquid cooling consumes 10% less energy¹⁵, improves power utilization, and reduces water usage.

Market Application

NVIDIA is actively supporting direct-to-chip liquid cooling for its high-performance data center GPUs. They have released their first data center PCIe GPU, which utilizes this cooling method. Additionally, NVIDIA plans to continue supporting liquid cooling in their GPUs and HGX platforms. In sum, this approach enhances efficiency, sustainability, and optimal system performance for AI workloads. Furthermore, NVIDIA was awarded $5 million by the United States Department of Energy (DOE) to develop a cooling solution combining two-phase direct-to-chip and immersion cooling techniques using environmentally compliant refrigerants¹⁶.

Hyperscalers Must Be Sustainability Innovators

Approximately 8,000 data centers currently operate in the world. The US leads with a third of the total data centers, followed by Europe (16%) and China (10%). Over the last decade, the nature of these data centers has evolved, moving from “traditional” to “Cloud” and “Hyperscale”¹⁷.

Cloud & Hyperscale data centers became the standard over the decade

Source: JP Morgan estimates, IEA

With the prominence of hyperscalers, (i.e, the biggest data center owners—Google, Microsoft and Amazon); data centers saw an uptick in efficiency as all of the aforementioned companies have set climate goals and face internal and external pressure to deliver on them. But the rise of AI is jeopardizing these corporate goals. Current practices of sourcing renewable energy or utilizing carbon credits/offsets are no longer sufficient.

Market Application

Alphabet Inc. In addition to ensuring the use of high-efficiency hardware, resilient power, and cooling systems to improve AI workloads, hyperscalers must also ensure the procurement of renewable energy sources. To do so, many are following Google’s lead to “load shifting.” Rather than relying solely on the grid’s mix of fossil fuels and renewables, hyperscalers are trying to shift, daily or even hourly, data center operations around the world to access excess renewable energy production operations across time zones. Google has taken a pioneering step by aligning its data center power usage with zero-carbon sources on an hourly basis. However, achieving uninterrupted clean energy remains elusive.

Information is Power

In our opinion, AI is a transformative technology, but its use is directly responsible for an uptick in carbon emissions and the consumption of millions of gallons of fresh water. Yet it can also be a positive, improving building efficiency, health care and climate modeling.

What is apparent is that the development of AI cannot come at the expense of our planet.

As such, many government representatives in the US and abroad are working to develop a standardized system for reporting AI impacts on society and the environment. Leading the way is the International Organization for Standardization (ISO) who will be issuing criteria for “sustainable AI,” which will include standards for measuring energy efficiency, raw material use, transportation, and water consumption, as well as practices for reducing AI impacts throughout its life cycle. “The ISO wants to enable AI users to make informed decisions about their AI consumption¹⁸.”

Calamos’ Sustainable Equities team recognizes the benefits and challenges AI brings. We will continue to seek global leaders who are working to advance this long-term secular trend, sustainably.