Hot hot hot

Or cold?

"Restoring balance will not be achieved through technical innovation alone." This quote is taken from a recent case study into "AI" data center water usage in Georgia.

That's right. I'm dipping my toes into this unfortunately contentious topic about data center water usage and its impact.

My lede here is specifically about The AI Data Center Water Management Challenge: A Georgia Case Study and what it tells us about actual water use, as opposed to theoretical or projected use cases.

Hot hot hot

In the mid nineties (yikes!), there was a recurring sketch on Saturday Night Live satirizing Celebrity Jeopardy. One of my favorites of these involved Keanu Reeves (in this case played by Tobey Maguire) trying to guess the clue, Hot or Cold.

The video clue was of Ricky Martin (was that Chris Kattan?) delivering this line "In this cup is some hot tea... it's hot hot hot.... The answer is, is it hot hot hot? Or cold? Hot hot hot, or cold?"

Keanu Reeves eventually buzzes in and asks, "Is it... iced tea?"

This disconnect is largely evident in discussions about data center water usage in lieu of these closed loop cooling systems.

In theory, these systems are meant to mitigate the potential for negative environmental impact from water usage.

What we do know is that these GPUs run very hot. (Hot hot hot, even.) Just look at NVIDIA's own tech specs, which denote that the average thermal temperature of one of these units at 87 degrees Celsius (188 F).

Hot hot hot.

Traditional data centers have used air conditioning to cool their servers, but this is just plain impossible with these modern GPUs. Imagine trying to cool tens to hundreds of thousands of these chips, all night and all day.

That's hot hot hot.

This is where "looped cooling" enters the ring. It is an idea that keeping water circulating in/out of the server heat farm, the cooling can happen without excessive water waste.

Currently, there are two conventional methods for cooling. Using a "cooling tower" or a "chiller". (See this summary for more details.)

I could go more into depth, but that has already been done in several places elsewhere, notably this particularly great write up by Masheika Allgood.

In short, regardless of either method, in warmer temperatures (a majority of places) either system requires evaporation to cool the hot liquid. (Spraying, or sending the liquid through a water-soaked medium). The heat evaporates with water, allowing the "closed loop" water to cool quickly.

Water evaporation is still needed in "closed loop" systems, which in turn, requires a steady stream of water to keep up demand. At scale, this can be quite costly.

According to Allgood, the xAI Memphis data center uses 12 million gallons of drinking water each day.

In Theory

What's generally been pretty annoying in this debate about water usage is gauging the effect of some of these techniques in theory as opposed to IRL (in the real world).

Ideally, to properly asses the impact of esoteric, technological solutions to something as mundane as cooling a GPU, it requires a wide lens of interdisciplinary cooperation.

It requires a deep understanding of theoretical vs practical, and what the tradeoffs are at scale.

It also depends on who is identified as the stakeholder, and what are the wider implications of using these cooling technologies.

For example, there are several advances in cooling technologies that are novel and interesting, particularly as they relate to the "chillers" and "cooling towers" referenced above.

Per the study cited:

Economizers are a class of free cooling innovations that utilize a series of fans and pumps to either pull ambient air directly into the facility (air-side), or to push ambient air through a refrigerant or evaporative heat exchange process that uses cooler outside air to remove the heat from the facility (water-side or refrigerant-based).

These "economizers," whether air, refrigerant, or water-based solutions, are used to minimize the impact. What impact?

However, as the study points out, the core function of these economizers are to "minimize a facility's electric bill."

Note how it is NOT their core function to reduce the amount of water that is needed, or to minimize the immediate (and longterm) environmental impact.

And sure, you could say, why not both?

If so, you're giving capitalists way too much credit. They can claim to care about the environment, about water usage, about community and environmental impact—but make no mistake, the dollar is the bottom line.

These techno-capitalists have no incentive to make the world a better place to live. They have every incentive to make you think that they do.

Rubber Meets Road

And that is why a report like this is so important.

If you actively endorse technological progress, in spite of the perceived cost to society at large, as well as to targeted communities — you should still be clamoring for transparency and impact studies that show clearly how communities are having to battle against the whims of oppressive billionaires.

This study provides insight into the current state of cooling technology, all referenced and available (as of now) to validate online. Additionally, the researchers attempted to talk with state, regional, and local agencies connected to the center. They also ran several case studies to support their research.

Their findings aren't entirely surprising.

The buildouts of these data centers in Georgia are far outpacing any sort of water infrastructure planning, and most of the decisions are being made blindly, without regard to downstream impacts.

Our most pressing finding is that AI data centers have not been explicitly considered at any level of Georgia's water management organizational structure. The risk of long-term capacity and technological underestimations is significant, particularly in cities with brittle infrastructure and cities and counties with understaffed local governments that struggle to perform accurate demand analysis.

Cities with "brittle infrastructure" and "understaffed local governments"? Umm. Isn't that most cities?

It's unfortunate that all of this comes off as a sort of knee-jerk reaction against "technological progress", or some alarmist "anti-AI" rhetoric.

But it's neither one of those things.

The recommendations in the findings are to create a cross-functional task force that studies the impacts based on best available science, with the ability to provide recommendations on how best to address gaps and mitigate negative impacts.

This should not be controversial.

Instead, you have individuals being arrested for clapping at a city meeting where zoning changes for a proposed data center project were passed.

Local To The Rescue

As the viability of the much venerated "local models" seems to improve, there should be no reason to stand aside and let these monsters of industry continue destroying these local communities.

Even if you've developed a new paradigm of working with nondeterministic software tools—if you care at all about what these communities are going through—you should also be vehemently against these data center build outs, at least until proper due discipline is in practice.

I opened with a quote from the study that references "restoring balance."

You may think that technology is what will get us there. Instead, technology continues to sever community trust (surveillance capitalism) and established norms (non-consensual training data).

We do not need more technology. Bigger. Faster. Hot hot hot.

If you're interested in local models, you should also be interested in local communities.

We need solidarity and involvement. It's hard to know where to start. But we can write (I mean, that's what I'm doing). We can increase awareness in our own neighborhoods. We can engage in local politics. We can support each other from afar.

But we must do something.