Pengfei Li, Jianyi Yang, Mohammad A. Islam, Shaolei Ren
2023 · arXiv
The growing carbon footprint of artificial intelligence (AI) has been undergoing public scrutiny. Nonetheless, the equally important water (withdrawal and consumption) footprint of AI has largely remained under the radar. For example, training the GPT-3 language model in Microsoft's state-of-the-art U.S. data centers can directly evaporate 700,000 liters of clean freshwater, but such information has been kept a secret. More critically, the global AI demand is projected to account for 4.2-6.6 billion cubic meters of water withdrawal in 2027, which is more than the total annual water withdrawal of 4-6 Denmark or half of the United Kingdom. This is concerning, as freshwater scarcity has become one of the most pressing challenges. To respond to the global water challenges, AI can, and also must, take social responsibility and lead by example by addressing its own water footprint. In this paper, we provide a principled methodology to estimate the water footprint of AI, and also discuss the unique spatial-temporal diversities of AI's runtime water efficiency. Finally, we highlight the necessity of holistically addressing water footprint along with carbon footprint to enable truly sustainable AI.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
9 selected excerpts · 9 total mentions
Average phone allegedly uses 30 kWh per year (charging + data transfer)[0], which in turn uses about 26 gallons of water[1]. Which I feel is well worth it for the value each user gets out, but does start to add up to lakes and power plants worth of water/energy when there are billions of phones. [0]: https://www.bryceenergyservices.com/2024/10/03/the-total-ene... [1]: Using same electricity water intensity as used in https://arxiv.org/pdf/2304.03271
According to this study[1] "With good water quality, roughly 80% of water withdrawal is evaporated and considered “consumption"" with the rest being discharged to wastewater facilities. [1] https://arxiv.org/pdf/2304.03271
Your source cites https://www.washingtonpost.com/technology/2024/09/18/energy-... which in turn claims to be based on https://arxiv.org/abs/2304.03271 but uses 0.14 kWh as the energy consumption for a 100-token request to GPT-4, which is an order of magnitude larger than any figure in that paper. Based on a speed of 18 tokens/s https://openrouter.ai/openai/gpt-4/performance the implied power draw is ≈91 kW, about two thirds of a 72-GPU rack https://www.supermicro.com/datasheet/datasheet_SuperCluster_... I somewhat doubt that the model is large enough to require an entire rack's worth of GPU memory, but even if that were the case, a single request is going to get batched with hundreds or thousands of others at the same time, so the true energy consumption should be much smaller than that.
I went down that “water use” rabbit hole a month ago and basically… it’s just a bunch of circular reporting that was based on some paper published in 2023[1]. For ChatGPT 3.5 they claimed “500ml for 10-50 responses”. In 2024, Washington Post published an article that took their claim and said “519 milliliters per email”[2] but didn’t source that from the original paper… that “shocking headline” took off and got widely circulated and cited directly, treating the WaPo calculation as if it were the original research finding. Then tech blogs and advocacy sites ran with it even harder, citing each other instead of any actual research[3]. If you look at the original paper they…
are quite upfront with the difficulty of estimating water use. It’s not public data—in fact it’s usually a closely held trade secret, plus it’s got all kinds of other issues like you don’t know where the training happened, when it happened, what the actual cooling efficiency was, etc. The researchers were pretty clear about these limitations in the actual paper. Basically, it’s urban legend at this point. When OpenAI’s CEO later said ChatGPT uses ~0.3ml per query, that’s roughly 100x less than the viral claims. [1] <https://arxiv.org/abs/2304.03271> [2] <https://www.washingtonpost.com/technology/2024/09/18/energy-...> [3] <https://www.seangoedecke.com/water-impact-of-ai>/
I published this as a comment as well, but it's probably worth nothing that the ChatGPT water/power numbers cited (the one that is most widely cited in these discussions) comes from an April 2023 paper (Li et al, arXiv:2304.03271) that estimates water/power usage based off of GPT-3 (175B dense model) numbers published from OpenAI's original GPT-3 2021 paper. From Section 3.3.2 Inference: > As a representative usage scenario for an LLM, we consider a conversation task, which typically includes a CPU-intensive prompt phase that processes the user’s input (a.k.a., prompt) and a memory-intensive token phase that produces outputs [37]. More specifically, we consider a…
medium-sized request, each with approximately ≤800 words of input and 150 – 300 words of output [37]. The official estimate shows that GPT-3 consumes an order of 0.4 kWh electricity to generate 100 pages of content (e.g., roughly 0.004 kWh per page) [18]. Thus, we consider 0.004 kWh as the per-request server energy consumption for our conversation task. The PUE, WUE, and EWIF are the same as those used for estimating the training water consumption. There is a slightly newer paper (Oct 2023) that directly measured power usage on a Llama 65B (on V100/A100 hardware) that showed a 14X better efficiency. [2] Ethan Mollick linked to it recently and got me curious since I've recently been running my own inference (performance) testing and it'd be easy enough to just calculate power usage. My results [3] on the latest stable vLLM from last week on a standard H100 node w/ Llama 3.3 70B FP8 was almost a 10X better token/joule than the 2023 V100/A100 testing, which seems about right to me. This is without fancy look-ahead, speculative decode, prefix caching taken into account, just raw token generation. This is 120X more efficient than the commonly cited "ChatGPT" numbers and 250X more efficient than the Llama-3-70B numbers cited in the latest version (v4, 2025-01-15) of that same paper. For those interested in a full analysis/table with all the citations (including my full testing results) see this o1 chat that calculated the relative efficiency differences and made a nice results table for me: https://chatgpt.com/share/678b55bb-336c-8012-97cc-b94f70919d... (It's worth point out that that used 45s of TTC, which is a point that is not lost on me!) [1] https://arxiv.org/abs/2304.03271 [2] https://arxiv.org/abs/2310.03003 [3] https://gist.github.com/lhl/bf81a9c7dfc4244c974335e1605dcf22
Your comment is addressed in the introduction section of the paper being discussed. Do give it a read. https://arxiv.org/pdf/2304.03271
You could argue that this is no longer the case once the model is done; the cost per request will go down over time, as the set amount of power and coolant pumped through data centres gets divided over more people. However, AI companies can't afford to stand still. They have to keep training or they risk being made irrelevant by whatever AI company comes next. Furthermore, a non-significant amount of energy and cooling is being used for generating responses as well. It's plainly obvious when you run even the very modest AI models at home how much power these things take. The paper[1] mentions the statistics used to calculate these numbers. It has a separate column for…
inference, with numbers ranging from 10mL to 50mL of water per inference depending on the data centre sampled. The numbers seem bad, but the authors also call out that more transparency is needed. With all the bad rep out there from independent estimations and no AI companies giving detailed environmental impact data, I have to assume the real cost is worse than estimated, or companies would've tried to greenwash themselves already. [1] https://arxiv.org/pdf/2304.03271
According to the paper (https://arxiv.org/pdf/2304.03271), the vast majority of water usage is not from cooling but indirectly from energy usage. The paper in turn gets its electricity water intensity from: https://www.wsp.com/-/media/insights/us/documents/wp_guidanc... (reference [20]) - which notes: > Even though hydropower water withdrawal and consumption intensities are usually orders of magnitude larger than other types and likely to skew overall regional averages, it is important to include hydropower in the factors to show not only the power sector’s dependency on water but also its vulnerability to water shortages. Some people have pointed out that using water for…
cooling does not destroy it - it'll all rain back down. I think it would've still been fair to consider how much processed/drinking water was being evaporated, since it'd need processing again, but I can't really see the justification for the article's framing when the figure is measuring water that would've just flowed into the sea had the hydroelectric dam not been there.
Link to the paper in question: https://arxiv.org/pdf/2304.03271 I don’t know why the URL was replaced with this much crappier Washington post article.
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
A. Pluchino. A. E. Biondo, A. Rapisarda
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics.