Skip to content
The methodOpen data
Back to the collection

The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink

David Patterson, Joseph Gonzalez, Urs Hölzle

Show all 10 authors

Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, Jeff Dean

2022 · arXiv

Machine learning
Read the paper
HN score
0
Mentions
4
Readers
3
Months
3
What’s in a score?

From the abstract

Machine Learning (ML) workloads have rapidly grown in importance, but raised concerns about their carbon footprint. Four best practices can reduce ML training energy by up to 100x and CO2 emissions up to 1000x. By following best practices, overall ML energy use (across research, development, and production) held steady at <15% of Google's total energy use for the past three years. If the whole ML field were to adopt best practices, total carbon emissions from training would reduce. Hence, we recommend that ML papers include emissions explicitly to foster competition on more than just model quality. Estimates of emissions in papers that omitted them have been off 100x-100,000x, so publishing emissions has the added benefit of ensuring accurate accounting. Given the importance of climate change, we must get the numbers right to make certain that we work on its biggest challenges.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

4 selected excerpts · 4 total mentions

kinston
Neutral mention · rule
People seem to miss the specific thing that was controversial about this paper: The paper gives inaccurate estimates of the carbon dioxide emissions that result from machine learning. My understanding is that this is the main reason why Google wanted the paper pulled. You can't have a paper by Google authors saying incorrect things about how much CO2 Google is emitting. The authors refused. They argued that they gave a citation for their numbers, and it's not their problem if the paper they cite is inaccurate. Eventually Google did publish a response: https://arxiv.org/abs/2204.05149 But this got totally buried...
Original comment On the dangers of stochastic parrots: Can language models be too big? (2021)
jeffbee
Neutral mention · rule
These conclusions are broadly compatible with "The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink" or, as I prefer, the PDF metadata title that they left in there, "Revamped Happy CO2e Paper". Despite the incredible focus by the press on this topic, Mistral's lifecycle emissions in 18 months were less than the typical annual emissions of a single A320neo in commercial service. https://arxiv.org/pdf/2204.05149
Original comment Mistral reports on the environmental impact of LLMs

Keep following the thread.

More in Machine learning

More from the same topics.