S. Ambrogio, P. Narayanan, A. Okazaki
A. Fasoli, C. Mackin, K. Hosokawa, A. Nomura, T. Yasuda, A. Chen, A. Friz, M. Ishii, J. Luquin, Y. Kohda, N. Saulnier, K. Brew, S. Choi, I. Ok, T. Philip, V. Chan, C. Silvestre, I. Ahsan, V. Narayanan, H. Tsai, G. W. Burr
2023 · Nature
Abstract Models of artificial intelligence (AI) that have billions of parameters can achieve high accuracy across a range of tasks 1,2 , but they exacerbate the poor energy efficiency of conventional general-purpose processors, such as graphics processing units or central processing units. Analog in-memory computing (analog-AI) 3–7 can provide better energy efficiency by performing matrix–vector multiplications in parallel on ‘memory tiles’. However, analog-AI has yet to demonstrate software-equivalent (SW eq ) accuracy on models that require many such tiles and efficient communication of neural-network activations between the tiles. Here we present an analog-AI chip that combines 35 million phase-change memory devices across 34 tiles, massively parallel inter-tile communication and analog, low-power peripheral circuitry that can achieve up to 12.4 tera-operations per second per watt (TOPS/W) chip-sustained performance. We demonstrate fully end-to-end SW eq accuracy for a small keyword-spotting network and near-SW eq accuracy on the much larger MLPerf 8 recurrent neural-network transducer (RNNT), with more than 45 million weights mapped onto more than 140 million phase-change memory devices across five chips.
Metadata: Crossref. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
4 selected excerpts · 4 total mentions
High costs are due to inefficient chips and overpowered models. It’s like we discovered tnt and now we’re using it to weed a garden. The purpose built chips and tpus and pruned models will be cheaper. You can do speech to txt on $10 chip. https://www.nature.com/articles/s41586-023-06337-5 Imagine this chip for JavaScript.
People are working on that [1]. In some sense, it's a step back to analog computing. Add/multiply is possible to do directly in memory with voltages, but it's less versatile (and stable) than digital computing. So you can't do all calculations in a neural network that way, meaning some digital components will always be necessary. But I'm pretty sure analog will make a comeback for AI chips sooner or later. [1] https://www.nature.com/articles/s41586-023-06337-5
No, because NVidia has to work for all the models. Nvidia has other constraints that they need to have for users like instructions, security etc. which openai doesn't have. e.g. As they have a fixed model which they know they would get billions of request to, they could even work with analogue chip which is significantly cheaper and faster for inference. [1] could achieve 10-100x flops/watt for fixed models compared to nvidia for their first gen chip. [1]: https://www.nature.com/articles/s41586-023-06337-5
The actual nature paper mentioned in the article: https://www.nature.com/articles/s41586-023-06337-5 IBM blog post: https://research.ibm.com/blog/analog-ai-chip-low-power Also I'd like to mention that those chips are for inference only. You still need GPUs for training.
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
A. Pluchino. A. E. Biondo, A. Rapisarda
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics.
Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai +3
Believable proxies of human behavior can empower interactive applications ranging from immersive environments to rehearsal spaces for interpersonal communication to prototyping tools.