A Comparison of the Cerebras Wafer-Scale Integration Technology with Nvidia GPU-based Systems for Artificial Intelligence

between tokens (autoregressive, so token 1 before token 2). This means most of your weights loaded in SRAM would be just sitting unused most of the time, and when they need to be used they need to be broadcasted to all chips from the SRAM of the chip that has the particular layer you care about, this is extremely fast, but external memory is certainly fast enough to do this if you fetch the layer in advance. So the way to get the best ROI on such a system would be to pack the biggest batch size you can (so many users' queries) and process them all in parallel, streaming the weights as needed. The more your SRAM is occupied by batch activations and not parameters, the better the compute density and thus $/flops. You can check the Cerebras doc to see how weight streaming works [2]. From the start, one of the selling point of Cerebras is the possibility to scale memory independently of compute, and they have developped an entire system specifically for weight streaming from that decoupled memory. Their docs seems to keep things fairly simple assuming you can only fit one layer in SRAM and thus they fetch things sequentially, but if you can store at least 2 layers in those 44GB of SRAM then you can simply fetch l+1 when l is starting to compute, completely masking latency cost. Its possible they already mask the latency even within a single layer by streaming by tiles for matmul though, unclear from their docs. They mention that in passing in [3] section 6.3. All of their doc is for training since it seems for inference play they have pivoted to selling API access rather than chips, but inference is really the same thing, just without the backprop (especially in their case were they aren't doing pipeline parallelism where you could claim doing fwd+back prop gives you better compute density). At the end of the day whether you are doing training or inference, all you care about is that your cores have the data they need in their registers at the moment they are free to compute, so streaming to SRAM works the same way in both cases. Ultimately I can't tell you how much it cost to run Qwen-3, you can certainly do it on a single chip + weight streaming, but their specs are just too light on the exact FLOPs and bandwidth to know what the memory movement cost would be in this case (if any), and we don't even know the price of single chip (everyone is saying 3M$ though, regardless of that comment on the other thread). But I can tell you that your math of doing `model_size/sram_per_chip * chip_cost` just isn't the right way to think about this, and so the 100M$ figure doesn't make sense. [1]: https://arxiv.org/html/2503.11698v1#S3. [2]: https://training-api.cerebras.ai/en/2.1.0/wsc/cerebras-basic.... [3]: https://8968533.fs1.hubspotusercontent-na2.net/hubfs/8968533...

build and operate digital infrastructure that addresses societal challenges and creates benefits for communities" (2024-06) https://blogs.microsoft.com/blog/2024/06/02/microsofts-datac... : > We will design and operate our datacenters to support society’s climate goals and become carbon negative, water positive and zero waste before 2030. [...] > By 2025, we will procure 100% renewable energy on a global scale, both significantly expanding and decarbonizing local electricity grids. > Our datacenter designs are more water efficient than traditional enterprise datacenters, and our plan by 2030 is to replenish more water than we consume locally. Here's this about CNT cooling: "Cyberpower begins selling desktop PCs with carbon nanotube CPU cooling" (2025) https://news.ycombinator.com/item?id=44899495 "A carbon-nanotube-based tensor processing unit" (2024) https://news.ycombinator.com/item?id=41322070 Graphene semiconductors should be at least 10X more energy efficient; but how much less water would graphene-based chips waste?