Summer Yue, Ebrahim M. Songhori, Joe Wenjie Jiang
Toby Boyd, Anna Goldie, Azalia Mirhoseini, Sergio Guadarrama
2022 · Proceedings of the 2022 International Symposium on Physical Design
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
5 selected excerpts · 5 total mentions
I assume you mean Fig. 6 here?[0] But that was explicitly limited to 8 hours for all setups. Do they have another paper that shows that you can't increase the number of hours of a smaller GPU setup to compensate? [0]https://dl.acm.org/doi/pdf/10.1145/3505170.3511478
See this ISPD 2022 paper where the AlphaChip authors dive more into the value of pre-training (Figure 7, Figure 8): https://dl.acm.org/doi/pdf/10.1145/3505170.3511478
The impact of pre-training is covered in Kahng's FAQ, "6. Did you use pre-trained models? How much does pre-training matter?" https://github.com/TILOS-AI-Institute/MacroPlacement#new-faq... It is also covered in the ISPD'22 paper, "Scalability and Generalization of Circuit Training for Chip Floorplanning", by Summer Yue, Ebrahim M. Songhori, Joe Wenjie Jiang, Toby Boyd, Anna Goldie, Azalia Mirhoseini, and Sergio Guadarrama, which was written by authors from the Nature paper. https://dl.acm.org/doi/pdf/10.1145/3505170.3511478 In their ISPD'22 paper, link above, Figure 7 shows a 5x speedup but 0.3% worse placement results with pre-training on diverse designs vs. running…
from scratch. Pre-training on previous versions of the same design does improve results by 1.7% vs. running from scratch, and has similar runtime benefit to pre-training with diverse designs. Given that the Circuit Training approach requires 41x to 646x wall time and an even higher factor of compute time vs. other standard techniques, the 5x speedup with pre-training is insufficient to close that very large gap. Furthermore, pre-training does not improve results vs. training from scratch, unless you are pre-training on previous versions of the same design but even then a 1.7% improvement in the circuit training results is not adequate to demonstrate improvement vs. standard techniques as can be seen by examining the results from Professor Kahng's group's work. This concern about pre-training has been answered in multiple forums, including the discussion in ISPD'23 Session 8, when Mr. Washauer-Baker raised the same issue. It's unfortunate to see Gabriel Warshauer-Baker, Anna Goldie's spouse, making repeated personal attacks on Professor Kahng. Please focus on the scientific discussion.
For a more thorough discussion on pre-training, see this ISPD 2022 paper by the AlphaChip people: https://dl.acm.org/doi/pdf/10.1145/3505170.3511478 As for external usage of the method - MediaTek is one of the largest chip design companies in the world, and they built on AlphaChip. There's a quote from a MediaTek SVP at the bottom of the GDM blog post: "AlphaChip's groundbreaking AI approach revolutionizes a key phase of chip design. At MediaTek, we've been pioneering chip design's floorplanning and macro placement by extending this technique in combination with the industry's best practices. This paradigm shift not only enhances design efficiency, but also sets new benchmarks for effectiveness, propelling the industry towards future breakthroughs."
See their ISPD 2022 paper, which goes into more detail about the value of pre-training (e.g. Figure 7): https://dl.acm.org/doi/pdf/10.1145/3505170.3511478 Sometimes training from scratch is able to match the results of pre-training, given ~5X more time to converge. Other times, though, it never does as well as a pre-trained model, converging to a worse final result. This isn't too surprising -- the whole point of the method is to be able to learn from experience.
More from the same topics.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski +10
We trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands.
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics.