Skip to content
The methodOpen data
Back to the collection

A high-bias, low-variance introduction to Machine Learning for physicists

Pankaj Mehta, Marin Bukov, Ching-Hao Wang

Show all 7 authors

Alexandre G. R. Day, Clint Richardson, Charles K. Fisher, David J. Schwab

2018 · arXiv

Machine learningStatistics & probabilityPsychology
Read the paper
HN score
0
Mentions
4
Readers
4
Months
4
What’s in a score?

From the abstract

Machine Learning (ML) is one of the most exciting and dynamic areas of modern research and application. The purpose of this review is to provide an introduction to the core concepts and tools of machine learning in a manner easily understood and intuitive to physicists. The review begins by covering fundamental concepts in ML and modern statistics such as the bias-variance tradeoff, overfitting, regularization, generalization, and gradient descent before moving on to more advanced topics in both supervised and unsupervised learning. Topics covered in the review include ensemble models, deep learning and neural networks, clustering and data visualization, energy-based models (including MaxEnt models and Restricted Boltzmann Machines), and variational methods. Throughout, we emphasize the many natural connections between ML and statistical physics. A notable aspect of the review is the use of Python Jupyter notebooks to introduce modern ML/statistical packages to readers using physics-inspired datasets (the Ising Model and Monte-Carlo simulations of supersymmetric decays of proton-proton collisions). We conclude with an extended outlook discussing possible uses of machine learning for furthering our understanding of the physical world as well as open problems in ML where physicists may be able to contribute. (Notebooks are available at https://physics.bu.edu/~pankajm/MLnotebooks.html )

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

4 selected excerpts · 4 total mentions

quanto
Neutral mention · rule
I am looking for a book and resource for (advanced) data science. I understand there are numerous ML books and articles discussing algorithms at all levels; there are also numerous introductory books on data science, e.g. how to call functions in sklearn. What I am looking for is a book on data exploration, data modeling, and prediction error analysis; with some level of rigor in both theory and implementation. Any recommendation for such a book or resource? DatabookV2 discussed here looks great for natural science + ML. It reminds me of this widely-cited paper A high-bias, low-variance introduction to Machine Learning for physicists https://arxiv.org/abs/1803.08823
Original comment Data-Driven Science and Engineering 2nd Edition [pdf]
ivan_ah
Neutral mention · rule
This is very, very, very good. Book is here: https://arxiv.org/abs/1803.08823 ____ My review: The authors provide a condensed summary of all central topics in machine learning. Topics include ML basics, ML theory, optimization algorithms, but also a detailed introduction to modern methods deep learning methods. Code examples and tutorials are provided as jupyter notebooks for each chapter [2]. The book uses three datasets (MNIST digit recognition, SUSY physics data, and simulated Nearest Neighbor Ising Model) as running examples throughout the book to help learners understand what different ML techniques can bring when analyzing the same problems from different…
Read the full comment
perspectives. The book has a high bias (since written from physics perspective), but low variance since assuming physics background allows authors to write a very focussed narrative that gets to the point, and communicates three-books-worth of information in 100 pages. This is somewhat of a repeat of the general physics-ML-explanations-for-the-win pattern established in Bishop's `Pattern Recognition and Machine Learning`. The authors are wrong to label this book as useful only to people with a physics background, and in fact it will be useful for everyone who wants to learn modern ML. An estimator with high-bias but high efficiency is always useful. ____ For all my hacker news peeps that wants to learn ML and/or DL, you need to drop everything right now, go print this on the office printer, and sit outside with coffee for the next two weeks and read through this entire thing. Turn off the computer and phone. Stop checking HN for two weeks. Trust me, nothing better than this will come around on HN anytime soon. [1] book pdf => https://arxiv.org/pdf/1803.08823 [2] jupyter notebooks zip => http://physics.bu.edu/~pankajm/ML-Notebooks/NotebooksforMLRe...
Original comment A high bias low-variance introduction to Machine Learning for physicists
orbifold
Neutral mention · rule
Whether you find something like this surprising largely depends on your prior knowledge. There are quite different sets of people that work with statistics, theoretical physicists are one of them. So a recent advance in one field (machine learning), will inevitably draw in the hyenas from other fields, especially if one of their favorite toys (building increasingly complicated high energy particle physics models) has been largely taken away from them given the recent zero results at the LHC. Which is how you get review articles like this one (https://arxiv.org/abs/1803.08823).
Original comment TherML – Thermodynamics of Machine Learning
drams
Neutral mention · rule
Yes; The following recent review article actually provides code samples: https://arxiv.org/abs/1803.08823 which use an open-source version of our software called 'paysage' (https://github.com/drckf/paysage). This has currently not been updated too recently, but we expect to put out a new update quite soon. The update will clean up code, docs, features, but might not yet contain the BEAM training code. The latter is pending some decisions about IP, etc.
Original comment Boltzmann Encoded Adversarial Machines

Keep following the thread.

More in Machine learning

More from the same topics.