Follow the ideas that keep turning up in Hacker News discussions.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski +10
We trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands.
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics.
Will Grathwohl, Kuan-Chieh Wang, Jörn-Henrik Jacobsen +3
We propose to reinterpret a standard discriminative classifier of p(y|x) as an energy based model for the joint distribution p(x,y).
Kenji Kawaguchi, Leslie Pack Kaelbling, Yoshua Bengio
This paper provides theoretical insights into why and how deep learning can generalize well, despite its large capacity, complexity, possible algorithmic instability, nonrobustness, and sharp minima, responding to an open question in the literature.
Ashish Vaswani, Noam Shazeer, Niki Parmar +5
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.
Leo Gao, Stella Biderman, Sid Black +9
Recent work has demonstrated that increased training dataset diversity improves general cross-domain knowledge and downstream generalization capability for large-scale language models.
Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy
Several machine learning models, including neural networks, consistently misclassify adversarial examples---inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed input results in the model outputting an incorrect answer with high confidence.
Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio
Neural machine translation is a recently proposed approach to machine translation.
Erik P. Verlinde
Starting from first principles and general assumptions Newton's law of gravitation is shown to arise naturally and unavoidably in a theory in which space is emergent through a holographic scenario.
Suyog Gupta, Ankur Agrawal, Kailash Gopalakrishnan +1
Training of large-scale deep neural networks is often constrained by the available computational resources.
Armand Joulin, Edouard Grave, Piotr Bojanowski +1
This paper explores a simple and efficient baseline for text classification.
Moritz Hardt, Eric Price, Nathan Srebro
We propose a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features.
Mostafa Dehghani, Stephan Gouws, Oriol Vinyals +2
Recurrent neural networks (RNNs) sequentially process data by updating their state with each new data point, and have long been the de facto choice for sequence modeling tasks.
Ewin Tang
We give a classical analogue to Kerenidis and Prakash's quantum recommendation system, previously believed to be one of the strongest candidates for provably exponential speedups in quantum machine learning.
Guilherme Penedo, Quentin Malartic, Daniel Hesslow +6
Large language models are commonly trained on a mixture of filtered web data and curated high-quality corpora, such as social media conversations, books, or technical papers.
Ryan Greenblatt, Carson Denison, Benjamin Wright +17
We present a demonstration of a large language model engaging in alignment faking: selectively complying with its training objective in training to prevent modification of its behavior out of training.
Andrew Gordon Wilson
Deep neural networks are often seen as different from other model classes by defying conventional notions of generalization.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis +3
Convolutional Neural Networks (CNNs) are commonly thought to recognise objects by learning increasingly complex representations of object shapes.
François Chollet
To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that enables comparisons between two systems, as well as comparisons with humans.
Jonathan Frankle, Michael Carbin
Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever +4
Deep neural networks are highly expressive models that have recently achieved state of the art performance on speech and visual recognition tasks.
Kaiming He, Xiangyu Zhang, Shaoqing Ren +1
Deeper neural networks are more difficult to train.
Nicholas Carlini, Jamie Hayes, Milad Nasr +6
Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images.
Dario Amodei, Chris Olah, Jacob Steinhardt +3
Rapid progress in machine learning and artificial intelligence (AI) has brought increasing attention to the potential impacts of AI technologies on society.
Grégoire Delétang, Anian Ruoss, Jordi Grau-Moya +8
Reliable generalization lies at the heart of safe ML and AI.
Henry W. Lin, Max Tegmark, David Rolnick
We show how the success of deep learning could depend not only on mathematics but also on physics: although well-known mathematical theorems guarantee that neural networks can approximate arbitrary functions well, the class of functions of practical interest can frequently be approximated through "cheap learning" with exponentially fewer
Alex Krizhevsky
I present a new way to parallelize the training of convolutional neural networks across multiple GPUs.
Pankaj Mehta, David J. Schwab
Deep learning is a broad set of techniques that uses multiple layers of representation to automatically learn relevant features directly from structured data.