For a rather complete view of what deep learning still does not do, I recommend the work of Gary Marcus and Ernest Davis. While seeming solely critical, I think they make very good points about the limits inherent in deep learning as we know it now, and how it needs to grow to overcome those deficits. "Rebooting AI" https://www.penguinrandomhouse.com/books/603982/rebooting-ai... And a few articles, for audiences both popular and technical: Deep Learning: A Critical Appraisal https://arxiv.org/abs/1801.00631 The Next Decade in AI https://arxiv.org/pdf/2002.06177.pdf How to Build AI We Can Trust https://www.nytimes.com/2019/09/06/opinion/ai-explainability... And a HBR podcast ("Beyond Deep Learning") https://hbr.org/podcast/2019/10/beyond-deep-learning-with-ga...
The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
Gary Marcus
2020 · arXiv
- HN score
- 1
- Mentions
- 3
- Readers
- 3
- Months
- 3
From the abstract
Recent research in artificial intelligence and machine learning has largely emphasized general-purpose learning and ever-larger training sets and more and more compute. In contrast, I propose a hybrid, knowledge-driven, reasoning-based approach, centered around cognitive models, that could provide the substrate for a richer, more robust AI than is currently possible.
Metadata: arXiv. Abstracts retain their original authors’ rights.
In the conversation
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
3 selected excerpts · 3 total mentions
A friend of mine works in AI professionally. He told me months ago that it is basically just all a scam and hype to garner investment money. He said the technology and paradigm itself will never lend toward AGI or anything like that. He sent me all these articles geared toward that end as well. https://garymarcus.substack.com/p/seven-replies-to-the-viral... https://substack.com/@cattelainf/note/c-135021342 https://arxiv.org/abs/2002.06177 https://garymarcus.substack.com/p/the-ai-2027-scenario-how-r... https://garymarcus.substack.com/p/25-ai-predictions-for-2025...
> This week I saw not one but three striking premonitions for how the scaling maximalism hypothesis might end. > 1. There might not be enough data in the world to make scaling maximalism work. https://arxiv.org/pdf/2209.12407.pdf > 2. There might not be enough available compute in the world to make scaling maximalism feasible. > 3. Some important tasks might simply not scale. https://arxiv.org/abs/2002.06177 Note: scaling maximalism hypothesis -> "that AGI might arise from larger scale without fundamental new innovation"
Keep following the thread.
More in PsychologyMore from the same topics.
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Understanding deep learning requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
Talent vs Luck: the role of randomness in success and failure
A. Pluchino. A. E. Biondo, A. Rapisarda
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.