Building high-level features using large scale unsupervised learning

handing on a wall. This painting was recreated as a cartoon, and the clock was shown as as a silly cartoonish clock hanging on the same spot on the wall. It seems trivial now, but at that moment I realized these models are capable of something profound (i.e. they understood a visual concept of a clock). A couple of years later, GPT-2 came out and I've read the story it wrote about English speaking unicorns. That was when I realized human level intelligence is probably much simpler to achieve than we thought. That was when my research interests switched to NLP, and about a year later (early 2020), I was discussing chatbot progress with someone here on HN. We made a $100 bet where I claimed that an AI model will be able to pass Turing test - properly conducted - by the end of 2022. Properly conducted means a model would successfully pretend to be an adult educated native English speaker, during a 2 hour chat session. Many people back then expressed how ridiculously implausible that seemed. Today I admit I lost the bet and I would pay up if contacted by the guy. But, keep in mind, ChatGPT was released in November and if OpenAI instead of integrating all the ethical safeguards, tried to optimize it to pass a Turing test, it would have fooled a lot of people. I feel that GPT-4 could have fooled most people, had the passing of TT been its training objective. Despite all that, despite my familiarity as an ML researcher with GPT-3 and other competing LLMs, I did feel the things you mentioned, the mixture of awe and fear for what's coming, after I watched the OpenAI live stream, and especially after I asked GPT-4 the same questions I asked ChatGPT-3.5 just a couple of weeks ago - the questions I ask job candidates applying to an ML researcher position to test both depth and breadth of their ML knowledge. GPT-4 produced satisfactory answers where GPT-3.5 didn't. That same day I asked it to write code for me - not some toy example but the actual code I needed for work (complicated rounding schemes when quantizing an ML model to a specific number format) - and it did eventually produced correct working code. I've spent roughly an hour on prompt engineering and progressive clarifications/corrections of what I'm trying to do, but at the end it would have taken at least 3 hours to write code like that, maybe more. For me, it's not so much about the current abilities of GPT-4, which are obviously a big deal. It's about what's coming next, in the near future (1-3 years). We (ML researchers) have not reached the plateau with LLMs yet - they will get better. We have just started looking into video generation, which has a huge potential of building very good world models. Video/audio and language modeling abilities will reinforce and amplify each other. Most importantly, models trained to predict the next video frame will be able to act in the physical world. When GPT-5 or GPT-6 is released inside a robot, and can do non-trivial physical actions - that's when things get really interesting. This will happen faster than people expect (again), and I am willing to bet we will see intelligent humanoid robots, capable of performing most forms of physical human labor, by the end of 2025. The main reason this might not happen is if humans don't want it to happen (e.g. ethical safeguards built into GPT-4).

images with no help from external labels (human intervention)? They missed the core of the discovery by not understanding that. The deep learning method is an unsupervised way to process raw input and transform it into useable features. This used to be done by a combination of domain knowledge and supervised training, but they could build an automated way to extract relevant features from images. This opened the window for hope that one day neural networks will be easily applied to any new domain if there is sufficient raw data to build a deep network for it. In the past there was a need for a large investment in human based data labeling and how to extract the best features from raw data (also described as voodoo magic by the same researchers - it was hard, it was domain locked and expensive).