Dual use of artificial-intelligence-powered drug discovery

and can only be undertaken by specialists in well-equipped, well-funded labs that are typically under the purview of an academic institution. You can't get access to dangerous chemical precursors or biological samples without credentials. Synthesis or development is hard, even if you have the plans. I'll only be worried if AI enables construction of these elements at sufficient yields without access to restricted chemical precursors or biological agents. If you can do this in your kitchen without BSL facilities, -80 freezers, etc., then we live in a truly "vulnerable world". That, however, remains to be seen. > 2. Bing chat threatening it's users almost immediately Have you heard of school bullies, gamer culture, or 4chan? Most humans have already been exposed to this negative behavior from peers, and it should only be a matter of time for products to move past this. > 3. All the stuff the red teams found so OpenAI could prevent them before chatGPT went live If anything, the complaints I hear most often is that they've neutered the behavior of their model. > Conversely you don't need an AGI to exist for some idiot to be convinced of a bad idea that gets nukes launched anyway Exactly. Humans are more liable to kill us than AGI at this point in time. We can reevaluate the risks when and if AGI actually comes about.

Ineffective way to kill off humans, but will annoy us, and will mess with power grid and logistics that it would benefit from. Conversely you don't need an AGI to exist for some idiot to be convinced of a bad idea that gets nukes launched anyway — Thule early warning radar incident is my favourite example of this, as it was caused by someone forgetting that the Moon doesn't have an IFF transponder and that's fine.

banned): Urbina, Fabio, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins. “Dual Use of Artificial-Intelligence-Powered Drug Discovery.” Nature Machine Intelligence 4, no. 3 (March 2022): 189–91. https://doi.org/10.1038/s42256-022-00465-9. Bilika, Domna, Nikoletta Michopoulou, Efthimios Alepis, and Constantinos Patsakis. “Hello Me, Meet the Real Me: Audio Deepfake Attacks on Voice Assistants.” arXiv, February 20, 2023. http://arxiv.org/abs/2302.10328 Mirsky, Yisroel, Ambra Demontis, Jaidip Kotak, Ram Shankar, Deng Gelei, Liu Yang, Xiangyu Zhang, Wenke Lee, Yuval Elovici, and Battista Biggio. “The Threat of Offensive AI to Organizations.” arXiv, June 29, 2021. http://arxiv.org/abs/2106.15764. I don't think most people have thought through all the ways perfect text, image, voice, and soon video generation/replication will upend society, or all the ways that the LLMs will be abused... As for AGI xrisk. I've done some reading, and since we don't know the limits of the current AI paradigm, and we don't know how to actually align an AGI, I think now is a perfectly cromulent time to be thinking about it. Based on my reading, I think the people ringing alarm bells are right to be worried. I don't think anyone giving this serious thought is being mendacious. Bowman, Samuel R. "Eight Things to Know about Large Language Models." arXiv preprint arXiv:2304.00612 (2023). https://arxiv.org/abs/2304.00612. Ngo, Richard, Lawrence Chan, and Sören Mindermann. “The Alignment Problem from a Deep Learning Perspective.” arXiv, February 22, 2023. http://arxiv.org/abs/2209.00626. Carlsmith, Joseph. “Is Power-Seeking AI an Existential Risk?” arXiv, June 16, 2022. http://arxiv.org/abs/2206.13353. I think Ian Hogarth's recent FT article https://archive.is/NdrNo is the best summary of where we are why we might be in trouble, for those that don't care for arXiv papers.

https://www.nature.com/articles/d41586-019-03228-6) * and many many, other examples There's a real sense of utopianism and not much consideration of misuse even on a very short timescale. Even if you don't think AI will become an existential risk in the future, there are enough problems caused by misuse of current systems that it warrants attention. All of the money I've seen flowing in the AI safety EA space has been put to extremely good use - $30k to a YouTuber making videos on AI safety who introduced me to the field and is my go-to for explaining topics like alignment (what you want vs what you say you want), grants to extremely productive AI safety researchers, grants to fund educational bootcamps and scholarships, etc. (Disclaimer: I'm an EA aligned person myself so I'm fairly biased!)