People cannot distinguish GPT-4 from a human in a Turing test

singural experiment protocol, but it's a well enough defined experimental scenario which for over half a century, it was kept as the benchmark of recognition of artificial intelligence, not by laymen (lol) but by major figures in AI research as well, figures like Minsky, McCarthy and others engaged with it. That researchers haven't done Turing-test studies (taking the setup from turing and even called them that) is patently false. Including openly testing LLMs: https://aclanthology.org/2024.naacl-long.290/ https://www.pnas.org/doi/10.1073/pnas.2313925121 https://arxiv.org/pdf/2503.23674 https://arxiv.org/pdf/2407.08853 https://arxiv.org/abs/2405.08007 https://www.sciencedirect.com/science/article/pii/S295016282...

students, not something that can act "disturbingly capable of emotions" or fake it. If you can't tell, does it matter? Students have been learning from increasingly capable simulacra of human intelligence and emotion since no later than the invention of writing, possibly even earlier with art. LLMs are good enough to be interesting, and good enough to fool a lot of people a lot of the time, even if last year's models weren't passing Turing test scenarios where people were actively looking for them. (But also, in those tests, humans rated other humans as human only 2/3rds of the time: https://arxiv.org/html/2405.08007v1). Although the newer models are generally better, I don't want to claim if they're specifically, at this task, better than the old models — still, that's how close they were on the only formal test I know of, rather than the mere vibes of listening to them synthesising a voice and me being surprised they'll um and err now. > AGI is nowhere near this (and there's no good definition of what AGI is, anyway). I made the parenthetical point myself. LLMs, however, are absolutely there. Think of it this way: neither one of us can tell by the other's words that the other is not an LLM, all we can do is trust because we're on a forum where it would be rude to use one. > whether the sociopath can pass knowledge well and can fake as if they had normal human emotions won't sway my opinion. Sure, but you also wouldn't be able to determine if you're looking at a sociopath or someone with normal human emotions. > Real human connection is what makes us human. Para-social relationships say otherwise. Fictional characters, let alone the actors portraying them. The electoral popularity of sociopaths in democracies. Shallow demagogues. Art from generations long-dead. These things give the illusion of a real connection — to the extent there's any connection, it's all one-directional. > AIs are not human by definition. Therefore, AIs cannot replace humans in those situations where human connection matters. (Parenthetically: would an uploaded human mind, count as human? I don't know). But it doesn't matter if they're human or not. Dogs and cats often provide a similar connection (both growing up as a companion, and a being to look after as an adult, though the dog owners I know really don't like anyone treating dogs as "fur babies"), as do TV shows: to the extent that the "connection" part matters, it doesn't need to be human; to the extent that the "human" part matters, it doesn't have to be real. > And we shouldn't want them to either, because otherwise what's the point of existence? Asimov's fictional planet of Solaria was a dystopia, not something to aspire to. The trouble with fiction is that it's as much of a shallow echo as current LLMs themselves: yes, it can produce an emotional affect, but for every Asimovian Solaria there's a Banks' Culture where the AI have us thrive. Both are unsound examples upon which to base actual expectations: even if we happen to develop down one or other of those paths, they are written to be interesting not plausible, and the similarity is mere coincidence. > [1] or the pop culture understanding of them, anyway. Let's not get caught up in definitions. I tend to agree, though I feel there's an irony here somewhere — shared human connection on the basis of common understanding of a shared meme about the mental processes of a sub-group of humans that's being Othered in the same way as LLMs… but I can't put it into words. And it would be super ironic to ask an LLM to try to do so for me, so I won't. -- Don't mistake the map for the territory; but also don't forget you can learn the territory from the map. Any given AI may be a mere simulacra, a P-zombie mimicking our inner world without having one of its own, but that doesn't mean the model isn't good enough to learn from.

the linked site repeats it. I'm complaining that the paper changed the game so it's no longer a binary choice, but continued to treat the 50% threshold as significant. Do you see why that's wrong? Or do you disagree? I'm not saying that any change to Turing's formulation would be bad, just that the paper's variation is specifically bad. It would be bad in isolation too, but I believe the reason for their confusion is most understandable with reference to Turing's original formulation. If you haven't read that paper then we're probably talking past each other. The link to that one also looks broken in the site, but it's also in the source, https://arxiv.org/pdf/2405.08007