Adversarial Policies Beat Superhuman Go AIs

The commenters here saying "humans cannot win anymore", "there is no chance that humans can beat the best Go AI anymore" are apparently unaware that this is no longer true. See https://arxiv.org/abs/2211.00241 and https://goattack.far.ai/ The best Go programs have a flaw that allows a good, but not championship-level, human to defeat them, by creating a group that encircles another group, which apparently confuses the AI's method of counting "liberties", which determine whether the group lives or not. Some appear to dismiss this as just a "trick", but it seems to me to point to a more fundamental deficiency in the architecture or training method.

Maybe we can extrapolate Godel where a complete ruleset cannot be consistent, and a consistent ruleset cannot be complete. Somewhere in there is the human ingenuity to adapt certain pathological patterns [1] that defeat (current) AI. But it's not always obvious, and the composer sapiens must defy their own understanding of the rules to create a purposeful deviation (sometimes with help from other AI) [2]. The idea of sharing GPT prompts promotes the implication that results are deterministic. That helps QA, but humans love to play. [1] Adversarial Policies Defeat Superhuman Go AIs: https://arxiv.org/abs/2211.00241 [2] Kellin Pelrine

Notably https://arxiv.org/abs/2211.00241 > We attack the state-of-the-art Go-playing AI system, KataGo, by training adversarial policies that play against frozen KataGo victims. Our attack achieves a >99% win rate when KataGo uses no tree-search, and a >77% win rate when KataGo uses enough search to be superhuman. Notably, our adversaries do not win by learning to play Go better than KataGo -- in fact, our adversaries are easily beaten by human amateurs. Instead, our adversaries win by tricking KataGo into making serious blunders. Our results demonstrate that even superhuman AI systems may harbor surprising failure modes

Congrats! I see by comparing the author list of v1 and V2 of this paper[1] that a Go player was consulted before making claims about defeating a Go AI. This is indeed a nontrivial exploit that passes muster with a Go player who was skeptical last time around [2]! [1] https://arxiv.org/abs/2211.00241v2 [2] https://news.ycombinator.com/item?id=33449600#33450069