Mihaly Barasz, Paul Christiano, Benja Fallenstein
Marcello Herreshoff, Patrick LaVictoire, Eliezer Yudkowsky
2014 · arXiv
We consider the one-shot Prisoner's Dilemma between algorithms with read-access to one anothers' source codes, and we use the modal logic of provability to build agents that can achieve mutual cooperation in a manner that is robust, in that cooperation does not require exact equality of the agents' source code, and unexploitable, meaning that such an agent never cooperates when its opponent defects. We construct a general framework for such "modal agents", and study their properties.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
5 selected excerpts · 5 total mentions
This is a relatively old manuscript, and I would recommend instead reading, or at least first reading: * http://intelligence.org/files/ProblemClassDominance.pdf * http://arxiv.org/abs/1401.5577 (The original TDT manuscript occurred via pleas for 'something, anything to be published' that were satisfied by cleaning up an unfinished book draft and publishing it as-was.)
With an optimal way of determining fair splitting of gains like Shapley value[0] you can cooperate or defect with a probability that maximizes other participants expected value when you act fairly. The ultimatum game is the simplest example; N dollars of prize to split, N/2 is fair, accept with probability M / (N /2) where M is what's offered to you; the opponents maximum expected value comes from offering N/2; trying to offer less (or more) results in expected value to them < N/2. Trust can be built out of clearly describing how you'll respond in your own best interests in ways that achieve fairness, e.g. assuming the other parties will understand the concept of fairness…
and also act to maximize their expected value given their knowledge of how you will act. If you want to solve logically harder problems like one-shot prisoners dilemma, there are preliminary theories for how that can be done by proving things about the other participants directly. It won't work for humans, but maybe artificial agents. https://arxiv.org/pdf/1401.5577 [0] https://en.wikipedia.org/wiki/Shapley_value
This is quite fun. From a very different starting point, the idea of Team Reasoning is another non-standard way of thinking about rationality, which also starts from the idea of "shared commitments": https://www.tandfonline.com/doi/abs/10.1080/1000200309853874... The paper underlying this discussion is here, by the way, and Eliezer Yudkowsky is an author: https://arxiv.org/abs/1401.5577
You keep hammering on with the same lazy slander. Yudkowsky was well known long before he wrote the popular Harry Potter fanfic, which incidentally is pedagogical / allegorical. Because his main roles are teacher and philosopher, and philosophers do that. Here's a sampling of his non-fiction writing: https://www.lesswrong.com/tag/sequences https://intelligence.org/files/EthicsofAI.pdf https://arxiv.org/abs/1710.05060 https://intelligence.org/files/AlignmentMachineLearning.pdf https://arxiv.org/abs/1401.5577 https://arxiv.org/abs/1401.5577 https://intelligence.org/files/Corrigibility.pdf https://intelligence.org/files/DefinabilityTruthDraft.pdf https://intelligence.org/files/IEM.pdf https://intelligence.org/files/TilingAgentsDraft.pdf But you'll just say there's nothing of value there, and it's somehow figuratively "fan fiction", because he didn't go to college, and he doesn't work much on ML, which is clearly the end-all of AI.
Read Eliezer himself, 4 years ago: https://www.reddit.com/r/xkcd/comments/2myg86/xkcd_1450_aibo... The "meat" of interest here for those too lazy to click (I encourage people click through, though, really, and maybe even take a glance at https://arxiv.org/abs/1401.5577) is probably this long parenthetical: > (But taking Roko's premises at face value, his idea would zap people as soon as they read it. Which - keeping in mind that at the time I had absolutely no idea this would all blow up the way it did - caused me to yell quite loudly at Roko for violating ethics given his own premises, I mean really, WTF? You're going to get everyone who reads your article tortured so…
that you can argue against an AI proposal? In the twisted alternate reality of RationalWiki, this became proof that I believed in Roko's Basilisk, since I yelled at the person who invented it without including twenty lines of disclaimers about what I didn't necessarily believe. And since I had no idea this would blow up that way at the time, I suppose you could even read the sentences I wrote that way, which I did not edit for hours first because I had no idea this was going to haunt me for years to come. And then, since Roko's Basilisk was a putatively a pure infohazard of no conceivable use or good to anyone, and since I didn't really want to deal with the argument, I deleted it from LessWrong which seemed to me like a perfectly good general procedure for dealing with putative pure infohazards that jerkwads were waving in people's faces. Which brought out the censorship!! trolls and was certainly, in retrospect, a mistake.) Edit: this plus more is all covered by the LW wiki's page someone else linked, check that out too if you really care. https://wiki.lesswrong.com/wiki/Roko%27s_basilisk
More from the same topics.
Haoran Xu, Fredrik Kjolstad
Fast compilation is important when compilation occurs at runtime, such as query compilers in modern database systems and WebAssembly virtual machines in modern browsers.
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics.
Donald E. Knuth
The author presents two tricks to accelerate depth-first search algorithms for a class of combinatorial puzzle problems, such as tiling a tray by a fixed set of polyominoes.
Jon Kleinberg, Sendhil Mullainathan, Manish Raghavan
Recent discussion in the public sphere about algorithmic classification has involved tension between competing notions of what it means for a probabilistic classification to be fair to different groups.