Ken Thompson
1984 · Communications of the ACM
To what extent should one trust a statement that a program is free of Trojan horses? Perhaps it is more important to trust the people who wrote the software.
Metadata: Crossref. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
20 selected excerpts · 23 total mentions
My immediate tangential thought is about Ken Thompson's paper, "Reflections on Trusting Trust". The gist is that our security issues could come layers away from where we may expect them to, all the way up to the compiler. It's a great paper, but who would have expected anything less from Ken Thompson. https://dl.acm.org/doi/10.1145/358198.358210
> I don't think good code has ever had an honest measurement of quality. Bad workmanship writes bad code in any language. Typically the "easier" the compiler is to use... the more complex the failure mode. =3 This list of rules applies to most languages: https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Dev... A bit of history where these rules came from, and why they matter. =3 "Why Fighter Jets Ban 90% of C++ Features" https://www.youtube.com/watch?v=Gv4sDL9Ljww The basis of the cited 10 rules helped form many standards for reliable code: https://www.stroustrup.com/JSF-AV-rules.pdf "Reflections on trusting trust" (Ken Thompson, 1984) https://dl.acm.org/doi/10.1145/358198.358210
I think one of the reasons that we sometimes think about different ways of compiler bootstrapping is that someone decided to spend their Turing Award lecture on the topic. Ken Thompson, "Reflections on Trusting Trust". https://dl.acm.org/doi/10.1145/358198.358210
With all due respect, I think that's a terrible idea for a couple reasons. First, a model that produces binary code would be impossible to vet. This is already an issue with "trusting trust" in compilers [1], and would be made worse in the case of companies that constantly tweak their black-box models however they like. At least my compiler can work offline. But then there's the idea of programming as "turn data into data" that forgets that software has to interface with humans too. If I ask my LLM to give me sentencing guidelines that are "fair" given a set of parameters, no one on Earth should trust a binary they cannot check themselves. In fact, those affected may even have a constitutional right to source code. [1] https://dl.acm.org/doi/10.1145/358198.358210
Can anyone explain to me why these two issues aren't considered deal breakers for introducing Rust into the kernel? 1. It doesn't map almost 1:1 to assembly the way C does, so it's not inherently clear if the code will necessarily do what it says it does. That seems questionable for something as important as a kernel and driver. 2. Only one real Rust compiler, that's a recursive compiler, which reminds me of the Trusting Trust problem: https://dl.acm.org/doi/abs/10.1145/358198.358210
Even open HW/SW requirements / standards don't protect against this. See Thompson's "Reflections on Trusting Trust": <https://dl.acm.org/doi/10.1145/358198.358210>. More recently, and very much part of the extant cybersecurity landscape are zero-day and backdoor vulnerabilities which are prevalent in both closed and open-source software.[1] And the typical end-use buyer is not equipped to audit code or even interpret audit assurances (e.g., some level of certification on a device). That might work for large-scale organisational buyers (corporations, national governments, major NGOs, militaries), but would challenge even many of these. TFA goes into depth on how HW…
characteristics already vary tremendously even within product lines and production runs, and firmware modifications could weaponise otherwise benign-seeming hardware. Open SW/HW might help, somewhat, but not against a motivated state-level or other advanced actor (e.g., major narco syndicate, independent militant organisations, and the like). The requirement is far from sufficient however. ________________________________ Notes: 1. I'm trying hard to not open that particular religious war, though my bias is that evidence tends to suggest F/OSS is more secure than closed source in general.
https://dl.acm.org/doi/10.1145/358198.358210 I don't really trust many DNSes and neither do many yet we all have few choices The lack of MitM isn't much comfort Neither are guarantees of the chain of trust
That is called the Ken Thompson hack In 1984, he published a paper called: Reflections on trusting trust https://dl.acm.org/doi/10.1145/358198.358210
The mechanism is a clever application of quines (self-reproducing programs), first explained in the classic lecture “Reflections on Trusting Trust” by Ken Thompson: https://dl.acm.org/doi/pdf/10.1145/358198.358210 Russ Cox obtained the actual code for Thompson’s compiler backdoor and presented it here: https://research.swtch.com/nih
From the ACM paper [0]: > Acknowledgment. I first read of the possibility of such a Trojan horse in an Air Force critique [4] of the security of an early implementation of Multics. I cannot find a more specific reference to this document. I would appreciate it if anyone who can supply this reference would let me know. ofc the US Gov was behind this. Incredible. [0]: https://dl.acm.org/doi/pdf/10.1145/358198.358210
From a security viewpoint; i wouldn't trust that code not to have embedded AI seeds, anyway. "Reflections on trusting trust" https://dl.acm.org/doi/10.1145/358198.358210
Funny, I thought about this today when installing Bun^. I read the bash script, saw that it was indeed downloading a release binary from the official GitHub repo, then gave up. The binary it downloaded could do absolutely anything when I run it, am I supposed to then go and verify every line of the source, and all its dependencies? Then verify the release package is the same as the one I make building it myself? Then verify the compiler source is a reasonable interpretation of the Zig spec (which doesn't seem to exist)? Then verify the complier binary I used is the same as the one created by a trusted compilers? Then verify all those binaries.... End of the day: the…
package is from a somewhat reputable source, and you don't keep life changing secrets user-accessible on your dev machine, fuck it. Also, resist installing as root. Figure out what the real permissions the thing needs are and go from there. ^https://bun.sh/ (if you haven't been introduced: https://dl.acm.org/doi/pdf/10.1145/358198.358210)
> It uses a homebrew encryption scheme Please stop with that. At some point every encryption scheme was "homebrew" (or rather, new). MTProto 1.0 had flaws, which were addressed in 2.0. The latter has been independently formally verified to be secure[0]. > Telegram is the least secure. Against who? What is your threat model? Such absolutist blanket statements are useless by design. > of all the major messengers Which are Messenger, Whatsapp, and SMS/RCS around here. Signal shows up from time to time, and Matrix doesn't even register above statistical noise so I won't count them as "major" (Signal, being, in a stroke of optimism, at best a challenger) Of these, SMS/RCS is a…
total clusterfuck, and the remainder is owned by Meta. Yes, Meta is high on my list of adversaries, and given their track record should probably be for anyone out there caring about their privacy. No, WhatsApp's E2E is not to be trusted[1][2]. So is Telegram the least secure of major messengers? definitely not. Is Telegram perfect security, certainly not either, because that doesn't exist[6], but their E2E is solid, even without E2E, extra steps are taken to thwart certain adversaries[3][4], they have reproducible builds for their client binary builds[5], and as can be observed so far their actions are veering on the complete opposite side of Meta's. Is Telegram an opportunity to convince non- {privacy,security}-minded random joes and janes to jump out of a bunch of terrible chat platform for a better (or less worse if you want) chat platform? I'd say probably. I mean, it really really looks like they're trying hard to get the job done. [0]: https://github.com/miculan/telegram-mtproto2-verification [1]: https://twitter.com/Shiftreduce/status/1347546599384346624 [2]: Meta controls the Es in this "E2E", and I won't trust that, ever. [3]: https://telegram.org/faq#q-do-you-process-data-requests [4]: https://security.stackexchange.com/questions/238562/how-does... [5]: https://core.telegram.org/reproducible-builds [6]: https://dl.acm.org/doi/10.1145/358198.358210
We could learn to write auditably correct software. Reflections on Trusting Trust explains why writing correct software is inadequate: https://dl.acm.org/doi/10.1145/358198.358210
"Reflections on Trusting Trust"[1] (Ken Thompson, 1984) is up there - it's a brief read, with high information density, and fairly profound implications. [1] - https://dl.acm.org/doi/pdf/10.1145/358198.358210
Put it into the compiler so it re-inserts itself when it sees that the compiler is being recompiled... Reflections on Trusting Trust: https://dl.acm.org/doi/pdf/10.1145/358198.358210
But it's similar to using a compiler, no? I almost never compile the compiler I use, so I'm implicitly trusting that the compiler actually spits out what I expect and not some kind of backdoor[1]. [1]: https://dl.acm.org/doi/10.1145/358198.358210
"I am a programmer. On my 1040 form, that is what I put down as my occupation. As a programmer, I write programs." -- Ken Thompson, in his 1983 Turing Award lecture, "Reflections on Trusting Trust". https://dl.acm.org/doi/10.1145/358198.358210
Talking of "honor" and "security" in the same breath reminds me of Ken Thompsons seminal Turing Award Lecture "Reflections on Trusting Trust" [1] [1] https://dl.acm.org/doi/10.1145/358198.358210
Unless you have nation-state level resources I don't think that's a question you can answer for certain. You can inspect the source code, but then you have to trust the compiler and underlying hardware as well (a la "Trusting Trust": https://dl.acm.org/doi/10.1145/358198.358210)
More from the same topics.
Dan Tong, David J. Farnham, Lei Duan +4
Abstract If future net-zero emissions energy systems rely heavily on solar and wind resources, spatial and temporal mismatches between resource availability and electricity demand may challenge system reliability.
Steven M. Paul, Daniel S. Mytelka, Christopher T. Dunwiddie +4