'Anonymisation' schemes are a little like encryption, in that they just get monotonically weaker over time as people work out attacks. But the attacks tend to be much worse. I work in academic open data publishing, and the netflix prize (https://arxiv.org/abs/cs/0610105) hangs over our heads. But what this illustrates to me is that researchers are just really careless, despite everything we make them agree to in data transfer agreements. It seems absurd to have little cubicles like this https://safepodnetwork.ac.uk/ (think Mission Impossible 1) but I do despair.
How To Break Anonymity of the Netflix Prize Dataset
Arvind Narayanan, Vitaly Shmatikov
2006 · arXiv
- HN score
- 0
- Mentions
- 5
- Readers
- 5
- Months
- 4
From the abstract
We present a new class of statistical de-anonymization attacks against high-dimensional micro-data, such as individual preferences, recommendations, transaction records and so on. Our techniques are robust to perturbation in the data and tolerate some mistakes in the adversary's background knowledge. We apply our de-anonymization methodology to the Netflix Prize dataset, which contains anonymous movie ratings of 500,000 subscribers of Netflix, the world's largest online movie rental service. We demonstrate that an adversary who knows only a little bit about an individual subscriber can easily identify this subscriber's record in the dataset. Using the Internet Movie Database as the source of background knowledge, we successfully identified the Netflix records of known users, uncovering their apparent political preferences and other potentially sensitive information.
Metadata: arXiv. Abstracts retain their original authors’ rights.
In the conversation
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
5 selected excerpts · 5 total mentions
We should have learned this lesson 20 years ago when researchers were able to deanonymize a lot of the Netflix Prize dataset, which contained nothing except movie ratings and their associated dates. https://arxiv.org/abs/cs/0610105 If movie ratings are vulnerable to pattern-matching from noisy external sources, then it should be obvious that location data is enormously more vulnerable.
> I'm telling you that Firefox is going to be out of business soon Do you not think everyone saying Firefox is going to be out of business soon plays a role in this? Regardless, I think you've ignored the root of my argument. I'm not trying to be a Firefox fanboy here but it's not like there's many options. The playing field is Chrome, Firefox, Safari. So only one of these is not "big tech". > Metrics can be transmitted without revealing the user. This is well known. This is not well known and I think you've kinda "told on yourself" here. It is fairly well known in the privacy community that it is difficult to transmit user data without accidentally revealing other…
Read the full comment
information. Here's a rather famous example[0,1]. I'd encourage you to read it and think carefully about how deanonymization might be possible after just reading a description of the datasets they deanonymize. > You can't suggest anything. I am done with this conversation. If you wish to disengage then that is your choice. I am really trying to engage with you faithfully here. I'm not even really attacking Brave here, as my critique is over the Chromium ecosystem. I think if you look at my points again you can see how they would dramatically shift if Brave were based off of Gecko or Webkit. Honestly, I would be encouraging Brave usage were it under those umbrella. Or even better, if it had its own engine! Because my point is about monopolization. [0] https://courses.csail.mit.edu/6.857/2018/project/Archie-Gers... [1] https://arxiv.org/abs/cs/0610105
LOL. I'm not going to deanonymize datasets for an internet stranger. Certainly not for free, but also not at any hourly rate unless I know that the organization employing me either has the original non-anonymized dataset or at least have very strict internal controls about how deanonymized data will be handled. (I also haven't done this work in a LONG time, and there's now a whole lot of academic work on the topic that didn't exist back then, so there are probably much better consultants for legitimate organizations looking to hire for this work.) If you want proof, you can use google to find LOTS of papers along these lines analyzing real datasets. I think https://arxiv.org/pdf/cs/0610105.pdf is a fairly typical example.
You're perhaps right. Of course, these people are on things like Facebook, Twitter and blogs and if you can narrow down age, gender, location and medical condition, you might be able to correlate with public posts. You can also look for specific people in there if you have certain kinds of prior information about them: For example, Aravind Narayanan was able to de-anonymize some part of the Netflix set [http://arxiv.org/abs/cs/0610105]. Maybe that won't translate to this data.
Keep following the thread.
More in DatabasesMore from the same topics.
The Case for Learned Index Structures
Tim Kraska, Alex Beutel, Ed H. Chi +2
Indexes are models: a B-Tree-Index can be seen as a model to map a key to the position of a record within a sorted array, a Hash-Index as a model to map a key to a position of a record within an unsorted array, and a BitMap-Index as a model to indicate if a data record exists or not.
A Critique of the CAP Theorem
Martin Kleppmann
The CAP Theorem is a frequently cited impossibility result in distributed systems, especially among NoSQL distributed databases.
Dissolving the Fermi Paradox
Anders Sandberg, Eric Drexler, Toby Ord
The Fermi paradox is the conflict between an expectation of a high {\em ex ante} probability of intelligent life elsewhere in the universe and the apparently lifeless universe we in fact observe.
Understanding deep learning requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.