Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai
2016 · arXiv
The blind application of machine learning runs the risk of amplifying biases present in data. Such a danger is facing us with word embedding, a popular framework to represent text data as vectors which has been used in many machine learning and natural language processing tasks. We show that even word embeddings trained on Google News articles exhibit female/male gender stereotypes to a disturbing extent. This raises concerns because their widespread use, as we describe, often tends to amplify these biases. Geometrically, gender bias is first shown to be captured by a direction in the word embedding. Second, gender neutral words are shown to be linearly separable from gender definition words in the word embedding. Using these properties, we provide a methodology for modifying an embedding to remove gender stereotypes, such as the association between between the words receptionist and female, while maintaining desired associations such as between the words queen and female. We define metrics to quantify both direct and indirect gender biases in embeddings, and develop algorithms to "debias" the embedding. Using crowd-worker evaluation as well as standard benchmarks, we empirically demonstrate that our algorithms significantly reduce gender bias in embeddings while preserving the its useful properties such as the ability to cluster related concepts and to solve analogy tasks. The resulting embeddings can be used in applications without amplifying gender bias.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
8 selected excerpts · 8 total mentions
Evidence suggests the opposite: they would write a self-congratulatory blog post about it. For example the work Google does on "de-biasing AI" is all about taking ML models and warping its understanding of the world to reflect ideological priorities. https://arxiv.org/abs/1607.06520
Yeah, besides the fact that this compositionality is relatively unique to word2vec, research on the biases pre-trained models express is pretty available. Linked a few below for those interested. Most of the issues are down to the same phenomenon discussed here in the context of ImageNet, the input texts were biased and the algorithm learned said bias. [0] https://arxiv.org/abs/1607.06520 "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings " [1] http://proceedings.mlr.press/v97/brunet19a/brunet19a.pdf "Understanding the Origins of Bias in Word Embeddings" [2] http://matthewkenney.site/biases.html "Google word2vec biases"
> The authors justed picked the 10th example in the list without clearly stating it. That's not an accurate description what Bolukbasi et al (2016) [0] did. In particular, they do not list x close to lovely + he - she and then pick arbitrarily from that list. Instead, they explicitly reject that approach (see appendix A), because they're looking for pairs of words that are maximally gendered. They do that by finding x and y such that the angle between x - y and she - he is minimized. Since the task they're solving is different, you can't fault them for getting different results. [0] https://arxiv.org/abs/1607.06520
How do you plan to counter the harmful societal biases that embeddings embody? See Bolukbasi (https://arxiv.org/pdf/1607.06520.pdf) and Caliskan (http://science.sciencemag.org/content/356/6334/183) While these examples are solely language based, it is easy to imagine the transfer to other domains.
It's interesting and slightly uncomfortable that the illustration for similar words uses the word "man" as an example, given the gender biases that result from learning word vectors solely from word distributions. To explain, although I’m sure the author himself is familiar with the issue: For any word that is disproportionately associated with one gender in the corpus, the model will learn that gender difference as part of the representation of the word, pretty much baking it into all applications. [1] It's all fun and games when this helps you find the difference between "king" and "queen", but it becomes a problem when the same difference appears between "genius" and…
"beautiful". I haven't evaluated these vectors for built-in biases, but I assume they would have similar problems to the pre-computed word2vec and GloVe embeddings. (If they don't -- if quantization is a natural way to counteract bias -- then that's an awesome result! But an unlikely one.) To the author: I don’t mean this to sound like an accusation that you haven’t done this yet; I know that short papers can’t tell two stories at the same time. But the next step is pretty clear, right? How do these vectors measure on the Word Embedding Association Test for implicit bias? Does Bolukbasi’s de-biasing method, or an analogue of it, work on quantized embeddings? [1] Bolukbasi et al., "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings." https://arxiv.org/abs/1607.06520
You are only talking about supervised learning. And the bias people are probably talking about is bias in the data, not bias in the "input", as you put it. See http://arxiv.org/abs/1607.06520 for a counterexample to your point.
I consider it very important to fight bias in AI, but I fundamentally do not understand this article's take. The thing stopping AI researchers from obtaining unbiased training data is not that we're waiting for laywers to give us permission. The thing stopping us is that the right training data is already hard to find, and unbiased training data is the hardest of all to find because it doesn't exist. Google, for example, does not need permission. They can acquire and train on basically any data they want if they put their mind to it. And Google is completely shit at deploying unbiased AI. (They write great blog posts and presentations about it! Then they don't do it in their actual products.) You won't just find a naturally unbiased dataset. The way to fight AI bias is deliberately and artificially, like in Bolukbasi et al. [1] [1] https://arxiv.org/abs/1607.06520
I think the OP might have been looking for code that already exists, not just techniques for how to build it. But I have some opinions on these techniques. LDA is not very controllable. It gives you a set number of clusters. Run it again and you get different clusters. It can't give you documents that are similar to a particular document that you asked for, except to say that all the documents that happen to be in a cluster at the time are similar. There are indeed up-to-date things you can do with word vectors -- though I'm sad that the tutorials always point to word2vec or GloVe, as if it's 2014. That's ancient in machine learning terms, and we now know of flaws in…
their outputs, such as [1]. If you want downloadable, pre-computed word vectors, ConceptNet Numberbatch [2] (part of the ConceptNet project that I develop) is the best in class right now, and you can download it in the same format as these older systems. And if you don't believe me tooting my own horn, at least use something else that's been updated in the last year, such as NASARI, or maybe fastText's precomputed English vectors. Or you can just compare "bags of words", which is still good enough for many applications and readily available out of the box. I believe this is what you get with a "More Like This" plugin for a search engine such as Lucene, for example. [1] https://arxiv.org/abs/1607.06520 [2] https://github.com/commonsense/conceptnet-numberbatch
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
A. Pluchino. A. E. Biondo, A. Rapisarda
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.