Neutral mention · rule
Politeness have impact (https://arxiv.org/abs/2402.14531) so I wouldn't be too fast to make any kind of claim with a technology we don't know exactly how it works.
Back to the collection
Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance
Ziqi Yin, Hao Wang, Kaito Horio, Daisuke Kawahara, Satoshi Sekine
2024 · arXiv
- HN score
- 0
- Mentions
- 5
- Readers
- 5
- Months
- 4
From the abstract
We investigate the impact of politeness levels in prompts on the performance of large language models (LLMs). Polite language in human communications often garners more compliance and effectiveness, while rudeness can cause aversion, impacting response quality. We consider that LLMs mirror human communication traits, suggesting they align with human cultural norms. We assess the impact of politeness in prompts on LLMs across English, Chinese, and Japanese tasks. We observed that impolite prompts often result in poor performance, but overly polite language does not guarantee better outcomes. The best politeness level is different according to the language. This phenomenon suggests that LLMs not only reflect human behavior but are also influenced by language, particularly in different cultural contexts. Our findings highlight the need to factor in politeness for cross-cultural natural language processing and LLM usage.
Metadata: arXiv. Abstracts retain their original authors’ rights.
In the conversation
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
5 selected excerpts · 5 total mentions
Neutral mention · rule
And just like a real employee, this makes it work worse. (Old study, I wonder if it holds up on newer models? https://arxiv.org/pdf/2402.14531)
Neutral mention · rule
Depends on the model, but e.g. [1] found many models perform better if you are more polite. Though interestingly being rude can also sometimes improve performance at the cost of higher bias Intuitively it makes sense. The best sources tend to be either of moderately high politeness (professional language) or 4chan-like (rude, biased but honest) 1: https://arxiv.org/pdf/2402.14531
Neutral mention · rule
Good creators tend to treat their tools with respect, and I can't imagine any reason we shouldn't feel gratitude toward our tools after a particularly satisfying session. Also, there may be selfish reasons to do this as well: (1) "Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance" https://arxiv.org/abs/2402.14531 (2) "Three Things to Know About Prompting LLMs" https://sloanreview.mit.edu/article/three-things-to-know-abo...
Neutral mention · rule
Basically always worse in languages other than English. Not sure if it's just from volume of dataset or if it has to do with dataset quality, or the GPT architecture is inherently English-centric, but LLMs don't have like, a universal subconscious with superficial English frontend wrapping UG, like such that would support !(sapir-whorf). LLMs so far are kind of English-based thinking machine(if we were to recognize their apparent behavior as "thinking"). Below is just cherry-picked search results, selected largely by whether last few lines in the abstracts support my narrative, but I mean, it's a problem obvious enough that the rest of the world just knows. 0: "Not All…
Read the full comment
Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting": https://arxiv.org/abs/2305.07004 1: "Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries", https://arxiv.org/abs/2310.13132 2: "Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Test": https://arxiv.org/abs/2402.02135 3: "Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance": https://arxiv.org/abs/2402.14531 4: "Exploring Multilingual Human Value Concepts in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages?": https://arxiv.org/abs/2402.18120 5: "How do Large Language Models Handle Multilingualism?": https://arxiv.org/abs/2402.18815
Keep following the thread.
More in Language modelsMore from the same topics.
2023arXiv
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
60 mentions14 months
3.5HN score
2020arXiv
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
83 mentions35 months
3HN score
2018arXiv
Talent vs Luck: the role of randomness in success and failure
A. Pluchino. A. E. Biondo, A. Rapisarda
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.
16 mentions14 months
2.5HN score
2018arXiv
Categorizing Variants of Goodhart's Law
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics.
8 mentions7 months
2.5HN score