Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts

That feeling of not knowing what to do with it is an example of humans being stupid. We are all victims of being "Johnny" in this paper: https://dl.acm.org/doi/full/10.1145/3544548.3581388

I love this whole series on misconceptions! Our expectations here are very much set by human-human interactions: we expect memory, introspection, that saying approximately-the-same-thing will give us approximately-the-same-result, that instructions are better than examples, that politeness helps, and many more [1] -- and some of these expectations are so deeply rooted that even when we know, intellectually, that our expectations are off, it can be hard to modify our behavior. That said, it will be super interesting to see how people's expectations shift -- and how we bring new expectations from human-AI interactions back to human-human interactions. [1]: https://dl.acm.org/doi/pdf/10.1145/3544548.3581388 (open access link)

For raw text completion I agree with you that it's a bit discordant. IMO text completion prompts work better when you use more of a first-person, here-is-the-beginning-of-some-transcript style. The OpenAI chat completion endpoint encourages the second-person prompting you describe, so that could be why you see it a lot. My understanding is that a transformation is applied to the user input prompts before being fed to the underlying model, so it's possible that the model receives a more natural transcription-style prompt. You might be interested in this paper, which explores ways to help non-experts write prompts https://dl.acm.org/doi/abs/10.1145/3544548.3581388.

If you are someone who was working in LLMs/Generative AI before it got "cool", The market has fundamentally changed and changed extremely in your favor. Talent is extremely scarce and expensive right now in this space. Ask for well above FAANG compensation. Try to shop around for offers. I had 4 offers and a bidding war for my talent (with no leetcode - but I have an extensive github and publications about LLMs), when previously these same companies would have brutally leetcoded me. I thought that there would be a ton of people getting really good really fast with Generative AI. It turns out that most people are terrible at using the tools, and there's even research about this right now - https://dl.acm.org/doi/10.1145/3544548.3581388