Evaluating Large Language Models Trained on Code

instructgpt/rlhf) https://arxiv.org/pdf/1707.06347.pdf). - 2018 lead author on GPT1 https://www.mikecaptain.com/resources/pdf/GPT-1.pdf - 2019 lead author on GPT2 https://insightcivic.s3.us-east-1.amazonaws.com/language-mod... blog https://openai.com/research/better-language-models - 2019 coauthor on RLHF https://arxiv.org/pdf/1909.08593.pdf) - 2020 coauthor on gpt3 https://arxiv.org/abs/2005.14165 - 2020 coauthor on scaling laws https://arxiv.org/pdf/2001.08361.pdf%E4%B8%AD%E5%BE%97%E5%88... - 2021 coauthor on DallE http://proceedings.mlr.press/v139/ramesh21a/ramesh21a.pdf - 2021 coauthor on Codex https://arxiv.org/pdf/2107.03374.pdf?trk=public_post_comment... - 2023 lead author on Whisper https://proceedings.mlr.press/v202/radford23a/radford23a.pdf name a more successful 7 year CS career post undergrad... Update: FYI openai just announced a "developer day" in Nov - somehow not blessed by the HN gods https://news.ycombinator.com/item?id=37408234

rock or just close to believe people are lying about how they’ve used it? Oh, absolutely. People are absolutely lying to themselves. OpenAI's and DeepMind's systematic testing of their code-geneator LLMs make it very clear that those systems produce incorrect code the majority of the time. The best results reported are 28.8% for Codex (in perpetual preprint: https://arxiv.org/abs/2107.03374) and 29.6% for AlphaCode (preprint: https://arxiv.org/abs/2203.07814 Science, paywalled: https://www.science.org/doi/10.1126/science.abq1158) and the latter is with their special 10@k metric which basically means the LLM gets 10 guesses. I have definitely observed people convince themselves online that CoPilot or ChatGPT even is helping them "improve their productivity" or some such. It is obvious that they are fooling themselves, badly. If you push them, they immediately say "oh yeah, it makes mistakes, but I can correct them" etc. So they just feel like it's useful, even when it's just making them do more work. In fact, that's exactly like homeopathic self-delusions: people use it because it makes them feel better, not because it has any measurable benefit. Btw: >> ... you’re either living under a rock ... Stop being a jerk.

image identification, speech recognition, etc.) so why is source code any different? The idea that "it's too much data" is absolutely a cop-out and absurd, especially for a company sitting on ~$100B in cash reserves. Your argument kind of demonstrates the underlying point here: They took the cheapest/easiest option and it's harmed the product. > A crawler simply believes that licenses, which are legally binding statements, are made by actual owners, rather than being fraud. It does seem reasonable to address the issue with takedowns, however. Yes, and to reiterate, they chose this method. They were not obligated to do this, they were not forced to pick this way of doing things, and given the complete lack of transparency it's a large leap of faith to assume that their training data simply looked at LICENSE files to determine which licenses were present. For what it's worth, it doesn't seem that that's what OpenAI did when they trained the model initially in their paper[1]: Our training dataset was collected in May 2020 from 54 mil- lion public software repositories hosted on GitHub, contain- ing 179 GB of unique Python files under 1 MB. We filtered out files which were likely auto-generated, had average line length greater than 100, had maximum line length greater than 1000, or contained a small percentage of alphanumeric characters. After filtering, our final dataset totaled 159 GB. I have not seen anything concrete about any further training after that, largely because it isn't transparent. [1]: https://arxiv.org/pdf/2107.03374.pdf

for 6B is very clearly a strength of GPT-J even above the 275B GPT-3. Note on that: ~8% of GPT-J training data is GitHub code, that's not the case for GPT-3 hence the difference. But OpenAI has a separate model avaiable in their API called Codex that is specifically tailored for code generation (also the model behind GH copilot) and that is much much better than GPT-J: Even the 300M params version of Codex outperforms it [2], and the API gives you access to a 12B version. I'm not trying to sell you OpenAI's API though, it has indeed pretty severe limitations, I'm only saying there are real reasons that people might want to use it contrary to what the comment I was replying to was saying, and just replicating what they do isn't exactly a walk in the park. [1]: https://arankomatsuzaki.wordpress.com/2021/06/04/gpt-j/ [2]: https://arxiv.org/pdf/2107.03374.pdf

software interview questions. > It is important for these tasks to be hand-written, since our models are trained on a large fraction of GitHub, which already contains solutions to problems from a variety of sources. For example, there are more than ten public repositories containing solutions to Codeforces problems, which make up part of the recently proposed APPS dataset So we take from this that the problems are simple and leet code style and that they have verified that the data is not in the training set by the simple nature of simply writing the code from scratch. If you aren't laughing now, you should be. So let's look and see if there are in fact samples of code that are exact or near to those in the test set that exist in public githubs prior to May 2020, their cutoff date. Now let's look at some of the test questions and see if we can find them on github. Github search is total garbage so I'm going to pull results from the last time I looked (search my comment history "godelski human eval") I apologize in advance for formatting. HumanEval/4: Prompt: from typing import List def mean_absolute_deviation(numbers: List[float]) -> float: """ For a given list of input numbers, calculate Mean Absolute Deviation around the mean of this dataset. Mean Absolute Deviation is the average absolute difference between each element and a centerpoint (mean in this case): MAD = average | x - x_mean | >>> mean_absolute_deviation([1.0, 2.0, 3.0, 4.0]) 1.0 """ canonical_solution: mean = sum(numbers) / len(numbers) return sum(abs(x - mean) for x in numbers) / len(numbers) Found on Github[2], commit date Oct 5, 2019: if reduction == "median": return np.median(scores) mean = sum(scores) / len(scores) if reduction == "mean": return mean return sum(abs(x - mean) for x in scores) / len(scores) A solution that is functionally equivalent. Swap numbers and scores and remove the if statement. This constitutes a near collision and ML models will preform very well on near collisions. If you look at the testing method for the evaluation you will also see that this code will pass the test. Thus our LLM can very easily simply copy paste this code and pass no problem. I'm not saying that's what happened, but that we cannot rule this out. What actually happened is an open question and we're far from ready as a community to call LLMs fuzzy copy machines. I also have this search query marked which still seems to be working[3] but you'll have to manually check the date. You can repeat this process for many examples in the HumanEval dataset. Or simply look the human eval dataset questions and answers and ask yourself "Have I written those exact lines of code?" The answer is probably. But note here, that overfitting is perfectly okay in certain circumstances. But HumanEval simply measures how good a LLM is at solving short leetcode style questions. It does not measure a LLMs ability to write code and certainly not write non-leetcode. It may very well do so, but this benchmark does not measure such things. This still can provide utility to people and these LLMs still learn a lot more than what HumanEval tests. My issue is with the metric and claims as to what the results indicate rather than the product itself. There is also the danger of chasing benchmarks like these as you will not be able to disentangle overfitting from desired training outcomes. I am not critiquing OP's network nor the work they did to create this. I'll explicitly state here, well done OP. This took a lot of hard work and you should feel very proud. I hope this question and context does not come off as pejorative nor overly cynical. I think your work is without a doubt, something to be proud of and useful to our community. This is a warning to all HN readers to help avoid snakeoil (I expect every ML person to already know this), scrutinize your metrics and know exactly what they measure. I mean precisely, there are no metrics that measure abstract things like "image quality", "performance in language", "code generation performance" and so on. Generative models are exceptionally difficult to determine what model is better and we are unfortunately at a point where many of our metrics (remember: metrics are proxies or more abstract goals. Metrics are models. All models are wrong, just some are more wrong than others) and you must do far more investigation to come to an even fuzzy answer to this question. Nuance is necessary. [0] https://huggingface.co/datasets/openai_humaneval [1]https://arxiv.org/abs/2107.03374 [2] https://github.com/danielwatson6/hate-speech-project/blob/8e... [3] https://github.com/search?q=abs%28x+-+mean%29+for+language%3...

closer?[3] There's plenty more examples[4] (does anyone know how actually limit the date to prior to 2021? `pushed:<2021` doesn't work nor does using the `created` keyword. Date searching doesn't seem to work well). In essence, we can still use this evaluation method to determine how good our model is at doing fuzzy searching. Which, mind you, is still a useful thing. But I would be careful in concluding that this means the model is good at generalizing arbitrary descriptions of code or novel pieces of code. That said, one may be able to argue that not many lines of code are actually that novel. Still, we need to be careful about our conclusions and understand the limitations of our metrics (something I am currently deeply troubled by) [0] https://arxiv.org/abs/2107.03374 [1] https://github.com/openai/code-align-evals-data/blob/97446d9... [2] https://github.com/bertomartin/stat4701/blob/ec2b64f629cbbf6... [3] https://github.com/danielwatson6/hate-speech-project/blob/64... [4] https://github.com/search?q=abs%28x+-+mean%29+for+language%3...

model to generate code by completing an initial "prompt" and have no way to control what kind of code is generated or whether it does what the programmer expects it to do. The task of checking correctness is left to the user. The result is code that may or may not contain bugs to an extent difficult to predict beforehand. Specifically about Copilot, it's based on Codex, a large language model that's a variant of GPT-3 fine-tuned on all of github. In formal testing, OpenAI found that Codex could generate the correct program about 30% of the time, which is pretty low: Evaluating Large Language Models Trained on Code https://arxiv.org/abs/2107.03374 Interestingly, if Codex was allowed 100 "guesses", the correct program was in there 75% of the time, instead. But that only goes to show that these kinds of systems are code firehoses without any ability to tell a correct program from garbage.

deductive program synthesis from a complete specification in natural lanaguage. I suspect the true contribution of the work is the demonstration of using natural language as a complete specification, where earlier work generally only demonstrated the use of natural language as incomplete specification (for example, comments describing intent rather than implementation) and the combination of natural language with code; as in the original Codex work [Edit: actually, now that I look again, the codex paper also has examples of comments that fully specify the target program, e.g. in Figure 2: https://arxiv.org/abs/2107.03374; so the work above is typically incremental]. On the other hand it's clear to me that the training has made the model memorise answers and all the work in prompt engineering, described under "Workflow" serves to find the right prompts to retrieve the desired memorisations, much like one must fire just the right SQL query to get back the right data. Certainly interesting to see in action and useful for everyday work, but far from "solving" anything in the gradniose way that it is announced by the authors (e.g. "These astounding results..." in section "Conclusion", etc).

other than to select the answer with the highest probability according to its training set which usually means the most common answer in its training set. So the probability that the answer you get is the "right" answer depends on the probability that the right anwser is the most common answer to the problem you give it. If that makes sense? __________ [1] https://arxiv.org/abs/2107.03374 See Figure 1.