Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipeline

Except of course it's not true lol. Horses are smart critters, but they absolutely cannot do arithmetic no matter how much you train them. These things are not horses. How can anyone choose to remain so ignorant in the face of irrefutable evidence that they're wrong? https://arxiv.org/abs/2507.15855 It's as if a disease like COVID swept through the population, and every human's IQ dropped 10 to 15 points while our machines grew smarter to an even larger degree.

a) no, gemini 2.5 was shown to "win" gold w/o tools. - https://arxiv.org/html/2507.15855v1 b) reductionism isn't worth our time. Planning works in the real world, today. (try any agentic tool like cc/codex/whatever). And if you're set on the purist view, there's mounting evidence from anthropic that there is planning in the core of an LLM. c) so ... not true? Long context works today. This is simply moving goalposts and nothing more. X can't do Y -> well, here they are doing Y -> well, not like that.

It's reasonable to question it, but there's a fun Chinese paper (https://arxiv.org/abs/2507.15855) where they attempt to create a system of prompts that can be used with commercial LLMs to solve hard maths problems. It turns out that they can use the same prompt system for all of them, with no changes and still solve 5/6 IMO problems. I think this is possibly iffy, since people might have updated the models etc., but it's pretty obvious that this kind of thing is how OpenAI are doing their multi-stage thinking thing for maths internally. Consequently if prompt systems are this transferable for these hard problems, why wouldn't both they and individual prompts, be highly transferable in general?

I think you meant to link to Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR https://arxiv.org/abs/2509.02522 not Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipeline https://arxiv.org/abs/2507.15855

A terse prompt and expecting a one-shot answer is really not how you'd get an LLM to solve complex problems. I don't know what Deepmind and OpenAI did in this case, but to get an idea of the kind of scaffolding and prompting strategy that one might want, have a look at this paper where some floks used the normal generally available Gemini Pro 2.5 to solve 5/6 of the 2025 IMO problems: https://arxiv.org/pdf/2507.15855

Hypothesis: If you had ~1M dollar to burn, I think we should try setting up an AI agent to explore and try to invent new mathematics. It turns out agents can get an IMO gold with Gemini 2.5 Pro production model only. Therefore I suspect a swarm of agents burning through tokens like there's no tomorrow can invent new math. Reference: https://arxiv.org/abs/2507.15855 Alternative: If Gemini Deep Think or GPT5-Pro people are listening, I think they should give free access to their models with potential scaffolding (ie. agentic workflow) to say some ~100 researchers to see if any of them can prove new math with their technology.

It is possible to take this too far, though - consider the OpenAI IMO proofs[1], for instance, and compare them to Gemini's.[2] [1] https://github.com/aw31/openai-imo-2025-proofs [2] https://arxiv.org/pdf/2507.15855 Appendix A

IMO is not breakthrough, if you craft proper prompts you can excel imo with 2.5 Pro. Paper : https://arxiv.org/abs/2507.15855. Google just put whole computational power with very high quality data. It was test-time scaling. Why it didn't solve problem 6 as well? Yes, it was breakthrough but saturated quickly. Wait for next breakthrough. If they can build adapting weights in llm we can talk different things but test time scaling coming to end with increasing hallucination rate. No sign for AGI.

This paper [1] shows that gemini pro 2.5 without data-contamination and some minimal prompting (llm orchestration) can solve problems 1-5 on imo 2025. [1] https://arxiv.org/pdf/2507.15855 For problems with 2 page solutions, the search space of solutions is likely limited, and hence could be brute-forced by a search. Combinatorics is likely the hold-out since there are no set approaches one could take ie training data wont cover the space of techniques - which could explain why problem 6 stumped the llms. Also, in the future the minimal diligence IMO problem setters could do is to test against these LLMs to ensure that they cant solve them. Further, one could expect these tools to become available to contestents just like calculators are allowed in tests these days. Still, it is impressive for the LLMs to be able to craft watertight arguments in math at the IMO level without the use of provers.