Get the latest tech news
Did xAI lie about Grok 3’s benchmarks?
OpenAI researchers accused xAI about publishing misleading Grok 3 benchmarks. The truth is a little more nuanced.
In a post on xAI’s blog, the company published a graph showing Grok 3’s performance on AIME 2025, a collection of challenging math questions from a recent invitational mathematics exam. But OpenAI employees on X were quick to point out that xAI’s graph didn’t include o3-mini-high’s AIME 2025 score at “cons@64.” But as AI researcher Nathan Lambert pointed out in a post, perhaps the most important metric remains a mystery: the computational (and monetary) cost it took for each model to achieve its best score.
Or read this on TechCrunch