Get the latest tech news
Grok 3: Another win for the bitter lesson
Congratulations to the xAI team—and the advocates of the scaling laws
): The game changed when companies realized that simply making models larger was yielding diminishing returns (the press was quick to misreport this as “scale is over” so I urge you to watch this talk by Ilya Sutskever at NeurIPS 2024 in December). Reinforcement learning combined with supervised fine-tuning proved to be highly effective —especially in structured domains like math and coding, where well-defined, verifiable reward functions exist. DeepSeek and xAI, in contrast, stood on the shoulders of those giants, leveraging the lessons painstakingly learned from their early efforts, and benefiting from the sheer luck of building their models at a moment when a paradigm shift made faster, more cost-effective progress possible (post-training era).
Or read this on Hacker News
