Get the latest tech news
DeepCoder: An Open-Source 14B Coder at O3-Mini Level
Fully Open-Source 14B Coder at O3-mini Level Through a joint collaboration between the Agentica team and Together AI, we release DeepCoder-14B-Preview, a code reasoning model finetuned from Deepseek-R1-Distilled-Qwen-14B via distributed RL. It achieves an impressive 60.6% Pass@1 accuracy on LiveCodeBench (+8% improvement), matching the performance of o3-mini-2025-01-031 (Low) and o1-2024-12-17 with just 14B parameters.
This technique masks out truncated sequences during training so that models aren’t penalized for generating thoughtful but lengthy outputs that exceed the current context limit. ⛰️ Baby, there ain’t no mountain high enough.Ain’t no context long enough.— Inspired by Marvin Gaye & Tammi Terrell We evaluate Deepcoder-14B-Preview on various coding benchmarks, including LiveCodeBench (LCB), Codeforces, and HumanEval+—and AIME2024. Finally, reward function calculation can take a long time, especially for coding related tasks, which require running thousands of unit tests per RL iteration.
Or read this on Hacker News