Get the latest tech news
Update on Reflection-70B
Reproducing Reflection-70B benchmark scores and postmortem on what happened.
On September 5th, Matt Shumer announced a new fine tuned model called Reflection 70B, finetuned on top of Llama 3.1 70B, showing SoTA benchmark numbers which was trained by me on Glaive generated data. I am publishing this post mortem to clearly communicate how to reproduce the model benchmark scores, what has happened since the launch, answer questions asked by the open source AI community and take responsibility for mistakes I made. I also gave access to our GPU nodes, the railway account and git repo which was running the proxy, to a few members of the community, including the OpenRouter team.
Or read this on Hacker News