Get the latest tech news
Sakana AI’s CycleQD outperforms traditional fine-tuning methods for multi-skill language models
CycleQD merges skills of experts models in clever ways to create many new models with multiple skills, no fine-tuning required.
The mutation operation uses singular value decomposition(SVD), a factorization method that breaks down any matrix into simpler components, making it easier to understand and manipulate its elements. “These results clearly show that CycleQD outperforms traditional methods, proving its effectiveness in training LLMs to excel across multiple skills,” the researchers write. Another exciting direction is the development of multi-agent systems, where swarms of specialized agents evolved through CycleQD can collaborate, compete and learn from one another.
Or read this on Venture Beat