Get the latest tech news
Small Language Models Are the New Rage, Researchers Say
Larger models can pull off a wider variety of feats, but the reduced footprint of smaller models makes them attractive tools.
But they can excel on specific, more narrowly defined tasks, such as summarizing conversations, answering patient questions as a health care chatbot, and gathering data in smart devices. One method, known as pruning, entails removing unnecessary or inefficient parts of a neural network —the sprawling web of connected data points that underlies a large model. Today’s pruning approaches trace back to a 1989 paper in which the computer scientist Yann LeCun, now at Meta, argued that up to 90 percent of the parameters in a trained neural network could be removed without sacrificing efficiency.
Or read this on Wired
