Get the latest tech news
Apple collaborates with Nvidia to research faster LLM performance
In a blog post today, Apple engineers have shared new details on a collaboration with NVIDIA to implement faster text...
In a blog post today, Apple engineers have shared new details on a collaboration with NVIDIA to implement faster text generation performance with large language models. It represents a new method for generating text with LLMs that is significantly faster and “achieves state of the art performance.” It combines two techniques: beam search (to explore multiple possibilities) and dynamic tree attention (to efficiently handle choices). “LLMs are increasingly being used to power production applications, and improving inference efficiency can both impact computational costs and reduce latency for users,” Apple’s machine learning researchers conclude.
Or read this on Hacker News
