Get the latest tech news
Infinite Tool Use
An LLM should never output anything but tool calls and their arguments. The tools hold the specific, instantiated state of what the model is doing and its goals, while the model itself holds only the information it requires for its immediate task and some additional context, leading to specialization between...
Methods like Entropix try to work around these issues by dynamically adapting token-sampling parameters like temperature, by branching and merging on demand, and even backtracking, all based on an external measurement of the model’s entropy. And the potential issue of going off-course is solved by simply refreshing the model’s memory about fine-grained details (specific sections, sentences, words, what the goal of the whole process is, …) through tool-use. But any LLM with a finite context window but with tools(and training to use them) can re-watch whatever part of the video it needs to understand what it has to, write down, edit, and revisit running notes, and more, without exploding costs.
Or read this on Hacker News