Get the latest tech news
The efficient frontier of LLM inference
Inference techniques either move a deployment along the latency–throughput frontier or push the entire frontier out, creating more efficiency to allocate.
None
Or read this on Hacker News