Predictive Speculative KV Replication for Bursty LLM Inference

Hetzner is working on LLM Inference

DeepSeek open sources DSpark, a new framework to speed up LLM inference by up to 85%

Real-time LLM Inference on Standard GPUs: 3k tokens/s per request