Get the latest tech news

Width vs. Depth: Speculating on the Margin

Some thinking about how to trade off batching and speculative decoding in a running inference engine.

None

Related news:

I love Baldur's Gate 3's queer representation, but triple-A games can still do so much more to explore lesbian stories with the depth they deserve

Meta's AI Glasses golf integration is so in-depth that it has me looking for clubs

Donut Lab claimed to have made the world's first production-ready solid state battery. An in-depth investigation suggests otherwise.