Get the latest tech news
The Bitter Lesson Is Misunderstood
Together, the Bitter Lesson and Scaling Laws reveal that the god of Compute we worship is yoked to an even greater one — the god of Data.
The heroic feats of computation — whether to radically increase memory, parallelize across thousands of GPUs or invent entirely new silicon just to speed up matmul — were trailing indicators. The shift to structured State-Space Models ( like Mamba) uses linear-time recurrence and selective state compression to handle vastly longer contexts with greater efficiency Between model autophagy, specification gaming and the sim-to-real gap, it’s easy to get stuck in dead-ends or get lost in degenerative feedback loops if you don’t have enough ground truth.
Or read this on Hacker News