XLSTM won't replace the Transformer. Two bitter lessons

Visualizing Attention, a Transformer's Heart [video]

Transformer as a general purpose computer

UC Berkeley’s transformer-based robot control system generalizes to unseen environments