Transformers Represent Belief State Geometry in Their Residual Stream

Transformers Can Do Arithmetic with the Right Embeddings

The Matrix: A Bayesian learning model for LLMs

Block* and AgentFormer – Playing with blocks and Transformers (yay)