Get the latest tech news
A Trick for Backpropagation of Linear Transformations
Linear transformations such as sums, matrix products, dot products, Hadamard products, and many more can often be represented using an [einsum](https://numpy.org/doc/stable/reference/generated/numpy.
This simple swapping trick makes it very easy to create formulas when backpropagating through einsums. If you swap around the letters, you can more easily interpret the einsum used for backpropagation, as a kind of matrix multiplication. Einsums are a powerful tool for representing and reasoning about linear transformations, and this simple swapping trick makes it very easy to backpropagate through them.
Or read this on Hacker News
