Get the latest tech news

The future of Deep Learning frameworks

Assumed audience: ML researchers who frequently work with PyTorch, but are interested in trying out JAX or have yet to be convinced. Introduction Usually, people start these ‘critiques’ with a disclaimer that they are not trying to trash the framework, and talk about how it’s a tradeoff.

It’s true that maintaining such a huge ecosystem will always have it’s problems, but the considering the case where devs shipped a built-in implementation of FSDP, and it didn’t work at all with their own torch.compile stack for months, really goes to show where their priorities lie. If, god forbid, Pytorch does end up going with the plan and commits to an XLA based compiler stack, then wouldn’t the ideal framework be the one that was specifically designed and built around it, as opposed to the one where it has just been crammed in with little thought and care? However, if they want PyTorch to stand the test of time, more focus has to be put in shoring up the foundations than shipping shiny new features that immediately crumble outside ideal tutorial conditions.

Get the Android app