Get the latest tech news
Trap – Transformers in APL
Autoregressive transformers in APL. Contribute to BobMcDear/trap development by creating an account on GitHub.
Existing transformer implementations generally fall under two broad categories: A predominant fraction depend on libraries carefully crafted by experts that provide a straightforward interface to common functionalities with cutting-edge performance - PyTorch, TensorFlow, JAX, etc. Some APL features relied upon by trap are only available in Co-dfns v5, which is unfortunately substantially less efficient than v4 and orders of magnitude slower than popular scientific computing packages such as PyTorch. The good news is that the team behind Co-dfns is actively working to resolve the issues that are inhibiting it from reaching peak performance, and PyTorch-like efficiency can be expected in the near future.
Or read this on Hacker News
