Get the latest tech news
Compress Better, Compute Bigger
How Blosc2 can compress data better and compute bigger
The developers of Blosc and Blosc2 have consistently focused on achieving compression and decompression speeds that approach or even exceed memory bandwidth limits. The reason for Numba's poor scaling with larger working set sizes is unclear, although it may require rewriting the expression with explicit loops for optimal performance, which falls outside the scope of this experiment. This integration enables you to benefit from Blosc2's powerful compression and decompression capabilities, while still leveraging NumPy's comprehensive function library.
Or read this on Hacker News