Get the latest tech news
Logical vs. Physical Data Types
The rationale behind the Vortex logical type system
It is perhaps no surprise that Wes McKinney, the creator of Pandas, helped create Apache Arrow to bring these benefits to the wider data ecosystem. Compressed Vortex arrays can be loaded from disk, network, object stores, and even memory-mapped with very little overhead, allowing us to choose precisely when and how much data to decompress. Read more in our post: Compressing strings with FSST... and many others, including FastLanes Delta, Frame of Reference, Dictionary, RunEnd, Adaptive Lossless Floating Point (ALP), and ZigZag.
Or read this on Hacker News