Get the latest tech news
Show HN: Xorq – open-source Python-first Pandas-style pipelines
deferred computational framework for multi-engine pipelines - xorq-labs/xorq
It enables us to write pandas-style transformations that never run out of memory, automatically cache intermediate results, and seamlessly move between SQL engines and Python UDFs—all while maintaining replicability. xorq orchestrates data movement between engines (e.g., Snowflake for initial extraction, DuckDB for transformations, and Python for ML model training). The CLI converts Ibis expressions into serialized artifacts that capture the complete execution graph, ensuring consistent results across environments.
Or read this on Hacker News