Get the latest tech news
Load Test GlassFlow for ClickHouse: Real-Time Dedup at Scale
Discover how GlassFlow enables real-time deduplication from Kafka to ClickHouse at scale. In this load test, we pushed 55K records/sec and processed over 9K/sec on a single MacBook—with sub-millisecond latency, no message loss, and full reproducibility.
Real-time deduplication(configurable window, event ID based) Stream joins between topics Exactly-once semantics Native ClickHouse sink with efficient batching and buffering While the setup supports running against cloud-hosted Kafka and ClickHouse, we chose to keep everything local to maintain control over the environment and ensure consistent test conditions. No complex joins or filters were applied in this run, keeping the focus on how well GlassFlow could handle high event volumes and real-time processing with exactly-once semantics.
Or read this on Hacker News
