Get the latest tech news
Optimizing ClickHouse for Intel's 280 core processors
Intel's latest processor generations are pushing the number of cores in a server to unprecedented levels. For analytical databases like ClickHouse, ultra-high core counts represent a huge opportunity and a complex challenge.
My work focused on using various profiling and analysis tools - including perf, emon, and Intel VTune - to analyze all 43 ClickBench queries on ultra-high core count servers systematically, identifying bottlenecks, and optimizing the ClickHouse accordingly. This requires a multi-pronged approach: reducing the duration of critical sections, replacing exclusive locks (mutexes) with more granular synchronization primitives, and in some cases, eliminating shared state entirely. Solving this issue requires careful algorithm design, smart data structure choices, and a deep understanding how hash tables behave under different load patterns.
Or read this on Hacker News
