Get the latest tech news
Binance built a 100PB log service with Quickwit
Binance Scales its Log Service to 100PB and Saves Millions with Quickwit
Our goal was ambitious: to create a new breed of full-text search engine that is ten times more cost-efficient than Elasticsearch, significantly easier to configure and manage, and capable of scaling to petabytes of data . As the world's leading cryptocurrency exchange, Binance handles an enormous volume of transactions, each generating logs that are crucial for security, compliance, and operational insights. Concretely speaking, you can tear down your cluster, recreate it in a minute without losing any data, ready to ingest at 1.6 PB/day or search through petabytes, and scale up and down to handle temporary spikes.
Or read this on Hacker News