Get the latest tech news

I solved a distributed queue problem after 15 years

Learn how queues make horizontal scaling, scheduling, and flow control easier in cloud systems, and how to make them durable and observable.

When I was responsible for the infrastructure at Reddit, the most important thing I maintained was Postgres, but a close second was RabbitMQ, our message broker. For example, if you upvoted a post, that was written to the queue and the cache, and then returned success to the user. With a durable queue, we could have resumed failed jobs from their last completed step and we wouldn’t have lost data when there were program crashes.

Get the Android app