Get the latest tech news

Synchronization Is Bad for Scale

In the early days of Mailgun I started working on a distributed lock service. Something I had worked on briefly at Rackspace. Even as I implemented the thing, I had the sneaky suspicion that it was a bad idea.

However, In a very high concurrency system where there are a non-trivial number of CPUs in use, kernel locking — regardless of how fast — eventually gets in the way of scaling. If you want, you can read about The Write Synchronization Problem which I used when on-boarding new devs, to explain some of the concepts I’m describing here, so there is a bit of an overlap. The nice thing about the Reservation Pattern in Queues is that much like a lock, you gain exclusive access to an item for a period of time.

Get the Android app