Get the latest tech news
Just make it scale: An Aurora DSQL story
AWS Senior Principal Engineers, Niko Matsakis and Marc Bowes, take us inside Aurora DSQL's development: scaling write operations without two-phase commit, overcoming garbage collection hurdles, and embracing Rust for both data and control planes.
What started with a push to make traditional relational databases easier to manage with the launch of Amazon RDS in 2009 quickly expanded into a portfolio of purpose-built options: DynamoDB for internet-scale NoSQL workloads, Redshift for fast analytical queries over massive datasets, Aurora for those looking to escape the cost and complexity of legacy commercial engines without sacrificing performance. Adding to the complexity, each layer has to provide a high degree of fan out (we want to be efficient with our hardware), but in the real world, subscribers can fall behind for any number of reasons, so you end up with a bunch of buffering requirements. The reality of distributed systems hit us hard here - when you need to read from every journal to provide total ordering, the probability of any host encountering tail latency events approaches 1 surprisingly quickly – something Marc Brooker has spent some time writing about.
Or read this on Hacker News