Get the latest tech news

ArcticDB: Why a Hedge Fund Built Its Own Database

James Munro discusses ArcticDB and the practicalities of building a performant time-series datastore and why transactions, particularly the Isolation in ACID is just not worth it.

It reminded me of this quote, the distance between insanity and genius is measured only by success, which after a bit of Googling I found out was made by someone called Bruce Feirstein, who apparently wrote some of the James Bond stories, an American author. Actually, you'll find that a lot of the time you're competing on this research productivity, for the trade ideas themselves, and how you manage risk and optimize portfolios, rather than the low latency aspects of it. For all the reasons discussed, we ended up with this fully client-side database machine that does all the work of deduplicating data, compressing, tiling, indexing, working with the storage system to create a fully featured shared database infrastructure on what could be shared file systems, cloud storage, or it could be very performant flash drives in your local data center, which is the way we run it.

Get the Android app