Get the latest tech news
Async hazard: MMAP is blocking IO
Memory-mapping a file is convenient, but it's a hazard when used with async/await concurrent code: it means a "simple" memory index does blocking IO.
Using either async/ await or operating system threads results in reading all files in less than 0.65 seconds, while the sequential single-threaded version takes only a bit longer (0.7s) but with little overlap in the distributions. I imagine because there’s less overhead: for conventional IO, there’s an extra memcpy of data from the operating system page cache into the buffer passed into the read call. It’s a hazard at a distance: a memory-mapped file can be coerced to a look like a normal byte array (&[u8], bytearray/ list[int], or void */ char *) and passed deep into the async code before being indexed.
Or read this on Hacker News