Get the latest tech news
Linux Lazy Unmap Flush "LUF" Reducing TLB Shootdowns By 97%, Faster AI LLM Performance
SK has been working on a Linux kernel feature dubbed Lazy Unmap Flush 'LUF' to defer TLB flushes until folios have been unmapped and freed are eventually allocated again.
The end result is what is most interesting and important: the LUF patches yielded TLB shootdown interrupts being reduced by around 97%. The most recent Lazy Unmap Flush patches were stressed for a week by running an AI LLM inference workload with 140GB of memory to prove its stability. The huge reduction in TLB shootdown interrupts and several percent gain to the Llama.cpp AI runtime as an example are quite promising.
Or read this on Phoronix