Get the latest tech news

Emulating the FMAdd Instruction, Part 1: 32-bit Floats

work is write an emulation of the FMAdd (fused multiply-add) instruction for hardware where it wasn't natively supported (specifically I was writing a SIMD implementation, but the idea is the same), and so I thought I'd share a little bit about how FMAdd works, since I've already been posting about how float rounding works. So, screw it, here we go with another unnecessarily technical, mathy post! What is the FMAdd Instruction? A fused multiply-add is basically doing a multiply and an add as a single operation, and it gives you the result as if it were computed with infinite precision and then rounded down at the final result.

Computing it normally (using the code above) for some values will get you double rounding(explained in a moment) which means you might be an extra bit off (or, more formally, one ULP) from where your actual result should be. (It can also be faster if it's supported by hardware but, as you'll see, computing it without a dedicated instruction on the CPU is actually surprisingly spendy, especially once you get into doing it for 64-bit floats, but sometimes you need precision instead of performance). The immediately obvious thing to try to get an accurate single-precision FMA is "hey, what if we do the multiply and add as doubles and then round the result back down to a single":

Get the Android app