Get the latest tech news
Binary vector embeddings are so cool
Vector embeddings by themselves are pretty neat. Binary quantized vector embeddings are extra impressive. In short, they can retain 95+% retrieval accuracy with 32x compression 🤯.
Calculating the Hamming distance between binary vectors takes only a couple of CPU clock cycles for XORing and counting the number of 1's. In contrast, calculating the cosine distance between float32 vectors requires floating point multiplication, addition, square roots, and division, which are all more computationally expensive. It seems somewhat counter-intuitive that you could reduce a 4-byte floating point number to a single bit and retain enough signal to implement vector similarity search with reasonably high accuracy.
Or read this on Hacker News