Get the latest tech news
I made a search engine worse than Elasticsearch (2024)
Integrating my BM25 pandas search library, SearchArray, into BEIR, in order to embarass myself in public.
Search engines can use an algorithm called Weak-AND or WAND to avoid work when combining multiple term scores into the final top N results. So the search engine can plan+cache the complete calculation, whereas SearchArray gives you all the tools to shoot yourself in the foot, performance wise (not to mention the earlier point about WAND, etc). In any case, I’m absolutely humbled by folks that work on big, large scale, distributed lexical search engines like (Vespa, Lucene, OpenSearch, Elasticsearch, Solr).
Or read this on Hacker News