Get the latest tech news

The Open-Source Software Saving the Internet From AI Bot Scrapers


Anubis, which block AI scrapers from scraping websites to death, has been downloaded almost 200,000 times.

Since she launched it in January, Anubis, a “program is designed to help protect the small internet from the endless storm of requests that flood in from AI companies,” has been downloaded nearly 200,000 times, and is being used by notable organizations including GNOME, the popular open-source desktop environment for Linux, FFmpeg, the open-source software project for handling video and other media, and UNESCO, the United Nations organization for educations, science, and culture. In theory, bot scrapers could pretend to be users with browsers as well, but the additional computational cost of doing so on the scale of scraping the entire internet would be huge. She told me she’s working on a non cryptographic challenge so it taxes users’ CPUs less, and also thinking about a version that doesn’t require JavaScript, which some privacy-minded disable in their browsers.

Get the Android app

Or read this on r/technology

Read more on:

Photo of internet

internet

Photo of source software

source software

Photo of AI bot scrapers

AI bot scrapers

Related news:

News photo

Cloudflare experiment will block AI bot scrapers unless they pay a fee

News photo

Using the Internet without IPv4 connectivity

News photo

Bot or human? Creating an invisible Turing test for the internet