Get the latest tech news

Cloudflare is taking a stand against AI website scrapers

Cloudflare has introduced a free tool for any of its clients to block all known AI website crawlers, even those that aren't following the rules.

Cloudflare has released a new free tool that prevents AI companies' bots from scraping its clients' websites for content to train large language models. In a blog post announcing this update, Cloudflare's team also shared some data about how its clients are responding to the boom of bots that scrape content to train generative AI models. The arms race to build models faster has led to instances of companies skirting or outright breaking the existing rules around blocking scrapers.

Get the Android app