Get the latest tech news

Perplexity is allegedly scraping websites it's not supposed to, again

Cloudflare reports that Perplexity's web crawlers are disguising themselves to access sites that have them blocked.

Cloudflare believes that Perplexity is getting around those obstacles by using "a generic browser intended to impersonate Google Chrome on macOS" when robots.txt prohibits its normal bots. Multiple websites reported in 2024 that Perplexity was still accessing their content despite them forbidding it in robots.txt — something the company blamed on the third-party web crawlers it was using at the time. Perplexity later partnered with multiple publishers to share revenue earned from ads displayed alongside their content, seemingly as a make-good for its past behavior.

Get the Android app