Get the latest tech news

AI haters build tarpits to trap and trick AI scrapers that ignore robots.txt

Attackers explain how an anti-spam defense became an AI weapon.

Site owners are discouraged from flagging when the malware has been deployed, forcing crawlers to face unknown "consequences" if they ignore robots.txt instructions. Soon, social media discussion drove users to inquire about Iocaine deployment, including not just individuals but also organizations wanting to take stronger steps to block scraping. Nagy used a reverse proxy to trap crawlers in an "infinite maze of garbage" in an attempt to slowly poison their data collection as much as possible for daring to ignore robots.txt.

Get the Android app