Get the latest tech news

A year of fighting scrapers on my 1.5 million-page website


A year of fighting scrapers on my 1.5 million-page website. What I tried, how I failed, and what's working now.

None

Get the Android app

Or read this on Hacker News

Read more on:

Photo of bots

bots

Photo of website traffic

website traffic

Related news:

News photo

Rogue AIs aren’t just outsmarting humans – they’re teaming up -- Bots caught plotting how to carry out cyber attacks as experts call recent incidents a ‘wake-up call’ for IT security

News photo

TIME Is Serving AI Bots a Different Website, with Ads Built In

News photo

Iowa-led states ask OpenAI to keep their bots on a leash