Get the latest tech news

Facebook's Fascination with My Robots.txt

Facebook is requesting my robots.txt thousands of times per hour.

None

Get the Android app

Or read this on Hacker News

Read more on:

Photo of Robots.txt

Robots.txt

Related news:

Fix your robots.txt or your site disappears from Google

Robots.txt is a suicide note (2011)

Show HN: Aura – Like robots.txt, but for AI actions

« VTT Test Donut Lab Battery Reaches 80% Charge in Under 10 Minutes [pdf]

I Donut Believe – Presenting Third Party Results (Donut Lab) »