Get the latest tech news
Baidu blocks Google, Bing from scraping content amid demand for data used on AI projects
Wikipedia-style service Baidu Baike recently barred the search engine crawlers of Google and Bing from indexing its online content.
It also showed that earlier on the same day Baidu Baike still allowed Google and Bing to browse and index its online repository of nearly 30 million entries, with only part of its website designated as off limits. Since OpenAI released ChatGPT on November 30, 2022, major search platforms Google and Microsoft have sought to obtain more data for use in their own generative artificial intelligence systems. Following Baidu Baike’s robots.txt update, the Post’s survey of Google and Bing on Friday found many entries – probably from older cached content – from the Wikipedia-style service still come up in the US search platforms’ results.
Or read this on r/technology
