Get the latest tech news

Cheap AI “video scraping” can now extract data from any screen recording

Researcher feeds screen recordings into Gemini to extract accurate information with ease.

Recently, AI researcher Simon Willison wanted to add up his charges from using a cloud service, but the payment values and dates he needed were scattered among a dozen separate emails. Microsoft demonstrated a similar " Copilot Vision" prototype concept earlier this month (based on OpenAI's technology) that will be able to "watch" your screen and help you extract data and interact with applications you're running. "The great thing about this video scraping technique is that it works with anything that you can see on your screen... and it puts you in total control of what you end up exposing to the AI model," Willison explained in his blog post.

Get the Android app