Get the latest tech news
AI that clicks for you: Microsoft’s research points to the future of GUI automation
Microsoft research reveals how AI agents are transforming computer interaction through GUI control, with major tech companies racing to deploy LLM-powered automation tools worth $68.9 billion by 2028.
A comprehensive new survey from Microsoft researchers and academic partners reveals that artificial intelligence agents powered by large language models (LLMs) are becoming increasingly capable of controlling graphical user interfaces (GUIs), potentially changing how humans interact with software. The research team provides a detailed roadmap for addressing these challenges, emphasizing the importance of developing more efficient models that can run locally on devices, implementing robust security measures, and creating standardized evaluation frameworks. The comprehensive survey suggests we’re at an inflection point where conversational AI interfaces could fundamentally change how humans interact with software — though realizing this potential will require continued advances in both the underlying technology and enterprise deployment practices.
Or read this on Venture Beat
