UFO
A Windows graphical user interface agent developed by Microsoft Research that can interpret Windows application interfaces and automatically execute user instructions, representing a significant contribution to the GUI automation agent domain.
Timeline (last 90 days)
The technical approach of GPT-6 Astra is similar to Anthropic's Claude Computer Use and Microsoft's UFO project.
Microsoft Research's UFO (UI-Focused Agent) combines GUI element detection and task planning capabilities.
Early representatives of browser-operating AI agents include Adept AI's ACT-1 and Microsoft's UFO.
基于视觉理解的移动端自动化方案代表包括Google DeepMind的CogAgent、微软的UFO项目以及Apple Intelligence中的App Intents框架
OpenAI 的 Operator、Anthropic 的 Computer Use 以及微软的 UFO Agent 框架均将 Accessibility Tree 作为主要的页面感知通道
All Facts (9)
微软的UFO（专注Windows应用操控）、SeeAct（基于网页截图的操作预测）、OS-Copilot（通用操作系统代理框架）等开源学术项目为Computer Use技术发展提供了基础
85%Unverified微软于2024年初开源了UFO（UI-Focused Agent）项目，基于GPT-4V的多模态能力，采用双Agent架构（AppAgent和HostAgent）
85%Unverified微软开发了UFO（UI-Focused Agent）开源项目
75%UnverifiedMicrosoft Research launched the UFO project, focusing on GUI agents on Windows systems.
70%UnverifiedThe technical approach of GPT-6 Astra is similar to Anthropic's Claude Computer Use and Microsoft's UFO project.
50%UnverifiedMicrosoft Research's UFO (UI-Focused Agent) combines GUI element detection and task planning capabilities.
50%UnverifiedEarly representatives of browser-operating AI agents include Adept AI's ACT-1 and Microsoft's UFO.
50%Unverified基于视觉理解的移动端自动化方案代表包括Google DeepMind的CogAgent、微软的UFO项目以及Apple Intelligence中的App Intents框架
50%UnverifiedOpenAI 的 Operator、Anthropic 的 Computer Use 以及微软的 UFO Agent 框架均将 Accessibility Tree 作为主要的页面感知通道
50%