Unverified50% confidenceFactExact time
On Windows, LU reads the list of buttons and fields of the target window (UI Automation); models that can see images get screenshots plus the list, while models that cannot see images work from the list alone.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWindows-MCP relies at its core on the Windows UI Automation (UIA) framework, translating the desktop interface into structured hierarchical text documents and extracting metadata and coordinates for every button and window.67% similarUnverifiedThe macOS Accessibility API allows programs to query UI element identifiers, positions, text, and interaction states and simulate operations without relying on image recognition; it is a mechanism of the same kind as Windows UI Automation and Linux AT-SPI.66% similarUnverifiedapplication-use implements desktop automation using the macOS Accessibility API, which can enumerate the UI element hierarchy tree in any application window and obtain attributes such as position, label, and state for buttons, text fields, and menu items.66% similarUnverifiedIf a casting tool wants to capture only a single macOS window, it usually needs to use screen recording permission to obtain the framebuffer and then locate the crop region via the window ID; or use the Accessibility API to read window geometry information.64% similarUnverifiedWindows UI Automation (UIA) is an accessibility framework built into the operating system by Microsoft since Windows Vista, originally designed to allow assistive technologies such as screen readers to programmatically access interface elements.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1064504API
curl https://kongchang.com/api/v1/knowledge/claims/1064504MCP
get_claim(id=1064504)