Unverified50% confidenceSolutionExact time
In the execution order of a computer-use agent, the tree should come first (listing interactive elements with names, roles and positions), text second (performing OCR on the window to match visible text when the tree has no labels), and screenshots last (for canvases, unnamed icons, or page verification).
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe role of OCR is to perform text detection and recognition on a screenshot of the current window, obtain the content and bounding box of each text block, and return the center of the text box to the model as clickable coordinates, serving as a text fallback layer rather than general visual recognition.69% similarUnverifiedPage-Agent 选择直接解析 DOM 树的路线，以结构化数据替代视觉感知65% similarUnverified感知层通过截图、DOM解析或可访问性树获取页面状态，执行层通过Playwright、Puppeteer等浏览器自动化框架模拟鼠标点击、键盘输入等操作64% similarUnverifiedPage state compression strategies for browser agents include selective DOM pruning, accessibility tree serialization, and screenshot + Vision Language Model (VLM) multimodal approaches.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1026541API
curl https://kongchang.com/api/v1/knowledge/claims/1026541MCP
get_claim(id=1026541)