Unverified50% confidenceSolutionExact time
Some mature visual agent frameworks (such as SeeAct and WebVoyager) incorporate Set-of-Mark (SoM) technology, which annotates interactive elements on screenshots with numbered markers, converting coordinate-based positioning into a multiple-choice problem.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/7/2026
First Seen
Valid until: 12/6/2026
Sources
Related Entities
Related Claims
UnverifiedSet-of-Mark (SoM) is a visual prompting method proposed by Microsoft Research that overlays numeric labels or colored bounding boxes on screenshots to annotate interactive elements, transforming the visual localization problem into a selection problem.79% similarUnverifiedAnnotate App的视觉标注模式与计算机视觉领域的Set-of-Mark（SoM）提示技术有相似之处75% similarUnverified图像条件化方案通常借助CLIP视觉编码器（ViT架构）将参考图映射到与文本嵌入共享的语义空间，再通过交叉注意力（Cross-Attention）机制将视觉特征注入UNet的去噪过程69% similarUnverifiedLovable支持上传截图、Figma线框图甚至手绘草图作为视觉参考来构建应用67% similarUnverified视觉模型接收每张屏幕截图并输出结构化信息，包括OCR识别文本、页面布局、UI控件的像素坐标框及元素状态67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/874382API
curl https://kongchang.com/api/v1/knowledge/claims/874382MCP
get_claim(id=874382)