gui-grounding

Mano-P: Open-source GUI-VLA agent for edge devices. #1 on OSWorld (specialized, 58.2%). Runs locally on Apple M4 Mac mini/MacBook — no data leaves your device.Mano-P 是一个开源 GUI-VLA 项目，支持在 Mac mini/MacBook 上或通过算力棒本地运行推理，实现纯视觉驱动的跨平台 GUI 自动化操作。数据完全本地处理，支持复杂多步骤任务规划与执行。

Qwen-UI-Agent: Towards Next-Generation Real-World Centric Foundation GUI Agent

UI-Venus is a general-purpose foundation GUI agent for mobile apps, web platforms, and desktop operating systems using only screenshots as input.

[AAAI 2026 Oral] Official repository for InfiGUI-G1. We introduce Adaptive Exploration Policy Optimization (AEPO) to overcome semantic alignment bottlenecks in GUI agents through efficient, guided exploration.

A 4B grounding foundation model for physical intelligence. Unifies detection, referring, pointing, OCR, GUI, and layout grounding, achieving SOTA overall across 34 benchmarks and improving robot generalization and data efficiency in controlled evaluations.

Official implementation of UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoning

A 4B diffusion grounding model for fast, precise visual evidence extraction. Unifies diverse grounding tasks through parallel decoding, with optional self-speculative decoding delivering 4.51x autoregressive throughput on COCO.

A Practical Zoom-in GUI Grounding and Behavior-Based Evaluation method.

DeskForge: Dense Supervision from Desktop Environments for Computer-Use Agents

Code for paper "Improved GUI Grounding via Iterative Narrowing"

Open, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.

Evaluation harness for GUI grounding models on ScreenSpot-Pro

Screen grounding for computer-use agents: find the target on a screenshot. Eyes-4B, 67.7 on ScreenSpot-Pro · 得心

Do frozen vision backbones transfer to GUIs? A dense-vs-global feature benchmark across pretraining objectives

Pixel-precise visual grounding for Claude Code — a drop-in skill for the coordinate-hallucination problem in visual agents.

WinDOM: Self-Family Distillation for Small-Model GUI Grounding (arXiv:2606.25964)

Jev 1.14's box tool on ScreenSpot-v2 and ScreenSpot-Pro with official scoring, every click logged. 84% and 19% raw; a zoomed second look takes Pro to 43%, a third to 60%.

Source-backed Computer-Use / GUI Agent catalog: papers, models, datasets, benchmarks, frameworks and tools. Searchable website + JSONL/SQLite.

Name a control on your Mac and a pointer arcs to it. Press Option Space, say or type what you want, and Thataway reads the frontmost app's accessibility tree first (12 of 12 targets on a Chrome window, 0.067 ms p50). H Company's Holo1.5-7B runs locally only when the tree has no answer. On-device speech, an editable exclusion list, lessons.