Weekly research scan [2026-09-07]

[jgoldfoot]

Automated weekly research scan. Draft — for human review. Do not merge without checking the Verification section.

Window covered: 2026-08-31 → 2026-09-07.

Note on the two open scan drafts. #164 (2026-08-24) and #167 (2026-08-31) are both still open, so their findings are not on main and this branch is cut from main without them. The overlap is again the single README WebMCP citation entry, plus its whitepaper twin. This PR's version is a superset of both: it keeps the 14/17 Aug executeTool() and inputSchema facts already on main, carries #167's ChatGPT Desktop implementation-status change (re-verified here from implementation-status.md at main), and adds the 3 Sept consequentialHint addition and the 4 Sept draft header. Whichever of the three merges last will need those two hunks rebased. Nothing else conflicts.

Findings

Category	Claim	Primary source
[NEW]	Computer-use agents for blind users: three-week IRB-approved diary study, 8 blind screen-reader users, 1,258 commands across 12 desktop applications, every model supplied with screenshots and UI trees. GPT-5 best at 52.5% (95% CI 49.8–55.3); Claude Sonnet 48.5%, Gemini 2.5 CU 43.9%, UI-TARS 39.8%, Qwen3-VL 37.9%. Failure classes: grounding, planning, constraint-tracking, termination.	arXiv:2609.00524
[NEW]	CUA-Universe: 16 desktop applications turned into hybrid GUI+CLI environments. A post-trained 9B model reaches 40.2% on OSWorld against its own GUI-only 23.4% (+16.8 pts), and 28.69% on OSWorld-MCP against the untuned base model's 20.90% (+7.79 pts), with 27% fewer steps and 30% fewer tokens.	arXiv:2609.05374
[CITATION]	WebMCP: ToolAnnotations gained consequentialHint on 3 Sept 2026 (PR #217). Draft header advanced to 4 September 2026. Implementation-status page now lists ChatGPT Desktop alongside Chrome 149 / Edge 150 origin trials and Brave Leo. README now states headless task completion as a goal, fully autonomous browser-less workflows as a non-goal.	spec · PR #217 · implementation-status.md

Why these matter to the framework

The blind-user study bounds a claim the framework currently states without a ceiling. §3.3 says a well-populated AOM "provides high-fidelity, reliable interaction targets for OS agents, bypassing the unreliability of pure vision models." This is the first deployment-scale evidence of what that buys in practice: agents given both screenshots and UI trees carrying roles and hierarchy still fail about half of real user commands, and the named failure classes — planning, constraint-tracking, termination — are not ones a structural layer reaches.

The honest framing is the restrictive one, and the text says so explicitly. The paper makes no with/without accessibility-metadata comparison; all five models received identical inputs. So this is not a measurement of Layer 2's contribution, and the added text states that outright rather than letting a low number imply a Layer 2 result in either direction. The claim carried is the narrow one: Layer 2 removes a class of failure, and does not remove the rest.

CUA-Universe is a check on reading the adoption gap as purely an interface problem. Screenshots or Tools? (already cited) found the reasoning model taking the tool route on only 23.9% of tool-reachable tasks, and the framework's Hybrid Agents caveat concludes the projection is "conditional on adoption, not on availability." CUA-Universe shows adoption is trainable: the same model, taught to orchestrate GUI+CLI, beats its own GUI-only self by 16.8 points on OSWorld. That cuts both ways for Layer 5 and is recorded that way — the interface is not the only lever, and 28.69% absolute on OSWorld-MCP says the binding constraint has moved elsewhere once adoption is trained in.

consequentialHint is the first WebMCP annotation that is about consequence rather than capability. readOnlyHint and untrustedContentHint describe what a tool touches; consequentialHint describes what it costs to get wrong ("booking a flight, transferring money"). It is recorded as a citation update only — no framework claim is built on it this week.

Verification

Every claim was confirmed against its primary source, and a refutation pass was run on each.

• arXiv:2609.00524 — title, full author list (Kodandaram, Padma Reddy, Bi, Zhou, Ramakrishnan, Ashok), v1 date (1 Sept 2026, 00:44:19 UTC), categories (cs.HC, cs.AI) and the complete abstract read from the abs page. The v1 full text was then put to an explicit refutation prompt. Confirmed TRUE with verbatim support: 52.5% is GPT-5's overall success rate ("95% CI: 49.8–55.3%"), and the per-model table (Claude Sonnet 48.5%, Gemini 2.5 CU 43.9%, UI-TARS 39.8%, Qwen3-VL 37.9%). Two questions returned negative and both are reflected in the framework text rather than omitted: whether the paper attributes failures to missing or poor-quality accessibility metadata (NOT FOUND), and whether it compares agents with vs. without accessibility metadata (FALSE — "all evaluated models received the same UI tree and screenshot inputs"). One inconsistency surfaced: the HTML render produced "88 blind screen-reader users" in one sentence against N=8 in its own participant table and in the abstract. The abstract's figure (8) is the one cited.
• arXiv:2609.05374 — title, full author list (Shi, Wang, Fang, Liang, Jin, Zhao, Liu, Chen, Wang), v1 date (4 Sept 2026), category (cs.AI) and the complete abstract read from the abs page. The refutation pass targeted the baselines specifically, because the abstract reports three deltas against three different comparisons and quoting them as one would be wrong. Confirmed from the body: OSWorld-MCP is trained-vs-untuned-base ("improves SR from 20.90% to 28.69%"), OSWorld is GUI+CLI-vs-GUI-only for the same model ("reaches 40.2% SR—up +16.8 points from its GUI-only 23.4%"), CUA-Verse is 0.189 → 0.582. The framework text carries the absolute numbers and names each baseline rather than the abstract's bare deltas. Two discrepancies preserved rather than smoothed: the abstract says "+7.84 pts" on OSWorld-MCP where the body's own figures give +7.79, so the absolute pair (20.90% → 28.69%) is cited and the body's delta used; and the paper was asked directly whether it makes any interface- or website-design claim — NOT FOUND, which is why both the README and whitepaper additions say so in bold.
• WebMCP — consequentialHint's definition quoted verbatim from the live specification document. PR #217 ("Add consequentialHint to ToolAnnotations") confirmed merged 2026-09-03T18:22:06Z, authored by beaufortfrancois, merged by spec editor domfarolino, via the GitHub API. Draft header date read from the spec document itself. Implementation status read from implementation-status.md at main. Goals/non-goals read from the repository README (PR #296, merged 2026-09-04).
• MCP — checked, no change. The releases API still tops out at 2026-07-28 (published 28 July 2026); no 2026-09-* release or RC exists. Existing README/whitepaper entries are correct as written.
• A2A — checked, no change. Latest release remains v1.0.1 (28 May 2026).
• schema.org — checked, no change. Latest release remains v30.0 (25 March 2026).
• WAI-ARIA — checked, no material change. The only commit in the window is editorial (ids for security and privacy considerations, 29 Aug).
• WCAG 3.0 — checked, still not published. Weekly research scan [2026-08-31] #167 flagged the w3c/wcag3 "Preparation for Aug 2026 publication" commit as a watch item. The published Working Draft at w3.org/TR/wcag-3.0 is still dated 03 March 2026. The August publication did not land. Remains a watch item.

Cut during the skeptic pass

• OSWorld-Verified leaderboard figures. Re-checked, per Weekly research scan [2026-08-31] #167's note that this is the one item that could make an existing framework claim stale. os-world.github.io now 301s to osworld-v1.xlang.ai, and that page still loads its table dynamically — the HTML contains "Loading verified benchmark data…" and no scores. Still unconfirmable from the primary source. Cut again, and still the item most worth a human look. The framework's OSWorld 2.0 figure (20.6%) and Operator figure (38.1% on OSWorld) both depend on it.
• arXiv:2609.02309, "Efficient GUI Agents: A Systems Survey." A survey, not a measurement — nothing verifiable to add to a claim.
• arXiv:2609.03438 (ConflictGUI / conflict-aware termination) and arXiv:2609.02057 (monitoring web agents without internal signals). Both real and both on-topic for agent safety, but the framework makes no claim either bears on. Noted, not proposed.
• arXiv:2608.30207 (SIR, red-teaming computer-use agents) and arXiv:2609.04850 (ElderBench). Outside the framework's current claim surface.

Settled conventions preserved

• Benchmark numbers remain directional projections synthesized from agent benchmarks. Both new papers are presented as measurements of agents — one of models on real user commands, one of a post-trained model — never as a BiModal A/B test.
• data-agent-* remains a supplementary layer (schema.org → WAI-ARIA → data-agent-*). Untouched.
• npm, not pnpm. No dated report files; DRIFT.md not modified this run (no new cross-artifact contradiction found).

Checks

• npm test — 155 passed, 7 suites.
• npm run lint — clean.
• npm run format:check — clean.
• No changes to package.json, package-lock.json, or .github/workflows. Only README.md and docs/whitepaper.md touched (+110 / −8).

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 46f61ad0c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Avoid attributing every failed command to planning

The 52.5% success rate only establishes that roughly half the commands failed; it does not establish that all of those failures arose from planning or termination. The preceding sentence explicitly lists grounding and constraint-tracking as additional failure classes, and the study has no accessibility-metadata ablation, so concluding that the failures occurred for reasons outside interface semantics overstates the evidence and undermines the stated caveat about not measuring Layer 2.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Do not infer the binding constraint from absolute success

A 28.69% OSWorld-MCP success rate does not show that tool availability is no longer the binding constraint. The reported comparison is between a post-trained model and its untuned base, not between complete and incomplete tool coverage, and the cited paper is expressly described here as making no interface-design claim. Missing or unsuitable tools could therefore still explain failures, so this causal conclusion should be removed or supported by a tool-availability ablation.

Useful? React with 👍 / 👎.