Weekly research scan [2026-09-27]

[jgoldfoot]

Automated weekly research scan. Draft for human review. Do not merge without checking the Verification section.

Window covered: 2026-09-07 → 2026-09-27. The 14 Sept run opened nothing and the 21 Sept run failed, so this scan covers three weeks.

Findings

Category	Claim	Primary source
[STALE]	Whitepaper §9.4 says Operator "leads in benchmarks" at 38.1% on OSWorld. On OSWorld-Verified the highest entry is now 90.19% (Intelligence-Indeed Agent, an agentic framework, 25 Jul 2026). The highest general model is 85.96% (claude-fable-5[1m], 1 Aug), followed by claude-opus-5[1m] at 83.39%. None of the 15 highest entries lists the accessibility tree as an additional input.	osworld_verified_results.xlsx (maintainers' data file behind the leaderboard, last-modified 7 Aug 2026)
[NEW]	Affora, a design system for interfaces read by both people and agents. On its primary set, ARIA + structured data 45/60 vs baseline 46/60 (no gain), while Affora rewrites and an agent.md file each reached 60/60. WebMCP-style tools scored 55/55 on an unmatched action set. Only small and mid-tier models were tested.	arXiv:2609.19125
[NEW]	Nudge susceptibility in GUI agents. 3,600 agents, 21,600 simulations, six models from three providers. Agents were susceptible to both default and social-influence nudges. More reasoning lowered the first and raised the second.	arXiv:2609.19843
[NEW]	MCP Skills Extension (SEP-2640) reached Final on 13 Sept 2026 (io.modelcontextprotocol/skills). It serves Agent Skills over Resources via skill://, with skills/list / skills/get.	SEP-2640 · PR #2640
[NEW]	WCAG 3.0 Working Draft, 10 Sept 2026, the first since 3 March. It "includes all requirements that have reached the developing status." It is still a draft.	WD-wcag-3.0-20260910
[CITATION]	WebMCP changes: ToolAnnotations.debugging (17 Sept, #253); toolactivated / toolcancel events (17 Sept, #245); Permissions-Policy: tools=() documented as the first security mitigation (15 Sept, #275); executeTool() inputObject retyped object → any (10 Sept, #251). Header date is now 26 Sept. Implementation status is unchanged.	spec · #253 · #245 · #275 · #251

Items that most need a human look

1. Affora partly contradicts a Layer 2 assumption. It is the first head-to-head test I've found of ARIA and structured data against other interventions, and on its primary set they did nothing (46→45/60). The added text treats this as a limit: Layer 2 is necessary but not sufficient. It does not frame it as a refutation. The paper supports that reading, since gains tracked the presence of a real deficit and rewriting already-sound components did nothing. Its scope is also narrow: small and mid-tier models, and admin/commerce apps only. Still, check that the framing is right before this merges. The paper is also the closest published analogue to the BiModal thesis ("a shared interface rather than a separate agent-only surface"), which may be worth a mention on the website.
2. The OSWorld-Verified accessibility-tree column. The leaderboard records whether each entry used the accessibility tree as additional input. None of the top 15 did, and the best one that did is autoglm-os-9b at 47.26%. That puts pressure on whitepaper §3.3's line that a populated AOM bypasses "the unreliability of pure vision models." I recorded the fact in §9.1 only and did not edit §3.3. Whether to soften §3.3 is a framework-claim decision, not a citation fix.
3. This resolves the long-running OSWorld watch item. Weekly research scan [2026-08-31] #167 and Weekly research scan [2026-09-07] #171 both cut the leaderboard figures because the page loads its table dynamically. The page's own JS fetches static/data/osworld_verified_results.xlsx, and I parsed that file directly.

Verification

• OSWorld-Verified: downloaded the xlsx the leaderboard page fetches (URL found in static/js/index.js and the page source). Parsed all 1,076 rows directly and sorted by success rate. Every figure cited is read from its row, including model name, institution, approach type, date and success/total. Cross-check: OSWorld-Pro (arXiv:2609.24890, NVIDIA) independently cites Opus 5 at "83.4% on OSWorld", which matches the file's 83.39%. The 72.36% human baseline is the original paper's number, measured before the Verified revision, and the text says so.
• arXiv:2609.19125: metadata from the arXiv API (v1 16 Sept 2026, cs.HC, single author Jin Gao). I read the full v1 HTML: all Table 2 figures are verbatim, and so are §6.2–6.3 and the §7.5 limitations. Refutation pass: I checked whether the WebMCP 55/55 could be read as a matched win. It cannot, because the author says "not a matched comparison", so the text carries that caveat. I also checked whether the null ARIA result was significant either way. The table note says "Colours do not indicate statistical significance", so no significance claim is made.
• arXiv:2609.19843: metadata from the API (v1 17 Sept, cs.AI, 4 authors). 3,600 / 21,600 / six models confirmed in the body, and the model list was read verbatim (GPT-5.4 and -mini, Gemini 3.5 Flash and 3.1 Flash Lite, Claude Sonnet 4.6 and Haiku 4.5). The direction of the reasoning moderation was taken from the abstract, and no effect sizes are cited.
• SEP-2640: status "Final", the extension identifier and the abstract were read from docs/seps/2640-skills-extension.mdx at main. PR merged 2026-09-13T21:27:51Z (API). PR #3372 was the source for "SDK implementation PRs remain open".
• WCAG 3.0: "This version: https://www.w3.org/TR/2026/WD-wcag-3.0-20260910/" and the status text were read from the TR document itself.
• WebMCP: each PR's merge time, merger (domfarolino) and author were checked via the GitHub API. The debugging definition, the §6.4.1 Permissions-Policy text, the §4.5 default allowlist and the header date were extracted verbatim from the live spec HTML. implementation-status.md was re-read at main: its last change was 4 Sept, so it is unchanged.
• Checked, no change: MCP releases (still 2026-07-28); A2A (v1.0.1); WAI-ARIA (editorial commits only). schema.org v30.1 (15 Sept) exists, but its release notes contain no content ("Release created automatically"), so nothing is verifiable to cite.

Cut during the skeptic pass

• OSWorld-Pro (arXiv:2609.24890). Its abstract says "top performers like Claude Opus 5 achieving only 75.7%", but its own Table 2 puts Opus 4.8 on top at 77.7%. That is an internal inconsistency, so I used it only as a cross-check.
• RecreationWorld (arXiv:2609.22000), "GPT-6 Astra leads at 58.1%": an agent-training and benchmark result with no interface claim.
• MCP registry census (arXiv:2609.10962): 58.8% tool-level safety-annotation omission on a random draw. It is relevant to annotations, but the framework makes no claim about MCP annotation adoption. Noted, not proposed.
• MCP publication/adoption study (arXiv:2609.14721), OdoBot (arXiv:2609.13491), WebMRE (arXiv:2609.27353), Motion-CAPTCHA against GUI agents (arXiv:2609.27461): all on-topic, but none bears on a current framework claim.

Settled conventions preserved

• Benchmark numbers remain directional projections. Every new figure is presented as a measurement of agents or of interventions in a third-party study, never as a BiModal result. The Affora text says explicitly that it "is not a test of BiModal Design."
• data-agent-* remains supplementary. Untouched.
• npm, not pnpm. DRIFT.md not modified, because no cross-artifact contradiction was introduced. The §3.3 tension is flagged above for decision rather than logged as drift.

Checks

• npm test: 155 passed, 7 suites.
• npm run lint: clean. npm run format:check: clean.
• Only README.md and docs/whitepaper.md touched. No changes to package.json, package-lock.json, or .github/workflows.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-09T16:15:44.411250Z	643d88d	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 643d88d122

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Limit the guidance to the nudges the study tested

The cited experiment tested only a preselected default and displayed social-influence information; it did not test scarcity cues. Including scarcity in the evidence-backed conclusion makes the ethics guidance appear supported by this study when it is not. Restrict the sentence to defaults and social proof, or cite separate evidence for scarcity cues.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Do not equate Skills with progressive tool discovery

The MCP roadmap's progressive-discovery effort specifically addresses large tool catalogs by exposing a small entry point and revealing tools as a conversation narrows. SEP-2640 instead transports Agent Skills through resources and skills/list/skills/get; it neither filters nor progressively exposes a server's tools. Calling it the protocol-level form of that roadmap work can mislead Layer 5 implementers into believing Skills solves tool-catalog context and selection costs, so describe it as a complementary discovery mechanism instead.

Useful? React with 👍 / 👎.