Skip to content

Proposal: 12-task privacy-rights category (PR #391) and co-authorship question #392

Open

Description

@nahatav

Target corpus

New isolated category (test-cases/privacy-rights/); V1 and V2 untouched.

Platform / Website

DataGrail privacy request centers (MyFitnessPal, Plex, Bed Bath & Beyond, Skillshare, Revolve, CookUnity, Harry's, ButcherBox, e.l.f. Beauty, FanDuel) and OneTrust webforms (Norton, AVG).

Task description

PR #391 adds a privacy-rights category of 12 tasks in which the agent files a privacy request for the benchmark persona: deletion, access, data portability, correction, opt-out of sale or sharing, and objection to processing.

What it adds to ClawBench:

  • Write tasks with a real-world purpose. US law lets people authorize an agent to file these requests for them, so whether a browser agent can do it correctly is a practical question.
  • Jurisdiction handling. The forms default to the location they detect, and the agent has to file under the persona's real one, Ontario. The submitted policy id shows whether it did.
  • Request-type selection across inconsistent, company-specific labels, with deliberate near misses.
  • Data minimisation. The persona files hold government, health and financial identifiers, and every rubric fails a request that includes any of them.

The interceptor blocks every submission on each company's form, so a wrong choice by the agent never reaches the company. The request details are graded by the judge. The PR description explains that decision, how the forms were analysed without submitting anything, the offline test coverage, and what is still pending: one human-mode run per task, because most of these forms use a CAPTCHA.

Category

privacy-rights (new; extends the one-task deletion-revocation metaclass in V1)

Is the task behind a payment wall?

No (free action, needs interception)

Questions for the maintainers

  1. Does privacy-rights count as a distinct new category in the sense of CONTRIBUTING.md?
  2. If these 10+ tasks are merged, does the contribution fall under the CONTRIBUTING.md policy of being proposed as a co-author on the relevant follow-up paper or workshop submission, and the README's "co-author invitation on the next paper revision"? I'm asking now so the expectation is clear before review. Merge and authorship of course stay your decision.
  3. Do you want the live interception runs in this PR before review, as feat: add twelve environmental observation request workflows #378 did, or after an initial review of the design?

This was built with Claude Code under my direction. I have reviewed the tasks, the analysis and the tests, and I take responsibility for them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions