[nahatav]

What does this PR do?

Adds a new shipping-quotes category: 12 tasks in test-cases/shipping-quotes/ in which the agent gets a price quote for mailing a letter or package, on the public rate tools of USPS, Canada Post, UPS and Purolator. No account, payment or CAPTCHA is involved.

The PR contains task files, a suite README, one offline test file with its fixtures, and a CHANGELOG line. No framework code changes; V1 and V2 are untouched.

Why this category

Mailing a package is an everyday errand the corpora don't cover. It also isolates something agents get wrong: the physical details. The tasks state weights and sizes the way people do, and each carrier's form wants them its own way.

Task	Carrier	What it tests
usps_domestic_package_pounds_ounces	USPS	Package shape; weight split into 3 lb 4 oz
usps_large_package_with_dimensions	USPS	Large Package rule (side over 12 in); dimensions
usps_package_weight_in_kilograms	USPS	2.5 kg into a pounds-and-ounces form
usps_square_envelope_letter	USPS	Letter, not large envelope; square nonmachinable surcharge
canada_post_domestic_parcel_metric	Canada Post	Postal codes; metric weight and dimensions
canada_post_parcel_to_us_imperial_units	Canada Post	US ZIP; pounds and inches to grams and centimetres
canada_post_international_parcel_to_uk	Canada Post	International country picker
canada_post_domestic_lettermail	Canada Post	Letter or document, not package
ups_toronto_to_new_york_package	UPS	Two-step form; customs value; inches and pounds
ups_domestic_canada_kilograms	UPS	Kilogram unit selector
purolator_toronto_to_montreal_pounds	Purolator	Postal-code lookup; pound weight
purolator_toronto_to_vancouver_kilograms	Purolator	Changing the default weight unit

The evaluated outcome is initiation of the correctly configured price request, in the same spirit as #378. Buying postage or a label is never required.

How matching works

Every request is a price lookup, so a wrong one is harmless and the matchers can be strict:

• USPS and Canada Post put every input in the query string. The url_pattern requires each constrained parameter, in any order, and rejects duplicates. Numbers accept equivalent formats. Only where a unit conversion makes rounding legitimate does the pattern use a tolerance band:

  • 2.5 kg is accepted as 5 lb with 8.x oz, or as 88.x oz;
  • 4.5 lb is accepted as 2030 to 2052 g.

  Both bands still reject unit errors such as 4.5 kg or a 2.5 lb package.

• UPS sends a form POST. The body filter requires origin, destination, customs value, packaging, weight, unit and dimensions. Requiring weight1 keeps UPS's earlier address-step POST to the same URL from counting.

• Purolator sends JSON and fires a warm-up estimate to the same endpoint on page load. The body filter requires the real sender, receiver and shipment blocks, so the warm-up never matches.

• Shipping dates are not constrained, so the tasks stay valid on any run date.

Validation

• Live interception 12/12. Each task was run with clawbench-run <task> --human in the official base runtime, built from 07eb50a. The carrier's UI was operated over CDP inside the container, and every run ended with the interceptor blocking the final price request (intercepted: true). The README lists each run with its interception.json SHA-256. The exact captured requests are committed as tests/fixtures/shipping_quotes_live_captures.json.
• Real prices returned by the same request shapes in exploratory sessions:
  • Canada Post: Regular Parcel $36.17, Xpresspost $84.39, Priority $129.28 (3.2 kg, Toronto to Vancouver);
  • UPS: Worldwide Express 197.07 CAD (Toronto to New York);
  • Purolator: Ground $80.20 to Express $121.36 (5 lb, Toronto to Montreal);
  • USPS: its full service list.
• tests/test_shipping_quotes_suite.py, 74 offline cases:
  • schema and runner validation;
  • each capture matches its own task and no other task;
  • equivalent formats and conversion rounding accepted;
  • near misses rejected: wrong shape, missing nonmachinable flag, unit errors, wrong origin, destination or country, intermediate wizard steps, a letter priced as a parcel, UPS's first-step POST, Purolator's warm-up, duplicated parameters, wrong method.
• Mutation check. Removing the shape constraint, the nonmachinable flag, the UPS weight fields, or the Purolator address blocks each makes the suite fail.
• Full suite 396 passed, 10 skipped; ruff and pyright clean. The one local failure, test_host_tasks.py::...[v1-lite], comes from Windows checking out the v1-lite symlinks as text files and fails the same way on main.

Verification environment, stated plainly:

• The shipped PurelyMail key is revoked (PurelyMail API token in the committed .env returns invalidToken #384), so the local runs used an unprovisioned address in place of mailbox creation. No task here uses email.
• On Windows, the runtime scripts had to be converted to LF line endings before building the image.

Neither change touches the runtime server or the interceptor, and neither is in this PR. These runs cover interception only: no model ran, and the Stage 2 judge was not exercised.

Known limits, also in the README

• Carriers redesign their calculators. The committed fixtures make the expected request explicit for whoever updates a task.
• UPS echoes postal codes as typed, so its instructions ask for them exactly as written.
• Purolator matches the whole shipment block, so its instructions say to leave dimensions and options unset.

Corpus

• v2
• v1
• both
• not applicable (new isolated category)

Related issues

Category proposal and authorship question: #394.

🤖 Generated with Claude Code