Skip to content

feat!: unify audit applicability and central index for v7.0.0 - #127

Open
k2kirov wants to merge 10 commits into
mainfrom
feat/v7-unified-audit-applicability
Open

k2kirov wants to merge 10 commits into
mainfrom
feat/v7-unified-audit-applicability

Conversation

@k2kirov

@k2kirov k2kirov commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Description

Common audits could skip readable interior pages when a scan had no readable homepage. Broad content classification also applied article obligations to general pages. This migration applies common checks to eligible pages and adds purpose-specific checks only to their selected populations.

  • Add article and unknown, classification provenance, stable page identity, and shared declaration validation across SDK, CLI, config, and MCP. Keep legacy content accepted as a general-page declaration; callers must use article to select article obligations.
  • Separate declared results from detected, informative advisoryResults. Evaluate evidence against selected URLs, retain unread pages and failed attempts, and count each audit once for scoring, progress, and tracing.
  • Show scope and coverage in reports, MCP output, and the website viewer. Preserve old saved reports without inventing missing coverage.
  • Add a generated, LLM-readable central index for all 215 audits in docs/evidence/audit-map.json, including purpose, features, applicability, evidence, scoring, and source/test/dossier paths. Enforce agreement with the registry.
  • Include the implementation plan, audit review ledger, migration notes, corpus comparison, and local release receipt. Changesets target 7.0.0 for all four published packages; versioning remains workflow-owned.

Validation

All seven repository gates passed locally in order: build, test, typecheck, lint, dossiers, requirements, and audit map. The committed product source matches the validated source fingerprint.

  • Full suite: 5,937 passed, 219 skipped; 350 files passed, 1 skipped, with network verification enabled.
  • Saved-response comparison against 6.0.0: 41 fixtures, 89 scenarios, 19,135 records per engine. All 6,948 changed records match reviewed transition patterns; 0 audit errors and 0 unexplained changes. No assessed pass became an assessed fail.
  • Three live scans completed with 0 report-invariant violations.
  • Isolated ESM/CommonJS consumers, built MCP stdio, CLI invalid-declaration handling, and legacy report readers passed.

The corpus replay uses saved responses and synthetic secondary-resource failures; it is not a full crawl or a public site-score comparison. Live invariants prove report consistency, not every finding's correctness. Existing ledger deferrals remain outside this migration. CI results are separate from this local proof.

Review entry points:

Type of Change

  • Breaking change
  • Documentation update
  • Refactor or internal chore

Affected Packages

  • @forkpoint/agent-lighthouse (CLI)
  • @forkpoint/agent-lighthouse-core
  • @forkpoint/agent-lighthouse-report
  • @forkpoint/agent-lighthouse-mcp
  • packages/website
  • Root configuration

Checklist

  • pnpm build
  • pnpm test
  • pnpm typecheck
  • pnpm lint
  • pnpm check:dossiers
  • pnpm check:requires
  • pnpm check:audit-map
  • Changeset files included for user-visible changes; release plan verified with pnpm exec changeset status.
  • Audit changes follow the documented evidence and applicability rules.

Apply common checks to eligible pages and separate declared page scope from detected advisory findings. Add selected-page coverage and a generated audit index.

BREAKING CHANGE: content becomes a legacy general-page declaration. Use article to select article obligations. Audit applicability and report contracts target 7.0.0 across all published packages.
Review fixes for the v7 applicability runner:
- Put the scan target first in each audit's page set and sort the rest by
  code point, so `pages[0]` is the requested URL again and the order no
  longer depends on the host locale (`CheckContext.targetUrl`).
- Tag a declared population whose every fetch failed `skipped:page-type`,
  so a 404 no longer counts toward the gated-mass unscored threshold.
- Turn a custom audit with conflicting page-type aliases into its own
  scan-error stub instead of aborting the scan.
- List per-audit coverage only for typed, advisory, partly unread or
  failing audits, so MCP and report output no longer carry ~215 URL lists.
- Drop the never-produced legacy `content` type from ten scope lists and
  remove the always-true `isArticlePage` helpers.

Enums:
- Every public string union in core is now a const object plus a type of
  the same name (CheckStatus, PageType, EvidenceKey, AuditTier, ...), and
  the Zod schemas read their values from those objects.
- 26 module-local unions became enums, 3 became constant lists, and 4
  duplicated unions now share one definition (HttpMethod, RuleStatus,
  SkipReason, StoreStatus).
- `AuditMetaSchema` validates page types against the enum.

`pnpm check:enums` uses the TypeScript checker to reject a string literal
wherever an enum is expected and any new string-literal union. It runs in
CI and is documented in CLAUDE.md.
Replace climbing relative imports (`../../types`) inside packages/core/src
with `#core/<path>`, a Node subpath import declared in the core
package.json. 518 files change; same-folder `./x` imports stay.

The mapping has three conditions:
- `agent-lighthouse-source` (a repo-only custom condition set in
  tsconfig.base.json) resolves TypeScript to `src/*.ts`;
- `types` resolves package users' TypeScript to `dist/*.d.ts`, because the
  emitted declarations keep the alias;
- `default` resolves Vite, tsx and esbuild to `src/*.ts`; the published
  bundle inlines every import, so it never resolves the alias at run time.

The named `#core/*` prefix is used instead of `#/*` because the published
declarations are read by package users' TypeScript, and `#/` resolves only
on TypeScript 6.0 or newer.
Map `#core/*` in the root package.json to packages/core/src, so scripts
import core source as `#core/<path>` instead of `../packages/core/src/...`
(16 files, plus the five scripts that imported the core entry as
`#core/index`).

Enforce both forms with oxlint's no-restricted-imports: a climbing `../`
import inside packages/core/src and a `packages/core/src` path inside
scripts are errors. The one test that imports a script from outside core
carries a commented suppression.
The fixture's package.json had no packageManager field, so the fixture ran
whatever pnpm the machine had globally. Under pnpm 11, `pnpm exec prettier`
(called by changesets) checks the dependency state first and refuses the
linked node_modules with ERR_PNPM_UNSAFE_MODULES_DIR. CI passed on pnpm 10;
local runs failed. Copy the root packageManager into the fixture.
A product page with a "recently viewed" rail read as a category page, so
every product audit was skipped. Four gaps combined: each part of one tile
that repeats the card class counted as a card, rail cards counted toward a
grid, carousel dots counted as result pagination, and a grid outranked the
page's own product evidence.

- The page's own product evidence (og:type=product, exactly one top-level
  product entity, or buy controls outside listing cards) outranks a grid.
- One top-level Product, ProductGroup, IndividualProduct or ProductModel,
  from JSON-LD, microdata or RDFa, and no listing schema, adds the strong
  signal `product-schema-primary`. Several top-level products stay a hint.
- Only outermost card matches count, and cards in recommendation, recently
  viewed, carousel, slider or aside regions do not count toward a grid.
- Carousel pagination (swiper, slick) is not result pagination.
- Buy controls inside listing cards do not count as the page's own. Once a
  page has two or more cards, each is a listing card at any wrapping depth;
  a lone element marked data-product-id, or a card holding the page's <h1>,
  keeps its controls.
…dits

Product audits read only Product nodes, so a brand or category declared
once on a ProductGroup was invisible, and offer-schema judged the first
Offer node in the page rather than the product's own offers. Both are
documented false findings: offer-schema's dossier already named the
short-circuit as a required fix.

- New `resolveProducts` (packages/core/src/product-schema.ts) returns each
  variant with the group's shared properties beneath its own, as Google's
  product variant documentation lays them out. Variants join through
  hasVariant, isVariantOf or inProductGroupWithID, and @id references in
  hasVariant and offers are followed. A ProductGroup with no variants is
  read as the product.
- offer-schema, product-identifiers, advanced-product-details,
  product-transaction-certainty, checkout-offer-field-mapping,
  landed-cost-and-returns and the product field summary use it.
- offer-schema passes on any priced offer reachable from the page's
  products: an Offer with price, an AggregateOffer with lowPrice, or an
  Offer whose itemOffered names the product, nested or by @id.
- checkout-offer-field-mapping reads an image list by its first URL.
- landed-cost-and-returns reads microdata and RDFa, as its dossier states,
  and takes the product's own offer before any other Offer node.

Dossiers record each change under Implementation deviations. Two corpus
verdicts move on the SaaS plans fixture: offer-schema fail -> pass (its
priced offers sat behind an empty first Offer stub) and
landed-cost-and-returns na -> fail (its products are microdata, which the
audit previously ignored).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant