Skip to content

ci(upstream): report how far the fork differs from upstream - #698

Merged
incognitojam merged 1 commit into
mainfrom
ci/upstream-divergence-report
Oct 10, 2026
Merged

incognitojam merged 1 commit into
mainfrom
ci/upstream-divergence-report

Conversation

@incognitojam

Copy link
Copy Markdown
Owner

Intake reports show how far the fork trails upstream, but not how much the fork differs from it. This adds scripts/upstream/divergence-report.ts, which compares a fork commit with upstream at the in-order tip, so upstream work the fork has not taken in yet does not count. It excludes .repos/ and pnpm-lock.yaml and keeps upstream files the fork changes or removes separate from fork-only files, since changed upstream files are what cause intake conflicts.

The report appears in three places:

  • Upstream lag report (daily): current divergence, daily charts since the merge base, and a breakdown by area.
  • Promote upstream intake: a projected comparison of main with the candidate SHA, next to the projected lag report.
  • Upstream divergence comment (new, pull_request_target): a single comment on each fork pull request, updated in place, showing the change from its base. It runs the report from the base branch and reads the PR's commits only as git data.

Example comment for fork #695:

Measure Base This PR Change
Upstream files the fork changes 1,187 1,187 0
Lines changed in them 63,644 63,707 +63
Upstream files the fork removes 37 37 0
Fork-only files 337 339 +2
Fork-only lines 47,127 47,265 +138

Below the table, the comment lists upstream files that start to differ, files that match upstream again, and files upstream deleted that the fork keeps. Per-file changes are in a collapsed table. Run against the last intake batch (5a4df64447..7abfa49110), the promotion comparison listed scripts/lint-restyle-ceiling.ts and its test as deleted upstream by pingdotgg#13210 and kept by the fork, which the fork feature ledger records as intentional until Fork CI stops calling them.

Today fork main changes 1,188 upstream files (+51,421 −12,343 lines) and removes 37, which is 27.4% of upstream's files and 5.3% of its lines. It also has 341 fork-only files (+47,373 lines).

Validation

  • divergence-report.test.ts builds disposable git repositories. It checks that the counts split into changed, removed and fork-only files with the excluded paths ignored, and that comparisons list files that start to differ, return to upstream, or survive an upstream deletion.
  • Ran each workflow's exact command against local refs: the full history report (about 17s), the promotion comparison for the last intake batch, and PR comments for fork fix(sidebar): the settled shelf closes once you move on #695 and fix(web): use component variants instead of className overrides in fork UI #691 (about 2s each), and checked the numbers against direct git diff output.
  • Not verified: the comment create/update step only runs in Actions. Because pull_request_target uses the base branch's workflow, this PR will not get a comment; the first comment appears on the next fork PR after merge.

Written by an agent (Claude Code, claude-opus-5-5).

Compare fork commits with upstream at the in-order tip, excluding
.repos/ and pnpm-lock.yaml, and separate upstream files the fork
changes or removes from fork-only files. The daily lag report charts
it, promotion validation projects the candidate's change, and a new
workflow posts the change on each fork pull request.
@incognitojam
incognitojam merged commit 9eb0c0b into main Oct 10, 2026
20 checks passed
@incognitojam
incognitojam deleted the ci/upstream-divergence-report branch October 10, 2026 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant