Repository navigation
fix(local-collector): publish the package so the packaging fix ships - #73
Merged
Merged
Conversation
…ases on it `packages/local-collector` — the workspace that builds `@pdpp/local-collector`, a command-line tool that copies personal data out of local applications — ships a test that can never pass in this repository. On a clean checkout its suite reports 218 tests, 212 passing and 1 failing. The cause is an import naming a path this repository does not contain. `test/connect-scope.test.ts` reached for `../../reference-contract/src/evidence/named-collection-scope.ts` to compare this package's date arithmetic against `@pdpp/reference-contract`, a shared package that defines how a data-collection time window is calculated. No such file exists here, so the import fails with `ERR_MODULE_NOT_FOUND`. Because it is a dynamic `import()` inside the test body rather than a static top-level import, the rejection stays contained to that one test and the rest of the file runs and passes. The result is one permanently failing test that cannot be fixed in place. This replaces the import with a fixed table of expected timestamps. The property being protected is that `recentSinceDays` returns a timestamp the given number of whole UTC days before now. Confirmed the table still detects drift rather than passing vacuously: changing the day constant in `recentSinceDays` from 86,400,000 to 86,400,001 milliseconds fails the test with its intended message. The table pins this package's arithmetic against fixed examples; it is not a live equivalence check and cannot observe a later change to the other repository's resolver. The five entries derive from the file-local `NOW` constant, so changing `NOW` means recomputing all five. The second change adds a `Verify local-collector package` step to the `quality` job in `.github/workflows/npm-release.yml`, the workflow that publishes this repository's npm packages. That job previously checked only `@pdpp/connector-protocol` and `@pdpp/collector-runtime`, the two packages it actually publishes. `@pdpp/local-collector` is `private: true` here and absent from the publishing configuration, but it compiles against both of those packages and copies their build output into its own release archive, so a change that breaks it would have reached a release with no job failing. The step runs the package's `verify` script, which runs the tests and `validate:package` — a check that inspects the code about to be published and fails when an import cannot resolve inside the packed archive. On where this package is published: older releases identify pdpp as their source. The package moved to data-connect; this checkout does not configure its publication. This change adds a test gate, not a fixed local-collector release. Publishing must be configured in the owning repository. Scope limit: this makes the existing suite green and gates releases on it. It does not change any shipped runtime code under `src/`. Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
…o 1.3.0-1.5.4 `packages/local-collector/scripts/validate-package.ts` — the script that inspects compiled code before it is packaged for release — recorded which published versions of `@pdpp/local-collector` shipped a broken import. Two places said the defect covered 1.5.1 through 1.5.4. The range starts four releases earlier, at 1.3.0. The defect is a compiled `import ... from "@pdpp/reference-contract"`, a package that exists only inside a linked development workspace and not on the public npm registry, so an ordinary install produces a command that exits immediately with `ERR_MODULE_NOT_FOUND`, Node's error for an imported module it cannot find on disk. Verified by downloading each candidate release archive from the registry with `npm pack` and searching its compiled output for that import: 1.2.7 contains no such import, while 1.3.0, 1.4.0, 1.4.1, 1.5.0, 1.5.1 and 1.5.4 each do. The understated range matters because one of the two places is the diagnostic message a maintainer reads when this check fires. Someone triaging a report on 1.4.1 would have read the old text as evidence that their version predates the problem and looked elsewhere. This changes a comment and the text of that diagnostic message. Validation logic is unchanged. Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
@pdpp/local-collector's packaging fix has been on main since the package
moved to this repository, but no user can install it. Nothing publishes the
package: pdpp's .releaserc.yaml records that publishing responsibility moved
here, and this repository's npm-release workflow publishes only
@pdpp/connector-protocol and @pdpp/collector-runtime, while the package
itself is marked private at version 0.0.0. Every published version from
1.3.0 through 1.5.4 therefore remains unrunnable: those tarballs carry an
undeclared bare import of @pdpp/reference-contract, which resolves through
the workspace link for developers and fails closed on every real install
with ERR_MODULE_NOT_FOUND, including on --version.
Add the package to the existing lockstep release set. It publishes last,
after the two packages its prepack builds against. Versions continue above
the 1.5.4 line without a manual version field: semantic-release derives the
version from this repository's v${version} tags, whose highest is already
v2.1.0 (the published @pdpp/connector-protocol@2.1.0), so the next release
is at least 2.1.1. The version field stays a placeholder that
@semantic-release/npm overwrites at publish time, matching both sibling
packages.
Admitting local-collector to the commit-analyzer scope gate requires
updating the exclusion rule in the same step: it is written as the exact
complement of the allowlist, and any divergence lets one commit match both,
where the exclusion wins and silently cancels the whole release batch.
The release job gains one quality gate that reuses the package's existing
verify script, now extended to run the packed-consumer smoke test. That test
packs the tarball, installs it into a clean project, and drives the real CLI,
which is what catches the undeclared-import class of defect before publish
rather than after. Because quality must pass before release, a failure there
blocks publication. Provenance is enabled to match how the two existing
packages publish.
npm trusted publishing is configured per package on the registry, so
authorizing this workflow for @pdpp/local-collector is an external
prerequisite this branch cannot perform.
This builds on the test repair in #71 and merges after it: that change
replaced a cross-package source import, which named a path absent from this
repository and left one test permanently failing, with a pinned expectation
table that runs everywhere. This branch keeps that test as written. The
commit-analyzer test is updated here because it asserted that
local-collector commits release nothing, which is the premise this change
reverses.
Assisted-by: AI
Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
tnunamak
force-pushed
the
feat/publish-local-collector
branch
from
September 9, 2026 14:33
d8e480f to
e3a09a1
Compare
Contributor
Author
|
This addresses PDP-Connect/pdpp#266, the Assisted-by: AI |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
@pdpp/local-collectoris broken on npm and the fix cannot reach anyone.Every published version from
1.3.0through1.5.4(currentlatest) is unrunnable. Those tarballs contain a compiled bare import of@pdpp/reference-contract, a package that is not declared as a dependency and does not exist on the registry. It resolved for developers through the workspace link and failed closed on every real install, so any invocation exits immediately:The packaging fix has been on
mainsince the package moved to this repository, and the built output now contains no@pdpp/*bare imports at all. It reaches no user, because nothing publishes this package:PDP-Connect/pdpp's.releaserc.yamlrecords that publishing responsibility for@pdpp/local-collectormoved here, and it retains nopkgRootentry for it.npm-releaseworkflow publishes only@pdpp/connector-protocoland@pdpp/collector-runtime.packages/local-collector/package.jsonisprivate: trueat version0.0.0, which blocks publishing. It does not blocknpm pack: a control run ofnpm pack --ignore-scriptsagainst the base's exact private manifest succeeds and producespdpp-local-collector-0.0.0.tgz.The commit-analyzer scope gate closed the loop:
fix(local-collector)was explicitly listed as a scope that releases nothing.Approach
The package joins the existing lockstep release set rather than getting a parallel pipeline.
Versioning. The next publish must exceed
1.5.4, and it does so without a manual version field. semantic-release derives the version from this repository'sv${version}tags; the highest isv2.1.0, which is the commit published as@pdpp/connector-protocol@2.1.0(gitHead 187838be1). The next release is therefore at least2.1.1, above the1.5.4line. Theversionfield stays a placeholder that@semantic-release/npmoverwrites at publish time, exactly as both sibling packages already do. The package name is unchanged.Publish set.
packages/local-collectoris added as an@semantic-release/npmpkgRoot, ordered last, after the two packages itsprepackbuilds against.publishConfig.provenanceflips totrueand the package is added to the workflow's existinggh attestation verifyloop.Plugin order determines invocation sequence, not atomicity. semantic-release tags before running publish hooks, so a failure partway through can leave a partial release, and a plain re-run is not guaranteed to repair it: the existing tag can make the next dry-run see no unreleased commit. The attestation loop runs after publication and cannot roll anything back.
Scope gate. Admitting
local-collectorto the allowlist required updating the exclusion rule in the same step. That rule is written as the exact complement of the allowlist, and the config's own header documents why: if a commit matches both a real rule and the exclusion, the exclusion'sfalsewins and cancels the entire release batch, including unrelated real releases.Release gate. The
qualityjob runs one step calling the package'sverifyscript, now extended to runpack-install-runalongsidevalidate:package. #71 added that step for downstream-breakage coverage while the package was unpublished; this branch keeps a single step and updates its comment for the package now being published here.validate:packagerejects any bare import the manifest does not declare.pack-install-runpacks the tarball, installs it into a clean project, and drives the real CLI through advertise, enroll, and run against an in-process reference server. Becausequalitymust pass beforerelease, a failure blocks publication.Verification
Run locally on the rebased head:
npm run --workspace packages/local-collector verifyexits 0: 218 tests, 213 pass, 0 fail, 5 skipped, ending inPASS pack-install-run local smoke. Wall time was 14-16s on this host across cold- and warm-build runs; an independent reviewer measured 113-119s on different hardware, so treat the duration as host-dependent rather than a fixed budget.@pdpp/reference-contractimport to a builtdist/file makesvalidate:packagethrow andverifyexit non-zero.npm packproduces a 92-file, 148.3 kB tarball. Installed into a clean temp project,pdpp-local-collector --helpprints usage.enroll --base-url http://127.0.0.1:9 --code probe-coderesolves its full module graph and fails only at the network call, with noERR_MODULE_NOT_FOUND. Omitting--codeexits 64 on argument validation before any network work, so the--codeargument is required for this to be a resolution test.1.5.4in a clean project reproduce the crash quoted above.npm publish --dry-run --provenance --access publicsucceeds. This exercises packing and manifest handling only; it is not an authenticated publish and proves nothing about registry authorization.@semantic-release/commit-analyzer, as the config header instructs. All 11 cases pass, including the batch case where a realfix(local-collector)commit sits alongside unrelated scoped and unscoped commits and still yieldspatch.npx vitest run scripts/npm-release-commit-analyzer.test.ts scripts/npm-release-signer-workflow.test.ts: 23 passed.A semantic-release dry-run confirms the config loads and registers all three npm publish roots in order. An independent reviewer, running the same dry-run against a scratch clone with the candidate tree named
main, observed the analyzer select2.1.1from tagv2.1.0. Attestation verification still cannot be exercised before a real release, unchanged from the note already in the workflow.npm trusted publishing: an external prerequisite
Publishing uses OIDC trusted publishing (
id-token: write), not a token.gh api /repos/PDP-Connect/data-connect/actions/secretsreturns{"total_count":0,"secrets":[]}, so there is noNPM_TOKENrepository secret and none is proposed.Trusted publishing is configured per package on npm, and this branch cannot configure it. Before merging, confirm or create for
@pdpp/local-collectora Trusted Publisher of type GitHub Actions with organizationPDP-Connect, repositorydata-connect, and workflow filenamenpm-release.yml— the filename alone in that field, not the full.github/workflows/path — with no environment restriction, since the release job declares none. The authorization must permit directnpm publish.The package's existing provenance attests to how it was published historically; it does not establish its current trusted-publisher settings, which were not inspected here.
If that authorization is missing or mismatched, the failure is fail-closed and early:
@semantic-release/npmverifies auth for each package duringverifyConditions, before any prepare, tag, or publish step, and theresolve-versiondry-run invokes verification too. An independent reviewer confirmed this against the installed plugin by simulating a 403 OIDC exchange, which threwENONPMTOKEN.Follow-up after this lands
Once a corrected version is published, deprecate the broken range, substituting the actual published version for
<NEXT>:This is an unexecuted maintainer step: it needs the real published version and npm write credentials.
Notes
The README documents installing and upgrading and records the affected
1.3.0-1.5.4range with its error signature. It states that@lateststill resolves to an affected version until a corrected release exists, and describes the gates as detecting the import failures they test for rather than as a guarantee against every packaging defect.The commit-analyzer test is updated here because it asserted that
local-collectorcommits release nothing, which is the premise this change reverses. Theconnect-scope.test.tsrepair belongs to #71 and is unchanged by this branch.Assisted-by: AI