Skip to content

fix(local-collector): publish the package so the packaging fix ships - #73

Merged
tnunamak merged 3 commits into
mainfrom
feat/publish-local-collector
Sep 9, 2026
Merged

tnunamak merged 3 commits into
mainfrom
feat/publish-local-collector

Conversation

@tnunamak

@tnunamak tnunamak commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Merges after #71. This branch is rebased on that PR's head, and keeps its pinned-expectation test exactly as approved there.

Problem

@pdpp/local-collector is broken on npm and the fix cannot reach anyone.

Every published version from 1.3.0 through 1.5.4 (current latest) is unrunnable. Those tarballs contain a compiled bare import of @pdpp/reference-contract, a package that is not declared as a dependency and does not exist on the registry. It resolved for developers through the workspace link and failed closed on every real install, so any invocation exits immediately:

$ npm i -g @pdpp/local-collector@1.5.4 && pdpp-local-collector --help
Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@pdpp/reference-contract'
  imported from .../@pdpp/local-collector/dist/polyfill-connectors/src/local-device-client.js

The packaging fix has been on main since the package moved to this repository, and the built output now contains no @pdpp/* bare imports at all. It reaches no user, because nothing publishes this package:

  • PDP-Connect/pdpp's .releaserc.yaml records that publishing responsibility for @pdpp/local-collector moved here, and it retains no pkgRoot entry for it.
  • This repository's npm-release workflow publishes only @pdpp/connector-protocol and @pdpp/collector-runtime.
  • packages/local-collector/package.json is private: true at version 0.0.0, which blocks publishing. It does not block npm pack: a control run of npm pack --ignore-scripts against the base's exact private manifest succeeds and produces pdpp-local-collector-0.0.0.tgz.

The commit-analyzer scope gate closed the loop: fix(local-collector) was explicitly listed as a scope that releases nothing.

Approach

The package joins the existing lockstep release set rather than getting a parallel pipeline.

Versioning. The next publish must exceed 1.5.4, and it does so without a manual version field. semantic-release derives the version from this repository's v${version} tags; the highest is v2.1.0, which is the commit published as @pdpp/connector-protocol@2.1.0 (gitHead 187838be1). The next release is therefore at least 2.1.1, above the 1.5.4 line. The version field stays a placeholder that @semantic-release/npm overwrites at publish time, exactly as both sibling packages already do. The package name is unchanged.

Publish set. packages/local-collector is added as an @semantic-release/npm pkgRoot, ordered last, after the two packages its prepack builds against. publishConfig.provenance flips to true and the package is added to the workflow's existing gh attestation verify loop.

Plugin order determines invocation sequence, not atomicity. semantic-release tags before running publish hooks, so a failure partway through can leave a partial release, and a plain re-run is not guaranteed to repair it: the existing tag can make the next dry-run see no unreleased commit. The attestation loop runs after publication and cannot roll anything back.

Scope gate. Admitting local-collector to the allowlist required updating the exclusion rule in the same step. That rule is written as the exact complement of the allowlist, and the config's own header documents why: if a commit matches both a real rule and the exclusion, the exclusion's false wins and cancels the entire release batch, including unrelated real releases.

Release gate. The quality job runs one step calling the package's verify script, now extended to run pack-install-run alongside validate:package. #71 added that step for downstream-breakage coverage while the package was unpublished; this branch keeps a single step and updates its comment for the package now being published here. validate:package rejects any bare import the manifest does not declare. pack-install-run packs the tarball, installs it into a clean project, and drives the real CLI through advertise, enroll, and run against an in-process reference server. Because quality must pass before release, a failure blocks publication.

Verification

Run locally on the rebased head:

  • npm run --workspace packages/local-collector verify exits 0: 218 tests, 213 pass, 0 fail, 5 skipped, ending in PASS pack-install-run local smoke. Wall time was 14-16s on this host across cold- and warm-build runs; an independent reviewer measured 113-119s on different hardware, so treat the duration as host-dependent rather than a fixed budget.
  • A sabotage control confirms the gate bites: appending an undeclared @pdpp/reference-contract import to a built dist/ file makes validate:package throw and verify exit non-zero.
  • npm pack produces a 92-file, 148.3 kB tarball. Installed into a clean temp project, pdpp-local-collector --help prints usage. enroll --base-url http://127.0.0.1:9 --code probe-code resolves its full module graph and fails only at the network call, with no ERR_MODULE_NOT_FOUND. Omitting --code exits 64 on argument validation before any network work, so the --code argument is required for this to be a resolution test.
  • The same commands against published 1.5.4 in a clean project reproduce the crash quoted above.
  • npm publish --dry-run --provenance --access public succeeds. This exercises packing and manifest handling only; it is not an authenticated publish and proves nothing about registry authorization.
  • The scope-gate rules were re-derived against the installed @semantic-release/commit-analyzer, as the config header instructs. All 11 cases pass, including the batch case where a real fix(local-collector) commit sits alongside unrelated scoped and unscoped commits and still yields patch.
  • npx vitest run scripts/npm-release-commit-analyzer.test.ts scripts/npm-release-signer-workflow.test.ts: 23 passed.

A semantic-release dry-run confirms the config loads and registers all three npm publish roots in order. An independent reviewer, running the same dry-run against a scratch clone with the candidate tree named main, observed the analyzer select 2.1.1 from tag v2.1.0. Attestation verification still cannot be exercised before a real release, unchanged from the note already in the workflow.

npm trusted publishing: an external prerequisite

Publishing uses OIDC trusted publishing (id-token: write), not a token. gh api /repos/PDP-Connect/data-connect/actions/secrets returns {"total_count":0,"secrets":[]}, so there is no NPM_TOKEN repository secret and none is proposed.

Trusted publishing is configured per package on npm, and this branch cannot configure it. Before merging, confirm or create for @pdpp/local-collector a Trusted Publisher of type GitHub Actions with organization PDP-Connect, repository data-connect, and workflow filename npm-release.yml — the filename alone in that field, not the full .github/workflows/ path — with no environment restriction, since the release job declares none. The authorization must permit direct npm publish.

The package's existing provenance attests to how it was published historically; it does not establish its current trusted-publisher settings, which were not inspected here.

If that authorization is missing or mismatched, the failure is fail-closed and early: @semantic-release/npm verifies auth for each package during verifyConditions, before any prepare, tag, or publish step, and the resolve-version dry-run invokes verification too. An independent reviewer confirmed this against the installed plugin by simulating a 403 OIDC exchange, which threw ENONPMTOKEN.

Follow-up after this lands

Once a corrected version is published, deprecate the broken range, substituting the actual published version for <NEXT>:

npm deprecate "@pdpp/local-collector@>=1.3.0 <NEXT" "Unrunnable: missing @pdpp/reference-contract dependency causes ERR_MODULE_NOT_FOUND on every install. Upgrade with npm i -g @pdpp/local-collector@latest"

This is an unexecuted maintainer step: it needs the real published version and npm write credentials.

Notes

The README documents installing and upgrading and records the affected 1.3.0-1.5.4 range with its error signature. It states that @latest still resolves to an affected version until a corrected release exists, and describes the gates as detecting the import failures they test for rather than as a guarantee against every packaging defect.

The commit-analyzer test is updated here because it asserted that local-collector commits release nothing, which is the premise this change reverses. The connect-scope.test.ts repair belongs to #71 and is unchanged by this branch.

Assisted-by: AI

…ases on it

`packages/local-collector` — the workspace that builds `@pdpp/local-collector`,
a command-line tool that copies personal data out of local applications —
ships a test that can never pass in this repository. On a clean checkout its
suite reports 218 tests, 212 passing and 1 failing.

The cause is an import naming a path this repository does not contain.
`test/connect-scope.test.ts` reached for
`../../reference-contract/src/evidence/named-collection-scope.ts` to compare
this package's date arithmetic against `@pdpp/reference-contract`, a shared
package that defines how a data-collection time window is calculated. No such
file exists here, so the import fails with `ERR_MODULE_NOT_FOUND`. Because it
is a dynamic `import()` inside the test body rather than a static top-level
import, the rejection stays contained to that one test and the rest of the
file runs and passes. The result is one permanently failing test that cannot
be fixed in place.

This replaces the import with a fixed table of expected timestamps. The
property being protected is that `recentSinceDays` returns a timestamp the
given number of whole UTC days before now. Confirmed the table still detects
drift rather than passing vacuously: changing the day constant in
`recentSinceDays` from 86,400,000 to 86,400,001 milliseconds fails the test
with its intended message. The table pins this package's arithmetic against
fixed examples; it is not a live equivalence check and cannot observe a later
change to the other repository's resolver. The five entries derive from the
file-local `NOW` constant, so changing `NOW` means recomputing all five.

The second change adds a `Verify local-collector package` step to the
`quality` job in `.github/workflows/npm-release.yml`, the workflow that
publishes this repository's npm packages. That job previously checked only
`@pdpp/connector-protocol` and `@pdpp/collector-runtime`, the two packages it
actually publishes. `@pdpp/local-collector` is `private: true` here and absent
from the publishing configuration, but it compiles against both of those
packages and copies their build output into its own release archive, so a
change that breaks it would have reached a release with no job failing. The
step runs the package's `verify` script, which runs the tests and
`validate:package` — a check that inspects the code about to be published and
fails when an import cannot resolve inside the packed archive.

On where this package is published: older releases identify pdpp as their
source. The package moved to data-connect; this checkout does not configure
its publication. This change adds a test gate, not a fixed local-collector
release. Publishing must be configured in the owning repository.

Scope limit: this makes the existing suite green and gates releases on it. It
does not change any shipped runtime code under `src/`.

Assisted-by: AI
Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
…o 1.3.0-1.5.4

`packages/local-collector/scripts/validate-package.ts` — the script that
inspects compiled code before it is packaged for release — recorded which
published versions of `@pdpp/local-collector` shipped a broken import. Two
places said the defect covered 1.5.1 through 1.5.4. The range starts four
releases earlier, at 1.3.0.

The defect is a compiled `import ... from "@pdpp/reference-contract"`, a
package that exists only inside a linked development workspace and not on the
public npm registry, so an ordinary install produces a command that exits
immediately with `ERR_MODULE_NOT_FOUND`, Node's error for an imported module
it cannot find on disk.

Verified by downloading each candidate release archive from the registry with
`npm pack` and searching its compiled output for that import: 1.2.7 contains
no such import, while 1.3.0, 1.4.0, 1.4.1, 1.5.0, 1.5.1 and 1.5.4 each do.

The understated range matters because one of the two places is the diagnostic
message a maintainer reads when this check fires. Someone triaging a report on
1.4.1 would have read the old text as evidence that their version predates the
problem and looked elsewhere.

This changes a comment and the text of that diagnostic message. Validation
logic is unchanged.

Assisted-by: AI
Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
@pdpp/local-collector's packaging fix has been on main since the package
moved to this repository, but no user can install it. Nothing publishes the
package: pdpp's .releaserc.yaml records that publishing responsibility moved
here, and this repository's npm-release workflow publishes only
@pdpp/connector-protocol and @pdpp/collector-runtime, while the package
itself is marked private at version 0.0.0. Every published version from
1.3.0 through 1.5.4 therefore remains unrunnable: those tarballs carry an
undeclared bare import of @pdpp/reference-contract, which resolves through
the workspace link for developers and fails closed on every real install
with ERR_MODULE_NOT_FOUND, including on --version.

Add the package to the existing lockstep release set. It publishes last,
after the two packages its prepack builds against. Versions continue above
the 1.5.4 line without a manual version field: semantic-release derives the
version from this repository's v${version} tags, whose highest is already
v2.1.0 (the published @pdpp/connector-protocol@2.1.0), so the next release
is at least 2.1.1. The version field stays a placeholder that
@semantic-release/npm overwrites at publish time, matching both sibling
packages.

Admitting local-collector to the commit-analyzer scope gate requires
updating the exclusion rule in the same step: it is written as the exact
complement of the allowlist, and any divergence lets one commit match both,
where the exclusion wins and silently cancels the whole release batch.

The release job gains one quality gate that reuses the package's existing
verify script, now extended to run the packed-consumer smoke test. That test
packs the tarball, installs it into a clean project, and drives the real CLI,
which is what catches the undeclared-import class of defect before publish
rather than after. Because quality must pass before release, a failure there
blocks publication. Provenance is enabled to match how the two existing
packages publish.

npm trusted publishing is configured per package on the registry, so
authorizing this workflow for @pdpp/local-collector is an external
prerequisite this branch cannot perform.

This builds on the test repair in #71 and merges after it: that change
replaced a cross-package source import, which named a path absent from this
repository and left one test permanently failing, with a pinned expectation
table that runs everywhere. This branch keeps that test as written. The
commit-analyzer test is updated here because it asserted that
local-collector commits release nothing, which is the premise this change
reverses.

Assisted-by: AI
Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
@tnunamak
tnunamak force-pushed the feat/publish-local-collector branch from d8e480f to e3a09a1 Compare September 9, 2026 14:33
@tnunamak
tnunamak merged commit b3c61de into main Sep 9, 2026
12 checks passed
@tnunamak
tnunamak deleted the feat/publish-local-collector branch September 9, 2026 15:15
@tnunamak

Copy link
Copy Markdown
Contributor Author

This addresses PDP-Connect/pdpp#266, the ERR_MODULE_NOT_FOUND report against published @pdpp/local-collector.

Assisted-by: AI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant