Release workflow

[acalcutt]

Please note, this is untested right now and I am giving it for review and/or ideas for #739.

This is a release workflow for node-pre-gyp. It is based on the node release workflow for maplibre-native at https://github.com/maplibre/maplibre-native/blob/main/.github/workflows/node-release.yml which I worked on getting working.

The idea behind this was, someone needs to approve and add changelog for each release, which should be a PR. When the version in package.json changes, this release workflow will check if that version has been published. If it has not published it, it will do that and also create a github release. the github release will also contain the text of the CHANGELOG.md for the version being published.

The workflow is also made to deal with pre-releases, so if the version is a pre-release, it will publish it as the next npm release

Whoever creates the PR for a new release would follow the instructions in
RELEASE.md

Needs 'NPM_TOKEN' Repository secret. A secret that contains a user npm token with permission to publish

[cclauss]

@mapsam Given all the progress that has been made in the past 24 hours, I believe that we need a provisional release of node-pre-gyp to https://www.npmjs.com/package/@mapbox/node-pre-gyp. This pre-release would allow multiple parties to kick the tires (for a week or two?) before we make a general release.

You or one of your colleagues would need to work with @acalcutt to add the required NPM_TOKEN as a repository secret.

If you have a different workflow for solving #739 then we would follow your lead.

• Needed: A GitHub Action to package and publish a new version #739

[acalcutt]

Just an FYI, if you did follow my instructions, when running the first command, any 'pre' version will make a pre-release/next release when published. So with a version incremted like
npm version preminor --preid pre --no-git-tag-version
would bring this package to 1.1.0-pre.0 from the current 1.0.11 and the workflow would publish it as a pre-release.

[acalcutt]

I gave this a test on my fork, and fixed a few issues and make it more similar to the ci. it should now be working correctly. I used it to make this test release

https://www.npmjs.com/package/@acalcutt/node-pre-gyp/v/2.0.0-pre.1
https://github.com/acalcutt/node-pre-gyp/releases/tag/v2.0.0-pre.1

it was made in this workflow run
https://github.com/acalcutt/node-pre-gyp/actions/runs/9735220985

[mapsam]

Nice work y'all 🙌 happy to get NPM_TOKEN added as a repo secret for the @mapbox/node-pre-gyp module. I'll report back when that has been added.

Update: see my comment in #739 (comment)

[mapsam]

@acalcutt @cclauss unsurprisingly, it's a little hard for me to get a static NPM token for our @mapbox/ namespace packages. Internally we're working on a better solution for public repos + npm + outside maintainers that will resolve this. For now I've set a reminder to refresh the token periodically before it is set to expire.

A couple questions:

• do you need me to modify any of the branch protections for the master branch? Currently no one is allowed to manually push to that branch
• are there protections to enforce only repo maintainers can trigger a workflow? Just want to be extra cautious outside contributors and forks cannot run the workflow themselves

[mapsam]

I've added an NPM_TOKEN secret to the repo 👍 I take it back - the tokens I have access to only have 1 hour lifecycles. Working on getting a longer term, static token.

[acalcutt]

do you need me to modify any of the branch protections for the master branch? Currently no one is allowed to manually push to that branch
This does not require any changes to branch protections.

are there protections to enforce only repo maintainers can trigger a workflow? Just want to be extra cautious outside contributors and forks cannot run the workflow themselves

The only way to trigger this would be to push a version change to the master branch on this repo into package.json. So either by a PR, which a maintainer would have to approve, or by a commit by someone who has admin rights to bypass branch protection.

If someone forks this repo, the workflow will not be able to publish because a valid 'NPM_TOKEN' will not exist in the new repository (it would only exist in this one). They could add their own NPM_TOKEN into their repo, but they would have to have permissions to update the package or it fails.

[mapsam]

Thanks for the quick response @acalcutt!

[acalcutt]

For example, to publish my test version, I had to do 3 things

1.) change my package.json package name
https://github.com/acalcutt/node-pre-gyp/blob/master/package.json#L2

2.) change the workflow to check my version of the package is published
https://github.com/acalcutt/node-pre-gyp/blob/master/.github/workflows/release.yml#L26

3.) Provide my own NPM_TOKEN repository secret with my own npm key, that has permission to @acalcutt/node-pre-gyp

[cclauss]

I agree that the branch protections should not be changed.

@acalcutt Would you be interested in becoming a maintainer of this repo?

[acalcutt]

I don't mind submitting PRs once and a while or helping out where I can, but I'm not sure how much of a maintainer I want to be. The things I have submitted so far are mainly stuff I know from trying to keep this limping along for my own projects.

I've mainly submitted these changes in hopes of seeing some up to date releases so I no longer need to maintain @acalcutt/node-pre-gyp. However in my current maplibre-native release I still publish node 16, so to move to this I would have to officially drop it (which i plan to do soon anyway), so using this package would be a breaking change once it is published.

[cclauss]

@mapsam Should this be merged or should something else be done in its place?

[mapsam]

@cclauss if the workflow works for you, it works for me!

My initial questions were just focused on blast-radius security because I wasn't able to get a well-scoped NPM token. I've since been able to generate an NPM_TOKEN only scoped to just the @mapbox/node-pre-gyp module, so my concerns there are settled. That token has been updated in the repo and should be available to test with now 👍

[acalcutt]

My feeling is it has been tested and works and can always be modified more in the future if another approach is found. It seems worth giving it a try.

…

On Fri, Jul 5, 2024 at 11:57 AM Sam Matthews ***@***.***> wrote: @cclauss <https://github.com/cclauss> if the workflow works for you, it works for me! My initial questions were just focused on blast-radius security because I wasn't able to get a well-scoped NPM token. I've since been able to generate an NPM_TOKEN only scoped to *just* the @mapbox/node-pre-gyp module, so my concerns there are settled. That token has been updated in the repo and should be available to test with now 👍 — Reply to this email directly, view it on GitHub <#744 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA454GC724A62KEYZHMEMNDZK267NAVCNFSM6AAAAABKDJPV2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJRGEYDONJXGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[benmccann]

@cclauss should we merge this and test it out to release a 2.0.0-rc.1 or something along those lines?

[acalcutt]

Sorry for the late reply. Let me know if you want me to make any more wording changes.

I am still around to help if this gets merged and there are any issues.

[cclauss]

Please move forward with this @benmccann

[acalcutt]

by the way, after merging this, if you wanted to do a ' 2.0.0-pre.0' prerelease from the current 1.x release, you would do the following in a PR (or directly in the repo if you are an admin)

1.) Bump the version to 2.0.0-pre.0
npm version premajor --preid pre --no-git-tag-version

2.) Add a changelog with the '2.0.0-pre.0' heading

3.) Commit the changes, so the workflow can pick it up

After that, you would wanted a '2.0.0-pre.1', you would do the same thing with
npm version prerelease --preid pre --no-git-tag-version

Or if you wanted a non-pre release of 2.0.0
npm version major--preid pre --no-git-tag-version

I know '2.0.0-rc.1' was mentioned above, but this isn't set up to make a pre-release for 'rc' only 'pre', so '2.0.0-pre.0' would be the expected result following the instructions here

[benmccann]

After that, you would wanted a '2.0.0-pre.1', you would do the same thing with
npm version prerelease --preid pre --no-git-tag-version

How does it know what version number to use if you've done a 2.0.0-pre.0 and 1.12.0-pre.0? Is it based off the package.json for the current branch?

Or if you wanted a non-pre release of 2.0.0
npm version major--preid pre --no-git-tag-version

Do you need to specify --preid pre here?

[acalcutt]

How does it know what version number to use if you've done a 2.0.0-pre.0 and 1.12.0-pre.0? Is it based off the package.json for the current branch?

The workflow reads it from the package.json from the branch it is running in. So if you submit a PR to master, this workflow in master would pick it up. The workflow is set to run on push to the master branch . npm publish in the workflow also reads the package.json in the branch the workflow is running on.

To run it on another branch, I usually just change this line to the new branch in the workflow file on that branch You can have two branches with different version number and it will work, provided the version being published doesn't already exist on npm (if it does the workflow will not run because the publish check will fail at https://github.com/mapbox/node-pre-gyp/pull/744/files#diff-87db21a973eed4fef5f32b267aa60fcee5cbdf03c67fafdc2a9b553bb0b15f34R22-R31 . You could also set the workflow to run on all braches by replacing 'master' with a '*', so when you make a branch that workflow would just run (but right now it is set to only run on master branch)

Do you need to specify --preid pre here?
I guess it doesn't matter if it is 'pre' here in this project, as long as it gets detected as a pre-release by this code in the workflow https://github.com/mapbox/node-pre-gyp/pull/744/files#diff-87db21a973eed4fef5f32b267aa60fcee5cbdf03c67fafdc2a9b553bb0b15f34R66-R74
(in the maplibre node project were I created this code for it matters because I am also using node-pre-gyp-github, which I added code to detect the 'pre'. to make a github pre-release... but for this project I guess it really doesn't matter as long as the workflow command picks it up as a pre-release)

[acalcutt]

I rebased this and ran a test in my forked repo. To test this I did the following

1.) In my forked repo I added my own npm key to the 'NPM_TOKEN' in repository secrets. This key does not copy over when you fork a repo (which is good)

2.) Because my npm key does not have write access to @mapbox/node-pre-gyp, I changed this so it would publish to @acalcutt/node-pre-gyp-test . I also changed the workflow to run in my branch 'test_release_12-7-2024'. Thoase changes look like this
WifiDB@e8e0dde

3.) I made a commit that changed the version number and updated the changelog header. I tested using 'rc' , so the command looked like
npm version premajor --preid rc --no-git-tag-version
WifiDB@b839bb5

4.) On commit, the workflow ran and published the changes
https://github.com/WifiDB/node-pre-gyp/actions/runs/12214759834

The github release got published with release notes
https://github.com/WifiDB/node-pre-gyp/releases/tag/v2.0.0-rc.1

On npm, the release was published as a 'next' release
https://www.npmjs.com/package/@acalcutt/node-pre-gyp-test/v/2.0.0-rc.1?activeTab=versions

[benmccann]

ok, let's try it!

[acalcutt]

Seems like it needs a plugin allowed (or changed to a supported version)

https://github.com/mapbox/node-pre-gyp/actions/runs/12214984091
martinbeentjes/npm-get-version-action@v1.3.1 is not allowed to be used in mapbox/node-pre-gyp. Actions in this workflow must be: within a repository that belongs to your Enterprise account, created by GitHub, or matching the following: anchore/*, aws-actions/*, bridgecrewio/*, cloudposse/*, docker/*, mapbox/*, octokit/*, smartlyio/*, 8398a7/action-slack@v3, amannn/action-semantic-pull-request@v5*, aquasecurity/trivy-action@0.12.0, arduino/setup-protoc@v1, ArtiomTr/jest-coverage-report-action@v2*, benchmark-action/github-action-benchmark@v1.14.0, codecov/codecov-action@v*, cycjimmy/semantic-release-action@v4*, dbt-checkpoint/dbt-checkpoint@v1*, gitleaks/gitleaks-action@v2, grafana/k6-action@v0.2.0, jenseng/dynamic-uses@v1, jwalton/gh-find-current-pr@v1*, ludeeus/action-shellcheck@2.0.0, ncipollo/release-action@v1, peter-evans/create-pull-request@v*, prisma-cloud-shiftleft/iac-scan-action@v*, ruby/setup-ruby@v*, slackapi/slack-github-action@v*, thollander/actions-comment-pull-request@...

[benmccann]

It looks like that action does almost nothing. Could we just copy that code into our workflow and remove use of that action?
https://github.com/martinbeentjes/npm-get-version-action/blob/main/entrypoint.sh

If you're up for sending a PR, I can merge it.

[acalcutt]

The action pulls the version number out of the commit package.json, which is used in the changelog and github publish steps steps (as steps.package-version.outputs.current-version in

node-pre-gyp/.github/workflows/release.yml

Lines 66 to 91 in 3eba8e4

	- name: Prepare release changelog
	id: prepare_release
	run: |
	RELEASE_TYPE="$(node -e "console.log(require('semver').prerelease('${{ steps.package-version.outputs.current-version }}') ? 'prerelease' : 'regular')")"
	if [[ $RELEASE_TYPE == 'regular' ]]; then
	echo "prerelease=false" >> "$GITHUB_OUTPUT"
	else
	echo "prerelease=true" >> "$GITHUB_OUTPUT"
	fi
	- name: Extract changelog for version
	run: |
	awk '/^##/ { p = 0 }; p == 1 { print }; $0 == "## ${{ steps.package-version.outputs.current-version }}" { p = 1 };' CHANGELOG.md > changelog_for_version.md
	cat changelog_for_version.md
	- name: Publish to Github
	uses: ncipollo/release-action@v1
	env:
	GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
	with:
	tag: v${{ steps.package-version.outputs.current-version }}
	name: v${{ steps.package-version.outputs.current-version }}
	bodyFile: changelog_for_version.md
	allowUpdates: true
	draft: false
	prerelease: ${{ steps.prepare_release.outputs.prerelease }}

)

[acalcutt]

It could possibly be modified to use the one pulled above in

node-pre-gyp/.github/workflows/release.yml

Line 28 in 3eba8e4

echo "currentVersion: $currentVersion"

like the prerelease tag does

[benmccann]

computing current-version is a 1-liner:

CURRENT_VERSION=$(cat ${PACKAGE_JSON_PATH}/package.json | jq '.version' | tr -d '"')

[acalcutt]

I added #887 to remove that action

[benmccann]

Awesome! Looks like it's working now as far as I can tell. Mind sending a PR for the release? I can't merge my own PR, but could merge yours 😄

I think we need to run npm version premajor --preid rc --no-git-tag-version and update Unreleased - 2.0.0 to 2.0.0-rc.0 in the CHANGELOG.md. I also noticed a typo for "automatically" on the last line of RELEASE.md that we could try to update at the same time