doc: clarify release process for new releasers

[RafaelGSS]

A few nitpicks I got while releasing v18.5.

cc: @nodejs/releasers

[nodejs-github-bot]

Review requested:

• @nodejs/tsc

[richardlau]

I'm not sure about including this list -- it's only accurate for current and not, for example, Node.js 14 or 16.

[MylesBorins]

I'm not sure why this was resolved. I also don't think we should be including this list as it changes over time and isn't always accurate. The release tool does let you know if there are any missing binaries when promoting and I think that is good enough. We could make not about checking that.

[RafaelGSS]

In terms of the "Release build list", I concur it wouldn't be easy to maintain. I've added it there because I was confused to identify which binary I would need to test (rhel8 wasn't clear to me, I thought it was Red Hat special binaries).

Perhaps we can use wildcards to suppress versions? Example: vs*-x86 = Windows x86

[MylesBorins]

This seems like something that could be checked via automation / unit test. If we had a regression here that should have failed before ever making it to a release. Not saying that we shouldn't quickly check this, but it does seem a bit unnecessary to do manually for every release if we can automate this.

[RafaelGSS]

I think this was added just for the sake of conscience. I can totally remove it

[BethGriggs]

I recall catching a regression with these quick checks before.

I believe this was mostly an attempt to explain what we mean by 'test the build' as I think what we do at this step has been transferred mostly by pairing (and the steps are somewhat arbitrary). I'm +1 to documenting this (or a subset) for now, but also +1 to trying to automate a test.

As an aside: when I worked in a team that built our own binaries, we had a slight hack to run node's test suite against the built binaries. I'd always wondered if it was worth us doing something similar at this step:

$ cd ~/node/out/Release
$ rm node # remove the node in out/Release/node
$ ln -S /path/to/node/to/test node # symlink to binary we want to test
$ cd ~/node
$ tools/test.py

^{IIRC some tests did fail due to not expecting node to be a symlink, but I'm sure we could patch those.}

[richardlau]

IIRC we should be able to run the test suite against an arbitrary Node.js binary with test.py --shell.

[RafaelGSS]

I was able to run it with:

~/repos/os/node$ ./tools/test.py --shell ./node

Amazing! I'll add it to the doc

[RafaelGSS]

Actually, I was testing it against this branch binary. Switching to the v18.5.0 proposal I got several errors:

rafaelgss@rafaelgss-desktop:~/repos/os/node$ ./tools/test.py --shell ~/Downloads/node-v18.5.0-linux-x64/bin/node
=== release test-assert-async ===
Path: parallel/test-assert-async
node:internal/process/promises:288
            triggerUncaughtException(err, true /* fromPromise */);
            ^

AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
+ actual - expected

  Comparison {
    code: 'ERR_INVALID_RETURN_VALUE',
+   message: 'Expected instance of Promise to be returned from the "promiseFn" function but got type undefined.',
-   message: 'Expected instance of Promise to be returned from the "promiseFn" function but got undefined.',
    name: 'TypeError'
  }

It might be because I'm using the tools/test.py from this PR Branch

[richardlau]

Yes, there may be test issues if the tests being used don't match the version of Node.js being tested.

[RafaelGSS]

I've removed the build list and included a mention to test the binary using tools/test.py --shell

[nodejs-github-bot]

Landed in 2f169ad