CI Job for testing NPM

[MylesBorins]

There is not current a CI job for testing NPM afaik.

Should we roll one up, it should be fairly simple to do, in fact we should likely be able to do a small modification to the citgm CI job to do this.

[Fishrock123]

@thealphanerd what's the benefit to using citgm for thus over make test-npm?

[zkat]

As of nodejs/node#4960 and nodejs/node#4958, make test-npm should pass on non-windows platforms without any modifications. Note that there's some discussion in #4958 about false negatives due to progress bar config, so track that.

[MylesBorins]

@Fishrock123 I more meant that the way citgm is setup we could modify the job to call make test-npm instead of citgm and we'd have a CI job ready to run. Just the way it is designed is easy to modify

edit: The idea being cloning the ci job into a new job specifically for testing npm

[MylesBorins]

I just went ahead and made it...

https://ci.nodejs.org/job/thealphanerd-npm/

[MylesBorins]

I'm going to re-examine this as a possibility next week... /cc @nodejs/npm

edit: likely only using osx + windows as a target for now

[gibfahn]

@thealphanerd nodejs/node#7867 should mean that make test-npm and vcbuild.bat nosign test-npm generate TAP files (called test-npm.tap) in the node directory.

[MylesBorins]

@gibfahn the problems we were having in CI were unrelated to the output. Simply getting the suite to pass was not happening. that being said, tap output is a great start

[maclover7]

Very similar to #234 -- should they be combined or closed?

[gibfahn]

We have gibfahn-test-npm and gibfahn-test-npm-win, which pass on my PR branch (nodejs/node#11540). Once that lands I'll take my name off the jobs and give collaborators access.

Unfortunately I've never seen a green CI run on Linux (macOS is pretty good).