/bin/importmap outdated

[xdmx]

Given the new bin command to pin and unpin dependencies, what do you think about adding a new outdated command to check what needs to be updated (like bundler / yarn / etc) in order to make it easier and not having to check each single dependency (given that integrations like dependabot wouldn't support that... at least initially, maybe at some point?)

[dhh]

I’d like to see an audit feature that checked all the external pins for know issues. Not entirely sure how we get there though. Might need to be a cloud integration as well.

[xdmx]

Maybe it could use what npm uses: https://docs.npmjs.com/cli/v7/commands/npm-audit#bulk-advisory-endpoint

I don't know if there is any restriction/limit to use that, from a very quick look at the implementation I also didn't see any auth or similar.

[dhh]

Remember we can't rely on local node. That's the whole point. So wherever the audit info is coming from, it has to come from the web. But I think we could well do that.

[xdmx]

Sorry it wasn't clear, I didn't mean to use the local npm audit, but to use their endpoint instead (/-/npm/v1/security/advisories/bulk).

I've just tried it out and unfortunately their --verbose mode doesn't show the full request, but yarn actually does, and it also seems working when sending the request from Insomnia, even without the integrity field.

The only unknown is if they'd be ok for other tools to use their endpoints, or if that'd make them add some extra authentication / limits.

Npm audit steps and results

To try it out:

mkdir test-audit-npm
cd test-audit-npm
npm install stimulus
npm audit --verbose

The output:

npm verb cli [ '/usr/bin/node', '/usr/bin/npm', 'audit', '--verbose' ]
npm info using npm@7.21.0
npm info using node@v16.7.0
npm timing npm:load:whichnode Completed in 0ms
npm timing config:load:defaults Completed in 4ms
npm timing config:load:file:/usr/lib/node_modules/npm/npmrc Completed in 2ms
npm timing config:load:builtin Completed in 2ms
npm timing config:load:cli Completed in 2ms
npm timing config:load:env Completed in 1ms
npm timing config:load:file:~/Temp/github/test-audit-npm/.npmrc Completed in 0ms
npm timing config:load:project Completed in 2ms
npm timing config:load:file:~/.npmrc Completed in 0ms
npm timing config:load:user Completed in 0ms
npm timing config:load:file:/usr/etc/npmrc Completed in 0ms
npm timing config:load:global Completed in 0ms
npm timing config:load:validate Completed in 0ms
npm timing config:load:credentials Completed in 1ms
npm timing config:load:setEnvs Completed in 1ms
npm timing config:load Completed in 14ms
npm timing npm:load:configload Completed in 15ms
npm timing npm:load:setTitle Completed in 1ms
npm timing npm:load:setupLog Completed in 0ms
npm timing config:load:flatten Completed in 4ms
npm timing npm:load:cleanupLog Completed in 2ms
npm timing npm:load:configScope Completed in 0ms
npm timing npm:load:projectScope Completed in 1ms
npm timing npm:load Completed in 29ms
npm timing arborist:ctor Completed in 1ms
npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 317ms
npm timing auditReport:getReport Completed in 319ms
npm timing auditReport:init Completed in 0ms
npm timing audit Completed in 358ms
found 0 vulnerabilities
npm timing command:audit Completed in 365ms
npm verb exit 0
npm timing npm Completed in 649ms
npm info ok

Here you can see that it sends a request to https://registry.npmjs.org/-/npm/v1/security/advisories/bulk, but it's not showing the payload, so it'd need to be discovered from the npm code (or by sniffing the request)

Yarn audit steps and results

To try it out:

mkdir test-audit-yarn
cd test-audit-yarn
yarn add stimulus
yarn audit --verbose

The output:

yarn audit v1.22.11
warning package.json: No license field
verbose 0.26114966 Checking for configuration file "~/Temp/github/test-audit-yarn/.npmrc".
verbose 0.26222021 Checking for configuration file "~/.npmrc".
verbose 0.262716255 Checking for configuration file "/usr/etc/npmrc".
verbose 0.263143051 Checking for configuration file "~/Temp/github/test-audit-yarn/.npmrc".
verbose 0.263490966 Checking for configuration file "~/Temp/github/.npmrc".
verbose 0.263723979 Checking for configuration file "~/Temp/.npmrc".
verbose 0.264268748 Checking for configuration file "~/.npmrc".
verbose 0.264574011 Checking for configuration file "/home/.npmrc".
verbose 0.266041481 Checking for configuration file "~/Temp/github/test-audit-yarn/.yarnrc".
verbose 0.266508415 Checking for configuration file "~/.yarnrc".
verbose 0.26685633 Found configuration file "~/.yarnrc".
verbose 0.267847162 Checking for configuration file "/usr/etc/yarnrc".
verbose 0.268313818 Checking for configuration file "~/Temp/github/test-audit-yarn/.yarnrc".
verbose 0.26879255 Checking for configuration file "~/Temp/github/.yarnrc".
verbose 0.269251526 Checking for configuration file "~/Temp/.yarnrc".
verbose 0.269662407 Checking for configuration file "~/.yarnrc".
verbose 0.270002503 Found configuration file "~/.yarnrc".
verbose 0.270578545 Checking for configuration file "/home/.yarnrc".
verbose 0.277376359 current time: 2021-09-02T08:05:58.593Z
warning No license field
verbose 0.333025574 Audit Request: {
  "install": [],
  "remove": [],
  "metadata": {},
  "requires": {
    "stimulus": "^2.0.0"
  },
  "dependencies": {
    "stimulus": {
      "version": "2.0.0",
      "integrity": "sha512-xipy7BS5TVpg4fX6S8LhrYZp7cmHGjmk09WSAiVx1gF5S5g43IWsuetfUhIk8HfHUG+4MQ9nY0FQz4dRFLs/8w==",
      "requires": {
        "@stimulus/core": "^2.0.0",
        "@stimulus/webpack-helpers": "^2.0.0"
      },
      "dependencies": {},
      "dev": false
    },
    "@stimulus/core": {
      "version": "2.0.0",
      "integrity": "sha512-ff70GafKtzc8zQ1/cG+UvL06GcifPWovf2wBEdjLMh9xO2GOYURO3y2RYgzIGYUIBefQwyfX2CLfJdZFJrEPTw==",
      "requires": {
        "@stimulus/mutation-observers": "^2.0.0"
      },
      "dependencies": {},
      "dev": false
    },
    "@stimulus/webpack-helpers": {
      "version": "2.0.0",
      "integrity": "sha512-D6tJWsAC024MwGEIKlUVYU8Ln87mlrmiwHvYAjipg+s8H4eLxUMQ3PZkWyPevfipH+oR3leuHsjYsK1gN5ViQA==",
      "requires": {},
      "dependencies": {},
      "dev": false
    },
    "@stimulus/mutation-observers": {
      "version": "2.0.0",
      "integrity": "sha512-kx4VAJdPhIGBQKGIoUDC2tupEKorG3A+ckc2b1UiwInKTMAC1axOHU8ebcwhaJIxRqIrs8//4SJo9YAAOx6FEg==",
      "requires": {
        "@stimulus/multimap": "^2.0.0"
      },
      "dependencies": {},
      "dev": false
    },
    "@stimulus/multimap": {
      "version": "2.0.0",
      "integrity": "sha512-pMBCewkZCFVB3e5mEMoyO9+9aKzHDITmf3OnPun51YWxlcPdHcwbjqm1ylK63fsoduIE+RowBpFwFqd3poEz4w==",
      "requires": {},
      "dependencies": {},
      "dev": false
    }
  },
  "dev": false
}
verbose 0.367830888 Performing "POST" request to "https://registry.yarnpkg.com/-/npm/v1/security/audits".
verbose 0.796965307 Request "https://registry.yarnpkg.com/-/npm/v1/security/audits" finished with status code 200.
verbose 0.798833883 Audit Response: {
  "actions": [],
  "advisories": {},
  "muted": [],
  "metadata": {
    "vulnerabilities": {
      "info": 0,
      "low": 0,
      "moderate": 0,
      "high": 0,
      "critical": 0
    },
    "dependencies": 5,
    "devDependencies": 0,
    "optionalDependencies": 0,
    "totalDependencies": 5
  }
}
0 vulnerabilities found - Packages audited: 5
Done in 0.58s.

Yarn on the other hand is kindly showing the full payload, and trying it from Insomnia confirms that it's working without any extra authentication:

And it's also working without the integrity field:

[dhh]

Please do investigate this further! Would be great to build in.

[xdmx]

I was able to sniff the request through mitmproxy. The payload is even simpler, just the dependency name and the version(s).

Details

POST https://registry.npmjs.org/-/npm/v1/security/advisories/bulk

{
    "@stimulus/core": [
        "2.0.0"
    ],
    "@stimulus/multimap": [
        "2.0.0"
    ],
    "@stimulus/mutation-observers": [
        "2.0.0"
    ],
    "@stimulus/webpack-helpers": [
        "2.0.0"
    ],
    "glob-parent": [
	"3.1.0"
    ],
    "stimulus": [
        "2.0.0"
    ]
}

[xdmx]

While I was there I also checked how they check for the outdated packages. Interestingly for both npm and yarn there is no specific endpoint, they send one request for each package in the package.json (eg. https://registry.npmjs.org/stimulus / https://registry.yarnpkg.com/stimulus) and then compare the version from there.

[dhh]

That would be a good one to have too. Basically "importmap update" that would just update all the external dependencies to latest. Yes, we clearly are rebuilding a basic package manager here 😄. It'll be just as important to know when to stop!

[dhh]

I think there's a very fair critique that "npm audit" in its current form isn't actually all that great: https://twitter.com/dan_abramov/status/1412376404738686984

[ghiculescu]

Yes, please don't copy npm audit - that Twitter thread is a pretty good summary, it's a huge nuisance even on not-that-big apps.

We have found the security updates that dependabot provides to be higher quality (ie. less noise, more signal). I dunno what they are doing differently but it's working. It would be very cool if dependabot worked with importmap-rails somehow.

[dhh]

Maybe we can see if dependabot has an API or a library we can use?

[ghiculescu]

Based on https://dependabot.com/javascript/ my guess is it looks for yarn.lock or package.json and then parses it.

I wonder how the dependabot team would feel about adding config/importmap.rb to that list 😆

[ghiculescu]

Hmmmmm, maybe it's not as bad as it sounds.

https://github.com/dependabot/dependabot-core/blob/e9131296b83013fd9be7d4b8e4004b3a9e52a0c5/npm_and_yarn/lib/dependabot/npm_and_yarn/file_fetcher.rb#L34

https://github.com/dependabot/dependabot-core/tree/main/npm_and_yarn/lib/dependabot/npm_and_yarn/file_parser

I don't think it'd work if you moved the config file away from the default location, but that can be handled with documentation.

[ghiculescu]

(I'm evaluating importmaps but probably a little while away from using them away internally, so if someone's interested in taking this on please do - I'm not volunteering right now.)

[xdmx]

By reading that thread it seems that the problem of npm/yarn audit is that it gives a lot of false positives that could be related to development/build libraries, which wouldn't be included in production environments and thus not really a security issue (as the audit sends the entire yarn/package.lock).

Wouldn't pinned libraries through this gem be way less prone to this since they'd be mainly production libraries already? It'd just send those to check, and not any other deep/build dependency.

If that'd still be an issue, there could even be a audit: false if you want to skip them to be checked (eg. pin "md5", to: "https://cdn.jsdelivr.net/npm/md5@2.3.0/md5.js", audit: false)