`dotnet list package --vulnerable` too verbose / poor SNR

[richlander]

I am using NuGet Audit via dotnet. I would expect the basic experience to have very good signal to noise characteristics, which is particularly important for large solutions. I believe that these is significant opportunity for improvement.

I noticed that the good folks at Jellyfin have adopted CPM, so I used their project as a test bed. In particular, I downgraded their 8.x reference to all 8.0.0 so that I'd find some vulnerabilities. For clarity, this is all artificial; Jellyfin doesn't have any of these vulnerabilities.

This is what I see:

rich@Richs-MacBook-Air jellyfin % dotnet list package --vulnerable

The following sources were used:
   https://api.nuget.org/v3/index.json

The given project `Jellyfin.Server` has no vulnerable packages given the current sources.
The given project `MediaBrowser.Controller` has no vulnerable packages given the current sources.
The given project `MediaBrowser.Common` has no vulnerable packages given the current sources.
Project `MediaBrowser.Model` has the following vulnerable packages
   [net8.0]: 
   Top-level Package       Requested   Resolved   Severity   Advisory URL                                     
   > System.Text.Json      8.0.0       8.0.0      High       https://github.com/advisories/GHSA-hh2w-p6rv-4g7w

The given project `MediaBrowser.Providers` has no vulnerable packages given the current sources.
The given project `MediaBrowser.XbmcMetadata` has no vulnerable packages given the current sources.
The given project `MediaBrowser.LocalMetadata` has no vulnerable packages given the current sources.
The given project `Jellyfin.Drawing` has no vulnerable packages given the current sources.
The given project `Emby.Photos` has no vulnerable packages given the current sources.
The given project `Emby.Server.Implementations` has no vulnerable packages given the current sources.
The given project `Emby.Naming` has no vulnerable packages given the current sources.
The given project `MediaBrowser.MediaEncoding` has no vulnerable packages given the current sources.
The given project `Jellyfin.Drawing.Skia` has no vulnerable packages given the current sources.
The given project `Jellyfin.Api` has no vulnerable packages given the current sources.
The given project `Jellyfin.Common.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.MediaEncoding.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.Naming.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.Api.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.Server.Implementations.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.Controller.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.Data` has no vulnerable packages given the current sources.
The given project `Jellyfin.Server.Implementations` has no vulnerable packages given the current sources.
The given project `Jellyfin.Networking` has no vulnerable packages given the current sources.
The given project `Jellyfin.XbmcMetadata.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.Model.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.Networking.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.Server.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.Server.Integration.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.Providers.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.Extensions` has no vulnerable packages given the current sources.
The given project `Jellyfin.Extensions.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.MediaEncoding.Keyframes` has no vulnerable packages given the current sources.
The given project `Jellyfin.MediaEncoding.Hls` has no vulnerable packages given the current sources.
The given project `Jellyfin.MediaEncoding.Hls.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.MediaEncoding.Keyframes.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.LiveTv.Tests` has no vulnerable packages given the current sources.
The given project `Jellyfin.LiveTv` has no vulnerable packages given the current sources.

This doesn't strike me as the desired output. I only want to see vuln hits and not have to search through pages of terminal data to find them. This also isn't a desired machine-readable format. If this was JSON, I could use jq to filter this ouput. grep won't really do it.

This is what Docker Scout does, in contrast.

rich@Richs-MacBook-Air jellyfin % docker scout cves mcr.microsoft.com/dotnet/runtime:8.0.7           
    ✓ SBOM of image already cached, 298 packages indexed
    ✗ Detected 11 vulnerable packages with a total of 17 vulnerabilities


## Overview

                    │              Analyzed Image               
────────────────────┼───────────────────────────────────────────
  Target            │  mcr.microsoft.com/dotnet/runtime:8.0.7   
    digest          │  01f928714573                             
    platform        │ linux/arm64                               
    vulnerabilities │    0C     0H     0M    15L     2?         
    size            │ 78 MB                                     
    packages        │ 298                                       


## Packages and Vulnerabilities

   0C     0H     0M     3L     2?  openssl 3.0.13-1~deb12u1
pkg:deb/debian/openssl@3.0.13-1~deb12u1?os_distro=bookworm&os_name=debian&os_version=12

    ✗ LOW CVE-2024-6119
      https://scout.docker.com/v/CVE-2024-6119
      Affected range : <3.0.14-1~deb12u2  
      Fixed version  : 3.0.14-1~deb12u2   
    
    ✗ LOW CVE-2024-4603
      https://scout.docker.com/v/CVE-2024-4603
      Affected range : <3.0.14-1~deb12u1  
      Fixed version  : 3.0.14-1~deb12u1   
    
    ✗ LOW CVE-2010-0928
      https://scout.docker.com/v/CVE-2010-0928
      Affected range : >=3.0.11-1~deb12u2  
      Fixed version  : not fixed           
    
// snip

17 vulnerabilities found in 11 packages
  UNSPECIFIED  2   
  LOW          15  
  MEDIUM       0   
  HIGH         0   
  CRITICAL     0   


What's next:
    View base image update recommendations → docker scout recommendations mcr.microsoft.com/dotnet/runtime:8.0.7

I can then follow that recommendation, too.

rich@Richs-MacBook-Air jellyfin % docker scout recommendations mcr.microsoft.com/dotnet/runtime:8.0.7
    ✓ SBOM of image already cached, 298 packages indexed

    i Base image was auto-detected. To get more accurate recommendations, build images with max-mode provenance attestations.
      Review docs.docker.com ↗ for more information.
      Alternatively, use  docker scout recommendations --tag <base image tag>  to pass a specific base image tag.
      
  Target   │  mcr.microsoft.com/dotnet/runtime:8.0.7   
    digest │  01f928714573                             

## Recommended fixes

  Base image is  debian:12-slim 

  Name            │  12-slim                                                                   
  Digest          │  sha256:19641d0a8330497653423d04744b1c92300f64dadbdd830172d98af7353592bb   
  Vulnerabilities │    0C     0H     0M    12L                                                 
  Pushed          │ 1 month ago                                                                
  Size            │ 29 MB                                                                      
  Packages        │ 125                                                                        
  Flavor          │ debian                                                                     
  OS              │ 12                                                                         
  Slim            │ ✓                                                                          

                                                                    
  │ The base image is also available under the supported tag(s)      
  │ bookworm-slim . If you want to display recommendations           
  │ specifically for a different tag, please re-run the command using
  │ the  --tag  flag.                                                



Refresh base image
  Rebuild the image using a newer base image version. Updating this may result in breaking changes.

That's pretty good!

I propose that NuGet audit move towards a default output with higher SNR that is oriented on vulnerabilities not on packages. This is what Docker Scout does. It would be good if there was a JSON output (perhaps I just missed it).

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[JonDouglas]

Some quick design thoughts:

1. We shouldn't output for a scanned project if nothing detected.
2. We should highlight the actual output for projects where there is detection.
3. We should summarize how many projects were scanned in the solution to inform user that things were scanned and they could validate that with another command to see how many projects exist in the solution like dotnet sln list or dotnet sln list | wc -l
4. Not really in love with the extended output to begin with. Should be able to simplify what audit source is/is not being used too given that feature recently launched.

Here's a design from 2021 that tries to do this: https://github.com/NuGet/Home/blob/0310d373a1c20240863ccbb0574aae481d45d873/proposed/2021/DotNetAudit.md