Show extra validation information during pack to allow for more effective means of communicating best practices such as README

[nkolev92]

Showing additional pack validation information

• Nikolche Kolev
• GitHub Issue - Show extra validation information during pack to allow for more effective means of communicating best practices such as README #12070

Summary

Package authoring best practices are an ever evolving set of rules and guidelines. With the addition of new features, there are new recommended practices.
Historically warnings have been the only way the tooling can communicate that certain metadata or package structure is suggested.
Warnings can be disruptive and not every new ideas warrants that alert level.
Adding a non-breaking validation message would allow NuGet to make soft recommendations for scenarios such as README.

Motivation

For some of the package authoring best practices, such as embedded licenses, a warning was added to alert the user.
TreatWarningsAsError isn't an uncommon thing, so warnings in these cases can be disruptive.
Furthermore, the warning itself wasn't really saying use an embedded license, but rather saying don't use licenseurl, use an embedded license instead, thus elevating the urgency.

Showing extra validation information during pack to allow for more effective means of communicating best practices such as READMEs.

Explanation

Functional explanation

Currently when you pack the package, you get the following output.

  Successfully created package 'C:\Code\Temp\Pack\bin\Debug\Pack.1.0.0.nupkg'.

When every package is create, 2 sets of validations are run:

• Mandatory validation, valid package id, valid version
• A set of warning validation rules is run and each one is usually associated with a warning code.
• This proposes a new set of rules. These rules are going to be independent from the other sets of rules. A violation of any of these rules will lead to an information message indicating that the metadata can be specified.
  The idea is that this feature will be completed by the package scoring.

Currently if you have a warning, you get the following:

C:\Program Files\dotnet\sdk\6.0.400\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Tasks.Pack.targets(221,5): warning NU5125: The 'licenseUrl' element will be deprecated. Consider using the 'license' element instead. [C:\Code\Temp\Pack\P
ack.csproj]
  Successfully created package 'C:\Code\Temp\Pack\bin\Debug\Pack.1.0.0.nupkg'.

The proposal is to add the following on normal verbosity.

The created package is missing a readme. Go to https://aka.ms/nugetpackreadme to learn why package readmes are important.
The created package does not specify a license. Go to https://aka.ms/nugetpackagelicense to learn why package licenses are important.
ack.csproj]
  Successfully created package 'C:\Code\Temp\Pack\bin\Debug\Pack.1.0.0.nupkg'.

The validations can be added on minimal or normal verbosity level.
msbuild /t:pack and nuget.exe pack default to normal verbosity, while dotnet.exe pack defaults to minimal verbosity.

It is considered a good practice to have 1 information output per project and per operation. As such, these messages will be added on normal verbosity.

Technical explanation

By default these validations are not going to raise any warnings or errors.
These validations will not something that can be disabled early on.
These validations cannot be elevated to warnings at this point.

Drawbacks

• Pack command noise. These best practices are not relevant for every package created. Some packages are easy.
• Not super visible. Non-breaking output from commandline commands is not always gonna be noticed by package authors.

Rationale and alternatives

• By going with the current minimal approach we allow ourselves the freedom to experiment and get this new set of validations correct.
• Do nothing.
• Add a warning. This is specific to a single rule, but we want to create a concept that would allow us to add non-breaking validations.

Prior Art

• Pack Validation rules are similar, but they are not a perfect paralell as they do not really require you to take action.

Unresolved Questions

• Is normal verbosity good enough?
• Is it ok to ship without a means to upgrade these validations to warnings.

Future Possibilities

• Add a means to upgrade these validations to warnings/errors.
  • We might need to define nuget log codes for each validation. Alternatively, we can use 1 warning code for all information validations that get upgraded to errors.

[nkolev92]

I've added some quick design ideas in the issue body.
The exact text in the output is up for discussion.

[zivkan]

while dotnet.exe pack defaults to minimal verbosity
Is normal verbosity good enough?

In my opinion no, normal verbosity is not good enough given dotnet pack defaults to minimal. Most customers will never have the messages output to their console or CI logs, so never even have an opportunity to see the message. That's before we consider the points you raised about customers ignoring anything that's not an error.

[heng-liu]

As the set of rules and guidelines keep evolving, people use different versions of NuGet client tooling will get different info. Will this cause confusion? Or shall we show something when it's not checking against latest rules?

[jwfx]

I am being spammed by
"The package xxxxxx is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important."

How can I get rid of all these educational warnings permanently?