Announcing System.CommandLine 2.0 Beta 4

[jonsequitur]

System.CommandLine 2.0 Beta 4

We've been working through feedback on the beta 2 and beta 3 releases of System.CommandLine and now we're ready with beta 4. Here's a quick overview of what's new.

Microsoft Docs

Probably the most important update isn't in the API itself but in the release of preview documentation for System.CommandLine. It's been live now since April and, like the library, is in preview. There's a good deal of API to cover so if there's something that's missing or unclear, please let us know.

SetHandler simplification

The most common feedback we've heard about the SetHandler API introduced in beta 2 is that it's confusing, mainly because you can pass any number of IValueDescriptor<T> instances (e.g. Argument<T>, Option<T>, or BinderBase<T>) to the params IValueDescriptor[] parameter on most of the overloads, and this number can differ from the count of the handler's generic parameters.

I'll use this simple parser setup in the following examples:

var option = new Option<int>("-i");
var argument = new Argument<string>("-s");
var rootCommand = new RootCommand
{
    option,
    argument
};

Here's an example of the old SetHandler signature:

// ⚠️ Now outdated
public static void SetHandler<T1, T2>(
    this Command command,
    Action<T1, T2> handle,
    params IValueDescriptor[] symbols);

The expectation was that in the common case, you would call this method like this:

rootCommand.SetHandler(
    (int someInt, string someString) => { /* Do something exciting! */ },
    option,
    argument);

But the fact that the final parameter to the old SetHandler methods was a params array meant that the following would compile and then fail at runtime:

// ⚠️ Now outdated
rootCommand.SetHandler((int someInt, string someString) => { /* Do something exciting! */ });

The logic behind this method signature was that any handler parameters without a matching IValueDescriptor would be satisfied instead by falling back to BindingContext.GetService. This has been a longstanding way that dependency injection was supported for both System.CommandLine types such as ParseResult as well as for external types. But it's not the common case, and it's confusing.

We've changed the SetHandler methods to require a single IValueDescriptor<T> for each generic parameter T, rather than the previous params array:

 public static void SetHandler<T1, T2>(
    this Command command,
    Action<T1, T2> handle,
    IValueDescriptor<T1> symbol1,
    IValueDescriptor<T2> symbol2);

This has a number of benefits. In addition to making it clearer that you also need to pass the Option<T> and Argument<T> instances corresponding to the generic parameters, it also works with C# type inference to make your code a little more concise. You no longer need to restate the parameter types, because they're inferred:

rootCommand.SetHandler(
    (someInt, someString) => { /* Do something exciting! */ },
    option,
    argument);

We've also added overloads accepting InvocationContext, so if you want more detailed access for DI or larger numbers of options and arguments, you can do this:

rootCommand.SetHandler(
    (context) => // InvocationContext, but it's inferred so you don't need to specify it.
    {
        var i = context.ParseResult.GetValueForOption(option);
        var s = context.ParseResult.GetValueForArgument(argument);
        var someDependency = context.GetService(typeof(ILogger)) as ILogger;
        /* Do something exciting! */ 
    });

Finally, we've reduced the number of SetHandler overloads. Where before there were overloads supporting up to sixteen generic parameters, they now only go up to eight.

Option and Argument (non-generic) are now abstract

Support has existed since the earliest previews of System.CommandLine for creating non-generic Option and Argument instances that could be late-bound to a handler parameter. For example, an Argument could be created that could later be bound to either a string or an int parameter. This relied heavily on the reflection code that powered CommandHandler.Create, which was moved out of System.CommandLine and into the System.CommandLine.NamingConventionBinder package in beta 2. Fully removing support for this late-binding behavior by making Option and Argument abstract has helped simplify some code, clarify the API, and provided additional performance benefits.

Response file format unification

Response file support is a common feature of a number of command line tools, allowing a token within the command line to be replaced by inlining the contents of a file. You can think of a response file as a sort of configuration file that uses the same grammar as the command line tool itself. For example, given a file called response_file.rsp, the following command line will interpolate the contents of that file into the command line as if you had run it directly:

> myapp @response_file.rsp

(The .rsp file extension is another common convention, though you can use any file extension you like).

Until this release, System.CommandLine supported two different formats for response files: line-delimited and space-delimited. Line-delimited response files are more common and thus were the default. If response_file.rsp contained the following:

# Make it verbose
--verbosity
very 

then it would be treated as though you had run this:

> myapp --verbosity very

(Note following a #, the rest of the text on that line is treated as a comment and the parser will ignore it.)

This also provided a mechanism for avoiding the sometimes confusing rules of escaping quotes, which differ from one shell to another. You could produce a single token with a line-delimited response file by putting multiple words on a single line.

If response_file.rsp contained the following:

--greeting
Good morning! 

then it would be treated as though you had run this:

> myapp --greeting "Good morning!"

System.CommandLine also supported another response file format, space-separated, where the content of the response file was expected to be all on a single line and was interpolated verbatim into the command line. This was not commonly used and we decided it would simplify the developer choice and make the behavior clearer for end users to unify the format.

The rules for the new format are as follows:

• Files can have many lines and content is treated as though concatenated into a single line.
• One or more tokens can appear on a single line.
• A line containing Good morning! will be treated as two tokens, Good and morning!.
• If multiple tokens are intended to be passed as a single token (e.g. "Good morning!"), you must enclose them in quotes.
• Shell-specific escaping rules are not used for the contents of a response file.
• Any text between a # symbol and the end of the line is treated as a comment and ignored.
• Tokens prefixed with @ can reference additional response files.

Custom token replacement

While we were trying to figure out how to make response file handling simpler, we saw that there are so many variations on how to parse response files that we could never provide a single implementation that could address them all. This could easily become a deal-breaker for people who have older apps that they would like to reimplement using System.CommandLine. So while we were consolidating response file handling, we also decided to generalize and hopefully future-proof the solution by adding a new extensibility point. After all, the core logic of response files is simple: during tokenization, a token beginning with @ is (possibly) replaced by zero or more other tokens.

If the default response file handling doesn't work for you, you can now replace it using the CommandLineBuilder.UseTokenReplacer method. The following example configures the parser to replace @-prefixed tokens with other tokens.

var parser = new CommandLineBuilder(command)
                .UseTokenReplacer((string tokenToReplace, out IReadOnlyList<string> replacementTokens, out string errorMessage) =>
                {
                    if (tokenToReplace == "fruits") // note that the leading '@' will already have been removed 
                    {
                        replacementTokens = new [] { "apple", "banana", "cherry" };
                        errorMessage = null;
                        return true;
                    }
                    else
                    {
                        replacementTokens
                        errorMessage = null; // no parse error is created
                        return false; // tokenToReplace is not replaced 
                    }
                })
                .Build();

This token replacement happens prior to parsing, so just as with response files, the new tokens can be parsed as commands, options, or arguments.

If you return true from the token replacer, it will cause the @-prefixed token to be replaced. If you return false, it will be passed along to the parser unchanged, and any value you assign to the errorMessage parameter will be displayed to the user.

Given the above token replacement logic, an app will parse as follows:

> myapp @fruits # is parsed as: myapp apple banana cherry
> myapp @not-fruits # is parsed as: myapp @not-fruits 

It's worth noting that this behavior is recursive, just as with response files. If a replacement token begins with @, the token replacer will be called again to potentially replace that token.

Added support for NativeAOT compilation

Continuing the work done in beta 3 to make apps built using System.CommandLine trimmable, they can now also be compiled with Native AOT. For more details, you can read about Native AOT compilation in the .NET 7 Preview 3 announcement here: https://devblogs.microsoft.com/dotnet/announcing-dotnet-7-preview-3/#faster-lighter-apps-with-native-aot.

[maxild]

If before beta 4, I used a lambda with "magic" IConsole parameter (for instance (IConsole console, ...) => { /* code here */ } will I now have to use the InvocationContext based lambda, and use the GetValueForXXXX API to get the parsed option args?

rootCommand.SetHandler(context =>
// Using this line I cannot get to the IConsole console...
// rootCommand.SetHandler((env, module) =>
//
// Before beta 4 I used this line
// rootCommand.SetHandler((Environment? env, Module module, IConsole console) =>
{
    var module = context.ParseResult.GetValueForOption(optionModule);
    var env = context.ParseResult.GetValueForOption(optionEnvironment);
    // blah blah
    context.Console.Out.Write(response);
}

[jonsequitur]

Yes. We wanted to limit the number of overloads. But it should be straightforward to define your own extension methods if you prefer to have the parameters for both injected objects and bound command line values in the same handler signature.

[bording]

@jonsequitur So with beta 4, if I want access to the IConsole or a cancellation token, I have to use the InvocationContext overload, correct?

That seems to mean that there no way to use a custom model binder and have access to a cancellation token at the same time.

If that is true, then this seems like an odd backstep to me.

[jonsequitur]

@bording The InvocationContext should give you access to everything you need. You can get to the IConsole instance using InvocationContext.Console and you can get a CancellationToken (triggering Ctrl+C cancellation handling) by calling InvocationContext.GetCancellationToken().

[bording]

@jonsequitur I think you're missing my point. If I'm using the InvocationContext overload, how do I pass in my custom model binder? I've been able to abstract away the creation of several things that require multiple parameters and just get an instance of it in my handler through custom model binders.

I'm currently using custom model binders, and injecting an IConsole and a cancellation token. In beta 4, it seems like an either/or scenario.

[bgever]

Before I was able to specify types on the handler that would automatically injected, and would resolve to the services registered in a middleware on startup.

So a Command class would look something like this:

using System.CommandLine;
using System.CommandLine.Binding;
using System.CommandLine.Invocation;
using System.CommandLine.IO;
using MyApp.Cli.Commands.User;
using MyApp.Cli.Primitives;
using MyApp.Cli.Services;

namespace MyApp.Cli.Commands;

public class UserCommand : Command
{
    public UserCommand() : base("user")
    {
        Description = "Display user authentication status";

        AddCommand(new LoginCommand());
        AddCommand(new LogoutCommand());

        var refreshOption = new Option<bool>("--refresh", "Force access token refresh even if hasn't expired yet");
        Add(refreshOption);

        this.SetHandler<bool, TokenService, Auth0Client, InvocationContext>(UserHandler, refreshOption);
    }

    public async Task UserHandler(
        bool refresh, TokenService tokenService, Auth0Client auth0Client, InvocationContext context)
    {
        var console = context.Console;
        var cancellationToken = context.GetCancellationToken();
        // [...]

But after this change I'm not able to easily inject both options/arguments and resolved services.
As a workaround, I made a small primitive class to inject services:

using System.CommandLine.Binding;
using Microsoft.Extensions.DependencyInjection;

namespace MyApp.Cli.Primitives;

public class Injectable<T> : BinderBase<T>
{
    protected override T GetBoundValue(BindingContext bindingContext)
    {
        return (T)bindingContext.GetRequiredService(typeof(T));
    }
}

That allows me to write the code like this, what looks even cleaner;

• no more need to provide the generic types for the SetHandler, and
• the service dependencies are clearly being marked as being injected.

using System.CommandLine;
using System.CommandLine.Invocation;
using System.CommandLine.IO;
using MyApp.Cli.Commands.User;
using MyApp.Cli.Primitives;
using MyApp.Cli.Services;

namespace MyApp.Cli.Commands;

public class UserCommand : Command
{
    public UserCommand() : base("user")
    {
        Description = "Display user authentication status";

        AddCommand(new LoginCommand());
        AddCommand(new LogoutCommand());

        var refreshOption = new Option<bool>("--refresh", "Force access token refresh even if hasn't expired yet");
        Add(refreshOption);

        this.SetHandler(
            UserHandler,
            refreshOption,
            new Injectable<TokenService>(),
            new Injectable<Auth0Client>(),
            new Injectable<InvocationContext>());
    }

    public async Task UserHandler(
        bool refresh, TokenService tokenService, Auth0Client auth0Client, InvocationContext context)
    {
        var console = context.Console;
        var cancellationToken = context.GetCancellationToken();
        // [...]

Maybe this type of Injectable helper can be added to the library?

[WeihanLi]

Should I use the Option<bool> instead of Option since the the Option is abstract now

[jonsequitur]

Should I use the Option<bool> instead of Option since the the Option is abstract now

Yes. When you declare options and arguments, you'll need to use Option<T> and Argument<T>.

You can use Option and Argument (or their base type, Symbol) when iterating over collections that might contain different generic types of Option<T> and Argument<T>, e.g.:

foreach (Option opt in command.Options)
{
}

[tomrus88]

Finally, we've reduced the number of SetHandler overloads. Where before there were overloads supporting up to sixteen generic parameters, they now only go up to eight.

So what should I do now if I used one of removed overloads? I was passing 11 of Option<T> into SetHandler.

[wuzzeb]

Now that options are required to be passed to SetHandler (I like this change), how about allowing us to omit also adding them directly to the command. In other words, any option passed to SetHandler is implicitly also added to the root command.

[jonsequitur]

Now that options are required to be passed to SetHandler (I like this change), how about allowing us to omit also adding them directly to the command.

We've looked into this and while it works for the simple case, there are a number of cases where this API would lead to ambiguities. I outlined a few here: #1537 (comment)

[jonsequitur]

So what should I do now if I used one of removed overloads? I was passing 11 of Option into SetHandler.

@tomrus88, you can either use a custom BinderBase<T> or the SetHandler overloads with a delegate that accepts an InvocationContext. Both approaches are described here: https://docs.microsoft.com/en-us/dotnet/standard/commandline/model-binding#parameter-binding-more-than-8-options-and-arguments

[NickStrupat]

Hi folks,

I needed the CancellationToken in my handlers but I wanted to use the SetHandler helper method. I came up with this class which provides the CancellationToken as an IValueDescriptor<T>.

var name = new Option<String>("name", "Your name!");
var ctvs = new CancellationTokenValueSource();
var root = new RootCommand("Root command!") { name };

root.SetHandler(async (name, ct) => {
	Console.Write("name: ");
	try { await Task.Delay(5000, ct); }
	finally { Console.WriteLine(name); }
}, name, ctvs);

var clb = new CommandLineBuilder(root);
clb.CancelOnProcessTermination();
return await clb.Build().InvokeAsync(args);

class CancellationTokenValueSource : IValueDescriptor<CancellationToken>, IValueSource
{
	public Boolean TryGetValue(IValueDescriptor valueDescriptor, BindingContext bindingContext, out Object? boundValue)
	{
		boundValue = (CancellationToken)bindingContext.GetService(typeof(CancellationToken))!;
		return true;
	}

	public Object? GetDefaultValue() => throw new NotImplementedException();
	public String ValueName => throw new NotImplementedException();
	public Type ValueType => throw new NotImplementedException();
	public Boolean HasDefaultValue => throw new NotImplementedException();
}

Run dotnet run -- name nick and use CTRL+C before the 5 seconds wait is up to see the behaviour.