Proposal: Command-line config

[ehuss]

This is a two-part proposal to make it easier to use Cargo config settings.

1. Add --config <key>=<value> command-line argument for setting config values for the current command.
2. Add include key to the config for including other config files.

It is intended that the use of these should be relatively rare and only for advanced use cases.

--config CLI option

The --config command-line option is a global option available to all commands. It sets the given config key to the given value, and takes precedence over all other sources (environment variables and all config files). The argument is parsed as TOML syntax. The --config flag may be specified multiple times, in which case the values are merged in left-to-right order, using the same merging logic that multiple config files use.

Some examples of what it looks like using Bourne shell syntax:

# Most shells will require escaping.
cargo --config http.proxy=\"http://example.com\" …

# Spaces may be used.
cargo --config "net.git-fetch-with-cli = false" …

# TOML array example. Single quotes make it easier to read and write.
cargo --config 'build.rustdocflags = ["--html-in-header", "header.html"]' …

# Example of a complex TOML key.
cargo --config "target.'cfg(all(target_arch = \"arm\", target_os = \"none\"))'.runner = 'my-runner'" …

# Example of overriding a profile setting.
cargo --config profile.dev.overrides.image.opt-level=3 …

include key

The include key may be used to include other config files. It may be a string for a path to the file, or a TOML array of strings to include multiple files. The paths are relative to the config file where it is defined, or from the current working directory if used from an environment variable or command-line option.

When used in a config file, the include is loaded and merged immediately after the file with the include key has been loaded and fully processed. When used from an environment variable (CARGO_INCLUDE), it is loaded after all config files have been loaded. When used from the command line (--config include=…), it is loaded after the environment variable has been processed. This will require special handling as normally config values replace and shadow previous ones.

include = "config-foo"

In conjunction with --config, this can be a convenient way to specify other config files on the command-line:

cargo --config 'include=".cargo/config-foo"' …

Users are allowed to use filenames of the form .cargo/config-*, though users are free to use other filenames outside of the .cargo directory.

Questions and concerns:

• Restricting the CLI to <key>=<value> may be tricky, because toml-rs does not provide a way to restrict that. It would be easy to restrict to a single value, but would still allow comments, or things like [foo]\nbar=1. I would prefer to be conservative and only allow a restricted syntax and not allow any free-form TOML, but that may be difficult. If that's a concern, should toml-rs be extended to support this case? How should this be handled?
• Does it make sense for include via env/cli to be relative to CWD? I think that seems natural, but perhaps the workspace root would be another option? Or maybe a limited env var substitution?
• I'm concerned the real-world use cases may not be strong enough to justify this addition (at least in this form).
• When an environment variable or command-line option are required to properly use a package, it can make a package more difficult to use. This may encourage overly clever solutions that make using Cargo a less pleasant experience. For example, requiring the --config option to properly compile something would make IDEs and other tools unable to work properly (or require arduous setup).

References:

• Specify config file on the command line #6648 — Specify config file on the command line

[alexcrichton]

This all sounds pretty great to me, thanks for writing this up @ehuss! I think an alternative to include is to have something like --config-file and an associated env var, but I think I'd lean a bit more towards include becaues it seems to fit nicely with the proposed --config flag.

I also think that this is definitely pretty hard to invoke on the command line for colorful syntaxes due to the usage of TOML, but that feels appropriate given the target use case here which is advanced configuration. I've seen some various tools take JSON on the command line before for similar use cases so it seems not so bad as well.

With regards to your questions:

toml-rs does not provide a way to restrict that

I think this'd be fine to add to the toml crate or otherwise just exposing more methods on the internal parser, it should be easy to say "parse a key, then an equal sign, then a value, only ignorning whitespace".

Does it make sense for include via env/cli to be relative to CWD?

Yes.

But more seriously, that's how lots of other Cargo things work :)

One thing to consider though is that if you specify a relative path in .cargo/config it's not actually relative to the config file but rather relative to the diretory containing the .cargo folder. (as if relative paths all start with ../ to be relative to the file itself). That may be somewhat surprising for config files defined in .cargo/config-foo?

I'm concerned the real-world use cases may not be strong enough to justify this addition

Could you elaborate a bit on this? Do you mean --config, include, or both?

When an environment variable or command-line option are required to properly use a package, it can make a package more difficult to use.

Agreed! Good to mention! Not a reason to not do it :)

[ehuss]

due to the usage of TOML

Yea, it's not super elegant. There are some advantages that I didn't mention. It makes it possible to safely express things like #6139 which is currently not possible.

Could you elaborate a bit on this? Do you mean --config, include, or both?

Both. I'm just thinking it's a significant addition for something that may not be used often.

[alexcrichton]

Ah ok, I feel that --config is something we've wanted for a really long time as it's just a super flexible way to specify configuration that's not env vars (and also things like #6139). I also think it's a great fit for the profile changes coming down the pike. Similarly I feel that include (or something like include) fits in the same bucket, so while it makes sense to acknowledge I feel these are long-awaited features in Cargo :)

[ijackson]

I have also just tripped a situation where this feature would have helped. I have an environment where I have a .cargo/config which is nearly always right but which I want to be able to selectively override. Currently there seems to be absolutely no way to do this!

[ijackson]

Relatedly, I have just filed #6728 "Want env var specifying overriding config file"