argparse++

[ThomasWaldmann]

Have a look at:

• https://jsonargparse.readthedocs.io/en/stable/
• https://github.com/bw2/ConfigArgParse
• https://pypi.org/project/confuse/
• https://github.com/tiangolo/typer and https://palletsprojects.com/p/click/
  • see also convert to click for command line parsing and dispatching #251
• https://github.com/omry/omegaconf
• https://github.com/google/python-fire

[RonnyPfannschmidt]

i would strongly recommend towards click/typer,
personally i have a slight preference for click as when i gave it a try,
typer wasnt as nicely composable as the underlying click, but that may not be relevant for borg

[ThomasWaldmann]

Yeah, I also like click. A problem might be that borg uses quite some argparse features, so we need something quite compatible with that if we want to avoid major rework / a dead end when trying to switch.

[anarcat]

i don't know. i filed #251 and now i'm not sure click is the right choice at all. i put more information in that other issue, but i don't like click very much anymore: whenever i encounter it, even in projects i wrote, it seems awkward and complicated. it's pretty limiting in tons of ways and not immediately obvious what the flow is in the program. it does fancy stuff with the terminal too.

adding an extra dependency just for argument parsing also seems frivolous. i think borg should aim at having less dependencies, not more.

these days i use argparse for everything. it's sometimes a little clunky, but it can do pretty much everything i need, and it's not some special framework someone needs to learn if they land in my program.

[ThomasWaldmann]

Something that opens up a way to deal with config file(s), env vars and cli arguments in some nicely integrated and well-defined way would be nice.

[anarcat]

so i actually went through a few of those, and here's a few comments...

• https://jsonargparse.readthedocs.io/en/stable/

This one is interesting. It's basically a wrapper (or reimplementation? not sure) of argparse, except that it can load stuff from config files and the environment. It also supports sub-commands (or subparsers in argparse's jargon). It supports JSON, YAML, and jsonnet as config file formats. And since it's compatible with argparse, stuff like argcomplete and other third-party argparse things just work fine.

It also has a click-like CLI wrapper to hook functions into the CLI parser, which is nice. I don't know how good that is, but I like that it has a more minimalist approach than click.

Oh, and it's type-safe: you can specify the expected type of an argument or config file item, and it will fail if invalid stuff is passed, which is really nice if you start setting up mypy because then you get type context from your args, something that's generally really annoying to fix with stdlib argparse.

Not packaged in Debian.

• https://github.com/bw2/ConfigArgParse

This one is similar, but less features than jsonargparse. It also seems to reimplement argparse which seems fraught with peril. It does pass the argparse unittests, but that still seems like quite an ordeal to maintain.

Packaged in Debian.

• https://pypi.org/project/confuse/

This is also really nice, but is more focused on configuration file parsing, and it does that excellently. The one problem I see is that while it can import settings from argparse, it doesn't actually generate a parser, so you need to generate your own parser and keep it in sync with the configuration file schema. Doesn't seem ideal, especially since we don't have much config file stuff to begin with now...

Packaged in Debian bookworm and above.

• https://github.com/tiangolo/typer and https://palletsprojects.com/p/click/

Those two are similar in spirit: make a function signature just so, and it will make the commandline for you and bring you to the moon. Works well until it doesn't do what you want. docopt is similar in spirit and limitation, in my opnion. I would avoid except for rather simple programs, a bridge that borg crossed a long time ago.

Click is packaged in Debian.

Typer is packaged in Debian bookworm and above.

• https://github.com/omry/omegaconf

A little odd. Also supports config/env/argument parser, but creates the context based on some sort of odd DSL based on lists and dicts (or a yaml file or a dataclass or... you get the idea). Not really obvious how it actually works, after skimming through the docs, which is kind of smelly for me.

Omegaconf is packaged in Debian bookworm and above.

• https://github.com/google/python-fire

similar to click, so similar limitations, but a little more powerful on the dispatch, while at the same time less featureful (which might be good). as far as i could tell, no config file / env support out of the box.

Fire is packaged in Debian.

---

all are licensed with either MIT/Apache/BSD, no GPL, if that's something you care about. (i care that it's free software of course, so that's okay with all of them.)

that's what i got. i'd suggest taking a deeper look at jsonargparse and see how healthy the project is, how good the code looks and so on. one red flag for me is that it's not packaged in Debian at all, which typically implies that it's not as widely used as the other packages. It will also mean borg adding that dependency would delay borg's update in Debian while packages ship that dependency as well.

[ThomasWaldmann]

master branch now automatically generates shell completions using shtab (which works by analyzing the argparse parser we currently have).

Guess this limits our choice to whatever shtab supports if we do not want to go back to manually writing the completion files (which is a pain).