[Feature Request] Completion?

[Pack3tL0ss]

Curious if this project currently has completion (couldn't find it in the docs or code) or if there are aspirations to provide completion in the future??

[BrianPugh]

I would love to add autocompletion, but have no experience in the area. I would certainly welcome any input, whether it be general recommendations, library suggestions, PRs, etc.

[BrianPugh]

Just a heads up, I am working on this, hoping to have something testable in the next week or two.

[BrianPugh]

shell completion is infinitely more complex than I was initially expecting 😄 . Still a WIP.

[mattangus]

This is somewhat of a deal breaker for me so I'm keen to help out to add this feature!

[BrianPugh]

Currently I'm working on adding bash and zsh support; i'm taking another direction on it and going to be basing the code off of shtab's implementation, which I think is a good reference.

Originally I was pursuing the idea of adding functionality to generate an argparse.ArgumentParser object from a cyclopts.App. However, some Cyclopts concepts don't directly translate (such as positional-or-keyword arguments). The benefits of this approach is that other off-the-shelf tooling could be used. I'm still considering this approach, but it does have its issues.

[BrianPugh]

I'm still thinking about this; I'm actually leaning towards the argparse solution (and maybe even also having a to_click method later) where I map postional-or-keyword arguments to be position-only to make them more compatible with those libraries/ecosystems.

[alextremblay]

Hi @BrianPugh

I've recently discovered cyclopts after becoming disillusioned with typer's slow (possibly stalled) development

I love everything you've done with it so far, and the only thing preventing me from adopting it in all my projects is the lack of shell completion. I have users leveraging shell completion in typer now, and taking that away from them is a hard no

I'm wondering if you'd be willing to publish your WIP on shell completion in another branch so that I or others could take a look at it and possibly help you finish it?

[alextremblay]

nvm, I just realized you already did that! hahaha

looking at the code, i think i see what you were going for. I agree that some kind of parser transformer (converting and App to an ArgumentParser or a click Group would be better

[BrianPugh]

Hey @alextremblay! I totally agree that shell completion should be the next important feature to implement

I'm actually working on a cyclopts v3.0.0 that does a fairly large revamp of all the internal structures. The big feature of v3.0.0 is that it will support dictionary-like parameters (dicts, typeddicts, dataclasses, attrs, pydantic, etc). Error messages will also be much more helpful because exact user-provided values and their source (CLI, config files, environment variables, etc) are now explicitly logged. Some of the lesser used cyclopts functionality will have a slight API change, but I don't think it'll impact most users.

In cyclopts v3.0.0 a lot more of the argument parsing/resolving is done up-front, which should make implementing shell-completion much easier. My current branch I'm working on is here. It's not functional yet, and a lot of the commits are sloppy due to the large changes. It's also going to change a lot, but you can track my progress a bit over there.

My initial attempts were converting cyclopts structures to something like ArgumentParser objects, but there are some incompatibilities that make it not work, so I sort of abandoned that idea. Benefits of that approach is that it could increase inter-operability with other CLI libraries. However, for now, I'll just be implementing direct shell completion without intermediate conversion.

I've been diligently working on v3.0.0, but no guarantees on a timeframe. I hope to have it done in about a month (and then start working on shell completion again), but you never know if life gets in the way 😄 .

[alextremblay]

That’s awesome!

Another option you may want to consider regarding shell completion, outsourcing!
I've been keeping an eye out for a while now on a tool called carapace-bin
It comes from the land of golang, but has since grown to encompass rust and python CLI frameworks.

The downside of using carapace-bin instead of doing your own shell completion Is that all users who want shell completion would have to install carapace (more on that later)
The upsides are many:

• shell support; carapace supports many more shells than any existing python CLI framework in existence, and certainly more than we can hope to support with an initial implementation
• Extensive type completion; carapace supports a dizzying array of different kinds of completion (file completion, folder completion, completion of files with a specific extension, completion of files/folders in a specific path, as well as an extensive array of other completion macros
• Performance; carapace-bin is a single-file go binary, it loads up custom Specs on shell startup. When it’s tasked with generating completions for a python CLI, it’s blazing fast. Far faster than any python-based shell completer could ever hope to be, no matter how well optimized
• I just think it’s really neat :)

The core idea here is that we could pre-generate completion specs for all the parts of our cyclopts command tree (including subcommands, options, positional args) which are known at compile-time, and compile them into an installable carapace-spec. For parts of the cyclopts CLI that are too dynamic to pregenerate, we have two options:

1. We could insert a “stub” in the carapace spec, what carapace calls an exec macro… (side note, there’s a lot of useful info on how we could use this here in this issue I opened
2. Less preferable, but still viable, we could implement completion logic in cyclopts similar to how click does it (looking for a _{cli name}_COMPLETE env var to tell us we need to parse sys.argv and generate completions) and then use carapace-bin’s Click bridge

We could also provide a mechanism for users to install carapace and install our CLI spec file

i guess all of this is to say: if you like this idea, I’d like to work with you to implement carapace-bin completion in cyclopts. Once the v3 codebase is semi-stable (to the point where the data structures involved are mostly stable), I can start working on a program that takes those data structures and generates a carapace Spec file.
closer to the release of v3, or after v3 release, I can work on integrating this spec generator into cyclopts itself

Thoughts?

[BrianPugh]

I like the idea! I'll post back in this thread in a week or two with a more meaningful follow up (with a hopefully solidified v3). I would love help on this!

[arogozhnikov]

@BrianPugh maybe shtab's developers would be interested in incorporating positional-or-keyword arguments?
It is a very pythonic way of treating arguments after all.

I feel like shtab is feature-rich, focused and not bloated.

cyclopts api + autosuggestion would be awesome.

cc @casperdcl

[casperdcl]

What's a "positional-or-keyword" arg?
app (<arg> | --arg=<arg>)?

[BrianPugh]

Hey @arogozhnikov @casperdcl !

Big fan of shtab, I was actually going to base Cyclopt's autocompletion on it; the codebase is very clean and minimal. Is there an API that Cyclopts could use (or vise-versa) to hook into shtab directly?

What's a "positional-or-keyword" arg?
app ( | --arg=)?

Exactly. Something like:

@app.default
def foo(src, dst):
  print(f"Copying {src} -> {dst}")

Has the following equivalent CLI invocations:

python my-script.py fizz buzz
python my-script.py --src fizz buzz
python my-script.py --src fizz --dst buzz
python my-script.py --dst buzz --src fizz
python my-script.py --dst=buzz fizz