How to define optional arguments?

[esafak]

I tried using Option (std/options) but it balked (type mismatch).

[c-blake]

Since every parameter with a default is an optional argument, I'm not entirely sure of your use case from the subject/title. I think of Option[T] as more for "answers" than for "arguments", though of course it can be natural to chain one procedure's output to another's input and the latter might want a CLI.

As described in README.md, the design is open enough that you don't really need me to add it. You just have to do your own argParse, argHelp after import cligen and before dispatch (examples in already linked thread).

That said, did you look at the discussion for #212 ? ¹. As mentioned in that thread, I'm not against adding an Option[T] to cligen/argcvt (probably hidden behind a when declared Option: so that we don't force the import on everyone, if we are willing to assume no one will hijack the stdlib name).

Footnotes

1. There really is a lot of guidance/help in the almost 200 closed issues, as per README.md. ↩

[esafak]

I see that discussion thanks to your guidance. Closed issues are normally hidden from search, so they have poor discoverability, hence the repetition. I should point out that the README does not mention the closed issues. The part you cite says:

cligen grew many features upon request & usage. The test suite has suggestively named examples. There's also a Wiki, generated docs for various modules (which include various system interfaces useful in writing CLIs) and a set of fully worked utilities which may be useful to you in their own right. I also try to mention breaking changes & new features in RELEASE-NOTES.

I think a traditional documentation site using something like MkDocs or Docusaurus would help a lot. The README is terse, if I may say so. I'm glad I created a discussion instead of an issue because now it won't get closed and future users will perhaps be less inclined to ask the same question.

As for the why, I think setting defaults in the parser is ugly and bad form by virtue of violating encapsulation. Defaults should live with the objects they are associated with, not the code (e.g., parser) that calls them. Suppose the user does not specify the argument because he has set his defaults in his config file. If we hard code its value in the parser, it will incorrectly override the config file. I would expect the parser to say that the user has entered nothing, and the procedure to use the config file or default value, as appropriate.

Thank you for listening.

[c-blake]

Fair point. I could swear it used to say that, but can't find it quickly. Maybe elsewhere. Sorry! Anyway, I updated the README to make the footnote true now. :-)

I've had many questions indicating people didn't read the README much, though. Ambiguous from them just not understanding it, of course. Others have said it's too long. So, I worry about making it more verbose. If we make it more structured (like clicking through links), then that "hides" things. The huge spectrum of patience across audiences make writing for any possible reader very hard.

I have literally never heard of MkDocs or Docusaurus before just now. So, I'm sure they're traditional to you, but not to me. :-) Anyway, I barely remember many little features and I wrote them and have a good memory. What I do is search the github issue / pr history. I often have responded in extreme detail in the past to the most common questions.

I do 100% support anyone trying to add how-to's to the Wiki, though. I wonder if people think to click on Discussions for docs? Interesting question. Many github repos I've visited don't even have a Discussion section. Anyway, whatever you want to do, github integration and ideally crowd sourcing is probably key, but I imagine Docusaurus / MkDocs put effort into such.

[esafak]

I wonder if people think to click on Discussions for docs?

People will simply use the search bar, and it will return results from code, issues, and discussions.

I've had many questions indicating people didn't read the README much, though. Ambiguous from them just not understanding it, of course. Others have said it's too long. So, I worry about making it more verbose. If we make it more structured (like clicking through links), then that "hides" things. The huge spectrum of patience across audiences make writing for any possible reader very hard.

The answer is not one or the other; it's both, and more. I suggest reading https://diataxis.fr/ and browsing https://docs.stripe.com/api as an example of documentation done right.

You could try using an LLM to generate the MkDocs; it's simpler than docusaurus. I could help, if you want.

[c-blake]

Re: your added on clarification (thanks!), the config file set up is as open architecture as the argParse / argHelp, really. You can have your own mergeParams framework with XML { not that I would enjoy using that :-) }. That said, I think you have a misunderstanding. The stock / in the box mergeParams from include cligen/mergeCfgEnv & friends results in an effective priority order of (lowest to highest) defaults-in-the-proc-entry-point, config files, environment variable, command line. So, the "most end"-user, the CLI invoker has the final say, and the entry point proc params are the weakest. So, you might like that or you might hate it. If the latter, you can probably do your own mergeParams.

parseOnly is also the "sledge hammer" of total control, but, in so being, it is also what imposes the most amount of work on the CLauthor. I think that is a fundamental tension - automate for common cases, but what is "common" just varies. parseOnly barely saves anything over manual cligen/parseopt3 work, TBH. I try to strike a balance in automation & defaults, but there is enough disagreement that it's not really possible to please everyone.

From the way you express your tastes (as far as I can tell), I suspect there is an initFromCL object-type based thing with defaults in the object itself that you might like better. I haven't done that work yet, though as this is a relatively new feature of Nim (I've been a user since like 2014...).

As to "documentation done right", as I said - I think it's very hard work to tune across broad spectrum audiences, anticipating misunderstanding and so on. So, I tend to shy away from it in favor of being as 1-on-1 supportive of my users as I can be where I can kind of iterate and learn about background, tastes, and priorities as with you here. I have said many times and think it's fundamentally true that there is no substitute for being the initially confused person who gets clarity, but sadly those people don't usually "write up" their experiences with answers. I am so open to better documentation and your help, though, that I'll just invite you to be a collaborator after pressing reply here. :-)

[c-blake]

I should say, though, that my own experiences with LLMs is that they are SO prone to just making things up that I have to spend more time separately validating what they say than I might have saved. E.g., I just tried Claude for this Option[T] question and got back several pages of 100% misinformation. So, I'd probably suggest you to wait on that as a text generation strategy, but I think this "how to document & config & general operation" is getting a bit off-topic. Maybe we should take it to email? My email is in all the commit logs. Or start a more specific discussion?

[c-blake]

I did mention this discussion in the commit, the way one does with issue threads, but it at least did not visually indicate that here (only in the issue -- some kind of player in the argument between Issues and Discussions, though I suppose Microsoft could always futz with how Github works). So, I will add it manually a3a1ef0 . It isn't and perhaps will never be a perfect feature for complicated T in Option[T], but something is better than nothing.

Something I didn't say over at #212 (but have in other Issue threads) is that one way to think of cligen is like a foreign function interface (FFI) but for seq[string] (CL & CL-like things) -> Nim proc calls. Nim proc calls themselves cannot tell within callees if a defaulted parameter was actually passed. This is only available at the call-site. Because this toolkit interposes itself in the call sequence, it has access to more data, but using that access can violate "Nim callability" as well as "CL callability", the duality of which is one of the charms of this approach.

I'm not ideologically opposed to it, as shown by test/UserError.nim which allows one to embrace a "this will only ever be a CL utility entry point" mentality (as seen like 11 uses of Help !! over in bu/), but it is tighter coupling than ideal. Loosening to have a Nim entry point and a CL entry point being possible, but perhaps not standard CLauthor practice due to laziness. E.g., pretty sure I skipped that factoring myself in those bu/ cases. The 2500x to 15e6 overhead slowdown is pretty objective, though (and not just a theoretical problem - it has bitten both me and many others over the decades. It's a big part of why "shell builtins" are a thing, for example.) Anyway, refactoring when needed isn't so awful. It just relates to this "using Option[T] to model input optionality idea" that opened this discussion.

Anyway, I will probably punch a new release of cligen soon to put that out there if you want to give it a test drive off of git HEAD.

[esafak]

I'll try the next release.