Manager for Haskell Cabal files

[ysangkok]

The Cabal build system supports version constraints, see example. It would be nice if Renovate could parse the Cabal files and bump the version bounds contained within. @secustor requested on #31434 that I start a discussion for this:

This needs a discussion first for the manager part

Cabal doesn't specify the versioning scheme that packages should use, but some operators are designed for use with PVP , for example the ^>= operator. It would be nice if Renovate supported PVP, but strictly speaking this is a separate concern from adding Cabal file parsing and dependency extraction.

I am a committer for the Hackage-server, which proves a web API that the Cabal client command line tools interacts with, e.g. the commands in cabal-install. Hackage has a JSON endpoint that lists versions available for a package, e.g. https://hackage.haskell.org/package/base.json . However, this is also not really relevant to the discussion of the manager, since this functionality would be covered by the datasource (subject to future discussion, but see the PR for an idea of how it could work). See also #8187, which has a discussion on a Hackage data source.

Have you had a look at the Cabal file parsing code in the linked PR, and do you have some ideas for how to implement a parser for the format? I would love to hear them.

Have you looked at the tests in the PR? Do you think they adequately test the minimum functionality? Let's start out with a solid MVP such that we can iterate on it with smaller PRs. What do you consider the most important features to support in a Cabal file? Does the linked PR attempt to handle those MVP features?

[secustor]

Cabal doesn't specify the versioning scheme that packages should use, but some operators are designed for use with PVP , for example the ^>= operator. It would be nice if Renovate supported PVP, but strictly speaking this is a separate concern from adding Cabal file parsing and dependency extraction.

We would need a haskel versioning implementation for that in any case as combining multiple ranges with && is not used by any other system right now.
So it is fine if it doesn't support PVP in the first iteration, though we have to document this.

Have you had a look at the Cabal file parsing code in the linked PR, and do you have some ideas for how to implement a parser for the format? I would love to hear them.

Yes, I have glanced through your code.
You should take inspiration of the other managers which use it such as bazel, though this is at a later stage.

The order of PRs is:

• versioning
• datasource ( as datasource a defines a default versioning )
• manager ( as the extraction results define a datasource to be used )

With that ordering users can also already use the data even without the manager via custom manager.

Have you looked at the tests in the PR? Do you think they adequately test the minimum functionality? Let's start out with a solid MVP such that we can iterate on it with smaller PRs. What do you consider the most important features to support in a Cabal file? Does the linked PR attempt to handle those MVP features?

It makes sense to start with a subset features, like you do, though foundation has to be still reliable and well tested.

Regarding tests, look at our development docs in /docs/development and look at other PRs which recently implemented new integrations to see how code looks like that we are feeling comfortable to merge.

Finally there shouldn't be any code changes for the general processing logic needed ( lib/workers ), that makes me suspicious that you do something which is not needed.

[ysangkok]

Thanks for your comments. As suggested I have started a discussion on the requirements for versioning. This code will not need any modification to lib/workers. Discussion is at: #31663

I will return to this discussion once I finish versioning.

[ysangkok]

combining multiple ranges with && is not used by any other system right now.

As far as I understand from the semver cheatsheet, semver does support ranges, though the && is implicit. Does Renovate currently aim to support these? It seems like supporting ranges across the codebase should be the first step. As I understand versioning, the versioning only covers part of this, since there seem to be assumptions that version bounds are for specific versions, and not ranges. For instance lookup/index calls versioning.isValid, but isValid for semver is defined to not accept ranges:

describe('modules/versioning/semver/index', () => {                                                                                                           
  it.each`                                                                                                                                                    
    version                                          | expected                                                                                               
[...]
    ${'1.2.3'}                                       | ${true}                                                                                                
[...]
    ${'~1.2.3'}                                      | ${false}                     
    ${'^1.2.3'}                                      | ${false}                     
    ${'>1.2.3'}                                      | ${false}

The reason I patched auto-replace was because I couldn't get it to accept the range that I outputted. If we tackle range support in a generic fashion, you're of course right that I shouldn't need to make any changes to lib/workers, which contains auto-replace.

It makes sense to start with a subset features, like you do, though foundation has to be still reliable and well tested.

I totally agree, but I am not sure what would be considered a reasonable subset. For example, I am worried that adding range support across the codebase might be too hard. Maybe it would make sense to add just NPM-style bounds first with semver. As far as I understand, this means that there would be no versioning changes. Is this correct?

One issue with adding PVP support, is that getMajor is defined in e.g. versioning/generic.ts to return an int. But in PVP, there are two numeric components in the 'major' part of the version. That's something that makes me hesitant to add PVP, and even if Renovate only supports semver ranges, and puts the updates in the wrong minor/major bucket, it would still add a lot of value.

But all of these concerns do not really affect the data source. I could add the data source, specifying semver as its versioning. It would be correct for some packages, incorrect for others. But it would be a start. Would this be acceptable? Or do you think one of the other previously mentioned areas should be done first?

[rarkins]

As far as I understand from the semver cheatsheet, semver does support ranges

Unfortunately this is incorrect. Semantic Versioning, as defined in https://semver.org/, has no concept of ranges/constraints at all. All ranges/constraints you see are adaptations and wrappers of semver, such as by npm.

though the && is implicit.

In npm semver, and probably most other adaptations, the && is implicit.

Does Renovate currently aim to support these?

Not as part of Renovate's "semver" versioning, because as mentioned above semver doesn't support ranges. Renovate already supports multiple semver flavors (npm, cargo, go, composer, etc) too, but none of those need &&. If there was a need to support && for a new flavor of semver like Haskell then yes.

It seems like supporting ranges across the codebase should be the first step.

If Haskell's versioning differs from other existing flavors of semver which Renovate supports then it means a new "versioning scheme" needs adding in Renovate for Haskell. Adding a new semver versioning flavor is reasonably well documented and precedented.

[ysangkok]

In my PR, I did add a stab at PVP support, see same-major-pvp/index.spec.ts. Is this broadly the right structure? It sounds like it might be supposed to contain more range logic. I could expand it to have stuff like >=1.2 && <1.3 in the range arguments. I could also have it omit the &&, such that it would be closer to NPM syntax. In the PR I submitted, I had the auto-replace module insert the && where needed, to keep the versioning module oblivious to the details of the encoding of the boolean expression. But it might be cleaner to just have the versioning module operate on snippets that are also valid snippets of Cabal file syntax. So how does that sound, I could make a new PR, with just versioning, that contains all those tests, and more (related to ranges). Depending on how difficult it would be, I could also support || and parens and version sets. Would it be ok for an MVP only to support e.g. expressions of form >=X && <Y?

[rarkins]

I'm not aware of the requirements for this new versioning so can't comment further. I suggest starting a new discussion on this versioning where we aim to capture all the requirements and ideally how it's similar or different to any existing semver flavors. Once I understand the requirements then I'm better placed to comment on the solution or suggest a solution.

Regarding your last question, it's perfectly ok for a versioning to support only a subset of syntax initially, as long as it's documented. Unsupported syntax would then be ignored (no updates provided) with skipReason=unsupported-value