Split dependency solver into a library

[grayjay]

It might be nice to split the solver into an internal library or new top-level library. See #3222 and commercialhaskell/stack#1615.

[BardurArantsson]

Thanks for creating the issue @grayjay.

I'll just summarize my understanding of the thread so any newcomers can hopefully avoid having to re-read the whole thread in detail.

The solver modules are currently split out into their own module namespace Distribution.Solver, but there may be issues with the exact API boundary and if the current API is enough for e.g. stack.

As I remember/understand it, the main sticking points for splitting out to a new top-level library were:

• Splitting tests out is non-trivial. (Or at least was at the time. Things may have changed once someone gets around to implementing this.)
• Concerns that the workflow would become more cumbersome. I think "new-build" (once fully stabilized) should address this neatly.
• Concerns about versioning in a consistent way. (For this you'll have to see the thread. Too much to summarize neatly.)

For the split into an internal library: This should be pretty easy, but needs to wait until all Cabal versions that we want to be able to bootstrap from actually support internal libraries. Motivation: Lets us easily avoid accidentally introducing new dependencies from the solver into cabal-install code.

[dcoutts]

BTW, if other people actively want to use this library, have they said they'll contribute any effort? I may have missed it but I've not seen much in the way of contributions to the Cabal lib from the stack maintainers despite the fact that stack greatly relies on it (and Cabal is full of issues/limitations that cause problems for both stack and cabal-install).

[phadej]

FWIW: I personally want to split solver into a library, to use it for "my own things ™️", but the parsec parser is the first priority atm.

[sjakobi]

Can someone clarify what "internal" vs. "top-level" library means? An "internal" libray would be a library component inside cabal-install while a "top-level" library would be a separate package?

In any case I'd be willing to contribute work to make the solver accessible as a library.

BTW why doesn't cabal-install simply move all its other-modules into a library component, similarly to how stack is structured?

For the split into an internal library: This should be pretty easy, but needs to wait until all Cabal versions that we want to be able to bootstrap from actually support internal libraries.

How long is that expected to take?

[BardurArantsson]

Can someone clarify what "internal" vs. "top-level" library means? An "internal" libray would be a library component inside cabal-install while a "top-level" library would be a separate package?

Yes, that's right.

BTW why doesn't cabal-install simply move all its other-modules into a library component, similarly to how stack is structured?

This has been discussed before. There are at least a few of problems with this: a) Suddently everyone and their mother might start depending on cabal-install-the-library, effectively freezing the API[1]. b) The API would be completely ad-hoc and probably just about as usable as the shell commands themselves[2, 3], so effectively nothing would be gained other than just calling cabal-install via the shell. c) Versioning still isn't a solved problem. d) It would lead to unnecessary overhead when developing cabal-install-the-app (mostly solved by new-build, but that still isn't quite ready for prime time.)

(See the lengthy discussion threads for more issues and concerns.)

[1] Warnings the the API notwithstanding, the risk here is that people just do it anyway and suddenly cabal-install-the-library developers become the Bad Guys when they break the API, nevermind if it's specified as "unstable" (or whatever).
[2] Maybe even less.
[3] I think the general feeling among the Cabal developers is that we want to Do It Right(TM), i.e. come up with a solver API that makes sense for others to use, doesn't have lots of incidental complexity, guides the users in the right direction via types, etc. etc. In other words: An API that ticks all the boxes of good API design.

How long is that expected to take?

I think the "internal" split will have to wait until 2.0 is out? (Please someone correct me, if I'm wrong about this.) Basically the "anonymous library" functionality will have had to have been part of at least 1-2 stable releases. (For bootstrap reasons, IIRC.)

[ezyang]

It's worth clarifying that the solver split can happen whenever; it's just a "simple" matter of deciding on an API, splitting it out into a separate package, and putting it on a release cycle.

[23Skidoo]

mostly solved by new-build, but that still isn't quite ready for prime time.

Maybe not for non-expert users, but all core developers are using it and it's currently the officially recommended way to build Cabal for development.

[BardurArantsson]

@23Skidoo True, true. I'm just going by the fact that I still see various minor tweaks/fixes going in from time to time. :)

FTR, I'm also currently using it for all my (non-Cabal) development.