Special treatment of pre-installed packages by the solver

[hasufell]

The cabal solver seems to treat pre-installed packages specially (e.g. those shipped with GHC).

To reproduce:

git clone https://github.com/hasufell/toto.git
cd toto
ghcup run --ghc 9.4.8 -- cabal build

This should cause a failure, because ghc-9.4.8 ships with filepath-1.4.2.2, but the package above uses modules from 1.4.100.1. The package has no upper bounds on filepath. For any other non-pre-installed package, the solver would pick the latest.

I understand that this is by design, but I question this design here, because:

• it makes it harder for core library maintainers to ship bugfixes
• it's a potential security risk

@mpickering found out that there used to be a --upgrade-dependencies switch, which is now disabled.

I argue that the default should be to pick the latest possible version anyway.

---

CCing some potentially interested parties: @simonpj @frasertweedale

[mpickering]

Branch which re-enables --upgrade-dependencies (in a unfinished way) - https://github.com/mpickering/cabal/tree/wip/upgrade-dependencies

It seems sensible to me to choose pre-installed packages, then you don't have to build them again. If the version constraints don't disallow it then the solver could choose that install plan anyway. If you really don't want a package to be part of the install plan then perhaps instead you want a means to instruct the solver to never choose a particular version as an additional constraint form.

[Mikolaj]

Regardless of the outcome of the discussion, it would make sense to find the PR that commented --upgrade-dependencies out, understand how this happened and prevent for the future, e.g., by guarding this functionality with tests and also by documenting it (better).

[hasufell]

@mpickering I'm worried about defaults here. I don't think there's an easy way to tell my users that there was a subtle bug in filepath in splitFileName. No one reads the changelog. GHC already ships the version.

What if this is a security bug? What do I do? I have no communication channels. A cabal update && cabal build should leave your project in the best possible state (bugfixes, security fixes) without further interventions/constraints required by the end users.

This is how all linux distributions work, afaik. "Saving compilation time" seems like a questionable priority, imo (as a default).

[Mikolaj]

@mpickering also says about --upgrade-dependencies: "There is the -prefer-oldest option already and this I suppose is a --prefer-newest ... decide how it should interact with --prefer-oldest".

[Mikolaj]

CC @grayjay, @gbaz

[ulysses4ever]

I agree that --prefer-newest (so, no special-casing the boot packages) would be a cleaner default. Perhaps, there should be --prefer-installed to get the current default if we change the default to --prefer-newest?

[Mikolaj]

While I like the default @hasufell proposed better, for the reasons given and also because it's more uniform, I worry it would hit hard the users with big sets of installed packages, which may include Nix users, Linux distribution users and v1-/cabal-env users (whether for teaching purposes or others). So we'd need some good backward compatibility scheme.

[Mikolaj]

Another solution, which unfortunately increases complication, might be to treat libraries installed together with GHC (and perhaps all installed not by the user directly, but by install/upgrade scripts) specially and apply --upgrade-dependencies to them, while keeping user-installed libraries immutable, on the premise that the user knows what the user is doing (and that installing libraries is rare and discouraged).

Edit: which somehow agrees with how we treat local packages even if newer versions are on Hackage, local packages being "user-installed" and so automatic upgrades being disabled (I think?). I remember apt keeps track which packages are directly requested by the user and which are only installed as dependencies, but this is a very distant analogy to installed Haskell packages and how cabal treats them.

[hasufell]

CCing @Ericson2314 @angerman wrt Nix

[michaelpj]

I don't think there's an easy way to tell my users that there was a subtle bug in filepath in splitFileName. No one reads the changelog. GHC already ships the version.

What happens if you deprecate the version with the bug (the installed version)? Will cabal still prefer it?

[Ericson2314]

Re Nix, I would like to have no notion of "preinstalled dependencies" because one should not be "preinstalling" things with Nix. So I like this.

• it shouldn't affect my ideal world for Nix, as described above
• insofar as users might expect "core libraries" to be preinstalled, the less special-casing Cabal does, the easier/less-surprising it is to switch those libraries to be non-pre-installed.

In conclusion, Proud Nix Hater @hasufell has proposed something that I think is actually great for Nix. Thank you! :)

[gbaz]

I'm a bit confused. In fact, the way we use nix at work involves "pre-installing" everything in the sense that everything goes into a package database which nix provides, instead of the cabal store, no?

So with the current behavior, if I am building foo which depends on bar-1 and the latter is in the package database nix provides, then even if bar-1.1 is released, cabal build will still use bar-1. With the new behavior, cabal-build would download and build bar-1.1 even though the nix configuration specified bar-1.

(the workaround for this, which is possible but slightly irritating, is to use a cabal.project or the like to disable hackage or any other package repository for packages developed in a nix provided environment)