Bootstrap of version 3.8.1.0 fails on macOS

[essandess]

Describe the bug

I'm the maintainer for the MacPorts cabal port and am trying to update to version 3.8.1.0.

I'm following the revised bootstrap instructions on https://github.com/haskell/cabal/blob/master/bootstrap/README.md#bootstrapping-cabal-install and am hitting this issue. The cabal binary is downloaded from https://downloads.haskell.org/~cabal/cabal-install-3.8.1.0/.

cabal new-update
cabal new-configure

cd ./bootstrap
cabal v2-run -v0 cabal-bootstrap-gen -- darwin-9.2.1.plan.json | tee darwin-9.2.1.json

The cabal error prior to the tee output is:

Error: cabal: Could not resolve dependencies:
[__0] trying: cabal-bootstrap-gen-0 (user goal)
[__1] next goal: base (dependency of cabal-bootstrap-gen)
[__1] rejecting: base-4.16.0.0/installed-4.16.0.0 (conflict:
cabal-bootstrap-gen => base^>=4.12.0.0 || ^>=4.13.0.0 || ^>=4.14.0.0)
[__1] skipping: base-4.15.0.0 (has the same characteristics that caused the
previous version to fail: excluded by constraint '^>=4.12.0.0 || ^>=4.13.0.0
|| ^>=4.14.0.0' from 'cabal-bootstrap-gen')
[__1] rejecting: base-4.14.1.0, base-4.14.0.0, base-4.13.0.0, base-4.12.0.0,
base-4.11.1.0, base-4.11.0.0, base-4.10.1.0, base-4.10.0.0, base-4.9.1.0,
base-4.9.0.0, base-4.8.2.0, base-4.8.1.0, base-4.8.0.0, base-4.7.0.2,
base-4.7.0.1, base-4.7.0.0, base-4.6.0.1, base-4.6.0.0, base-4.5.1.0,
base-4.5.0.0, base-4.4.1.0, base-4.4.0.0, base-4.3.1.0, base-4.3.0.0,
base-4.2.0.2, base-4.2.0.1, base-4.2.0.0, base-4.1.0.0, base-4.0.0.0,
base-3.0.3.2, base-3.0.3.1 (constraint from non-upgradeable package requires
installed instance)
[__1] fail (backjumping, conflict set: base, cabal-bootstrap-gen)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: base, cabal-bootstrap-gen

Expected behavior

I expect the file darwin-9.2.1.json to be created to be able to bootstrap cabal on macOS.

System information
macOS 12.5 21G72 x86_64
Xcode 13.4.1 13F100

cabal-install version 3.8.1.0
compiled using version 3.8.1.0 of the Cabal library

The Glorious Glasgow Haskell Compilation System, version 9.2.1

[Mikolaj]

That you for the report. The bounds on cabal-bootstrap-gen seem to be outdated. A workaround is to compile cabal-bootstrap-gen with an older GHC and execute the compiled version (possible while GHC is set to a new one; I don't know it that matters). A proper fix would be to relax bounds in the .cabal file for `cabal-bootstrap-gen to include all new GHCs up to 9.4.

[robx]

You shouldn't need to regenerate the bootstrap files, I believe we check that the linux bootstrap files are good for bootstrapping macOS in CI. Instead, you should be able to bootstrap using the normal bootstrapping command,

./bootstrap/bootstrap.py -d ./bootstrap/linux-ghcver.json -w /path/to-ghc

[Mikolaj]

Oh, I didn't think about that. @essandess: please let us know if that works in your case.

@robx: I don't remember if I asked you and I can't find you on IRC/Matrix: would you like to drop in sometimes into the fortnightly cabal dev's chats (starts in 40min)?

[essandess]

please let us know if that works in your case.

Thanks for the suggestions. Unfortunately, swapping in linux-9.2.3.json doesn't work for ghc version 9.2.1:

ghc-pkg: cannot find package base-4.16.2.0

We have explicitly not updated ghc beyond version 9.2.1 yet because these versions require the Haskell build tool hadrian and we (I) just haven't gotten around to figuring out how to build and use hadrian to bootstrap ghc. When we do, ghc's latest version is 9.4.1, which isn't in the predefined linux-*.json files in the cabal bootstrap directory.

Is there an alternate approach by simply ripping out the version requirements in cabal-bootstrap-gen.cabal? Any suggestion on which requirements to relax, or simply remove all of them?

[essandess]

Is there an alternate approach by simply ripping out the version requirements in cabal-bootstrap-gen.cabal? Any suggestion on which requirements to relax, or simply remove all of them?

That apparently doesn't work either. Simply removing all the requirements from cabal-bootstrap-gen.cabal yields the error:

> :info:build src/Main.hs:39:11: error:
> :info:build     • Couldn't match expected type: Map.Map C.PackageName I.PackageI
nfo
> :info:build                   with actual type: (FilePath, Map.Map C.PackageName
 I.PackageInfo)
> :info:build     • In the first argument of ‘main2’, namely ‘meta’
> :info:build       In a stmt of a 'do' block: main2 meta plan
> :info:build       In the expression:
> :info:build         do meta <- I.cachedHackageMetadata
> :info:build            plan <- P.decodePlanJson planPath
> :info:build            main2 meta plan
> :info:build    |
> :info:build 39 |     main2 meta plan
> :info:build    |           ^^^^

[Mikolaj]

Any suggestion on which requirements to relax, or simply remove all of them?

I think just add base-4.16 and actually all other base version up to the one in GHC 9.4, in the same way older base version ranges are specified so far. If that works, we'd accept this as a PR.

[Mikolaj]

BTW, I think there is a linux-9.2.1.json somewhere in the repo history, half a year ago perhaps.

[essandess]

linux-9.2.1.json

BTW, I think there is a linux-9.2.1.json somewhere in the repo history, half a year ago perhaps.

Thanks, but no such luck:

git log --all --full-history -- "*linux-9.2.1.json*"

I'll try setting the version on base-4.16 and failing that, move over to work on bootstrapping the latest ghc with hadrian.

[essandess]

I've updated to ghc-9.4.1. All approaches discussed to bootstrap cabal appear to fail:

Using linux-9.2.3.json:

:debug:build system -W /opt/local/var/macports/build/_opt_local_ports_lang_cabal
/cabal/work/cabal-install-3.8.1.0: ./bootstrap/bootstrap.py -d ./bootstrap/darwi
n-9.4.1.json -w /opt/local/bin/ghc
:info:build ghc-pkg-9.4.1: cannot find package ghc-prim-0.8.0

Trying to build darwin-9.4.1 per the instructions:

:debug:build system -W /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-install-3.8.1.0: /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/bin/cabal v2-build --with-compiler=/opt/local/bin/ghc --dry-run cabal-install:exe:cabal
:info:build Resolving dependencies...
:info:build Error: cabal: Could not resolve dependencies:
:info:build [__0] trying: Cabal-3.8.1.0 (user goal)
:info:build [__1] trying: base-4.17.0.0/installed-4.17.0.0 (dependency of Cabal)
:info:build [__2] trying: Cabal-described-3.8.1.0 (user goal)
:info:build [__3] next goal: rere (dependency of Cabal-described)
:info:build [__3] rejecting: rere-0.2 (conflict: base==4.17.0.0/installed-4.17.0.0, rere
:info:build => base>=4.3.0.0 && <4.17)
:info:build [__3] skipping: rere-0.1 (has the same characteristics that caused the
:info:build previous version to fail: excludes 'base' version 4.17.0.0)

[Mikolaj]

Yes, linux-9.2.3.json has no chance to work with GHC 9.4. I wonder, perhaps spin a Linux VM, generate linux-9.2.1.json on that (make something-bootstrap) and copy over?

[Mikolaj]

BTW, I forgot the backlog, but don't you have any old cabal binary lying around? Is it the first time you bootstrap for that architecture? That should really be the only time using the bootstrap files makes sense.

[essandess]

BTW, I forgot the backlog, but don't you have any old cabal binary lying around? Is it the first time you bootstrap for that architecture? That should really be the only time using the bootstrap files makes sense.

I'm following the revised bootstrap instructions on https://github.com/haskell/cabal/blob/master/bootstrap/README.md#bootstrapping-cabal-install and am hitting this issue. The cabal binary is downloaded from https://downloads.haskell.org/~cabal/cabal-install-3.8.1.0/.

I downloaded the cabal 3.8.1.0 binary and am following the bootstrap instructions. The issue here is to get cabal bootstrapping reliably working for a MacPorts package install. One-off’s like specifying some previous version or relying on a VM won’t scale.

[Mikolaj]

I downloaded the cabal 3.8.1.0 binary

And the binary works for you? Forgive my clueless questions, I'm trying to understand the "what for" to be able to help with "how".

[essandess]

And the binary works for you? Forgive my clueless questions, I'm trying to understand the "what for" to be able to help with "how".

I’m the maintainer of the cabal port on MacPorts. Ideally, package managers like MacPorts want to download the source code and compile the target so that the binary can be built and deployed across a broad range of OS versions and architectures. Simply distributing a prebuilt binary is limiting fallback option.

Bootstrapping cabal from source worked in previous versions. Apparently, this no longer works. I would like to be able to bootstrap cabal from source. Is there a way to accomplish this with the latest versions of cabal and ghc?

[Mikolaj]

Thank you for the explanation. So you'd like to bootstrap cabal using ghc binary, but not cabal binary? Why not bootstrap GHC without using ghc binary? Why not bootstrap cabal using ghc and cabal binary?

Bootstrapping cabal from source worked in previous versions. Apparently, this no longer works.

It always worked only for a selection of GHC version. I hope the selection we have now works. I'm sorry that's not enough for your use case. We might accept a PR that makes more GHC versions available to boostrap with.

[essandess]

I’m simply following the bootstrap instructions provided in this repo. If there are other methods that work that would be great.

Would you please post cabal bootstrap instructions?

Thank you for the explanation. So you'd like to bootstrap cabal using ghc binary, but not cabal binary? Why not bootstrap GHC without using ghc binary? Why not bootstrap cabal using ghc and cabal binary?

Bootstrapping cabal from source worked in previous versions. Apparently, this no longer works.

It always worked only for a selection of GHC version. I hope the selection we have now works. I'm sorry that's not enough for your use case. We might accept a PR that makes more GHC versions available to boostrap with.

[Mikolaj]

I think the best way to bootstrap new cabal using a ghc and an old cabal is via cabal install cabal-install, as mentioned at https://github.com/haskell/cabal/blob/master/README.md. Alternatively, say, cabal install cabal-install-3.8.1.0 to make sure the result is reasonably (but not completely) immutable.

[essandess]

Here's the error message for cabal install cabal-install:

sudo -u macports bash -c 'export CABAL_CONFIG=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/.home/.cabal/config ; \
cd /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-install-3.8.1.0/ ; \
/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/bin/cabal update ; \
/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/bin/cabal install cabal-install'
Downloading the latest package list from hackage.haskell.org
Package list of hackage.haskell.org has been updated.
The index-state is set to 2022-08-18T15:48:58Z.
Error: cabal: Could not resolve dependencies:
[__0] trying: Cabal-3.8.1.0 (user goal)
[__1] trying: base-4.17.0.0/installed-4.17.0.0 (dependency of Cabal)
[__2] trying: Cabal-described-3.8.1.0 (user goal)
[__3] next goal: rere (dependency of Cabal-described)
[__3] rejecting: rere-0.2 (conflict: base==4.17.0.0/installed-4.17.0.0, rere
=> base>=4.3.0.0 && <4.17)
[__3] skipping: rere-0.1 (has the same characteristics that caused the
previous version to fail: excludes 'base' version 4.17.0.0)
[__3] fail (backjumping, conflict set: Cabal-described, base, rere)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: base, Cabal, rere, Cabal-described
Try running with --minimize-conflict-set to improve the error message.

I think the best way to bootstrap new cabal using a ghc and an old cabal is via cabal install cabal-install, as mentioned at https://github.com/haskell/cabal/blob/master/README.md. Alternatively, say, cabal install cabal-install-3.8.1.0 to make sure the result is reasonably (but not completely) immutable.

[essandess]

It always worked only for a selection of GHC version. I hope the selection we have now works.

Is it advisable to use stack to get reliable builds of cabal? This would make our cabal port dependent on our stack port.

[Mikolaj]

[__3] rejecting: rere-0.2 (conflict: base==4.17.0.0/installed-4.17.0.0, rere => base>=4.3.0.0 && <4.17)

Indeed, cabal-install, or rather its dependencies, don't yet build with GHC 9.4. I'd suggest either using GHC 9.2 or, if you feel adventurous, running with --allow-newer=base (and if there are more such problems, just with --allow-newer).

Is it advisable to use stack to get reliable builds of cabal? This would make our cabal port dependent on our stack port.

Depends on how reliable you need it. If very much so, it's probably worth the effort to use Stackage LTS. The newest GHC it lets you use at this time is GHC 9.0.2, which is understandable, given the focus on testing, reproducibility and the need to coordinate a lot of package updates. Also, usually only point version 2 or 3 of GHC is stable enough, so 9.4.1 is not the best bet, though GHC 9.2.5 should probably be and 9.2.2 is probably good enough if it supports your architecture.

[essandess]

Thanks for the advice. I’ll refactor to use stack.

[Mikolaj]

Good luck. Out of curiosity, is your stack package bootstrapped from ghc alone?

[essandess]

Out of curiosity, is your stack package bootstrapped from ghc alone?

I've set things up so that each Haskell thing bootstraps itself: stack bootstraps stack, and so forth.

Thinking on this, the real reason I'm pushing ahead with the latest cabal and ghc is that they both support native arm64 builds. However, stack only supports x86_64 at present.

I'd be willing to try a few more wacky things the get cabal to bootstrap itself so that it will also support native arm64. Do you have suggestions before I throw in the towel and use stack?

[Bodigrim]

    [__0] trying: Cabal-3.8.1.0 (user goal)
    [__1] trying: base-4.17.0.0/installed-4.17.0.0 (dependency of Cabal)
    [__2] trying: Cabal-described-3.8.1.0 (user goal)
    [__3] next goal: rere (dependency of Cabal-described)
    [__3] rejecting: rere-0.2 (conflict: base==4.17.0.0/installed-4.17.0.0, rere
    => base>=4.3.0.0 && <4.17)

@essandess this does not look right, why is Cabal-described a "user goal"? Neither Cabal package nor cabal-install executable depend on it.

[Bodigrim]

This works for me:

cabal install cabal-install-3.8.1.0 -w ghc-9.4 --allow-newer='cabal-install:time,cabal-install:base,lukko:base,cabal-install:text,hackage-security:base,hackage-security:template-haskell,cabal-install-solver:base'

[essandess]

This works for me:

cabal install cabal-install-3.8.1.0 -w ghc-9.4 --allow-newer='cabal-install:time,cabal-install:base,lukko:base,cabal-install:text,hackage-security:base,hackage-security:template-haskell,cabal-install-solver:base'

Thanks. I see these issues running with the targets cabal-install-3.8.1.0 and new:

$ sudo -u macports bash -c 'export CABAL_CONFIG=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/.home/.cabal/config ; cd /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-install-3.8.1.0/ ; /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/bin/cabal install cabal-install-3.8.1.0 -w ghc-9.4.1 --allow-newer="cabal-install:time,cabal-install:base,lukko:base,cabal-install:text,hackage-security:base,hackage-security:template-haskell,cabal-install-solver:base"'
Error: cabal: No targets given and there is no package in the current
directory. Use the target 'all' for all packages in the project or specify
packages or components by name or location. See 'cabal build --help' for more
details on target options.

sudo -u macports bash -c 'export CABAL_CONFIG=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/.home/.cabal/config ; cd /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-install-3.8.1.0/ ; /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/bin/cabal install all -w ghc-9.4.1 --allow-newer'
…
Wrote tarball sdist to
/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/dist-newstyle/sdist/cabal-benchmarks-3.tar.gz
Resolving dependencies...
Error: cabal: Cannot build the executables in the package Cabal because it
does not contain any executables. Check the .cabal file for the package and
make sure that it properly declares the components that you expect.
Cannot build the executables in the package Cabal-QuickCheck because it does
not contain any executables. Check the .cabal file for the package and make
sure that it properly declares the components that you expect.
Cannot build the executables in the package Cabal-described because it does
not contain any executables. Check the .cabal file for the package and make
sure that it properly declares the components that you expect.
Cannot build the executables in the package Cabal-syntax because it does not
contain any executables. Check the .cabal file for the package and make sure
that it properly declares the components that you expect.
Cannot build the executables in the package Cabal-tree-diff because it does
not contain any executables. Check the .cabal file for the package and make
sure that it properly declares the components that you expect.
Cannot build the executables in the package cabal-install-solver because it
does not contain any executables. Check the .cabal file for the package and
make sure that it properly declares the components that you expect.

You don’t need to download anything manually, just execute cabal update && cabal install cabal-install-3.8.1.0 outside of a project (e. g., in an empty folder).

This is what happens when I do that. Also, I don't see how this could bootstrap cabal—there's no source code in the empty directory. We don't want to simply download another prebuilt binary.

sudo -u macports bash -c 'export CABAL_CONFIG=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/.home/.cabal/config ; cd /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/tmp ; /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/bin/cabal update ; /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/bin/cabal install cabal-install-3.8.1.0'
Downloading the latest package list from hackage.haskell.org
Package list of hackage.haskell.org is up to date.
The index-state is set to 2022-08-19T00:37:09Z.
Resolving dependencies...
Error: cabal: Could not resolve dependencies:
[__0] trying: cabal-install-3.8.1.0 (user goal)
[__1] next goal: base (dependency of cabal-install)
[__1] rejecting: base-4.17.0.0/installed-4.17.0.0 (conflict: cabal-install =>
base>=4.10 && <4.17)
[__1] skipping: base-4.17.0.0 (has the same characteristics that caused the
previous version to fail: excluded by constraint '>=4.10 && <4.17' from
'cabal-install')
[__1] rejecting: base-4.16.3.0, base-4.16.2.0, base-4.16.1.0, base-4.16.0.0,
base-4.15.1.0, base-4.15.0.0, base-4.14.3.0, base-4.14.2.0, base-4.14.1.0,
base-4.14.0.0, base-4.13.0.0, base-4.12.0.0, base-4.11.1.0, base-4.11.0.0,
base-4.10.1.0, base-4.10.0.0, base-4.9.1.0, base-4.9.0.0, base-4.8.2.0,
base-4.8.1.0, base-4.8.0.0, base-4.7.0.2, base-4.7.0.1, base-4.7.0.0,
base-4.6.0.1, base-4.6.0.0, base-4.5.1.0, base-4.5.0.0, base-4.4.1.0,
base-4.4.0.0, base-4.3.1.0, base-4.3.0.0, base-4.2.0.2, base-4.2.0.1,
base-4.2.0.0, base-4.1.0.0, base-4.0.0.0, base-3.0.3.2, base-3.0.3.1
(constraint from non-upgradeable package requires installed instance)
[__1] fail (backjumping, conflict set: base, cabal-install)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: cabal-install, base

[Mikolaj]

Retry the last one with the whole --allow-newer='cabal-install thing. It downloads the source code of an official release of cabal-install-3.8.1.0 from the official repository of Haskell packages, called Hackage. No binaries involved.

[andreasabel]

Is there an alternate approach by simply ripping out the version requirements in cabal-bootstrap-gen.cabal? Any suggestion on which requirements to relax, or simply remove all of them?

That apparently doesn't work either. Simply removing all the requirements from cabal-bootstrap-gen.cabal yields the error:

> :info:build src/Main.hs:39:11: error:
> :info:build     • Couldn't match expected type: Map.Map C.PackageName I.PackageI
nfo
> :info:build                   with actual type: (FilePath, Map.Map C.PackageName
 I.PackageInfo)
> :info:build     • In the first argument of ‘main2’, namely ‘meta’
> :info:build       In a stmt of a 'do' block: main2 meta plan
> :info:build       In the expression:
> :info:build         do meta <- I.cachedHackageMetadata
> :info:build            plan <- P.decodePlanJson planPath
> :info:build            main2 meta plan
> :info:build    |
> :info:build 39 |     main2 meta plan
> :info:build    |           ^^^^

This PR might help here in the future:

• Bump cabal-bootstrap-gen to GHC 9.2.4 ecosystem. #8403

[essandess]

Retry the last one with the whole --allow-newer='cabal-install thing. It downloads the source code of an official release of cabal-install-3.8.1.0 from the official repository of Haskell packages, called Hackage. No binaries involved.

Great thank you! 🇺🇦

I'm getting very close. I'm able to build with new-install cabal-install-3.8.1.0 --with-compiler=/opt/local/bin/ghc --allow-newer.

But there's a DESTDIR issue: cabal tries to install the build's binary in /opt/local/bin. I need it to install in a DESTROOT directory, say _build/bin. I tried issuing the flag --prefix=_build, but this didn't work. Package managers like Macports need a GNU DESTDIR capability or workaround to put intermediate build targets in a temporary directory, and manage installing these targets themselves.

How to I tell cabal to build the target and put the binary and _build? If I invoke new-install, I run into this DESTDIR issue. If I invoked new-build, I get an error telling me that I haven't run an init. And if I run init I get a separate error.

cd "/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-install-3.8.1.0" && \
/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal \
--store-dir=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/.home/.cabal/store \
--logs-dir=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/.home/.cabal/logs new-install cabal-install-3.8.1.0 \
--prefix=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-install-3.8.1.0/_build \
--with-compiler=/opt/local/bin/ghc \
--allow-newer
…
:info:build Completed    cabal-install-3.8.1.0 (lib)
:info:build Starting     cabal-install-3.8.1.0 (exe:cabal)
:info:build Building     cabal-install-3.8.1.0 (exe:cabal)
:info:build Installing   cabal-install-3.8.1.0 (exe:cabal)
:info:build Completed    cabal-install-3.8.1.0 (exe:cabal)
:info:build Copying 'cabal' to '/opt/local/bin/cabal'
/opt/local/bin: copyFile:atomicCopyFileContents:withReplacementFile:openBinaryTempFile: permission denied (Operation not permitted)

[Mikolaj]

cabal install --help|grep -i2 dir suggests a few promising options. Would --installdir help?

[Mikolaj]

--install-method=copy looks relevant, too.

[essandess]

• Bump cabal-bootstrap-gen to GHC 9.2.4 ecosystem. #8403

Thanks. I patched the source per #8403, but still see this issue running the first bootstrap command in the instructions:

:debug:build system -W /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-install-3.8.1.0: /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/bin/cabal v2-build --with-compiler=/opt/local/bin/ghc --dry-run cabal-install:exe:cabal
:info:build Resolving dependencies...
:info:build Error: cabal: Could not resolve dependencies:
:info:build [__0] trying: Cabal-3.8.1.0 (user goal)
:info:build [__1] trying: base-4.17.0.0/installed-4.17.0.0 (dependency of Cabal)
:info:build [__2] trying: Cabal-described-3.8.1.0 (user goal)
:info:build [__3] next goal: rere (dependency of Cabal-described)
:info:build [__3] rejecting: rere-0.2 (conflict: base==4.17.0.0/installed-4.17.0.0, rere
:info:build => base>=4.3.0.0 && <4.17)
:info:build [__3] skipping: rere-0.1 (has the same characteristics that caused the
:info:build previous version to fail: excludes 'base' version 4.17.0.0)
:info:build [__3] fail (backjumping, conflict set: Cabal-described, base, rere)
:info:build After searching the rest of the dependency tree exhaustively, these were the
:info:build goals I've had most trouble fulfilling: base, Cabal, rere, Cabal-described

[Mikolaj]

What you describe suggests you run from the cloned repo. If so, https://github.com/haskell/cabal/blob/master/README.md suggests, for your use case, to do cabal install --project-file=cabal.project.release cabal-install (and then add --allow-newer if you use an unsupported GHC). Is that what you do?

[essandess]

What you describe suggests you run from the cloned repo. If so, https://github.com/haskell/cabal/blob/master/README.md suggests, for your use case, to do cabal install --project-file=cabal.project.release cabal-install (and then add --allow-newer if you use an unsupported GHC). Is that what you do?

The instructions at https://github.com/haskell/cabal/blob/master/bootstrap/README.md say to run the command:

./bootstrap/bootstrap.py -d ./bootstrap/darwin-9.4.1.json -w /opt/local/bin/ghc-9.4.1

That's what I'm doing. I included the flag --allow-newer and with the patches from #8403 can now successfully generate the file darwin-9.4.1.json.

Unfortunately, the bootstrap.py command above still fails with the error:

:info:build [1 of 2] Compiling Main             ( Setup.hs, /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/dists/cabal-install-solver-3.8.1.0/setup/Main.o )
:info:build [2 of 2] Linking /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/dists/cabal-install-solver-3.8.1.0/setup/Setup
:info:build Configuring cabal-install-solver-3.8.1.0...
:info:build Error: Setup: Encountered missing or private dependencies:
:info:build base >=4.10 && <4.17
…
info:build bootstrap: running cwd=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/cabal-install-solver /opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/dists/cabal-install-solver-3.8.1.0/setup/Setup configure --builddir=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/dists/cabal-install-solver-3.8.1.0 --package-db=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/packages.conf --prefix=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/pseudostore --bindir=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/bin --with-compiler=/opt/local/bin/ghc --with-hc-pkg=/opt/local/bin/ghc-pkg --with-hsc2hs=/opt/local/bin/hsc2hs --flags=-debug-conflict-sets -debug-expensive-assertions -debug-tracetree
:info:build Traceback (most recent call last):
:info:build   File "/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/./bootstrap/bootstrap.py", line 399, in <module>
:info:build     main()
:info:build   File "/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/./bootstrap/bootstrap.py", line 367, in main
:info:build     bootstrap(info, ghc)
:info:build   File "/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/./bootstrap/bootstrap.py", line 262, in bootstrap
:info:build     install_dep(dep, ghc)
:info:build   File "/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/./bootstrap/bootstrap.py", line 205, in install_dep
:info:build     install_sdist(dist_dir, sdist_dir, ghc, dep.flags)
:info:build   File "/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/./bootstrap/bootstrap.py", line 235, in install_sdist
:info:build     check_call([setup, 'configure'] + configure_args)
:info:build   File "/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/./bootstrap/bootstrap.py", line 228, in check_call
:info:build     subprocess_run(args, cwd=sdist_dir, check=True)
:info:build   File "/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/./bootstrap/bootstrap.py", line 396, in subprocess_run
:info:build     return subprocess.run(args, **kwargs)
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 524, in run
:info:build     raise CalledProcessError(retcode, process.args,
:info:build subprocess.CalledProcessError: Command '[PosixPath('/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/dists/cabal-install-solver-3.8.1.0/setup/Setup'), 'configure', '--builddir=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/dists/cabal-install-solver-3.8.1.0', '--package-db=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/packages.conf', '--prefix=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/pseudostore', '--bindir=/opt/local/var/macports/build/_opt_local_ports_lang_cabal/cabal/work/cabal-Cabal-v3.8.1.0/_build/bin', '--with-compiler=/opt/local/bin/ghc', '--with-hc-pkg=/opt/local/bin/ghc-pkg', '--with-hsc2hs=/opt/local/bin/hsc2hs', '--flags=-debug-conflict-sets -debug-expensive-assertions -debug-tracetree']' returned non-zero exit status 1.
:info:build Command failed: ./bootstrap/bootstrap.py -d ./bootstrap/darwin-9.4.1.json -w /opt/local/bin/ghc
:info:build Exit code: 1
:error:build Failed to build cabal: command execution failed

[Mikolaj]

:info:build base >=4.10 && <4.17

This is indeed present in cabal-install-solver.cabal and it rules out GHC 9.4. We'd need to either modify this in all 4 packages or tell Setup (in bootstrap.py) to allow-newer, but I don't know if the latter is even possible. The ability to run bootstrap generation on GHC 9.4 doesn't yet enable you to compile cabal for GHC 9.4, unfortunately. For this we'd need another PR, which is blocked on the ability to set up GHC 9.4 CI for cabal, attempted in #8264.

[essandess]

:info:build base >=4.10 && <4.17

This is indeed present in cabal-install-solver.cabal and it rules out GHC 9.4. We'd need to either modify this in all 4 packages or tell Setup (in bootstrap.py) to allow-newer, but I don't know if the latter is even possible. The ability to run bootstrap generation on GHC 9.4 doesn't yet enable you to compile cabal for GHC 9.4, unfortunately. For this we'd need another PR, which is blocked on the ability to set up GHC 9.4 CI for cabal, attempted in #8264.

Based on the feedback in this thread, I have a cabal-based bootstrap using this comment. Please see macports/macports-ports#15795.

I'm confused by the status of bootstrapping cabal. Apparently Python script bootstrap.py doesn't currently work with ghc-9.4.1. But this command does work to bootstrap cabal with cabal and ghc-9.4.1:

cabal install --project-file=cabal.project.release cabal-install -w ghc-9.4.1 --allow-newer

What's the Python script even for if the above command works?

[ulysses4ever]

@essandess "bootstrapping" is the case when you don't have another version of cabal-install to build your new shiny cabal-install --- that's when you have to use bootstrap.py. In contrast, you're doing

cabal install --project-file=cabal.project.release cabal-install -w ghc-9.4.1 --allow-newer

which indicates that you have a previous version of cabal-install (the first word cabal in this command). That's not bootstrapping, it's a mere building of cabal-install with cabal-install.

But it's good that it works for you. This thread shows how important it is to understand constraints of your environment!

[essandess]

Thanks for the explanation. But we need an exisiting cabal to create the file darwin-9.4.1.json per the bootstrap README.md, which is required for bootstrapping. So with either method, isn't a pre-existing cabal binary always required?

[ulysses4ever]

With the trick that worked for you (cabal install cabal-install), you have to have a functioning cabal on the same platform where you want to use a new cabal. In contrast, the .json file can be generated (with cabal, as you rightly point out) on any platform. You can then copy the .json file to the desired platform and use the Python script to build a cabal there for the first time.

[Mikolaj]

But this command does work to bootstrap cabal with cabal and ghc-9.4.1

Congrats. Do you run it at tag cabal-install-v3.8.1.0?

I still think taking the source package from Hackage is more obvious (and even more secure cryptographically, because the repo can be tampered with more easily), but you need to make sure you are not installing it when in the repo checkout directory, because then the local cabal.project takes over.

[essandess]

I still think taking the source package from Hackage is more obvious

I'd like to implement the install using this approach but am still stuck because cabal install cabal-install-v3.8.1.0 tries to install the binary outside DESTDIR.

I would like to be able to run

cabal build cabal-install-v3.8.1.0
cabal haddock cabal-install-v3.8.1.0

then copy the binaries and documentation produced by these commands to DESTDIR, and have the package manager take over from there.

But running the first command cabal build cabal-install-v3.8.1.0 in an empty directory throws an error because there's no project file.

What are the correct cabal commands to build cabal and documentation with itself, but without the final target copy outside the build directory?

[Mikolaj]

I'd like to implement the install using this approach but am still stuck because cabal install cabal-install-v3.8.1.0 tries to install the binary outside DESTDIR.

Did my idea to use --installdir fail?

What are the correct cabal commands to build cabal and documentation with itself, but without the final target copy outside the build directory?

That would probably be what you listed (without the -v3.8.1.0 suffixes), but prepended with cabal get cabal-install-v3.8.1.0. Not 100% sure if it would work, but I'd love to know, because that's how distros should probably do it (optionally applying their own patches after the first step).

[essandess]

Did my idea to use --installdir fail?

This idea works, thank you. But the command cabal haddock cabal-install-v3.8.1.0 … fails with the error:

No cabal.project file or cabal file matching the default glob './*.cabal' was found.
Please create a package description file <pkgname>.cabal or a cabal.project file referencing the packages you want to build.

Is there a comparable cabal command for building and installing the documentation in DESTDIR?

[Mikolaj]

Is there a comparable cabal command for building and installing the documentation in DESTDIR?

Oh, I missed this. Does any of cabal haddock --help|grep -i dir sound plausible? E.g., does any the following work for you?

 --docdir=DIR                   installation directory for documentation
 --htmldir=DIR                  installation directory for HTML documentation
 --haddockdir=DIR               installation directory for haddock interfaces

[essandess]

Is there a comparable cabal command for building and installing the documentation in DESTDIR?

Oh, I missed this. Does any of cabal haddock --help|grep -i dir sound plausible? E.g., does any the following work for you?

 --docdir=DIR                   installation directory for documentation
 --htmldir=DIR                  installation directory for HTML documentation
 --haddockdir=DIR               installation directory for haddock interfaces

Sorry, none of these options get around the error:

No cabal.project file or cabal file matching the default glob './*.cabal' was found.

[Mikolaj]

My bad, I mixed up which variant we are currently discussing. cabal install cabal-install-v3.8.1.0, not in cabal source code directory, with proper options, may build haddocks without any more invocations, I think. And should not give this error.

The variant with cabal get, cabal build and cabal haddock runs in source directory and you should not need to specify output directories and you should not get the error above.

BTW, do you package just the executable or the library as well? If only executable, what are the haddocks for? Would not the user manual be a better option? If the library, cabal install won't help you at all.

[Mikolaj]

cabal install cabal-install-v3.8.1.0 ... may build haddocks without any more invocations

To be specific, it should build haddocks for the Cabal library, if that's what you are after, and many more. Executables don't have any substantial haddocks as a rule (no library API in their case).

[essandess]

Thanks. Yes, we would prefer just the binary and the user manual.

The command

cabal install cabal-install-3.8.1.0 --installdir=./bin --docdir=./doc --htmldir=./html …

creates the binary in ./bin, but not the user manual in either ./doc or ./html. How does one create the cabal-install-3.8.1.0 user manual?

[ulysses4ever]

@essandess as described in doc/README.md:

Building the documentation requires Python 3 be installed

• $ make users-guide
• Python on Mac OS X dislikes LC_CTYPE=UTF-8, unset the env var in terminal preferences and instead set LC_ALL=en_US.UTF-8 or something

[Mikolaj]

Let me think what kinds of users manuals we have. The main one seems to be https://cabal.readthedocs.io/en/3.8. A minor one is the automatically generated man page from cabal man. I suppose you can somehow convince man cabal to invoke cabal man, but I don't know how. The former manual at that address is compiled automatically by readthedocs.io, perhaps with some help from our CI. In addition our CI builds it locally somehow. The easiest instructions seem to be at https://github.com/haskell/cabal/blob/master/doc/README.md and, I'm afraid, you need source code from github, not an official source package from Hackage, to build it.

[gbaz]

is this an action item for cabal to fix or just at this point a user question?