clean up bootstrap JSON generation by geekosaur · Pull Request #11637 · haskell/cabal

geekosaur · 2026-03-19T01:59:30Z

The primary entrypoint is now make bootstrap-jsons, which is now documented and called out in bootstrap/README.md. It automatically tries to obtain ghc versions from first ghcup, falls back to Nix if nix-shell is found, and if all else fails warns the user to install the appropriate ghcs. The Nix script can still be used directly, but now obtains the list of ghc versions from the Makefile.

Manual QA:

make bootstrap-jsons with ghcup in $PATH
make bootstrap-jsons with no ghcup (renaming it is fine) but with Nix (this may take several hours due to the existing Nix setup using ghc.nix instead of nixpkgs)
extra credit: make bootstrap-jsons with neither, verifying it warns that you need to install each veriioned GHC manually

Template B: This PR does not modify behaviour or interface

E.g. the PR only touches documentation or tests, does refactorings, etc.

Include the following checklist in your PR:

Patches conform to the coding conventions.
~~Is this a PR that fixes CI? If so, it will need to be backported to older cabal release branches (ask maintainers for directions).~~

geekosaur · 2026-03-19T02:00:27Z

I tested both the ghcup and Nix paths locally.

philderbeast · 2026-03-19T15:47:21Z

Manual QA:

make bootstrap-jsons with ghcup in $PATH

I gave the manual QA a run, with ghc-9.14.1:

$ git rev-parse HEAD
53cdf3e85efb65b5a98a9aa932b4765179cfaac3

$ which ghc
/home/.../.ghcup/bin/ghc

$ ghc --numeric-version
9.14.1

$ make bootstrap-jsons
ghcup install ghc "9.2.8"
[ Warn  ] [GHCup-00110] ghc-9.2.8 is already installed; if you really want to reinstall it, you may want to run 'ghcup install ghc --force 9.2.8'
make "bootstrap-json-do-9.2.8"
make[1]: Entering directory '/home/.../cabal'
rm -rf dist-bootstrap
cabal build --distdir=dist-bootstrap --project-file=cabal.bootstrap.project --with-compiler=ghc-9.2.8 --dry-run \
	cabal-install:exe:cabal
Warning: this is a debug build of cabal-install with assertions enabled.
Resolving dependencies...
Build profile: -w ghc-9.2.8 -O1
In order, the following would be built (use -v for more details):
 - Cabal-syntax-3.17.0.0 (lib) (first run)
 - process-1.6.26.0 (lib) (requires build)
 - random-1.3.1 (lib) (requires build)
 - open-browser-0.4.0.0 (lib) (requires build)
 - hsc2hs-0.68.10 (exe:hsc2hs) (requires build)
 - echo-0.1.4 (lib) (requires build)
 - Cabal-3.17.0.0 (lib) (first run)
 - edit-distance-0.2.2.1 (lib) (requires build)
 - zlib-0.7.1.0 (lib) (requires build)
 - network-3.2.7.0 (lib) (requires build)
 - cabal-install-solver-3.17.0.0 (lib) (first run)
 - hackage-security-0.6.3.1 (lib) (requires build)
 - HTTP-4000.4.1 (lib) (requires build)
 - cabal-install-3.17.0.0 (lib) (first run)
 - cabal-install-3.17.0.0 (exe:cabal) (first run)
cp dist-bootstrap/cache/plan.json bootstrap/linux-9.2.8.plan.json
cd bootstrap && CABAL_CONFIG=/home/.../.config/cabal/config cabal run -v0 cabal-bootstrap-gen -- linux-9.2.8.plan.json \
	| python3 -m json.tool > linux-9.2.8.json
Warning: this is a debug build of cabal-install with assertions enabled.
Error:
    Dependency on unbuildable library from ghc-bignum
    In the stanza 'library'
    In the package 'base-4.19.2.0'

Expecting value: line 1 column 1 (char 0)
make[1]: *** [Makefile:292: bootstrap-json-do-9.2.8] Error 1
make[1]: Leaving directory '/home/.../cabal'
make: *** [Makefile:283: bootstrap-json-9.2.8] Error 2

Then again with ghc-9.12.2, one of the BOOTSTRAP_GHC_VERSIONS := 9.2.8 9.4.8 9.6.7 9.8.4 9.10.2 9.12.2:

$ ghcup set ghc 9.12.2
[ Info  ] GHC 9.12.2 successfully set as default version

$ make bootstrap-jsons
ghcup install ghc "9.2.8"
[ Warn  ] [GHCup-00110] ghc-9.2.8 is already installed; if you really want to reinstall it, you may want to run 'ghcup install ghc --force 9.2.8'
make "bootstrap-json-do-9.2.8"
make[1]: Entering directory '/home/.../cabal'
rm -rf dist-bootstrap
cabal build --distdir=dist-bootstrap --project-file=cabal.bootstrap.project --with-compiler=ghc-9.2.8 --dry-run \
	cabal-install:exe:cabal
Warning: this is a debug build of cabal-install with assertions enabled.
Resolving dependencies...
Build profile: -w ghc-9.2.8 -O1
In order, the following would be built (use -v for more details):
 - Cabal-syntax-3.17.0.0 (lib) (first run)
 - process-1.6.26.0 (lib) (requires build)
 - random-1.3.1 (lib) (requires build)
 - open-browser-0.4.0.0 (lib) (requires build)
 - hsc2hs-0.68.10 (exe:hsc2hs) (requires build)
 - echo-0.1.4 (lib) (requires build)
 - Cabal-3.17.0.0 (lib) (first run)
 - edit-distance-0.2.2.1 (lib) (requires build)
 - zlib-0.7.1.0 (lib) (requires build)
 - network-3.2.7.0 (lib) (requires build)
 - cabal-install-solver-3.17.0.0 (lib) (first run)
 - hackage-security-0.6.3.1 (lib) (requires build)
 - HTTP-4000.4.1 (lib) (requires build)
 - cabal-install-3.17.0.0 (lib) (first run)
 - cabal-install-3.17.0.0 (exe:cabal) (first run)
cp dist-bootstrap/cache/plan.json bootstrap/linux-9.2.8.plan.json
cd bootstrap && CABAL_CONFIG=/home/.../.config/cabal/config cabal run -v0 cabal-bootstrap-gen -- linux-9.2.8.plan.json \
	| python3 -m json.tool > linux-9.2.8.json
Warning: this is a debug build of cabal-install with assertions enabled.
Error: [Cabal-7107]
Could not resolve dependencies:
[__0] trying: cabal-bootstrap-gen-0 (user goal)
[__1] next goal: base (dependency of cabal-bootstrap-gen)
[__1] rejecting: base-4.21.0.0/installed-ae91 (conflict: cabal-bootstrap-gen => base^>=4.12.0.0 || ^>=4.13.0.0 || ^>=4.14.0.0 || ^>=4.15.0.0 || ^>=4.16.0.0 || ^>=4.17.0.0 || ^>=4.18.0.0 || ^>=4.19.0.0 || ^>=4.20.0.0)
[__1] skipping: base; 4.22.0.0, 4.21.1.0, 4.21.0.0 (has the same characteristics that caused the previous version to fail: excluded by constraint '^>=4.12.0.0 || ^>=4.13.0.0 || ^>=4.14.0.0 || ^>=4.15.0.0 || ^>=4.16.0.0 || ^>=4.17.0.0 || ^>=4.18.0.0 || ^>=4.19.0.0 || ^>=4.20.0.0' from 'cabal-bootstrap-gen')
[__1] rejecting: base; 4.20.2.0, 4.20.1.0, 4.20.0.1, 4.20.0.0, 4.19.2.0, 4.19.1.0, 4.19.0.0, 4.18.3.0, 4.18.2.1, 4.18.2.0, 4.18.1.0, 4.18.0.0, 4.17.2.1, 4.17.2.0, 4.17.1.0, 4.17.0.0, 4.16.4.0, 4.16.3.0, 4.16.2.0, 4.16.1.0, 4.16.0.0, 4.15.1.0, 4.15.0.0, 4.14.3.0, 4.14.2.0, 4.14.1.0, 4.14.0.0, 4.13.0.0, 4.12.0.0, 4.11.1.0, 4.11.0.0, 4.10.1.0, 4.10.0.0, 4.9.1.0, 4.9.0.0, 4.8.2.0, 4.8.1.0, 4.8.0.0, 4.7.0.2, 4.7.0.1, 4.7.0.0, 4.6.0.1, 4.6.0.0, 4.5.1.0, 4.5.0.0, 4.4.1.0, 4.4.0.0, 4.3.1.0, 4.3.0.0, 4.2.0.2, 4.2.0.1, 4.2.0.0, 4.1.0.0, 4.0.0.0, 3.0.3.2, 3.0.3.1 (constraint from non-reinstallable package requires installed instance)
[__1] fail (backjumping, conflict set: base, cabal-bootstrap-gen)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: base, cabal-bootstrap-gen
Expecting value: line 1 column 1 (char 0)
make[1]: *** [Makefile:292: bootstrap-json-do-9.2.8] Error 1
make[1]: Leaving directory '/home/.../cabal'
make: *** [Makefile:283: bootstrap-json-9.2.8] Error 2

I added some diagnostics, if that helps?

$ git diff
diff --git a/Makefile b/Makefile
index cd5d0d20d..74473dfed 100644
--- a/Makefile
+++ b/Makefile
@@ -270,6 +270,9 @@ BOOT_NIX := $(shell nix-shell --version 2>/dev/null)
 CABALCONF := $(shell if test -f $$HOME/.config/cabal/config; then echo CABAL_CONFIG=$$HOME/.config/cabal/config; fi)
 
 bootstrap-json-%: phony
+       $(info INFO: ghcup --version is '$(BOOT_GHCUP)')
+       $(info INFO: nix-shell --version is '$(BOOT_NIX)')
+       $(info INFO: cabal config is '$(CABALCONF)')

$ make bootstrap-jsons
INFO: ghcup --version is 'The GHCup Haskell installer, version 0.1.50.2'
INFO: nix-shell --version is 'nix-shell (Nix) 2.25.3'
INFO: cabal config is 'CABAL_CONFIG=/home/.../.config/cabal/config'
...

philderbeast · 2026-03-19T15:50:45Z

Here's what I have installed:

$ ghcup list --tool ghc --show-criteria installed
   Tool Version Tags                      Notes                 
✓  ghc  8.2.2   base-4.10.1.0                                   
✓  ghc  8.10.7  base-4.14.3.0                                   
✓  ghc  9.2.8   base-4.16.4.0                                   
✓  ghc  9.4.8   base-4.17.2.1                                   
✓  ghc  9.6.7   recommended,base-4.18.3.0                       
✓  ghc  9.8.4   base-4.19.2.0             2024-11-27            
✓  ghc  9.10.3  base-4.20.1.0             hls-powered           
✔✔ ghc  9.12.2  base-4.21.0.0             hls-powered,2025-03-12
✓  ghc  9.14.1  latest,base-4.22.0.0      hls-powered,2025-12-18

$ ghcup list --tool cabal --show-criteria installed
   Tool  Version  Tags   Notes
✔✔ cabal 3.16.1.0 latest

geekosaur · 2026-03-19T15:59:24Z

The second one is just nobody ever bumping the permitted base versions on cabal-bootstrap-gen. I recently had to bump it for 9.10 when I most recently regenerated the JSONs, which is one of the things that set me itching on this topic. (Note that, being a tool, it's built with your default ghc, not the one being bootstrapped.) Maybe I'll add something about it to the release list, even though it's not strictly release-related.

Temporarily setting your default ghc to 9.10 or earlier will get around it.

geekosaur · 2026-03-19T16:02:02Z

General question for cabal devs: maybe the base dependency on cabal-bootstrap-gen should simply be < 5? I don't think anyone is expecting fundamentals to break on us.

philderbeast · 2026-03-19T16:16:15Z

Temporarily setting your default ghc to 9.10 or earlier will get around it.

Yes that works.

geekosaur · 2026-03-19T16:31:55Z

Another general question: various PRs might bump revisions in cabal files, but we only normally regenerate the JSONs before releases. Should PRs perhaps regenerate them as part of CI? And does this suggest that bootstrapping happens so rarely that it's all pretty much a waste of time?

Bodigrim · 2026-03-19T20:55:17Z

General question for cabal devs: maybe the base dependency on cabal-bootstrap-gen should simply be < 5? I don't think anyone is expecting fundamentals to break on us.

Yeah, I think it would reduce the friction.

Another general question: various PRs might bump revisions in cabal files, but we only normally regenerate the JSONs before releases. Should PRs perhaps regenerate them as part of CI? And does this suggest that bootstrapping happens so rarely that it's all pretty much a waste of time?

Bootstrapping is used to build cabal-install on a new platform / distro, but I can hardly imagine anyone doing it with an unreleased version. So it would be a redundant friction to regenerate bootstrap plans on every PR.

geekosaur · 2026-03-20T01:52:14Z

Bootstrapping is used to build cabal-install on a new platform / distro, but I can hardly imagine anyone doing it with an unreleased version.

Thing is, we only support bootstrapping from master. IIRC we cut a few corners with the generated cabal (although maybe I misunderstood what it was doing; I didn't look that closely, just noticed some slightly shady stuff going on), and the README.md tells you you should only use it to build a full released cabal and then use that.

geekosaur · 2026-03-20T05:17:22Z

Anyway I think this is ready to merge, if someone would like to QA and review.

Bodigrim · 2026-03-20T19:17:17Z

Something went wrong with manual testing on my machine:

$ make bootstrap-jsons
ghcup install ghc "9.2.8"
[ Info  ] downloading: https://raw.githubusercontent.com/haskell/ghcup-metadata/master/ghcup-0.0.9.yaml as file /Users/andrew/.ghcup/cache/ghcup-0.0.9.yaml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  591k  100  591k    0     0  2361k      0 --:--:-- --:--:-- --:--:-- 2356k
[ Info  ] downloading: https://raw.githubusercontent.com/haskell/ghcup-metadata/refs/heads/master/ghcup-prereleases-0.0.9.yaml as file /Users/andrew/.ghcup/cache/ghcup-prereleases-0.0.9.yaml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  324k  100  324k    0     0  1318k      0 --:--:-- --:--:-- --:--:-- 1320k
[ Warn  ] New stack version available. If you want to install this latest version, run 'ghcup install stack 3.9.3'
[ Warn  ] [GHCup-00110] ghc-9.2.8 is already installed; if you really want to reinstall it, you may want to run 'ghcup install ghc --force 9.2.8'
/Library/Developer/CommandLineTools/usr/bin/make "bootstrap-json-do-9.2.8"
ghcup install ghc "do-9.2.8"
[ Warn  ] New stack version available. If you want to install this latest version, run 'ghcup install stack 3.9.3'
[ Error ] [GHCup-00010] Unable to find a download for GHC version 'do-9.2.8' on detected platform aarch64-darwin
[ Error ] Also check the logs in /Users/andrew/.ghcup/logs
make[1]: *** [bootstrap-json-do-9.2.8] Error 3
make: *** [bootstrap-json-9.2.8] Error 2

geekosaur · 2026-03-20T19:24:01Z

Argh. I know what went wrong and will have to do some renaming in the Makefile to fix it. Figures. (I do wonder how it passed here, though.)

geekosaur · 2026-03-20T19:32:08Z

Kinda wish I could disable the warnings and such, but apparently the current behavior of ghcup is already the "non-verbose" one.

geekosaur · 2026-03-20T22:09:25Z

And this is demonstrating why I'm trying to move magic numbers into a config.yml (part of validate-actions): I forgot to update bootstrap.yml when I switched 9.10 minor releases.

philderbeast · 2026-03-25T18:13:46Z

Nice. Running the make subroutine by itself is really fast:

$ make bootstrap-subr-9.12.2
rm -rf dist-bootstrap
cabal build --distdir=dist-bootstrap --project-file=cabal.bootstrap.project --with-compiler=ghc-9.12.2 --dry-run \
        cabal-install:exe:cabal
Warning: this is a debug build of cabal-install with assertions enabled.
Resolving dependencies...
Build profile: -w ghc-9.12.2 -O1
In order, the following would be built (use -v for more details):
 - Cabal-syntax-3.17.0.0 (lib) (first run)
 - edit-distance-0.2.2.1 (lib) (requires build)
 - process-1.6.26.0 (lib) (requires build)
 - open-browser-0.4.0.0 (lib) (requires build)
 - hsc2hs-0.68.10 (exe:hsc2hs) (requires build)
 - echo-0.1.4 (lib) (requires build)
 - Cabal-3.17.0.0 (lib) (first run)
 - zlib-0.7.1.0 (lib) (requires build)
 - network-3.2.7.0 (lib) (requires build)
 - cabal-install-solver-3.17.0.0 (lib) (first run)
 - hackage-security-0.6.3.1 (lib) (requires build)
 - HTTP-4000.4.1 (lib) (requires build)
 - cabal-install-3.17.0.0 (lib) (first run)
 - cabal-install-3.17.0.0 (exe:cabal) (first run)
cp dist-bootstrap/cache/plan.json bootstrap/linux-9.12.2.plan.json
cd bootstrap && CABAL_CONFIG=/home/.../.config/cabal/config cabal run -v0 cabal-bootstrap-gen -- linux-9.12.2.plan.json \
        | python3 -m json.tool > linux-9.12.2.json
Warning: this is a debug build of cabal-install with assertions enabled.

Do we need both of these?

bootstrap/linux-9.6.6.json
bootstrap/linux-9.6.7.json

Deleting bootstrap/*.json only to show those we track:

philderbeast · 2026-03-25T18:16:59Z

Sorry, fumbled my keyboard. Didn't mean to close this.

geekosaur · 2026-03-25T18:34:11Z

No, that was a fumble and I thought I'd pushed removal of the old one. I'll do that shortly.

geekosaur · 2026-03-25T19:05:40Z

Hm. Considering that this confusion has come up multiple times (most recently @Bodigrim, but it also came up a couple months ago), should I rename generate_bootstrap_plans to generate_nix_bootstrap_plans?

Bodigrim · 2026-03-25T19:10:31Z

Hm. Considering that this confusion has come up multiple times (most recently @Bodigrim, but it also came up a couple months ago), should I rename generate_bootstrap_plans to generate_nix_bootstrap_plans?

Yes please.

geekosaur · 2026-03-25T19:32:25Z

Okay, that (and comments in the script, and information about where people can get ghc if they don't have either ghcup or Nix) will be in the next push.

geekosaur · 2026-03-25T21:25:40Z

I think we just got nailed by a new tasty release deprecating things we use, with -Werror=deprecations.

Bodigrim · 2026-03-25T21:32:02Z

Ouch, sorry :) You can just replace sequentialTestGroup with dependentTestGroup, there are no semantical changes.

The primary entrypoint is now `make bootstrap-jsons`, which is now documented and called out in `bootstrap/README.md`. It automatically tries to obtain ghc versions from first `ghcup`, falls back to Nix if `nix-shell` is found, and if all else fails warns the user to install the appropriate ghcs. The Nix script can still be used directly, but now obtains the list of ghc versions from the `Makefile`.

mergify · 2026-03-28T16:22:48Z

geekosaur added the attention: needs-review label Mar 19, 2026

Bodigrim linked an issue Mar 19, 2026 that may be closed by this pull request

Rewrite generate_bootstrap_plan, so that it does not require Nix #11464

Closed

geekosaur force-pushed the bootstrap-json-gen-cleanup branch 2 times, most recently from 89ec9cb to 28b0307 Compare March 20, 2026 00:52

geekosaur force-pushed the bootstrap-json-gen-cleanup branch 2 times, most recently from 18aab6b to 254f428 Compare March 20, 2026 19:31

philderbeast reviewed Mar 20, 2026

View reviewed changes

Comment thread Makefile

Bodigrim reviewed Mar 20, 2026

View reviewed changes

Comment thread bootstrap/cabal-bootstrap-gen.cabal Outdated

Bodigrim reviewed Mar 20, 2026

View reviewed changes

Comment thread Makefile Outdated

geekosaur force-pushed the bootstrap-json-gen-cleanup branch 3 times, most recently from 2ec47e5 to a4b9f6c Compare March 20, 2026 22:07

Bodigrim approved these changes Mar 20, 2026

View reviewed changes

geekosaur force-pushed the bootstrap-json-gen-cleanup branch from a4b9f6c to bfc0f7f Compare March 20, 2026 23:51

philderbeast closed this Mar 25, 2026

philderbeast reopened this Mar 25, 2026

philderbeast approved these changes Mar 25, 2026

View reviewed changes

geekosaur force-pushed the bootstrap-json-gen-cleanup branch 2 times, most recently from 941842e to fcd4104 Compare March 25, 2026 18:53

geekosaur removed the attention: needs-review label Mar 25, 2026

geekosaur force-pushed the bootstrap-json-gen-cleanup branch from fcd4104 to 0d777e8 Compare March 25, 2026 19:41

geekosaur force-pushed the bootstrap-json-gen-cleanup branch from 0d777e8 to 4d15200 Compare March 26, 2026 00:01

geekosaur added the merge me Tell Mergify Bot to merge label Mar 26, 2026

mergify Bot added ready and waiting Mergify is waiting out the cooldown period merge delay passed Applied (usually by Mergify) when PR approved and received no updates for 2 days queued labels Mar 26, 2026

mergify Bot added a commit that referenced this pull request Mar 28, 2026

Merge of #11637

e3c5820

mergify Bot mentioned this pull request Mar 28, 2026

merge queue: embarking master (df5520c), #11634 and #11637 together #11672

Closed

86 tasks

mergify Bot merged commit f309a44 into haskell:master Mar 28, 2026
207 of 231 checks passed

mergify Bot removed the queued label Mar 28, 2026

philderbeast mentioned this pull request May 29, 2026

tasty -Werror=deprecations of sequentialTestGroup #11901

Closed

Uh oh!

Conversation

geekosaur commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

geekosaur commented Mar 19, 2026

Uh oh!

philderbeast commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

philderbeast commented Mar 19, 2026

Uh oh!

geekosaur commented Mar 19, 2026

Uh oh!

geekosaur commented Mar 19, 2026

Uh oh!

philderbeast commented Mar 19, 2026

Uh oh!

geekosaur commented Mar 19, 2026

Uh oh!

Bodigrim commented Mar 19, 2026

Uh oh!

geekosaur commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

geekosaur commented Mar 20, 2026

Uh oh!

Bodigrim commented Mar 20, 2026

Uh oh!

geekosaur commented Mar 20, 2026

Uh oh!

geekosaur commented Mar 20, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

geekosaur commented Mar 20, 2026

Uh oh!

philderbeast commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

philderbeast commented Mar 25, 2026

Uh oh!

geekosaur commented Mar 25, 2026

Uh oh!

geekosaur commented Mar 25, 2026

Uh oh!

Bodigrim commented Mar 25, 2026

Uh oh!

geekosaur commented Mar 25, 2026

Uh oh!

geekosaur commented Mar 25, 2026

Uh oh!

Bodigrim commented Mar 25, 2026

Uh oh!

mergify Bot commented Mar 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Queue Status

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

geekosaur commented Mar 19, 2026 •

edited

Loading

philderbeast commented Mar 19, 2026 •

edited

Loading

geekosaur commented Mar 20, 2026 •

edited

Loading

philderbeast commented Mar 25, 2026 •

edited

Loading

mergify Bot commented Mar 28, 2026 •

edited

Loading