flake: add ci.buildbot output

[MattSturgeon]

Background

I've long complained that buildbot is too tightly coupled to our flake's checks output; the same one used by nix flake check. Being unable to customise what buildbot builds has been a pain point for a while.

Turns out around seven months ago buildbot added a per-repo config file in nix-community/buildbot-nix#318.

The available options are documented on its readme, here. Currently only lock_file and attribute, but the latter enables this PR!

Core change

This PR adds an explicit ci.buildbot output to the flake, which we configure buildbot to use instead of the default checks output via a buildbot-nix.toml file.

This allows us to specify a different set of tests for nix flake check and buildbot.

Additional changes

This PR is not focused on fully taking advantage of the new separation, however it does make some minor tweaks:

• packages are now only built on x86_64-linux, but are built by nix flake check on all systems 🚀
  • Dropped for now due to feedback. Will address docs packages specifically in future work.
• Many tests imported by tests/default.nix are also only built on x86_64-linux.
  • "Platform" tests and "module" tests (generated by fetch-tests) are still built on all platforms.

Without this PR buildbot had 288 builds, with just tests it has 205, with tests and packages it has 214, with tests (mostly single system) and packages (all systems), it has 235.

Future work:

• Some builds may be better off done by a GitHub Actions workflow instead of buildbot. This will further reduce the load on buildbot while also improving the UX of how other tests get displayed in PR status checks.
  • The ci.buildbot option/output has been designed to accommodate ci.* sibling options, such as GitHub Actions job matrices.
• We can further iterate on deciding which tests should/shouldn't be built by CI and/or nix flake check
  • E.g., do we really need packages to be built by nix flake check?

Related:

• [RFC] Stripping down the test set (i.e. make the CI go brrrrr) #3238

[mergify]

This pull request, with head sha 6c456efc96f60213019460046d07f5691e0fafa3, has been successfully merged with fast-forward by Mergify.

This pull request will be automatically closed by GitHub.

As soon as GitHub detects that the sha 6c456efc96f60213019460046d07f5691e0fafa3 is part of the main branch, it will mark this pull request as merged.

It is possible for this pull request to remain open if this detection does not happen, this usually happens when a force-push is done on this branch ci/buildbot-output, this means GitHub will fail to detect the merge.

[nixvim-ci]

Successfully created backport PR for nixos-25.05:

• [Backport nixos-25.05] flake: add ci.buildbot output #3419

[nixvim-ci]

Backport failed for nixos-24.11, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin nixos-24.11
git worktree add -d .worktree/backport/nixos-24.11/3410 origin/nixos-24.11
cd .worktree/backport/nixos-24.11/3410
git switch --create backport/nixos-24.11/3410
git cherry-pick -x e18d3fb2804908fc00629e9139059d26a3ff4dcb 74e6ada9d1545d3a3c515ea3d85ad6766dc00df5 7d0ac00557dd97d4353cfcad8ffb65163ac1d782 6c456efc96f60213019460046d07f5691e0fafa3

[nixvim-ci]

Git push to origin failed for nixos-25.05 with exitcode 1

[nixvim-ci]

Backport failed for nixos-24.11, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin nixos-24.11
git worktree add -d .worktree/backport/nixos-24.11/3410 origin/nixos-24.11
cd .worktree/backport/nixos-24.11/3410
git switch --create backport/nixos-24.11/3410
git cherry-pick -x e18d3fb2804908fc00629e9139059d26a3ff4dcb 74e6ada9d1545d3a3c515ea3d85ad6766dc00df5 7d0ac00557dd97d4353cfcad8ffb65163ac1d782 6c456efc96f60213019460046d07f5691e0fafa3

[MattSturgeon]

Backport failed for nixos-24.11, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

Looked into it, there's too many differences between nixvim 24.11 and 25.05 to be worth backporting when we'll be dropping 24.11 support soon anyway.

[MattSturgeon]

Git push to origin failed for nixos-25.05 with exitcode 1

This seems to be caused by the backport workflow running twice. We should be using the same conditions as nixpkgs, but maybe something is wrong?