source-repository-package clone not disabled by --offline flag or active-repositories

[peterbecich]

Describe the bug
Similar to #5346

cabal.project source-repository-package is cloned from GitHub, regardless of the --offline flag or active-repositories: https://cabal.readthedocs.io/en/stable/cabal-project.html#cfg-field-active-repositories

To Reproduce
example project: https://github.com/peterbecich/cabal2nix-project-issue
This example clones the haskell/text repository as a source-repository-package.
The nix files are irrelevant to this issue.

• git clone https://github.com/peterbecich/cabal2nix-project-issue.git
• cabal build --enable-tests --offline

Cabal will clone the source-repository-package:

Cloning into '/home/peterbecich/nix/cabal2nix-project-issue/dist-newstyle/src/text-8e814f5854032da8'...
remote: Enumerating objects: 11518, done.
remote: Counting objects: 100% (11518/11518), done.
remote: Compressing objects: 100% (4352/4352), done.
.
.

I have tested

active-repositories: none

and

active-repositories: :none

To re-test,

rm -rf dist-newstyle

Expected behavior

The --offline flag and/or active-repositories work perfectly to block access to Hackage. Can the same be done for source-repository-package?

System information

• Operating system: Debian
• cabal 3.10.2.1
• ghc 9.6.4

Additional context

This may help to solve a different issue with Nix. Example Nix build from a different repository:

       > Writing default configuration to
       > /build/source-halogen-chess/.config/cabal/config
       > Error: cabal: The program 'git' is required but it could not be found.
       >

In this Nix build, Cabal is attempting to clone the source-repository-package but Nix is correctly not providing Git.
One way to solve this may be to have Nix invoke Cabal with the --offline flag.
I think that may go somewhere in make-package-set.nix: https://github.com/NixOS/nixpkgs/blob/46d10fdfe87d34e1641d627c31dbedd719024c91/pkgs/development/haskell-modules/make-package-set.nix#L4

[Mikolaj]

Just to be sure, it would make sense to test this with a cabal master branch nightly binary (or a local build from the master branch).

[michaelpj]

I wouldn't expect active-repositories to affect source-repository-package, but I do think --offline should. But I'm not sure what it should do - arguably it should just fail since it doesn't have the repository and we need it?

[peterbecich]

@Mikolaj , yes good idea, here is a test from a recent build of Cabal; same issue

$ ~/.cabal/bin/cabal build --offline    
Cloning into '/home/peterbecich/nix/cabal2nix-project-issue/dist-newstyle/src/text-8e814f5854032da8'...
remote: Enumerating objects: 11518, done.
remote: Counting objects: 100% (11518/11518), done.
remote: Compressing objects: 100% (4352/4352), done.
remote: Total 11518 (delta 6805), reused 11389 (delta 6741), pack-reused 0
Receiving objects: 100% (11518/11518), 2.93 MiB | 3.08 MiB/s, done.
Resolving deltas: 100% (6805/6805), done.
HEAD is now at 73620de Bump version to 2.1
....


$ ~/.cabal/bin/cabal --version      
cabal-install version 3.11.0.0
compiled using version 3.11.0.0 of the Cabal library 

[peterbecich]

not sure what it should do - arguably it should just fail since it doesn't have the repository and we need it?

i.m.o. if the source-repository-package is already on the disk, it should succeed; otherwise it should fail.

I'd prefer it works like building the example twice:

$ cabal build --enable-tests --offline
Build profile: -w ghc-9.6.4 -O1
In order, the following will be built (use -v for more details):
 - text-2.1 (lib) (requires build)
 - baz-0.1.0.0 (exe:bar) (first run)
Starting     text-2.1 (lib)
Building     text-2.1 (lib)

Ultimately what I'd like is for Nix to provide the source-package-repository and cabal --offline build to skip over the Git clone. Example:
purescript-bridge is provided by Nix: https://github.com/peterbecich/halogen-chess/blob/master/flake.nix#L15-L17
For the Nix build, Cabal should ignore the source-repository-package: https://github.com/peterbecich/halogen-chess/blob/master/cabal.project

[michaelpj]

More specifically, I think you mean:

• If --offline is provided, do not create or update the repository
• If the requested commit is not the in the repository, then fail

Ultimately what I'd like is for Nix to provide the source-package-repository and cabal --offline build to skip over the Git clone

We have this problem in haskell.nix too. We solve it in a very ugly way, we strip out the s-r-ps and replace them: https://github.com/input-output-hk/haskell.nix/blob/master/lib/call-cabal-project-to-nix.nix#L259

[peterbecich]

If the requested commit is not the in the repository, then fail

Referring to the remote repository?

If so, this error will occur in the Nix build:

       > Writing default configuration to
       > /build/source-halogen-chess/.config/cabal/config
       > Error: cabal: The program 'git' is required but it could not be found.
       >

I want to prevent Cabal in the Nix build from invoking Git. I guess this also applies to Git local repository on disk.

Thanks for the haskell.nix reference: https://github.com/input-output-hk/haskell.nix/blob/master/lib/call-cabal-project-to-nix.nix#L259
I think haskell-flake uses make-package-set.nix, but I have not identified exactly where Cabal is invoked by haskell-flake. haskell-flake can pass the offline flag to Cabal so I'm trying to solve it that way. I believe @srid has ruled out a haskell-flake bug (srid/haskell-flake#212 (comment)); I assume it is in some Nix library which haskell-flake uses.

[srid]

haskell-flake just uses the callCabal2nix function from nixpkgs, which then calls the cabal2nix program.

You should be able to reproduce this issue using callCabal2nix directly (by using the flake from this tutorial for example).

[peterbecich]

Does callCabal2nix support cabal.project? This issue says it does not: NixOS/cabal2nix#286
I am reading the source: https://github.com/NixOS/nixpkgs/blob/c8208f98aba6824fadcac393245eebb77dcfd7a0/pkgs/development/haskell-modules/make-package-set.nix#L225

The source-repository-package can only be specified in a cabal.project: https://cabal.readthedocs.io/en/3.4/cabal-project.html#specifying-packages-from-remote-version-control-locations
Is there some other way of defining a remote dependency in *.cabal without having to use cabal.project?

To reproduce the issue with the todo-app example (https://github.com/peterbecich/todo-app), I can provide the cabal.project and roll-my-own cabal.project reader which calls callCabal2nix.

The issue of Cabal ignoring the --offline flag for source-repository-package while building in Nix would be exposed that way.

---

However, isn't it sufficient to show the issue exists with just plain Cabal and cabal.project, without Nix?

---

I will try to implement this fix, starting here:

cabal/cabal-install/src/Distribution/Client/ProjectBuilding.hs

Line 355 in a703871

| fromFlagOrDefault False (projectConfigOfflineMode config) && not (null packagesToDownload) = return offlineError

[peterbecich]

source-package-repository might be enabled/disabled by the --offline flag here:

cabal/cabal-install/src/Distribution/Client/ProjectConfig.hs

Lines 1371 to 1379 in a703871

	-- For syncing we don't care about different 'SourceRepo' values that
	-- are just different subdirs in the same repo.
	syncSourceRepos
	verbosity
	vcs
	[ (repo, repoPath)
	| (repo, _, repoPath) <- repoGroupWithPaths
	]

TODO for me:

• use projectConfigOfflineMode there
• maybe print an error/warning like here

  cabal/cabal-install/src/Distribution/Client/ProjectBuilding.hs

  Lines 435 to 436 in a703871

  	offlineError :: BuildOutcomes
  	offlineError = Map.fromList . map makeBuildOutcome $ packagesToDownload

• the build has to proceed without error if the source-package-repository is available offline
  • in my use case, it will be provided by Nix

[peterbecich]

This makes the offline flag apply to source-repository-package, please review: #9771

I have not confirmed this will fix my original issue with Nix. I believe it will; Nix should provide the remote resource, Nix will apply the --offline flag, and the --offline flag will ensure Cabal does not attempt to pull the source-repository-package. There could be an issue if Nix does not place this resource where it is expected as a source-repository-package; that may be an issue in make-package-set.nix.