--offline flag doesn't seem to affect new-build

[chris-martin]

This is an excerpt from the output of running cabal new-build --offline:

Downloading th-abstraction-0.2.6.0...
Configuring ansi-terminal-0.8.0.4 (lib)...
Configuring cookie-0.4.4 (lib)...
Configuring bsb-http-chunked-0.0.0.2 (lib)...
Configuring blaze-builder-0.4.1.0 (lib)...
Downloading wai-extra-3.0.22.1...
Downloading scotty-0.11.1...
Building bsb-http-chunked-0.0.0.2 (lib)...
Building cookie-0.4.4 (lib)...
Building ansi-terminal-0.8.0.4 (lib)...

Either the --offline flag isn't working, or that is some misleading build output?

$ cabal --version
cabal-install version 2.2.0.0
compiled using version 2.2.0.1 of the Cabal library 

Here's another trial, this time with my network link disabled and ~/.cabal directory cleared:

$ cabal new-build --offline
Config file path source is default config file.
Config file /home/chris/.cabal/config not found.
Writing default configuration to /home/chris/.cabal/config
Warning: Caught exception during _mirrors lookup:res_query: does not exist
(Connection refused)
Warning: No mirrors found for http://hackage.haskell.org/
dieVerbatim: user error (cabal: Couldn't establish HTTP connection. Possible cause: HTTP proxy server
is down.

Again here it seems like it's trying to make HTTP requests despite the --offline flag which I thought was supposed to be an instruction to the contrary.

For context: My Nix shell environment should be providing all of the necessary dependencies, so I don't ever need or want cabal-install to download anything.

[23Skidoo]

Looks like an oversight.

[parsonsmatt]

This is affecting our workflow at the day job. Sometimes folks will update our .cabal file with a new dependency that isn't provided by nix. We'd like for the build to fail. Instead, it tries to download and build the missing dependency from Hackage. Passing --offline would fix this, but it doesn't appear to work.

[fgaz]

@parsonsmatt if you have cabal >= 3.4, you can use active-repositories:

-- "Offline" mode
active-repositories: none

Maybe --offline could just be sugar for that

[gbaz]

Furthermore, I note that there's a global --active-repositories flag (although it doesn't show up in cabal help, because we don't show all global flags by default, irritatingly). cabal --active-repositories=":none" configure will work. What's the right design choice here -- do we want to deprecate the --offline flag and teach people the existing, working flag? Do we want to preserve it as an alias? (Note that the --offline flag is not a global flag (although arguably that would make more sense these days) but instead a command specific flag, though shared across all v2 commands.

I'd like discussion to confirm first what I think -- i.e. that the --active-repositories=":none" flag gives all the semantics we want. If so, then it would be nice to figure out the cleanest ergonomic way of directing people to that. (Note also that the :none syntax instead of none is going to be surprising to people...)

[gbaz]

Ok, there's a pr in place to get the active repositories flag documented in --help (including the :none syntax).

So as a distinct discussion we should just figure out the best solution for the existing --offline flag -- I propose that we change it for now to just tell you to do the other thing instead. Supporting a bunch of different interacting aliases seems like a path to code spaghetti.

[fgaz]

I'd like discussion to confirm first what I think -- i.e. that the --active-repositories=":none" flag gives all the semantics we want

There might be problems with "file" repositories. Technically, those are "offline", but will still be deactivated by that flag I think

[parsonsmatt]

That should fix our use case, thanks!

I think it'd be nice to have --offline alias the --active-repositories work-around and emit a warning, something like "this is an alias for --active-repositores=":none", does not guarantee that it does not access the network - if you really cabal to not touch the network, comment on #5346 on github".

Having --offline error out feels bad, for a few reasons - right now, it doesn't do what it's supposed to do, but it's essentially a no-op. A command with --offline completes fine, even if it hits the network. And ideally --offline will eventually work. For a tool to fail, and suggest a work-around, feels strictly worse (from a UX perspective) than just doing the work-around for me.

If the intent is to remove the --offline feature entirely, then erroring seems like the way forward.

[chris-martin]

By the way, one really nice thing that the Nix tools now do is automatically detect whether you're online, so when you're on the airplane or whatnot it behaves sensibly and gives "you can't do this without the internet" error messages.

[arrowd]

Another problem is source repositories in the cabal.project files. These are checked out even with --offline flag.

[gbaz]

We definitely should find a better interaction for the source repository issue! And this motivates keeping the offline flag -- to both handle that situation, and also set active repositories to either none, or only file-based ones. 🤔

[cbclemmer]

I want to try looking into this

[cbclemmer]

I'm trying to write a test for this. Is there a way to force cabal to check a specific directory for a local package? That way I can write a script that deletes that directory every time and forces cabal to re download the package

[fgaz]

See "Hermetic tests" and possibly "How can I add a dependency on a package from Hackage in a test?" in cabal-testsuite

[cbclemmer]

@fgaz I can add a dependency but I assume that would pull from my ~/.cabal directory. I don't think there is a way to explicitly remove a package from there so I'd rather just start from a fresh directory.

[ulysses4ever]

@cbclemmer

Is there a way to force cabal to check a specific directory for a local package?

Local as in the current host, in the Cabal store? I’m asking because “local package” means something completely different in Cabal’s parlance. To answer that (according to my understanding of your question), there’s a --store-dir option, which comes close. Otherwise, you could use cabal.project file with source-repository-package, I think.

More generally, it’s not clear to me that your question necessarily needs to be solved to write a test for this. Could you perhaps lay out your idea for a test in more detail? That way you will have a better chance for helpful comments.

[cbclemmer]

@ulysses4ever I am trying to write a test that will probably go in the cabal-testsuite/PackageTests folder to simulate running a build for a project that will use the --offline option when building. If the build output has something like Downloading foo-bar-1.0.0... in it instead of throwing an error saying that it can't find the package locally then the test should fail.

[cbclemmer]

I assume that's what the flag is supposed to do

[cbclemmer]

I need to have control of the package directory because I need to remove the folder before it runs and I don't want it to be affected by my local packages in my ~/.cabal folder.

[ulysses4ever]

@cbclemmer okay, there are several conditions one may test here. For example, assuming --offline flag is on:

1. The dependency is in the cache -- we should succeed.
2. The dependency is not in the cache -- we should fail.

By default, the cache is empty, so it seems straightforward to test (2). How about we stop here and leave (1) for future work?

More concretely, how about adding an integration test with a package current that depends on a package p (p should be in the local repository, see documentation for the testsuite on how to do it). The test simply tries to build current with the --offline test and assertFails on that (step 1)? For more assurance, you can then try to build current without the flag and check that this succeeds (step 2). I think, this would test (2) quite thoroughly? As a bonus point (step 3), try to build current again with the flag on: it may be the case that p is in the cache now, and it will succeed. In that case, we are even able to test (1)!

[Mikolaj]

@cbclemmer: no pressure ;) but do you require assistance?

[cbclemmer]

@Mikolaj I was working on making a test for it and it took longer than I thought. Now I'm not doing much oss stuff until the holidays are over. Someone is free to take it if they want to but otherwise I'll get back to it in January

[Mikolaj]

@cbclemmer: no rush. I wish you that the holidays are merry for you. :)

[cbclemmer]

@ulysses4ever I've been work on this on and off and I got a test that should create a new store and retrieve from a local repo, but I can't get the output to say that it is "downloading". I tried to do it as you outlined before but that's what I end up with. Can you look at my test and see if I'm missing something? Here is the branch and the test folder is cabal-testsuite/PackageTests/I5346 The test passes because I'm not sure exactly what the output should be at the moment.

[ulysses4ever]

@cbclemmer

I can't get the output to say that it is "downloading"

That's the crux of the issue and the reason I proposed what I proposed above: afaik you can't get an explicit message that the cache is used or not used. My proposal above is supposed to work around it.

I looked at your test case: it looks pretty good. It's just I think you have to assertFail with --offline and then normally build the package without --offline, and that would be it.

Thank you for working on it!

[peterbecich]

I believe this is still an issue for source-repository-package: #9641