[Feature] remote freeze files

[hasufell]

If you want to use the same freeze file for a plethora of internal projects, then a "global" freeze file might make sense. Currently this would require copying and synchronising such a file across repositories, or having a mono-repo. Being able to set a remote location as a freeze file in cabal.project would solve this use case.

Cli options could allow to say what to do in case fetching fails:

• abort resolution
• continue without a freeze file

[Kleidukos]

This is a great idea and will certainly help bringing cabal to feature parity with stack in enterprise environments.

[merijn]

@hvr spent considerable time to keep the critical path to produce a plan.json fast, so that a no-op cabal build and cabal run latency would be <100ms in order to keep unconditional cabal build and cabal run usable in interactive settings or when integrating with other build tools.

Such a remote fetch for a freeze file would be and almost certainly ruin this criticial path.

[cdsmith]

IMO, it's very reasonable to be concerned about re-fetching a remote freeze file every time you run cabal build, and this is a convincing reason not to do it that way.

There are other ways to do it:

1. A cache that would only re-fetch a freeze file occasionally. Taking an extra couple seconds once a day isn't anything like the same barrier. But the unpredictable timing of picking up changes would be unfortunate.
2. My favorite: Give names to packaged constraint sets, and fetch them as part of cabal update. This opens the door to future work mirroring them in case one source is down, migrating hosting or even hosting on Hackage, integrating with signing and other security measures, etc.

It's not like the latency concern is a dead end for the concept. It's just an argument against one of many possible implementations.

[hasufell]

Wrt 2.: Sounds like a controversial choice to me, since remote freeze files very likely would be per project (although there might be a global one too). cabal update so far has no project specific semantics afaik, so this would be a pretty drastic change in workflow.

As for the latency concerns: I do not share them.

1. This is an optional feature
2. Users who rely on it will care more about the feature than latency. It has zero impact on hackage and no impact on repos you use through source-repository-package. If you don't want it, it doesn't impact you.
3. There are a bunch of reasons cabal today is already pretty slow. Fetching of source-repository-package is one of them. I have never seen people complain about that.
4. Fetch strategies could be added to the configuration, to allow for immutable remote freeze files (uniquely identified by the name). Then you would only fetch once.

I believe that configure would be the correct place for this, since it also fetches remote repos at that stage.

[gbaz]

Recent discussions have convinced me this would be a great feature to have. I think we should give the current freeze workflows a once-over as a first pass to make sure they're sufficiently ergonomic, then just tweak them to allow http fetch with caching. There are at least three features to reduce repeated fetch latency -- cache-control, etag, and last-modified checks. Our current transport backends should have decent support for them, last I checked :-)

[cdsmith]

I believe that configure would be the correct place for this, since it also fetches remote repos at that stage.

I agree with this. I was thinking too complex and missing a simpler answer here. I think it needs to be possible to do cabal build and cabal run without internet access, but if network fetching can already happen during configure, then adding remote freeze files at the same time sounds good to me.

[Kleidukos]

I too agree with this.

[Mikolaj]

@ptkato: does v2-configure still fetch remote repos or did we focus it exclusively on translating build flags to caba.project syntax and overwriting project files?

Even if it still does, given that the use of cabal configure is deprecated in normal workflows and that fetching remote repos is a part of cabal build, wouldn't it be more natural to call cabal build --fetch-remote-only or something similar and include repos and freeze files in that? [Edit: the same as for cabal build applies to cabal install.]

I didn't read closely about your use cases in this issue, so pardon me if I don't make sense.

[hasufell]

Even if it still does, given that the use of cabal configure is deprecated in normal workflows and that fetching remote repos is a part of cabal build, wouldn't it be more natural to call cabal build --fetch-remote-only or something similar and include repos and freeze files in that?

I was under the impression that configure is an internal phase in cabal-install that is run by both cabal configure and cabal build, while the former seems to do additional work.

[Mikolaj]

I was under the impression that configure is an internal phase in cabal-install that is run by both cabal configure and cabal build, while the former seems to do additional work.

Yes, that was a misguided attempt at backward compatibilty with v1-configure, which was a mandatory first step in the standard v1- workflow and which was not duplicated in v1-install. Now the overlap of cabal v2-configure and cabal v2-build (and the same in the alternative cabal v2-install) is gone (in cabal 3.6, I think), which makes it easier to tell which action fits into which command and so I'm just trying to establish exactly that. :)

[hasufell]

Well in that case, the user definitely needs a way to:

1. pre-fetch everything before running cabal build, even if that's a completely different command
2. tell cabal build to not fetch anything ever (I believe --offline exists for that purpose)

[Mikolaj]

I think if cabal build consists of stage1 and stage2, it makes perfect sense to have cabal build --stage1-only and cabal build-stage2-only. The remaining work it delineating the stages sensibly (re-ordering some things, if needed), agreeing on self-explanatory names, porting the flags to cabal install and possibly elsewhere. I'm guessing this is quite tangential to your original issue and your needs and can be done in a separate bike-shedding ticket. However, if you can refine what precise stage1 would fit your use cases best, that would fit in here, certainly.

[fgaz]

pre-fetch everything before running cabal build, even if that's a completely different command

That'd be the new --only-download flag