build dependencies for custom Setup.hs

[kowey]

Projects often require custom Setup.hs files which in turn may rely on Haskell packages. Those packages may in turn depend on Cabal, perhaps without being too fussy about which version of Cabal they want.

It might be useful to be able to express the idea that the Setup script itself has dependencies. Rogan Creswick suggested maybe something that looks like

   CustomBuild
        Setup-is: Setup.hs
        Build-Depends:  ...
        Build-tools:    ...
        ...

Here was my musing about it: https://plus.google.com/114991347543804898741/posts/2Wpy3WWvGyH (not sure if I understood correctly, and my user claims that upping the cabal-version constraint [which should be unnecessary in that case] fixes it for them)

This appears to have come up for several projects:

• GUI programmers on Mac might use a cabal-macosx package to help create application bundles for their executables
• Pandoc's Setup.hs at one point needed something from a new version of the process package
• Rogan Creswick filed this GSoC suggestion http://hackage.haskell.org/trac/summer-of-code/ticket/1602
• Rogan also commented something about cabal-dev (which I didn't fully process, sorry, Rogan!)

To ensure that these packages are present, I sometimes abuse the build-depends for my executable, which appears to be insufficient because when a Haskell-naive user upgrades from one version of Cabal to another, you get into a kind of messy situation where the Setup.hs file wants to use one version of Cabal but it relies on the package which assumes another version of Cabal. It's the kind of thing that a Haskeller can just sort of muddle/bulldoze through, but for somebody who knows enough to type cabal install type commands but isn't otherwise a Haskeller, it can be seriously confusing.

[creswick]

Cabal-dev's sandbox abstraction breaks a little because we can't specify explicit Setup.hs dependencies.

Specifically, when cabal builds the Setup.hs to generate a build tool that compiles the rest of the project, it uses the user and global package databases, without acknowledging any package-db path settings/flags (which is how cabal-dev controls the sandbox). Thus, when building setup.hs, cabal is unaware of the cabal-dev sandbox. Presumably, a fix to this would also enable the Setup.hs compilation step to use a custom package database. (I could see that being a separate issue, too.)

One of the common hacks to get around the absence of setup.hs dependencies is to put those deps. in the standard build-depends for your library/executable. Cabal-install will install them first, then build setup.hs. This often works if you are not using a sandbox, because those deps go into either the user or global package db. However, this does not work with cabal-dev (it may work with virthualenv, I haven't tried).

This also introduces other potential issues, as the dependencies for Setup.hs really have no relationship to the dependencies for your actual program/library, so spurious conflicts can incorrectly cause build failures, although that's somewhat of a separate bug... (each build artifact sortof needs it's own sandbox, at least morally.)

I encourage anyone reading this to look at the summer of code ticket Erik linked too, as it outlines other benefits of explicit Setup.hs dependencies (such as sharing complex extensions to Cabal via hackage).

[dcoutts]

So I'm starting work on this.

I've been pondering the question of whether we should go for a "maximal" or "minimal" approach. By maximal I mean we treat the Setup a like other executables in the package, allowing all or almost all the options/fields allowed for executables. The minimal approach would be to allow the minimum necessary set of fields.

While maximal seems attractive at first, here's this issue: building the Setup.hs is the first thing any build/packaging system needs to do. Remember it is not just cabal-install that builds Setup.hs, it's also the scripts for distros and other custom build/package systems.

At the moment the Cabal spec makes clear that the primary interface to the package build system is the Setup.hs and that it's expected to be able to just be run directly. This proposed feature changes that: it means that the .cabal file must be consulted first. Tools will have to look at the .cabal file and see what build instructions are needed to build the Setup.hs and then do that.

While I think it's ok for the Setup.hs interface to become primarily a machine interface rather than a user interface (we've been moving in this direction for some time), we do still need to keep in mind that there are multiple implementations that need to build and then invoke the Setup.hs.

So that suggests we should go for a minimal approach. What is the minimum we can get away with / what is the most we can reasonably expect bash scripts invoking Setup.hs to do?

I propose:

• build-depends (ie Haskell packages)
• CPP macros to let the Setup.hs do conditional compilation based on the versions of packages its being built against (e.g. Cabal lib version)

and that's it. The build-depends is obviously the minimum we could consider, that's the main point. The CPP one is perhaps a bit of a burden on build/package tools (we'll see if we can minimise that by making it as simple as possible) but it's also another big pain point for custom Setup.hs.

The "Simple" build system already provides some quite nice package version macro support, creating a .h file like:

#define MIN_VERSION_filepath(major1,major2,minor) (\
  (major1) <  1 || \
  (major1) == 1 && (major2) <  3 || \
  (major1) == 1 && (major2) == 3 && (minor) <= 0)

I think this is probably a bit much to expect every tool to do. So I'm thinking of something simpler like:

ghc Setup.hs -cpp -DVERSION_filepath=010300

[creswick]

@dcoutts fantastic!

I'm ok with the minimalistic approach. I think it's still generally a bad idea to have a really complex Setup.hs--that complexity should be moved into "real" packages, which you can specify in the build-depends.

The maximal approach does seem ideal, and I'd like to see that eventually, but I don't think it's strictly necessary. (The biggest real drawback I see to the minimal approach is that people may expect to have the same expressiveness in a setup build stanza as in an executable or libraries stanza, but that's more an annoyance than a real problem, and it may not actually be an issue at all in practice.)

Thanks!

[dagit]

@dcoutts Thanks for taking a stab at this!

[23Skidoo]

@dcoutts Awesome!

[ekmett]

I now have a gl package that tries to smuggle a fair bit of work into the Setup.hs (it generates ~740 modules from scratch) that needs a couple of dependencies set up that really could benefit from having proper 'Setup' dependencies as well.

[cies]

Does this mean we can also require, for instance, alex and happy binaries to be available?

And how about requiring other binaries and or libraries, for instance, postgres development headers; could we find a way to test for them as well "a la the traditional configure script"?

Please excuse my ignorance, I'm rather new to Haskell, but I was just wondering about these things and did not find the answers for them by reading this thread and google'ing.

[bennofs]

@cies You can already specify library dependencies for your package in the project.cabal file (there are pkgconfig-depends and extra-libraries fields). Build tools are currently not possible afaik.

This issue, however, is about dependencies for a special file, Setup.hs, which can be used to configure the build process in haskell (for example, to add additional checks to the configure phase). It's not about dependencies for the rest of your haskell code, only about dependencies of Setup.hs.

[23Skidoo]

See #2515.

[enolan]

So, since #2515 is merged, can we close this?

[23Skidoo]

I guess we can.