Missing dependencies for adjunctions

[cocreature]

When I try to build a toyproject I'm getting the following error on nix-shell --command 'sh $setup'. I run cabbage -a before. I also tried deleting .cabbages, cabal.config, cabal clean and cabal sandbox delete but sadly with no effect. Generating a shell.nix using cabal2nix --shell . > shell.nix and then running nix-shell --command 'cabal build' works just fine. Sadly I don't really know what I am doing and this bash script is scaring me so I'm a bit lost. I'd be happy to provide more information if needed.

building path(s) ‘/nix/store/y1v7r6176rqc3f024gn217rs4kiy0ndg-haskell-adjunctions-4.2’
Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
Resolving dependencies...
Configuring adjunctions-4.2...
cabal: At least the following dependencies are missing:
comonad ==4.*,
contravariant ==1.*,
distributive >=0.4 && <1,
free ==4.*,
mtl >=2.0.1 && <2.3,
profunctors ==4.*,
semigroupoids ==4.*,
semigroups >=0.11 && <1,
tagged >=0.7 && <1,
void >=0.5.5.1 && <1
builder for ‘/nix/store/wiww14fmb7c8q328d6701kdjfs5b1jdd-haskell-adjunctions-4.2.drv’ failed with exit code 1
error: build of ‘/nix/store/wiww14fmb7c8q328d6701kdjfs5b1jdd-haskell-adjunctions-4.2.drv’ failed
/home/moritz/.nix-profile/bin/nix-shell: failed to build all dependencies

Not quite sure which files are relevant so here is a selection :)
cabal.config

constraints: ConfigFile ==1.1.4,
             HUnit ==1.2.5.2,
             MissingH ==1.3.0.1,
             PSQueue ==1.1,
             QuickCheck ==2.7.6,
             adjunctions ==4.2,
             ansi-wl-pprint ==0.6.7.1,
             asn1-encoding ==0.9.0,
             asn1-parse ==0.9.0,
             asn1-types ==0.3.0,
             base16-bytestring ==0.1.1.6,
             base64-bytestring ==1.0.0.1,
             byteable ==0.1.1,
             case-insensitive ==1.2.0.4,
             cereal ==0.4.1.1,
             cipher-aes ==0.2.10,
             cipher-des ==0.0.6,
             cipher-rc4 ==0.1.4,
             clock ==0.4.1.3,
             connection ==0.2.4,
             cookie ==0.4.1.4,
             crypto-cipher-types ==0.0.9,
             crypto-numbers ==0.2.7,
             crypto-pubkey ==0.2.8,
             crypto-pubkey-types ==0.4.3,
             crypto-random ==0.0.8,
             cryptohash ==0.11.6,
             doctest ==0.9.13,
             errors ==1.4.7,
             hourglass ==0.2.8,
             hslogger ==1.2.8,
             hspec ==2.1.4,
             hspec-core ==2.1.4,
             hspec-discover ==2.1.4,
             hspec-expectations ==0.6.1.1,
             http-client ==0.4.8.1,
             http-client-tls ==0.2.2,
             http-types ==0.8.6,
             iCalendar ==0.4.0.2,
             kan-extensions ==4.2.1,
             lens ==4.8,
             lens-aeson ==1.0.0.3,
             logict ==0.6.0.2,
             mime ==0.4.0.1,
             mime-types ==0.1.0.5,
             network ==2.6.0.2,
             network-uri ==2.6.0.1,
             optparse-applicative ==0.11.0.2,
             parallel ==3.2.0.6,
             pem ==0.2.2,
             publicsuffixlist ==0.1,
             quickcheck-io ==0.1.1,
             reflection ==1.5.1.2,
             regex-base ==0.93.2,
             regex-compat ==0.95.1,
             regex-posix ==0.95.2,
             regex-tdfa ==1.2.0,
             regex-tdfa-rc ==1.1.8.3,
             securemem ==0.1.7,
             setenv ==0.1.1.3,
             smallcheck ==1.1.1,
             socks ==0.5.4,
             streaming-commons ==0.1.10.0,
             tasty ==0.10.1,
             tasty-hspec ==1.1,
             tasty-quickcheck ==0.8.3.2,
             tasty-smallcheck ==0.8.0.1,
             tf-random ==0.5,
             tls ==1.2.16,
             unbounded-delays ==0.1.0.9,
             wreq ==0.3.0.1,
             x509 ==1.5.0.1,
             x509-store ==1.5.0,
             x509-system ==1.5.0,
             x509-validation ==1.5.1,
             zlib ==0.5.4.2

default.nix

{ stdenv, lib, haskellBuildTools, pkgs, callPackage }:
let cabalTmp = "cabal --config-file=./.cabal/config";
    frozenCabbages = rec {
      ConfigFile = callPackage .cabbages/ConfigFile-1.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      HUnit = callPackage .cabbages/HUnit-1.2.5.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      MissingH = callPackage .cabbages/MissingH-1.3.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      PSQueue = callPackage .cabbages/PSQueue-1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      QuickCheck = callPackage .cabbages/QuickCheck-2.7.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      adjunctions = callPackage .cabbages/adjunctions-4.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      ansi-wl-pprint = callPackage .cabbages/ansi-wl-pprint-0.6.7.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      asn1-encoding = callPackage .cabbages/asn1-encoding-0.9.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      asn1-parse = callPackage .cabbages/asn1-parse-0.9.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      asn1-types = callPackage .cabbages/asn1-types-0.3.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      base16-bytestring = callPackage .cabbages/base16-bytestring-0.1.1.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      base64-bytestring = callPackage .cabbages/base64-bytestring-1.0.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      byteable = callPackage .cabbages/byteable-0.1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      case-insensitive = callPackage .cabbages/case-insensitive-1.2.0.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cereal = callPackage .cabbages/cereal-0.4.1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cipher-aes = callPackage .cabbages/cipher-aes-0.2.10 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cipher-des = callPackage .cabbages/cipher-des-0.0.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cipher-rc4 = callPackage .cabbages/cipher-rc4-0.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      clock = callPackage .cabbages/clock-0.4.1.3 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      connection = callPackage .cabbages/connection-0.2.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cookie = callPackage .cabbages/cookie-0.4.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      crypto-cipher-types = callPackage .cabbages/crypto-cipher-types-0.0.9 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      crypto-numbers = callPackage .cabbages/crypto-numbers-0.2.7 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      crypto-pubkey = callPackage .cabbages/crypto-pubkey-0.2.8 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      crypto-pubkey-types = callPackage .cabbages/crypto-pubkey-types-0.4.3 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      crypto-random = callPackage .cabbages/crypto-random-0.0.8 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cryptohash = callPackage .cabbages/cryptohash-0.11.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      doctest = callPackage .cabbages/doctest-0.9.13 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      errors = callPackage .cabbages/errors-1.4.7 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hourglass = callPackage .cabbages/hourglass-0.2.8 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hslogger = callPackage .cabbages/hslogger-1.2.8 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hspec = callPackage .cabbages/hspec-2.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hspec-core = callPackage .cabbages/hspec-core-2.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hspec-discover = callPackage .cabbages/hspec-discover-2.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hspec-expectations = callPackage .cabbages/hspec-expectations-0.6.1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      http-client = callPackage .cabbages/http-client-0.4.8.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      http-client-tls = callPackage .cabbages/http-client-tls-0.2.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      http-types = callPackage .cabbages/http-types-0.8.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      iCalendar = callPackage .cabbages/iCalendar-0.4.0.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      kan-extensions = callPackage .cabbages/kan-extensions-4.2.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      lens = callPackage .cabbages/lens-4.8 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      lens-aeson = callPackage .cabbages/lens-aeson-1.0.0.3 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      logict = callPackage .cabbages/logict-0.6.0.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      mime = callPackage .cabbages/mime-0.4.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      mime-types = callPackage .cabbages/mime-types-0.1.0.5 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      network = callPackage .cabbages/network-2.6.0.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      network-uri = callPackage .cabbages/network-uri-2.6.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      optparse-applicative = callPackage .cabbages/optparse-applicative-0.11.0.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      parallel = callPackage .cabbages/parallel-3.2.0.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      pem = callPackage .cabbages/pem-0.2.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      publicsuffixlist = callPackage .cabbages/publicsuffixlist-0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      quickcheck-io = callPackage .cabbages/quickcheck-io-0.1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      reflection = callPackage .cabbages/reflection-1.5.1.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      regex-base = callPackage .cabbages/regex-base-0.93.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      regex-compat = callPackage .cabbages/regex-compat-0.95.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      regex-posix = callPackage .cabbages/regex-posix-0.95.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      regex-tdfa = callPackage .cabbages/regex-tdfa-1.2.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      regex-tdfa-rc = callPackage .cabbages/regex-tdfa-rc-1.1.8.3 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      securemem = callPackage .cabbages/securemem-0.1.7 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      setenv = callPackage .cabbages/setenv-0.1.1.3 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      smallcheck = callPackage .cabbages/smallcheck-1.1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      socks = callPackage .cabbages/socks-0.5.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      streaming-commons = callPackage .cabbages/streaming-commons-0.1.10.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tasty = callPackage .cabbages/tasty-0.10.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tasty-hspec = callPackage .cabbages/tasty-hspec-1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tasty-quickcheck = callPackage .cabbages/tasty-quickcheck-0.8.3.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tasty-smallcheck = callPackage .cabbages/tasty-smallcheck-0.8.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tf-random = callPackage .cabbages/tf-random-0.5 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tls = callPackage .cabbages/tls-1.2.16 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      unbounded-delays = callPackage .cabbages/unbounded-delays-0.1.0.9 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      wreq = callPackage .cabbages/wreq-0.3.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      x509 = callPackage .cabbages/x509-1.5.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      x509-store = callPackage .cabbages/x509-store-1.5.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      x509-system = callPackage .cabbages/x509-system-1.5.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      x509-validation = callPackage .cabbages/x509-validation-1.5.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      zlib = callPackage .cabbages/zlib-0.5.4.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
    };
    mkCmd = pkg: let nm = lib.strings.removePrefix "haskell-" pkg.name;
                     p = pkg.outPath;
                     pkgPath = ".cabal-sandbox/x86_64-linux-ghc-7.8.4-packages.conf.d";
                 in ''ln -sFf ${p}/${pkgPath}/*.conf $out/${pkgPath}/
                    '';
    mkSetupCmd = pkg: let nm = lib.strings.removePrefix "haskell-" pkg.name;
                          p = pkg.outPath;
                      in "ln -sFf ${pkg.outPath}/.cabal-sandbox/x86_64-linux-ghc-7.8.4-packages.conf.d/*.conf /home/moritz/code/haskell/trccbot-hs/.cabal-sandbox/x86_64-linux-ghc-7.8.4-packages.conf.d/\n";
in
stdenv.mkDerivation rec {
  name = "haskell-trccbot-hs-0.1.0.0";
  src = ./dist/trccbot-hs-0.1.0.0.tar.gz;
  cabbageDeps = with frozenCabbages; [ ConfigFile HUnit MissingH PSQueue QuickCheck adjunctions ansi-wl-pprint asn1-encoding asn1-parse asn1-types base16-bytestring base64-bytestring byteable case-insensitive cereal cipher-aes cipher-des cipher-rc4 clock connection cookie crypto-cipher-types crypto-numbers crypto-pubkey crypto-pubkey-types crypto-random cryptohash doctest errors hourglass hslogger hspec hspec-core hspec-discover hspec-expectations http-client http-client-tls http-types iCalendar kan-extensions lens lens-aeson logict mime mime-types network network-uri optparse-applicative parallel pem publicsuffixlist quickcheck-io reflection regex-base regex-compat regex-posix regex-tdfa regex-tdfa-rc securemem setenv smallcheck socks streaming-commons tasty tasty-hspec tasty-quickcheck tasty-smallcheck tf-random tls unbounded-delays wreq x509 x509-store x509-system x509-validation zlib ];
  systemDeps = (with pkgs; [  ]) ++
               lib.lists.unique (lib.concatMap (lib.attrByPath ["systemDeps"] []) cabbageDeps);
  propagatedBuildInputs = systemDeps;
  buildInputs = [ stdenv.cc ] ++ haskellBuildTools ++ cabbageDeps ++ systemDeps;

  # Build the commands to merge package databases
  cmds = lib.strings.concatStrings (map mkCmd cabbageDeps);
  setupCmds = lib.strings.concatStrings (map mkSetupCmd cabbageDeps);
  setup = builtins.toFile "setup.sh" ''

    # Takes a GHC platform string, an array of add-source dependency
    # directories, and a string of old timestamps. Produces a new
    # timestamp string.
    updateTimeStamps() {
      local -a DEPS=("''${!2}")
      local CUR_TIME=$(date +%s)
      local i
      local STAMPED
      for ((i = 0; i < "''${#DEPS[@]}"; ++i)); do
        STAMPED[$i]="(\"''${DEPS[$i]}\",$CUR_TIME)"
      done
      local LIST=$(printf ",%s" "''${STAMPED[@]}")
      LIST=''${LIST:1}
      local NEWSTAMP="(\"$1\",[$LIST])"
      if echo "$3" | grep -q "$1"; then
        echo "$3" | sed "s:(\"$1\",[^]]*\]):$NEWSTAMP:"
      elif echo "$3" | grep -q "]\\$"; then
        echo "$3" | sed "s:\]\$:,$NEWSTAMP]:"
      else
        echo "[$NEWSTAMP]"
      fi
    }
    eval "$setupCmds"
    ${cabalTmp} sandbox hc-pkg recache
    SRCS=($(cabal sandbox list-sources | sed '1,/^$/ d' | sed '/^$/,$ d'))
    OLDTIMESTAMPS=$(cat .cabal-sandbox/add-source-timestamps)
    updateTimeStamps "x86_64-linux-ghc-7.8.4" SRCS[@] "$OLDTIMESTAMPS" > .cabal-sandbox/add-source-timestamps
  '';

  builder = builtins.toFile "builder.sh" ''
    source $stdenv/setup
    mkdir $out

    if [ -d "$src" ]; then
      cp -R "$src"/* .
      #*/
      if [ -f $src/cabal.config.bak ]; then
        cp "$src/cabal.config.bak" $out
      fi
    else
      tar xf "$src" --strip=1
    fi

    chmod -R u+w .
    ${cabalTmp} sandbox --sandbox=$out/.cabal-sandbox init -v0
    mkdir -p $out/.cabal-sandbox/lib/x86_64-linux-ghc-7.8.4
    eval "$cmds"
    ${cabalTmp} sandbox hc-pkg recache

    ${cabalTmp} --builddir=$out/dist --bindir=$out/bin --libdir=$out/.cabal-sandbox/lib --with-gcc=$CC configure $(echo $NIX_LDFLAGS | awk -e '{ for(i=1;i <= NF; i++) { if(match($(i), /^-L/)) printf("--extra-lib-dirs=%s ", substr($(i),3)); } }')
    echo "Building..."
    ${cabalTmp} --builddir=$out/dist build -v0
    ${cabalTmp} --builddir=$out/dist copy
    ${cabalTmp} --builddir=$out/dist register
    ${cabalTmp} --builddir=$out/dist clean || true
  '';    
  meta = {
    description = "";
  };
}

[acowley]

So something seems to have gone wrong in that the cabal.config isn't completely there. Your deletion technique seems commendably cautious, but something didn't take as the cabal.config is incomplete: It should mention all the transitive dependencies of what you're building, yet comonad, for example, isn't there. I need to bake some kind of reset command into cabbage as deleting things is too verbose; what I do is rm -r .cabbages shell.nix cabal.config && cabal sandbox delete.

With a fresh checkout of your repository, running cabbage -a then nix-shell --command 'sh $setup' did work for me on darwin with the following cabbage.config file added to the project directory.

flags:
  transformers-compat: three

Another thing that needs to be automated is special handling of transformers-compat: if your cabal.config doesn't mention transformers, then it's using the one in the global package database, which, with GHC 7.8, is version 3, thus necessitating that flag on transformers-compat. The good news is, your tests are all passing! :)

The main thing to understand about what cabbage is trying to do is that it freezes the downstream package (the thing you're trying to build), then intersects that build plan with frozen build plans of every upstream package you depend on. This results in a build plan for each upstream package that is consistent with the one needed for the downstream package, but restricted to the transitive dependencies of the upstream package. This build plan is part of the hash for the upstream package, so we can pull it out of the Nix store if we've built it before.

When things go wrong, it is either on the Nix side, or the Cabal side. If it's on the Cabal side, you can look at the cabal.config files for each package to see if they make sense. If it's on the Nix side, there are two parts:

1. The default.nix file in your downstream package defines attributes for every package that will be used. These are versioned packages.
2. The default.nix file for every upstream package has a cabbageDeps attribute that lists unversioned package names for every transitive dependency .The versions come from the downstream build plan, as supplied by the Nix expression for the downstream package.

The bash is just madness to make all that happen while only using cabal-install to drive the ship.

[cocreature]

First of all, thank you for your detailed explanation.
Sadly, I'm still too stupid to get it to working. I removed everything again according to your instructions.
This is what ls -la shows

total 76
drwxr-xr-x  6 moritz moritz 4096 Mar 14 19:21 .
drwxr-xr-x 31 moritz moritz 4096 Mar 14 10:23 ..
-rw-r--r--  1 moritz moritz   35 Mar 14 18:20 cabbage.config
-rw-r--r--  1 moritz moritz  101 Sep 28 13:01 config
-rw-r--r--  1 moritz moritz   94 Sep 28 12:36 config~
drwxr-xr-x  2 moritz moritz 4096 Nov  2 13:12 data
-rw-r--r--  1 moritz moritz  357 Mar  7 09:13 Dockerfile
-rw-r--r--  1 moritz moritz  447 Nov  2 12:52 Dockerfile~
drwxr-xr-x  8 moritz moritz 4096 Mar 14 19:21 .git
-rw-r--r--  1 moritz moritz  152 Mar 14 11:31 .gitignore
-rw-r--r--  1 moritz moritz   61 Oct  8 07:46 .gitignore~
-rw-r--r--  1 moritz moritz 1057 Sep 23 16:02 LICENSE
-rw-r--r--  1 moritz moritz  188 Oct  8 08:02 README.md
-rw-r--r--  1 moritz moritz  187 Oct  8 08:02 README.md~
drwxr-xr-x  2 moritz moritz 4096 Mar 13 12:07 src
drwxr-xr-x  3 moritz moritz 4096 Mar 13 11:37 test
-rw-r--r--  1 moritz moritz  283 Oct  8 08:37 .travis.yml
-rw-r--r--  1 moritz moritz 1980 Mar 13 11:38 trccbot-hs.cabal
-rw-r--r--  1 moritz moritz 1526 Sep 30 23:47 trccbot-hs.cabal~

the cabbage.config file looks like this

flags:
  transformers-compat: three
systemDeps:
  zlib: zlib

Interestingly I still get the message that zlib might require a system package

Writing a default package environment file to
/home/moritz/code/haskell/trccbot-hs/cabal.sandbox.config
Creating a new sandbox at /home/moritz/code/haskell/trccbot-hs/.cabal-sandbox
Resolving dependencies...
Froze downstream package at /home/moritz/code/haskell/trccbot-hs
Getting dependency: ConfigFile-1.1.4
Unpacking to .cabbages/ConfigFile-1.1.4/
Getting dependency: HUnit-1.2.5.2
Unpacking to .cabbages/HUnit-1.2.5.2/
Getting dependency: MissingH-1.3.0.1
Unpacking to .cabbages/MissingH-1.3.0.1/
Getting dependency: PSQueue-1.1
Unpacking to .cabbages/PSQueue-1.1/
Getting dependency: QuickCheck-2.7.6
Unpacking to .cabbages/QuickCheck-2.7.6/
Getting dependency: adjunctions-4.2
Unpacking to .cabbages/adjunctions-4.2/
Getting dependency: ansi-wl-pprint-0.6.7.1
Unpacking to .cabbages/ansi-wl-pprint-0.6.7.1/
Getting dependency: asn1-encoding-0.9.0
Unpacking to .cabbages/asn1-encoding-0.9.0/
Getting dependency: asn1-parse-0.9.0
Unpacking to .cabbages/asn1-parse-0.9.0/
Getting dependency: asn1-types-0.3.0
Unpacking to .cabbages/asn1-types-0.3.0/
Getting dependency: base16-bytestring-0.1.1.6
Unpacking to .cabbages/base16-bytestring-0.1.1.6/
Getting dependency: base64-bytestring-1.0.0.1
Unpacking to .cabbages/base64-bytestring-1.0.0.1/
Getting dependency: byteable-0.1.1
Unpacking to .cabbages/byteable-0.1.1/
Getting dependency: case-insensitive-1.2.0.4
Unpacking to .cabbages/case-insensitive-1.2.0.4/
Getting dependency: cereal-0.4.1.1
Unpacking to .cabbages/cereal-0.4.1.1/
Getting dependency: cipher-aes-0.2.10
Unpacking to .cabbages/cipher-aes-0.2.10/
Getting dependency: cipher-des-0.0.6
Unpacking to .cabbages/cipher-des-0.0.6/
Getting dependency: cipher-rc4-0.1.4
Unpacking to .cabbages/cipher-rc4-0.1.4/
Getting dependency: clock-0.4.1.3
Unpacking to .cabbages/clock-0.4.1.3/
Getting dependency: connection-0.2.4
Unpacking to .cabbages/connection-0.2.4/
Getting dependency: cookie-0.4.1.4
Unpacking to .cabbages/cookie-0.4.1.4/
Getting dependency: crypto-cipher-types-0.0.9
Unpacking to .cabbages/crypto-cipher-types-0.0.9/
Getting dependency: crypto-numbers-0.2.7
Unpacking to .cabbages/crypto-numbers-0.2.7/
Getting dependency: crypto-pubkey-0.2.8
Unpacking to .cabbages/crypto-pubkey-0.2.8/
Getting dependency: crypto-pubkey-types-0.4.3
Unpacking to .cabbages/crypto-pubkey-types-0.4.3/
Getting dependency: crypto-random-0.0.8
Unpacking to .cabbages/crypto-random-0.0.8/
Getting dependency: cryptohash-0.11.6
Unpacking to .cabbages/cryptohash-0.11.6/
Getting dependency: doctest-0.9.13
Unpacking to .cabbages/doctest-0.9.13/
Getting dependency: errors-1.4.7
Unpacking to .cabbages/errors-1.4.7/
Getting dependency: hourglass-0.2.8
Unpacking to .cabbages/hourglass-0.2.8/
Getting dependency: hslogger-1.2.8
Unpacking to .cabbages/hslogger-1.2.8/
Getting dependency: hspec-2.1.4
Unpacking to .cabbages/hspec-2.1.4/
Getting dependency: hspec-core-2.1.4
Unpacking to .cabbages/hspec-core-2.1.4/
Getting dependency: hspec-discover-2.1.4
Unpacking to .cabbages/hspec-discover-2.1.4/
Getting dependency: hspec-expectations-0.6.1.1
Unpacking to .cabbages/hspec-expectations-0.6.1.1/
Getting dependency: http-client-0.4.8.1
Unpacking to .cabbages/http-client-0.4.8.1/
Getting dependency: http-client-tls-0.2.2
Unpacking to .cabbages/http-client-tls-0.2.2/
Getting dependency: http-types-0.8.6
Unpacking to .cabbages/http-types-0.8.6/
Getting dependency: iCalendar-0.4.0.2
Unpacking to .cabbages/iCalendar-0.4.0.2/
Getting dependency: kan-extensions-4.2.1
Unpacking to .cabbages/kan-extensions-4.2.1/
Getting dependency: lens-4.8
Unpacking to .cabbages/lens-4.8/
Getting dependency: lens-aeson-1.0.0.3
Unpacking to .cabbages/lens-aeson-1.0.0.3/
Getting dependency: logict-0.6.0.2
Unpacking to .cabbages/logict-0.6.0.2/
Getting dependency: mime-0.4.0.1
Unpacking to .cabbages/mime-0.4.0.1/
Getting dependency: mime-types-0.1.0.5
Unpacking to .cabbages/mime-types-0.1.0.5/
Getting dependency: network-2.6.0.2
Unpacking to .cabbages/network-2.6.0.2/
Getting dependency: network-uri-2.6.0.1
Unpacking to .cabbages/network-uri-2.6.0.1/
Getting dependency: optparse-applicative-0.11.0.2
Unpacking to .cabbages/optparse-applicative-0.11.0.2/
Getting dependency: parallel-3.2.0.6
Unpacking to .cabbages/parallel-3.2.0.6/
Getting dependency: pem-0.2.2
Unpacking to .cabbages/pem-0.2.2/
Getting dependency: publicsuffixlist-0.1
Unpacking to .cabbages/publicsuffixlist-0.1/
Getting dependency: quickcheck-io-0.1.1
Unpacking to .cabbages/quickcheck-io-0.1.1/
Getting dependency: reflection-1.5.1.2
Unpacking to .cabbages/reflection-1.5.1.2/
Getting dependency: regex-base-0.93.2
Unpacking to .cabbages/regex-base-0.93.2/
Getting dependency: regex-compat-0.95.1
Unpacking to .cabbages/regex-compat-0.95.1/
Getting dependency: regex-posix-0.95.2
Unpacking to .cabbages/regex-posix-0.95.2/
Getting dependency: regex-tdfa-1.2.0
Unpacking to .cabbages/regex-tdfa-1.2.0/
Getting dependency: regex-tdfa-rc-1.1.8.3
Unpacking to .cabbages/regex-tdfa-rc-1.1.8.3/
Getting dependency: securemem-0.1.7
Unpacking to .cabbages/securemem-0.1.7/
Getting dependency: setenv-0.1.1.3
Unpacking to .cabbages/setenv-0.1.1.3/
Getting dependency: smallcheck-1.1.1
Unpacking to .cabbages/smallcheck-1.1.1/
Getting dependency: socks-0.5.4
Unpacking to .cabbages/socks-0.5.4/
Getting dependency: streaming-commons-0.1.10.0
Unpacking to .cabbages/streaming-commons-0.1.10.0/
Getting dependency: tasty-0.10.1
Unpacking to .cabbages/tasty-0.10.1/
Getting dependency: tasty-hspec-1.1
Unpacking to .cabbages/tasty-hspec-1.1/
Getting dependency: tasty-quickcheck-0.8.3.2
Unpacking to .cabbages/tasty-quickcheck-0.8.3.2/
Getting dependency: tasty-smallcheck-0.8.0.1
Unpacking to .cabbages/tasty-smallcheck-0.8.0.1/
Getting dependency: tf-random-0.5
Unpacking to .cabbages/tf-random-0.5/
Getting dependency: tls-1.2.16
Unpacking to .cabbages/tls-1.2.16/
Getting dependency: unbounded-delays-0.1.0.9
Unpacking to .cabbages/unbounded-delays-0.1.0.9/
Getting dependency: wreq-0.3.0.1
Unpacking to .cabbages/wreq-0.3.0.1/
Getting dependency: x509-1.5.0.1
Unpacking to .cabbages/x509-1.5.0.1/
Getting dependency: x509-store-1.5.0
Unpacking to .cabbages/x509-store-1.5.0/
Getting dependency: x509-system-1.5.0
Unpacking to .cabbages/x509-system-1.5.0/
Getting dependency: x509-validation-1.5.1
Unpacking to .cabbages/x509-validation-1.5.1/
Getting dependency: zlib-0.5.4.2
Unpacking to .cabbages/zlib-0.5.4.2/

You may need to supply system dependencies!

See the cabbage documentation for how to do this with a 'systemDeps'
section in a cabbage.config file.

Potentially necessary extra-libraries: z
Press any key to continue...
Making cabbage: ConfigFile-1.1.4
Making cabbage: HUnit-1.2.5.2
Making cabbage: MissingH-1.3.0.1
Making cabbage: PSQueue-1.1
Making cabbage: QuickCheck-2.7.6
Making cabbage: adjunctions-4.2
Making cabbage: ansi-wl-pprint-0.6.7.1
Making cabbage: asn1-encoding-0.9.0
Making cabbage: asn1-parse-0.9.0
Making cabbage: asn1-types-0.3.0
Making cabbage: base16-bytestring-0.1.1.6
Making cabbage: base64-bytestring-1.0.0.1
Making cabbage: byteable-0.1.1
Making cabbage: case-insensitive-1.2.0.4
Making cabbage: cereal-0.4.1.1
Making cabbage: cipher-aes-0.2.10
Making cabbage: cipher-des-0.0.6
Making cabbage: cipher-rc4-0.1.4
Making cabbage: clock-0.4.1.3
Making cabbage: connection-0.2.4
Making cabbage: cookie-0.4.1.4
Making cabbage: crypto-cipher-types-0.0.9
Making cabbage: crypto-numbers-0.2.7
Making cabbage: crypto-pubkey-0.2.8
Making cabbage: crypto-pubkey-types-0.4.3
Making cabbage: crypto-random-0.0.8
Making cabbage: cryptohash-0.11.6
Making cabbage: doctest-0.9.13
Making cabbage: errors-1.4.7
Making cabbage: hourglass-0.2.8
Making cabbage: hslogger-1.2.8
Making cabbage: hspec-2.1.4
Making cabbage: hspec-core-2.1.4
Making cabbage: hspec-discover-2.1.4
Making cabbage: hspec-expectations-0.6.1.1
Making cabbage: http-client-0.4.8.1
Making cabbage: http-client-tls-0.2.2
Making cabbage: http-types-0.8.6
Making cabbage: iCalendar-0.4.0.2
Making cabbage: kan-extensions-4.2.1
Making cabbage: lens-4.8
Making cabbage: lens-aeson-1.0.0.3
Making cabbage: logict-0.6.0.2
Making cabbage: mime-0.4.0.1
Making cabbage: mime-types-0.1.0.5
Making cabbage: network-2.6.0.2
Making cabbage: network-uri-2.6.0.1
Making cabbage: optparse-applicative-0.11.0.2
Making cabbage: parallel-3.2.0.6
Making cabbage: pem-0.2.2
Making cabbage: publicsuffixlist-0.1
Making cabbage: quickcheck-io-0.1.1
Making cabbage: reflection-1.5.1.2
Making cabbage: regex-base-0.93.2
Making cabbage: regex-compat-0.95.1
Making cabbage: regex-posix-0.95.2
Making cabbage: regex-tdfa-1.2.0
Making cabbage: regex-tdfa-rc-1.1.8.3
Making cabbage: securemem-0.1.7
Making cabbage: setenv-0.1.1.3
Making cabbage: smallcheck-1.1.1
Making cabbage: socks-0.5.4
Making cabbage: streaming-commons-0.1.10.0
Making cabbage: tasty-0.10.1
Making cabbage: tasty-hspec-1.1
Making cabbage: tasty-quickcheck-0.8.3.2
Making cabbage: tasty-smallcheck-0.8.0.1
Making cabbage: tf-random-0.5
Making cabbage: tls-1.2.16
Making cabbage: unbounded-delays-0.1.0.9
Making cabbage: wreq-0.3.0.1
Making cabbage: x509-1.5.0.1
Making cabbage: x509-store-1.5.0
Making cabbage: x509-system-1.5.0
Making cabbage: x509-validation-1.5.1
Making cabbage: zlib-0.5.4.2

The output of nix-shell --command 'sh $setup' after that is exactly the same. Also I should add that I am not on darwin but on x86_64 Archlinux and I'm using nixpkgs from git master.

[acowley]

Hm, we need to figure out why some dependencies aren't getting included there. Could they be globally installed? It's surprising that comonad isn't showing up. You can look for it with ghc-pkg list --global. Or maybe it's in the user package db, though I would have thought that wouldn't interfere.

[cocreature]

Ah of course, I have quite a lot in my global database as I am using ghcWithPackages for binaries (e.g. ghc-mod). I guess I should either remove all of them (which I don't really want to do) or not use cabbage? This is the complete list

/nix/store/a0hpww6gbpdhm28hsk7v05iasqdv9l2l-ghc-7.8.4/lib/ghc-7.8.4/package.conf.d:
    Cabal-1.18.1.5
    MonadRandom-0.3.0.1
    aeson-0.8.0.2
    ansi-terminal-0.6.2.1
    applicative-quoters-0.1.0.8
    array-0.5.0.0
    async-2.0.2
    attoparsec-0.12.1.3
    base-4.7.0.2
    bifunctors-4.2.1
    bin-package-db-0.0.0.0
    binary-0.7.1.0
    blaze-builder-0.3.3.4
    rts-1.0
    bytestring-0.10.4.0
    cmdargs-0.10.12
    comonad-4.2.3
    containers-0.5.5.1
    contravariant-1.2.2.1
    cpphs-1.18.9
    data-default-0.5.3
    data-default-class-0.0.1
    data-default-instances-base-0.0.1
    data-default-instances-containers-0.0.1
    data-default-instances-dlist-0.0.1
    data-default-instances-old-locale-0.0.1
    deepseq-1.3.0.2
    descriptive-0.9.3
    directory-1.2.1.0
    distributive-0.4.4
    djinn-ghc-0.0.2.3
    djinn-lib-0.0.1.2
    dlist-0.7.1
    either-4.3.3.2
    exceptions-0.8.0.1
    extra-1.1
    filepath-1.3.0.2
    foreign-var-0.1
    free-4.11
    (ghc-7.8.4)
    ghc-mod-0
    ghc-paths-0.1.0.9
    ghc-prim-0.3.1.0
    ghc-syb-utils-0.2.3
    hashable-1.2.3.1
    haskeline-0.7.1.2
    haskell-src-exts-1.16.0.1
    haskell-src-meta-0.6.0.8
    (haskell2010-1.1.2.0)
    (haskell98-2.0.0.3)
    hasktags-0.69.1
    hindent-4.4.0
    hlint-1.9.17
    hoopl-3.10.0.1
    hpc-0.6.0.1
    hscolour-1.21
    integer-gmp-0.5.1.0
    io-choice-0.0.5
    json-0.9
    lifted-base-0.2.3.6
    monad-control-1.0.0.4
    monad-journal-0.7
    monad-loops-0.4.2.1
    mtl-2.1.3.1
    nats-1
    old-locale-1.0.0.6
    old-time-1.1.0.2
    parsec-3.1.8
    polyparse-1.11
    prelude-extras-0.4
    pretty-1.1.1.1
    primitive-0.5.4.0
    process-1.2.0.0
    profunctors-4.4.1
    random-1.1
    safe-0.3.8
    scientific-0.3.3.7
    semigroupoids-4.3
    semigroups-0.16.2.2
    split-0.2.2
    stm-2.4.4
    syb-0.4.4
    tagged-0.7.3
    template-haskell-2.9.0.0
    temporary-1.2.0.3
    terminfo-0.4.0.0
    text-1.2.0.4
    th-expand-syns-0.3.0.5
    th-lift-0.7.2
    th-orphans-0.11.0
    th-reify-many-0.1.3
    time-1.4.2
    transformers-0.3.0.0
    transformers-base-0.4.4
    transformers-compat-0.4.0.4
    uniplate-1.6.12
    unix-2.7.0.1
    unordered-containers-0.2.5.1
    utf8-string-0.3.8
    vector-0.10.12.2
    void-0.7
    xhtml-3000.2.1

[acowley]

(Skip to the last paragraph if you don't want the background.)

Right, so that is the problem. First off, you can absolutely use executables like ghc-mod, I certainly do. With regard to your setup, maybe this could be made to work, but it's not easy. A tricky issue that cabbage has to work around is that Cabal and GHC don't do a good job at package management. Specifically, the dependencies any package is built against make up part of its identity; name and version number are not enough. Even worse, GHC can be a bit funny about global packages in that hiding them is buggy (this may be improved in 7.10), so cabbage works around them.

What cabbage does, is if a globally installed package is being updated, then it instead uses so-called "cabbage patched" versions of globally-installed packages to ensure that the full dependency chain is consistent. This will be visible in the build plan as versions of packages with a 4552 suffix on their version numbers. If you need a newer version of unix than came with GHC, for example, a cabbage patched version of directory will appear in the build plan and be used for the build.

If, however, all dependencies available in the global DB are the right versions, then they will be used. So what's happening, it seems, is that those packages are getting picked up when doing the cabbage preparation, but are not getting picked up when we're running the build in the Nix store. Interestingly, transformers-0.3.0.0 is getting picked up (notice it's not mentioned in the original error message). Which means that something is going wrong on the GHC level regarding its global package DB.

Cabbage works in NixOS and Darwin with global package databases as come with GHC, but perhaps the haskell-ng infrastructure is doing something with package databases that is giving us these conflicting reports of the global DB. It is probably because we are running cabal in your environment, which has these globally-installed packages, but then building in a more minimal environment that lacks them. You can try switching to a clean environment to see if cabbage works there. If so, then we could try to work out how to make cabbage's cabal usage occur in a similarly clean environment.

[cocreature]

You are excellent at providing explanations, thanks!

I set up a clean environment and everything is working just fine. I looked at what I had in ghcWithPackages and it turns out that I could remove almost everything. However (at least to my knowledge) ghc-mod should stay in there as it uses the ghc api and will break otherwise because ghc can update without a ghc-mod rebuild. Sadly putting ghc-mod in there already pulls in a long list of dependencies which results in the same problem as I had previously.

[acowley]

For myself, I built ghc-mod with cabbage and installed it in my environment. That gives me the executable without any baggage. Since I don't use ghcWithPackages, I'm not sure what the right thing to do is. To clarify, I guess the problem is when switching versions of GHC, as you say. Perhaps we'd need something like ghcWithPackages that doesn't install things in the global package DB, but is just an alias for related executables.

I feel like there should be a pretty lightweight way to address this, but I haven't put my finger on it yet.