"malformed mach-o: load commands size" with a multi-package cabal.project

[phadej]

We have a project with over 50 packages we run into infamous OSX linker problem.

In otool -l dump of some dylib there are

Load command 21
          cmd LC_LOAD_DYLIB
      cmdsize 80
         name @rpath/libHSdpndnt-mp-0.2.4.0-b3458e24-ghc8.2.2.dylib (offset 24)
   time stamp 2 Thu Jan  1 02:00:02 1970
      current version 0.0.0

for each dependency, which is fine.

Local libraries names aren't mungled

Load command 45
          cmd LC_LOAD_DYLIB
      cmdsize 72
         name @rpath/libHSenv-config-0-inplace-ghc8.2.2.dylib (offset 24)
   time stamp 2 Thu Jan  1 02:00:02 1970
      current version 0.0.0
compatibility version 0.0.0

local package names aren't mungled: we have "long" names futurice-this and futurice-that, servant-algebraic-graphs ...

Each local library's dylib is in own dir

While there is one LC_RPATH for store:

Load command 330
          cmd LC_RPATH
      cmdsize 56
         path /Users/toku/.cabal/store/ghc-8.2.2/lib (offset 12)

There are one per local dependency:

Load command 320
          cmd LC_RPATH
      cmdsize 96
         path /Users/toku/hmr/dist-newstyle/build/x86_64-osx/ghc-8.2.2/log-cloudwatch-0/build (offset 12)
Load command 321
          cmd LC_RPATH
      cmdsize 88
         path /Users/toku/hmr/dist-newstyle/build/x86_64-osx/ghc-8.2.2/periocron-0/build (offset 12)
...

I'd say that fixing the second part, having single libdir in dist-newstyle (on OSX?) would save this problem too. For now it prevents splitting big "industrial size" repository into smaller packages (we want to have separate packages, as it's easier to manage).

We could use internal libraries to workaround this, but the problem would still persist for cases like https://github.com/phadej/acme-kmett (I didn't tried to compile it on OSX).

otool-hyperloglog.txt

The above dump is for hyperloglog .dylib from acme-kmett, it has 111 commands, small-dep app in our repo has 332, biggest (which overflows 32k limit) has 420.

cc @christiaanb

[angerman]

Lovely... so that's the boundary where the library-name munging hits its limits. So acme-kmett should be a good test case? I'll try building that. Let's see what happens.

[phadej]

@angerman acme-kmett doesn't fail, it's just a repo with enough "deps" to highlight the problem.

Maybe if you pull all deps of yesod locally into single cabal.project, it will actually fail. I don't have a osx machine myself, so cannot try that.

[angerman]

Just learned so myself. Alright, I’ll try yesod.

I vaguely remember having the everything in the same lib idea implemented somewhere. But it would break package relocatability. I’m however more inclined to see if we can just link direct dependencies only in ghc and circumvent the limit that way; might still need to symlink(?) all libs into a common folder in cabal.

I’ll give

[cartazio]

so the new-build version of the linker sadness of OSX has finally happened?

[cartazio]

what about having a lib symlinks directory in the new-dist folder for new builds, then we can get really short relative paths, but they should be safe in the face of relocations

lets ignore the issue with lack of symlinks on window for this :))))

[angerman]

Alright, i got a reproduction case:

$ brew install stack
$ stack new test-project yesod-sqlite
$ cd test-project && cabal new-build # this won't fail, but will populate the package database appropriately.

$ ghc-pkg --package-db ~/.cabal/store/ghc-8.4.1/package.db dot|tred > graph.dot
$ awk -F\  '{ print $1 }' graph.dot|grep \" >> pkgs
$ awk -F\  '{ print $3 }' graph.dot|grep \"|sed s/\"\;/\"/g >> pkgs

now pkgs will contain all packages from the package database.

$ echo "packages: ." > cabal.project
$ cat pkgs|sort -u|awk -F\" '{ print "          "$2 }' >> cabal.project

will create the cabal.project file containing all the libs. This might need a tiny bit of touch up...

And now...

$ cat cabal.project|grep -v "^packages"|xargs cabal unpack
$ cabal new-build

will likely error out with:

ghc: panic! (the 'impossible' happened)
  (GHC version 8.4.1 for x86_64-apple-darwin):
	Loading temp shared object failed: dlopen(/var/folders/fv/xqjrpfj516n5xq_m_ljpsjx00000gn/T/ghc53894_0/libghc_19.dylib, 5): no suitable image found.  Did find:
	/var/folders/fv/xqjrpfj516n5xq_m_ljpsjx00000gn/T/ghc53894_0/libghc_19.dylib: malformed mach-o: load commands size (34104) > 32768

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

[angerman]

Reproduction repo: https://github.com/angerman/macos-ghc-dylib-blowup

[angerman]

So I see the #4656 fixed the libraries in $HOME/.cabal/store only. Not in the the local dist-newbuild. That also explains why you need to specifically have them all in your project to trigger the issue.

[jship]

At work, we have worked around the mach-o "load commands size" issue by following in the Nix folks' footsteps and using a wrapper script around ld to recursively subdivide the dependencies into a tree of re-exporting delegate libraries.

The script, an example, and more info about this issue is available here:
https://github.com/Simspace/ld-wrapper-macos

[23Skidoo]

@jship's suggestion is probably the best way forward. PRs welcome.