Skip to content

stack setup installs GHC for aarch64 on M1 Macs #5562

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 29, 2021
Merged

Conversation

borsboom
Copy link
Contributor

@borsboom borsboom commented Jun 6, 2021

THIS IS UNTESTED because I don't have an M1 Mac. However, stack --arch=aarch64 --resolver=ghc-8.10.5 setup does download the correct bindist and tries to install it on my Intel Mac (but of course this fails with "Bad CPU type in executable").

Note: this will only work with GHC 8.10.5 right now, since that's the only GHC version with an aarch64 bindist for macOS. You can use stack --resolver=ghc-8.10.5 setup to test.

One potential downside of this change: stack setup on M1 Macs will not, by default, work with any GHC <8.10.5, since there aren't aarch64 bindists available (before this change, stack setup always uses the x86 bindist even on M1 Macs, so this wasn't an issue). Users will have to use --arch=x86_64 to force the use of Rosetta. Mac users are pretty accustomed to system or OS upgrades breaking old GHC versions, so not sure if this matters.

We could potentially implement automatic fallback to x86, if that turns out to be desirable. I personally think I would want to explicitly enable Rosetta emulation if I had an M1 Mac, but I could see others preferring it to be implicit and I'm not sure there's a really clear case to be made for either way. There will almost certainly come a day when Apple completely stops supporting Intel, even through emulation (as they did with PowerPC and Motorola 68000), at which point a fallback would stop working.

@yairchu
Copy link

yairchu commented Jun 8, 2021

Got this output:

yairchu@Yairs-M1-MBP-4 stack % stack exec -- stack --arch=aarch64 --resolver=ghc-8.10.5 setup
Preparing to install GHC to an isolated location.
This will not interfere with any system-level installation.
Downloaded ghc-8.10.5.                                      
ld: unknown option: --version                                                     
ld: unknown option: --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Installed GHC.    
The GHC located at /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/bin/ghc-8.10.5 failed to compile a sanity check. Please see:

    http://docs.haskellstack.org/en/stable/install_and_upgrade/

for more information. Exception was:
Received ExitFailure 1 when running
Raw command: /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/bin/ghc-8.10.5 /private/var/folders/_y/nt_hv7k57bq6yjznsyd87sp40000gn/T/stack-sanity-check-1e82db3595630220/Main.hs -no-user-package-db
Run from: /private/var/folders/_y/nt_hv7k57bq6yjznsyd87sp40000gn/T/stack-sanity-check-1e82db3595630220/
Standard output:

[1 of 1] Compiling Main             ( /private/var/folders/_y/nt_hv7k57bq6yjznsyd87sp40000gn/T/stack-sanity-check-1e82db3595630220/Main.hs, /private/var/folders/_y/nt_hv7k57bq6yjznsyd87sp40000gn/T/stack-sanity-check-1e82db3595630220/Main.o )
Standard error:


<no location info>: error:
    Warning: Couldn't figure out LLVM version!
             Make sure you have installed LLVM between 10 and 12
ghc: could not execute: opt

Regardless of the error message above,
/Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/bin/ghci does successfully run and evaluates Haskell expressions and shows up as an ARM process in the macOS Activity Monitor

And if I try to use /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/bin/ghc to compile things it errors with:

Missing file: /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/lib/settings

@borsboom
Copy link
Contributor Author

borsboom commented Jun 8, 2021

@yairchu Have you tried installing LLVM? (e.g. brew install llvm@12)

I wonder if Stack will end up having to manage LLVM installations, since in the past different versions of GHC have required different LLVM versions. This has been a barrier to full ARM support on Linux.

@yairchu
Copy link

yairchu commented Jun 8, 2021

It appears I've already had llvm installed via brew, so I tried upgrading it to 12.0.0_1, and got interesting output:

yairchu@Yairs-M1-MBP-4 stack % brew upgrade llvm
==> Upgrading 1 outdated package:
llvm 12.0.0 -> 12.0.0_1
==> Upgrading llvm 12.0.0 -> 12.0.0_1 
...
==> Pouring llvm--12.0.0_1.arm64_big_sur.bottle.tar.gz
==> Caveats
To use the bundled libc++ please add the following LDFLAGS:
  LDFLAGS="-L/opt/homebrew/opt/llvm/lib -Wl,-rpath,/opt/homebrew/opt/llvm/lib"

llvm is keg-only, which means it was not symlinked into /opt/homebrew,
because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.

If you need to have llvm first in your PATH, run:
  echo 'export PATH="/opt/homebrew/opt/llvm/bin:$PATH"' >> ~/.zshrc

For compilers to find llvm you may need to set:
  export LDFLAGS="-L/opt/homebrew/opt/llvm/lib"
  export CPPFLAGS="-I/opt/homebrew/opt/llvm/include"
...

So following the above message I've tried running stack with an augmented path, and got:

% PATH="/opt/homebrew/opt/llvm/bin:$PATH" stack exec -- stack --arch=aarch64 --resolver=ghc-8.10.5 setup

The GHC located at /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/bin/ghc-8.10.5 failed to compile a sanity check. Please see:

    http://docs.haskellstack.org/en/stable/install_and_upgrade/

for more information. Exception was:
Received ExitFailure 1 when running
Raw command: /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/bin/ghc-8.10.5 /private/var/folders/_y/nt_hv7k57bq6yjznsyd87sp40000gn/T/stack-sanity-check-e347b83a76856d2f/Main.hs -no-user-package-db
Run from: /private/var/folders/_y/nt_hv7k57bq6yjznsyd87sp40000gn/T/stack-sanity-check-e347b83a76856d2f/
Standard output:

[1 of 1] Compiling Main             ( /private/var/folders/_y/nt_hv7k57bq6yjznsyd87sp40000gn/T/stack-sanity-check-e347b83a76856d2f/Main.hs, /private/var/folders/_y/nt_hv7k57bq6yjznsyd87sp40000gn/T/stack-sanity-check-e347b83a76856d2f/Main.o )
Linking /private/var/folders/_y/nt_hv7k57bq6yjznsyd87sp40000gn/T/stack-sanity-check-e347b83a76856d2f/Main ...
Standard error:

You are using an unsupported version of LLVM!
Currently only 10 to 12 is supported. System LLVM version: 12.0.0
We will try though...
ld: warning: ignoring file /private/var/folders/_y/nt_hv7k57bq6yjznsyd87sp40000gn/T/stack-sanity-check-e347b83a76856d2f/Main.o, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/Cabal-3.2.1.0/libHSCabal-3.2.1.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/text-1.2.4.1/libHStext-1.2.4.1.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/parsec-3.1.14.0/libHSparsec-3.1.14.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/process-1.6.9.0/libHSprocess-1.6.9.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/template-haskell-2.16.0.0/libHStemplate-haskell-2.16.0.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/ghc-boot-th-8.10.5/libHSghc-boot-th-8.10.5.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/pretty-1.1.3.6/libHSpretty-1.1.3.6.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/mtl-2.2.2/libHSmtl-2.2.2.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/transformers-0.5.6.2/libHStransformers-0.5.6.2.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/directory-1.3.6.0/libHSdirectory-1.3.6.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/filepath-1.4.2.1/libHSfilepath-1.4.2.1.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/time-1.9.3/libHStime-1.9.3.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/unix-2.7.2.2/libHSunix-2.7.2.2.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/containers-0.6.4.1/libHScontainers-0.6.4.1.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/binary-0.8.8.0/libHSbinary-0.8.8.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/bytestring-0.10.12.0/libHSbytestring-0.10.12.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/integer-gmp-1.0.3.0/libHSinteger-gmp-1.0.3.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/rts/libHSrts.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/deepseq-1.4.4.0/libHSdeepseq-1.4.4.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/base-4.14.2.0/libHSbase-4.14.2.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/array-0.5.4.0/libHSarray-0.5.4.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/ghc-prim-0.6.1/libHSghc-prim-0.6.1.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/yairchu/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/rts/libCffi.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
Undefined symbols for architecture x86_64:
  "_hs_atomicwrite64", referenced from:
     -u command line option
  "_hs_atomicwrite32", referenced from:
     -u command line option
  "_hs_atomicread64", referenced from:
     -u command line option
  "_hs_atomicread16", referenced from:
     -u command line option
  "_hs_cmpxchg64", referenced from:
     -u command line option
  "_hs_atomic_xor64", referenced from:
     -u command line option
  "_hs_atomicwrite16", referenced from:
     -u command line option
  "_hs_atomic_xor8", referenced from:
     -u command line option
  "_hs_atomic_or32", referenced from:
     -u command line option
  "_hs_atomic_nand64", referenced from:
     -u command line option
  "_hs_cmpxchg32", referenced from:
     -u command line option
  "_hs_atomic_nand8", referenced from:
     -u command line option
  "_hs_atomic_and64", referenced from:
     -u command line option
  "_hs_atomic_nand16", referenced from:
     -u command line option
  "_hs_atomic_and32", referenced from:
     -u command line option
  "_hs_atomic_and16", referenced from:
     -u command line option
  "_hs_atomic_sub32", referenced from:
     -u command line option
  "_hs_atomic_or64", referenced from:
     -u command line option
  "_hs_atomic_sub16", referenced from:
     -u command line option
  "_hs_atomic_sub64", referenced from:
     -u command line option
  "_hs_atomic_sub8", referenced from:
     -u command line option
  "_hs_atomic_add64", referenced from:
     -u command line option
  "_hs_atomic_or16", referenced from:
     -u command line option
  "_hs_atomic_add8", referenced from:
     -u command line option
  "_hs_atomicread8", referenced from:
     -u command line option
  "_base_GHCziWord_W64zh_con_info", referenced from:
     -u command line option
  "_hs_atomicwrite8", referenced from:
     -u command line option
  "_hs_atomic_add16", referenced from:
     -u command line option
  "_base_GHCziWord_W8zh_con_info", referenced from:
     -u command line option
  "_base_GHCziInt_I64zh_con_info", referenced from:
     -u command line option
  "_base_GHCziInt_I32zh_con_info", referenced from:
     -u command line option
  "_base_GHCziStable_StablePtr_con_info", referenced from:
     -u command line option
  "_hs_atomic_nand32", referenced from:
     -u command line option
  "_base_GHCziInt_I16zh_con_info", referenced from:
     -u command line option
  "_base_GHCziInt_I8zh_con_info", referenced from:
     -u command line option
  "_hs_atomicread32", referenced from:
     -u command line option
  "_ghczmprim_GHCziTypes_Dzh_con_info", referenced from:
     -u command line option
  "_ghczmprim_GHCziTypes_Czh_con_info", referenced from:
     -u command line option
  "_hs_atomic_and8", referenced from:
     -u command line option
  "_base_GHCziTopHandler_runMainIO_closure", referenced from:
     -u command line option
  "_base_GHCziConcziIO_ioManagerCapabilitiesChanged_closure", referenced from:
     -u command line option
  "_base_GHCziTopHandler_flushStdHandles_closure", referenced from:
     -u command line option
  "_hs_atomic_xor16", referenced from:
     -u command line option
  "_base_GHCziConcziSignal_runHandlersPtr_closure", referenced from:
     -u command line option
  "_defaultRtsConfig", referenced from:
      _main in ghc_8.o
  "_base_GHCziTopHandler_runIO_closure", referenced from:
     -u command line option
  "_base_GHCziIOziException_cannotCompactPinned_closure", referenced from:
     -u command line option
  "_hs_atomic_or8", referenced from:
     -u command line option
  "_base_GHCziWeak_runFinalizzerBatch_closure", referenced from:
     -u command line option
  "_base_ControlziExceptionziBase_nonTermination_closure", referenced from:
     -u command line option
  "_base_GHCziWord_W16zh_con_info", referenced from:
     -u command line option
  "_base_GHCziConcziSync_runSparks_closure", referenced from:
     -u command line option
  "_base_GHCziPtr_FunPtr_con_info", referenced from:
     -u command line option
  "_base_GHCziPtr_Ptr_con_info", referenced from:
     -u command line option
  "_base_GHCziEventziThread_blockedOnBadFD_closure", referenced from:
     -u command line option
  "_base_GHCziIOziException_cannotCompactFunction_closure", referenced from:
     -u command line option
  "_ghczmprim_GHCziTypes_Izh_con_info", referenced from:
     -u command line option
  "_base_GHCziConcziIO_ensureIOManagerIsRunning_closure", referenced from:
     -u command line option
  "_hs_main", referenced from:
      _main in ghc_8.o
  "_base_GHCziIOziException_stackOverflow_closure", referenced from:
     -u command line option
  "_ghczmprim_GHCziTypes_Wzh_con_info", referenced from:
     -u command line option
  "_base_GHCziIOziException_blockedIndefinitelyOnMVar_closure", referenced from:
     -u command line option
  "_base_GHCziIOziException_allocationLimitExceeded_closure", referenced from:
     -u command line option
  "_base_GHCziIOziException_cannotCompactMutable_closure", referenced from:
     -u command line option
  "_hs_cmpxchg16", referenced from:
     -u command line option
  "_base_GHCziTopHandler_runNonIO_closure", referenced from:
     -u command line option
  "_hs_atomic_add32", referenced from:
     -u command line option
  "_base_GHCziIOziException_blockedIndefinitelyOnSTM_closure", referenced from:
     -u command line option
  "_hs_cmpxchg8", referenced from:
     -u command line option
  "_ghczmprim_GHCziTypes_Fzh_con_info", referenced from:
     -u command line option
  "_ghczmprim_GHCziTuple_Z0T_closure", referenced from:
     -u command line option
  "_ghczmprim_GHCziTypes_False_closure", referenced from:
     -u command line option
  "_hs_atomic_xor32", referenced from:
     -u command line option
  "_base_ControlziExceptionziBase_nestedAtomically_closure", referenced from:
     -u command line option
  "_ghczmprim_GHCziTypes_True_closure", referenced from:
     -u command line option
  "_base_GHCziWord_W32zh_con_info", referenced from:
     -u command line option
  "_base_GHCziIOziException_heapOverflow_closure", referenced from:
     -u command line option
  "_base_ControlziExceptionziBase_absentSumFieldError_closure", referenced from:
     -u command line option
  "_ZCMain_main_closure", referenced from:
      _main in ghc_8.o
  "_base_GHCziPack_unpackCString_closure", referenced from:
     -u command line option
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
`gcc' failed in phase `Linker'. (Exit code: 1)

@BlakeRain
Copy link
Member

I can report that I'm having the same issue as @yairchu, with the same output.

Stack is first reporting that the GHC failed to compile a sanity check. This seemed a little odd, so I ran the same GHC version myself. I first created a source file with the contents I think are the same as the sanity check:

import Distribution.Simple
main = putStrLn "Hello World"

I ran this through the GHC that stack indicated it used to compile the sanity check:

$ /Users/blakerain/.stack/programs/aarch64-osx/ghc-8.10.5/bin/ghc-8.10.5 $(pwd)/simple.hs -no-user-package-db
[1 of 1] Compiling Main             ( /Users/blakerain/simple.hs, /Users/blakerain/simple.o )

<no location info>: error:
    Warning: Couldn't figure out LLVM version!
             Make sure you have installed LLVM between 10 and 12
ghc: could not execute: opt

Adding the LLVM location to PATH allowed GHC to find the opt that comes with the LLVM installed via brew:

$ PATH="/opt/homebrew/opt/llvm/bin:$PATH" /Users/blakerain/.stack/programs/aarch64-osx/ghc-8.10.5/bin/ghc-8.10.5 $(pwd)/simple.hs -no-user-package-db
[1 of 1] Compiling Main             ( /Users/blakerain/simple.hs, /Users/blakerain/simple.o )
You are using an unsupported version of LLVM!
Currently only 10 to 12 is supported. System LLVM version: 12.0.0
We will try though...
Linking /Users/blakerain/simple ...

Whilst it complains about the LLVM version, it still links an arm64 executable that runs as expected. I must be missing something that stack is doing to the environment or something similar.

@BlakeRain
Copy link
Member

It seems I can reproduce the output for the failed sanity check when I run GHC with arch and select the x86-64 architecture:

$ PATH="/opt/homebrew/opt/llvm/bin:$PATH" arch -x86_64 /Users/blakerain/.stack/programs/aarch64-osx/ghc-8.10.5/bin/ghc-8.10.5 $(pwd)/simple.hs -no-user-package-db
[1 of 1] Compiling Main             ( /Users/blakerain/simple.hs, /Users/blakerain/simple.o )
You are using an unsupported version of LLVM!
Currently only 10 to 12 is supported. System LLVM version: 12.0.0
We will try though...
Linking /Users/blakerain/simple ...
ld: warning: ignoring file /Users/blakerain/.stack/programs/aarch64-osx/ghc-8.10.5/lib/ghc-8.10.5/Cabal-3.2.1.0/libHSCabal-3.2.1.0.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/blakerain/simple.o, building for macOS-x86_64 but attempting to link with file built for unknown-arm64

... [warnings as before]

Undefined symbols for architecture x86_64:
  "_hs_atomicwrite64", referenced from:
     -u command line option
  "_hs_atomicwrite32", referenced from:
     -u command line option

... [missing symbols as before]

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
`gcc' failed in phase `Linker'. (Exit code: 1)

@borsboom
Copy link
Contributor Author

borsboom commented Jun 9, 2021

If you omit the --arch=aarch64 argument from stack, what happens? Does it just install and use the x86 GHC?

I suspect that because you're using an x86 stack binary, Rosetta gets activated and stays active when spawning out to GHC, and that causes the problems you're seeing. I think we may need to bootstrap an M1-native stack binary to get this to work.

@BlakeRain
Copy link
Member

Hi @borsboom, when I run without the --arch=aarch64 option the setup instruction succeeds:

$ PATH="/opt/homebrew/opt/llvm/bin:$PATH" stack exec -- stack --resolver=ghc-8.10.5 setup
Preparing to install GHC to an isolated location.
This will not interfere with any system-level installation.
Downloaded ghc-8.10.5.
ld: unknown option: --version
ld: unknown option: --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/blakerain/.stack/programs/x86_64-osx/ghc-8.10.5/lib/ghc-8.10.5/rts/libHSrts.a(Disassembler.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/blakerain/.stack/programs/x86_64-osx/ghc-8.10.5/lib/ghc-8.10.5/rts/libHSrts.a(LdvProfile.o) has no symbols

... [lost more of the same]

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/blakerain/.stack/programs/x86_64-osx/ghc-8.10.5/lib/ghc-8.10.5/rts/libHSrts_debug_p.a(plt.debug_p_o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/blakerain/.stack/programs/x86_64-osx/ghc-8.10.5/lib/ghc-8.10.5/rts/libHSrts_debug_p.a(plt_aarch64.debug_p_o) has no symbols
Installed GHC.
stack will use a sandboxed GHC it installed
For more information on paths, see 'stack path' and 'stack exec env'
To use this GHC and packages outside of a project, consider using:
stack ghc, stack ghci, stack runghc, or stack exec

(full output is here)

I agree that the issue is likely due to the propagation of Rosetta. I had similar issues with other software before their Apple Silicon version (e.g. VSCode).

@ncihnegn
Copy link

ncihnegn commented Jun 9, 2021

My M1 setup has Rosetta removed. My stack binary is built with native ghc and cabal.

$ stack setup --resolver=ghc-8.10.5 -v
...
[debug] Downloading /~ghc/8.10.5/ghc-8.10.5-aarch64-apple-darwin.tar.bz2
...
$ stack ghci --resolver ghc-8.10.5
...
GHCi, version 8.10.5: https://www.haskell.org/ghc/  :? for help
...

It seems working well.

Will Stackage add 8.10.5 support?

@borsboom
Copy link
Contributor Author

borsboom commented Jun 9, 2021

@ncihnegn If you don't have Rosetta, how are you running stack? Did you build an M1-native binary somehow?
[EDIT: now I see you already answered that in an edit]

@borsboom
Copy link
Contributor Author

borsboom commented Jun 9, 2021

@ncihnegn Are you able to build the stack project itself using the stack binary you built using GHC and Cabal?

@ncihnegn
Copy link

ncihnegn commented Jun 9, 2021

@ncihnegn Are you able to build the stack project itself using the stack binary you built using GHC and Cabal?

Not at all. I tried changing resolver in stack.yaml to ghc-8.10.5 and adding packages in stack.yaml.lock to extra-deps. But there are still compiling errors in pantry.
But stack new hello builds well.

@ncihnegn
Copy link

Whilst it complains about the LLVM version, it still links an arm64 executable that runs as expected. I must be missing something that stack is doing to the environment or something similar.

It is ghc who complains about LLVM version.

@yairchu
Copy link

yairchu commented Jun 10, 2021

It appears that if I change ~/.stack/programs/aarch64-osx/ghc-8.10.5/bin/ghc-8.10.5 to make the last line

exec arch -arch arm64 "$executablename" -B"$topdir" ${1+"$@"}

(added the arch -arch arm64 there)

then another invocation of PATH="/opt/homebrew/opt/llvm/bin:$PATH" stack exec -- stack --arch=aarch64 --resolver=ghc-8.10.5 setup does work

@ncihnegn
Copy link

@ncihnegn Are you able to build the stack project itself using the stack binary you built using GHC and Cabal?

Not at all. I tried changing resolver in stack.yaml to ghc-8.10.5 and adding packages in stack.yaml.lock to extra-deps. But there are still compiling errors in pantry.
But stack new hello builds well.

After I dump the dependency from plan.json into extra-deps, I can build the stack project. So stack itself is fine. What is missing is Stackage support.

@borsboom
Copy link
Contributor Author

@ncihnegn I don't suppose you could upload the M1 stack binary somewhere? It would be really handy to have available since it'll save a step for bootstrapping in the future.

@ncihnegn
Copy link

@ncihnegn I don't suppose you could upload the M1 stack binary somewhere? It would be really handy to have available since it'll save a step for bootstrapping in the future.

You may want to use the native haskell-stack bottle in homebrew.

@liam-ly
Copy link

liam-ly commented Aug 29, 2021

I have a mac M1 without rosetta2 installed. I built this branch locally using stack from brew and everything appears to work as I would expect:

  1. Install stack and ghc from brew.
    % brew install haskell-stack ghc
    ...
    # stack-2.7.3 from brew doesn't install ghc
    % stack --version
    Version 2.7.3 aarch64
    ...
    % ghc --version
    The Glorious Glasgow Haskell Compilation System, version 8.10.7
    
  2. stack from brew fails, as expected.
    % cd stack
    % stack build
    I don't know how to install GHC for (OSX,AArch64), please install manually
    
  3. Compile stack using tools provided by brew.
    % stack --compiler ghc-8.10.7 --system-ghc build
    ...
    % echo $?
    0
    
  4. Use the newly compiled stack executable. GHC in yaml file is not supported on apple silicon.
    % .stack-work/install/aarch64-osx/****/8.10.7/bin/stack build
    No setup information found for ghc-8.10.4 on your platform.
    This probably means a GHC bindist has not yet been added for OS key 'macosx-aarch64'.
    Supported versions: ghc-8.10.5, ghc-8.10.6
    
  5. Build stack with supported ghc.
    % .stack-work/install/aarch64-osx/****/8.10.7/bin/stack --compiler ghc-8.10.6 build
    ...
    Preparing to download ghc-8.10.6 ...
    ...
    
    % echo $?
    0
    

(Edited to make it easier to read)

@borsboom
Copy link
Contributor Author

I'm going to merge this because there don't seem to be any effects on pre-existing behaviour, but it does improve the situation for Mac M1 users who aren't using Rosetta2.

@borsboom borsboom merged commit 708b516 into master Aug 29, 2021
@borsboom borsboom deleted the macosx-aarch64 branch August 29, 2021 16:48
@ChrisPenner
Copy link

ChrisPenner commented Sep 4, 2021

Chiming in here with what worked for me in case it's helpful for others who find this issue:

  1. brew install haskell-stack
  2. install ghc-8.10.7 with ghcup
  3. brew install llvm@12
  4. Set up your environment in the current shell to use the new llvm from brew (it's not linked by default)
export PATH="/opt/homebrew/opt/llvm/bin:$PATH"
export LDFLAGS="-L/opt/homebrew/opt/llvm/lib"
export CPPFLAGS="-I/opt/homebrew/opt/llvm/include"
  1. In the shell with your llvm env vars, build latest stack from master: stack --compiler ghc-8.10.7 --system-ghc install

Now you have stack installed under ~/.local/bin/stack 😄


Update:

So this works, and you CAN compile code with this, but it's incredibly slow.
Compiling the unison after a stack clean took me 2h and 3min with this setup, compared to ~9mins with x86 😬

@liam-ly
Copy link

liam-ly commented Sep 11, 2021

So this works, and you CAN compile code with this, but it's incredibly slow.
Compiling the unison after a stack clean took me 2h and 3min with this setup, compared to ~9mins with x86 😬

I was curious if I could reproduce this slow-down, but I was not able to.

I compiled unison on an M1 mac mini (8GB ram) and it took under 20 minutes. The only difference I see is that I used ghc from brew to compile stack as shown in my comment above, instead of ghcup. I rebuilt stack using the master branch (708b516) and then proceeded to make sure I was using the executable that I just built. The master branch build of stack can now download ghc for M1.

% stack=/Users/.../stack/.stack-work/install/aarch64-osx/.../8.10.7/bin/stack --compiler ghc-8.10.7
% $stack build
... downloads ghc-8.10.7 and builds everything ...
% $stack clean
% time $stack build
...
 --compiler ghc-8.10.7 build  1026.39s user 101.54s system 103% cpu 18:07.38 total

with --fast after clean, it's around 5 minutes:

 --compiler ghc-8.10.7 build --fast  303.73s user 30.22s system 110% cpu 5:01.41 total

image

@hasufell
Copy link
Contributor

The only difference I see is that I used ghc from brew to compile stack as shown in my comment above, instead of ghcup.

ghcup uses the official GHC binary distribution and so does the brew formula for bootstrapping: https://github.com/Homebrew/homebrew-core/blob/996835f510d4084f6a2c31a1353887945ba2cf45/Formula/ghc.rb#L54

Could you please test whether the builds are slower with the GHC upstream provided bindist? I doubt it, because we would have seen this in CI, but who knows.

@liam-ly
Copy link

liam-ly commented Sep 11, 2021

tl;dr

  • stack (master branch) and ghc-8.10.7 seems to be working just fine for me in all combinations I've tried. I look forward to the next release that contains this PR.
  • Any slow-downs due to GHC itself probably should be reported to GHC issue tracker with more detailed logging information.

details

ghcup uses the official GHC binary distribution and so does the brew formula for bootstrapping: https://github.com/Homebrew/homebrew-core/blob/996835f510d4084f6a2c31a1353887945ba2cf45/Formula/ghc.rb#L54

Note that the brew vs ghcup difference is only used to compile stack, not unison. The ghc used for unison is the one that stack downloads for aarch64-osx, which also matches what brew and ghcup use.

  • stack-setup2.yaml:

    url: "http://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-aarch64-apple-darwin.tar.bz2"
    
  • homebrew:

    url "https://downloads.haskell.org/ghc/8.10.7/ghc-8.10.7-aarch64-apple-darwin.tar.xz"
    
  • ghcup

    [ Info  ] downloading: https://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-aarch64-apple-darwin.tar.xz
    

Could you please test whether the builds are slower with the GHC upstream provided bindist? I doubt it, because we would have seen this in CI, but who knows.

Which URL does this correspond to?

I downloaded the archive from the CI job https://gitlab.haskell.org/ghc/ghc/-/jobs/772824 and unzipped/untarred the archive:

# Note: These 2 commands take ~2 minutes
% ./configure --prefix=$HOME/ghc-8.10.7
% make install

Then went back to unison

% export PATH=$HOME/ghc-8.10.7/bin:$PATH
% stack=/.../stack/.stack-work/install/aarch64-osx/.../8.10.7/bin/stack --compiler ghc-8.10.7 --system-ghc
% $stack build
% $stack clean
% time $stack build
---
 --compiler ghc-8.10.7 --system-ghc build  1034.28s user 106.65s system 103% cpu 18:19.22 total

I wasn't sure which ghc it was actually using, so I re-ran with -v and I see --with-ghc=$HOME/ghc-8.10.7/bin/ghc-8.10.7 in the output.

@fwcd
Copy link
Contributor

fwcd commented Nov 9, 2021

For anyone reading this thread looking to get Stack + GHC up and running natively on Apple Silicon before the next Stack version is released, installing the recentmost Stack from source using Homebrew did work for me (this will install LLVM 12 automatically):

brew install --HEAD haskell-stack

Compiling with GHC however yielded the following error message:

<no location info>: error:
    Warning: Couldn't figure out LLVM version!
             Make sure you have installed LLVM between [9 and 13)
ghc: could not execute: opt

The solution here is to make sure the LLVM 12 installation that brew installed earlier is on the PATH (note the use of llvm@12):

export PATH="/opt/homebrew/opt/llvm@12/bin:$PATH"
export LDFLAGS="-L/opt/homebrew/opt/llvm@12/lib"
export CPPFLAGS="-I/opt/homebrew/opt/llvm@12/include"

@simnalamburt
Copy link

For anyone reading this thread, brew install --HEAD haskell-stack is currently broken. I've sent a PR to fix this issue by #5677 and Homebrew/homebrew-core#95032, but both haven't merged yet.

Until then, you can use my patched version of stack like this:

brew install simnalamuburt/x/haskell-stack
Reference

simnalamburt added a commit to simnalamburt/homebrew-core that referenced this pull request Feb 17, 2022
- Fixed building from source.

  Due to recent update of aeson-2.0.0.0, stack can no longer be built
  with cabal-install. So I patched stack to freeze cabal dependencies
  using stackage 17.15 LTS.

- Add Apple Silicon support.

  HEAD version of stack has already been patched to support Apple
  Silicon. However, the next release containing that patch hasn't
  release yet. So I manually patched stack v2.7.3 to support Apple
  Silicon.

- Add llvm@12 as a dependency only on ARM

  All ghc versions before 9.2.1 requires LLVM Code Generator as a
  backend on ARM.

Fixes Homebrew#94796

Reference:
  commercialhaskell/stack#5562
  commercialhaskell/stack#5677
carlocab pushed a commit to Homebrew/homebrew-core that referenced this pull request Feb 18, 2022
- Fixed building from source.

  Due to recent update of aeson-2.0.0.0, stack can no longer be built
  with cabal-install. So I patched stack to freeze cabal dependencies
  using stackage 17.15 LTS.

- Add Apple Silicon support.

  HEAD version of stack has already been patched to support Apple
  Silicon. However, the next release containing that patch hasn't
  release yet. So I manually patched stack v2.7.3 to support Apple
  Silicon.

- Add llvm@12 as a dependency only on ARM

  All ghc versions before 9.2.1 requires LLVM Code Generator as a
  backend on ARM.

Fixes #94796

Reference:
  commercialhaskell/stack#5562
  commercialhaskell/stack#5677
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants