Support running on M1 Macs [Python]

[FabianBruengerPexon]

Hey,

i am running into issues with python azure function under VSCode. FYI the same example I will present is working fine with JS.

Set up:
Mac M1 Big Sur v11.2.1
python 3.9.1
Core Tools Version: 3.0.3284

I am following the usual workflow: func init, func new, func start. When I start the function, this is what I get:

Found Python version 3.9.1 (python3).

Azure Functions Core Tools
Core Tools Version:       3.0.3284 Commit hash: 98bc25e668274edd175a1647fe5a9bc4ffb6887d 
Function Runtime Version: 3.0.15371.0

Ä2021-02-16T20:28:10.779ZÅ Traceback (most recent call last):
Ä2021-02-16T20:28:10.783ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/worker.py", line 84, in <module>
Ä2021-02-16T20:28:10.783ZÅ     from azure_functions_worker import main
Ä2021-02-16T20:28:10.783ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/azure_functions_worker/main.py", line 8, in <module>
Ä2021-02-16T20:28:10.783ZÅ     from . import dispatcher
Ä2021-02-16T20:28:10.783ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/azure_functions_worker/dispatcher.py", line 21, in <module>
Ä2021-02-16T20:28:10.783ZÅ     import grpc
Ä2021-02-16T20:28:10.783ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/__init__.py", line 23, in <module>
Ä2021-02-16T20:28:10.783ZÅ     from grpc._cython import cygrpc as _cygrpc
Ä2021-02-16T20:28:10.784ZÅ ImportError: dlopen(/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so, 2): no suitable image found.  Did find:
Ä2021-02-16T20:28:10.785ZÅ  /opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so: mach-o, but wrong architecture
Ä2021-02-16T20:28:10.788ZÅ  /opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so: mach-o, but wrong architecture
Ä2021-02-16T20:28:10.788ZÅ Traceback (most recent call last):
Ä2021-02-16T20:28:10.788ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/worker.py", line 84, in <module>
Ä2021-02-16T20:28:10.788ZÅ     from azure_functions_worker import main
Ä2021-02-16T20:28:10.794ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/azure_functions_worker/main.py", line 8, in <module>
Ä2021-02-16T20:28:10.794ZÅ     from . import dispatcher
Ä2021-02-16T20:28:10.794ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/azure_functions_worker/dispatcher.py", line 21, in <module>
Ä2021-02-16T20:28:10.794ZÅ     import grpc
Ä2021-02-16T20:28:10.794ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/__init__.py", line 23, in <module>
Ä2021-02-16T20:28:10.799ZÅ     from grpc._cython import cygrpc as _cygrpc
Ä2021-02-16T20:28:10.799ZÅ **ImportError: dlopen(/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so, 2): no suitable image found.  Did find:**
Ä2021-02-16T20:28:10.799ZÅ  /opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so: mach-o, but wrong architecture
Ä2021-02-16T20:28:10.799ZÅ  /opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so: mach-o, but wrong architecture
Ä2021-02-16T20:28:10.799ZÅ Traceback (most recent call last):
Ä2021-02-16T20:28:10.800ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/worker.py", line 84, in <module>
Ä2021-02-16T20:28:10.800ZÅ     from azure_functions_worker import main
Ä2021-02-16T20:28:10.800ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/azure_functions_worker/main.py", line 8, in <module>
Ä2021-02-16T20:28:10.800ZÅ     from . import dispatcher
Ä2021-02-16T20:28:10.800ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/azure_functions_worker/dispatcher.py", line 21, in <module>
Ä2021-02-16T20:28:10.800ZÅ     import grpc
Ä2021-02-16T20:28:10.800ZÅ   File "/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/__init__.py", line 23, in <module>
Ä2021-02-16T20:28:10.800ZÅ     from grpc._cython import cygrpc as _cygrpc
Ä2021-02-16T20:28:10.800ZÅ **ImportError: dlopen(/opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so, 2): no suitable image found.  Did find:**
Ä2021-02-16T20:28:10.800ZÅ  /opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so: mach-o, but wrong architecture
Ä2021-02-16T20:28:10.801ZÅ Exceeded language worker restart retry count for runtime:python. Shutting down and proactively recycling the Functions Host to recover
Ä2021-02-16T20:28:10.808ZÅ  /opt/homebrew/Cellar/azure-functions-core-toolsÉ3/3.0.3284/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so: mach-o, but wrong architecture

Functions:

 HttpTrigger: ÄGET,POSTÅ http://localhost:7071/api/HttpTrigger

For detailed output, run func with --verbose flag.

I would appreciate some advice here!
Thanks and all the best

[apawast]

@FabianBruengerPexon we are waiting to get the machine to be able to test this.

[isa56k]

I'm also seeing this on M1 Mac Mini running:
Python 3.9.2
Core Tools Version: 3.0.3477 Commit hash: 5fbb9a76fc00e4168f2cc90d6ff0afe5373afc6d (64-bit)
Function Runtime Version: 3.0.15584.0

[simon-tarr]

Also experiencing this issue. Please see Azure/azure-functions-core-tools#2680

Returning M1 Macbook, sadly, as this issue has clearly been rumbling on for many months!

[jamwil]

Still experiencing this on M1 MacBook Pro, Python 3.9.7, Core Tools 3.0.3785.

[lks-hrsch]

The issue is still present.
M1 MacBook Pro

Core Tools Version: 4.0.3780 Commit hash: 2b9e0e618f69993ff119009e48587b67ac85450e (64-bit)
Function Runtime Version: 4.0.0.16394

ImportError: dlopen(/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.3780/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so, 0x0002): tried: '/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.3780/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cygrpc.cpython-39-darwin.so' (no such file), '/usr/lib/cygrpc.cpython-39-darwin.so' (no such file)

[boring-joey]

The issue is still present. M1 MacBook Pro

Core Tools Version: 4.0.3780 Commit hash: 2b9e0e6 (64-bit)
Function Runtime Version: 4.0.0.16394

ImportError: dlopen(/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.3780/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so, 0x0002): tried: '/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.3780/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cygrpc.cpython-39-darwin.so' (no such file), '/usr/lib/cygrpc.cpython-39-darwin.so' (no such file)

Still got the exact same error. Running two macbooks for this right now, waiting to solve this problem. Lets hope it comes quick in some way.

[anthonychu]

Looks like Core Tools is able to run in x64 emulation but it’s starting the python worker in ARM. The worker uses x64 binaries in some places and that’s where it appears to be failing. Are you able to run the entire stack with x64 emulation? I only have an Intel Mac so I can’t try this out.

/cc @vrdmr

[anthonychu]

For Node.js we were able to fix this by using a pure JS implementation of gRPC that doesn’t rely on platform specific binaries.
Not sure if same exists for Python. Azure/azure-functions-nodejs-worker#386

[simon-tarr]

Unable to test this at present but (thinking out loud here): I'm wondering if running everything in a conda environment of some description would help in the interim? Using conda environments has solved a ton of (unrelated) problems for me over the past few months while x86 and arm64 code is living side by side.

[lks-hrsch]

Looks like Core Tools is able to run in x64 emulation but it’s starting the python worker in ARM. The worker uses x64 binaries in some places and that’s where it appears to be failing. Are you able to run the entire stack with x64 emulation? I only have an Intel Mac so I can’t try this out.

/cc @vrdmr

This was a good hint, thank you! I was able to start func start host without an error on my M1 Macbook Pro.

What have I done?

1. Enable Rosetta in iTerm so that you get the following output:

$ arch
i386

2. install the intel homebrew with:

$ arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

refer to https://stackoverflow.com/questions/64882584/how-to-run-the-homebrew-installer-under-rosetta-2-on-m1-macbook
3. install azure-functions-core-tools@4 and python in this homebrew
4. setting python path to the rosetta python
5. run the command:

$ arch -x86_64 /usr/local/bin/func start host

[ejizba]

Cleaning up M1 Mac issues at the moment. This one seemed like the most detailed Python issue, so I've transferred it here to track any Python-specific work. Main core tools issue is here: Azure/azure-functions-core-tools#2834

Also this comment from @vrdmr seems particularly relevant: Azure/azure-functions-core-tools#2394 (comment)