Support running on M1 Macs

[ejizba]

Update: Core tools support for M1 macs has been released for both npm and brew.

In addition to work on core tools, each language worker also needs to add support individually:

• Done:
  • Node.js - tracked here
  • .NET
  • .NET Isolated
  • PowerShell - tracked here.
• Still to do
  • Python - tracked here. Should be included with the release of support for Python 3.10
  • Java: blocked because the Java worker only supports Java 8 & 11 and AFAIK Java itself doesn't support Apple Silicon until at least Java 17

[toklive]

Does this work for .NET 6 and Azure function v4? if not any timeline when M1 support will be available?

[RichMercer]

Does this work for .NET 6 and Azure function v4? if not any timeline when M1 support will be available?

I never got it working but compiled the source myself to target arm64 and copied it over the top of the brew install location. Works lovely now. :)

[EdiWang]

Adding related issue #2180

[toklive]

Does this work for .NET 6 and Azure function v4? if not any timeline when M1 support will be available?

I never got it working but compiled the source myself to target arm64 and copied it over the top of the brew install location. Works lovely now. :)

That is awesome to hear. If it is working fine, any idea when this will be available officially for Dotnet ?

[ajonno]

same, we really need this official M1 support too

[Arash-Sabet]

Any ETA on when this fix will be released?

[natdan]

Does this work for .NET 6 and Azure function v4? if not any timeline when M1 support will be available?

I never got it working but compiled the source myself to target arm64 and copied it over the top of the brew install location. Works lovely now. :)

Do you have any tips how to do it? (to compile it on M1 mac)

I've been trying to do it whole morning and couldn't move from starting point of adding --runtime osx-arm64 to build.sh. (I tried with some other combinations, but still nothing). All with .NET 6 I installed (6.03.xxx, 6.02.xxx or 6.0.1.xxx) and one installed by dotnet-install.sh script (with --architecture arm64 parameter). All I am getting is:

Users/daniel/Documents/Projects/Other/azure-functions-core-tools/src/Azure.Functions.Cli/Azure.Functions.Cli.csproj : error : NETSDK1056: Project is targeting runtime 'osx-arm64' but did not resolve any runtime-specific packages. This runtime may not be supported by the target framework.

(I even tried to add osx-arm64 to src/Azure.Functions.Cli/Azure.Functions.Cli.csproj but it didn't make any difference...

[RichMercer]

I've changed the dotnet-install.sh as you have, and also updated the Azure.Functions.Cli.csproj and changed the RuntimeIdentifiers to osx-arm64 (I think I still had some issues building the script if that wasn't changed). Then you can run the script and it should build. Comparing my local version to source those are the only two changes.

I had previously installed the core tools using brew, so I copied the output from /tmp/cli to the brew instal location. Once I'd done those two things func version would work again.

[natdan]

OK. It is much simpler than I expected and only if I know what I know now:

1. switch to v4.x branch (!)
2. run sh dotnet-install.sh --architected arm64
3. ensure ~/.dotnet is in the path
4. change linux-x64 in build.sh to osx-arm64
5. and just run sh build.sh

It very nicely generated func which is universal 64bit mac executable

$ file func
func: Mach-O 64-bit executable arm64

and did what you said:

cp -rp * /opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.3971/

(needed to remove func from destination dir first)

Now I have different problem:

[2022-02-11T15:47:57.805Z] Failed to initialize worker provider for: /opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.3971/workers/python
[2022-02-11T15:47:57.805Z] Microsoft.Azure.WebJobs.Script: Architecture Arm64 is not supported for language python.

Path workers/python/3.9/OSX/ has only X64 only. Eh.

[natdan]

I've fixed above by copying workers/python/3.9/OSX/X64 to workers/python/3.9/OSX/Arm64, updated workers/python/worker.config.json to include Arm64 and manually replaced existing libraries in my version of python (workers/python/3.8/OSX/Arm64) with locally installed through pip in some temp venv...

And so far so good! :D

It seems it'll hold water until official version does all of it. So far what I've seen it is not far off :)

[ejizba]

Hi folks, as @natdan mentioned, the work to support arm64 is already in the v4.x branch. It will be included in the next release of core tools (I would guess that's in the next few weeks, but don't hold me to that). It will be available through npm first and homebrew shortly thereafter

That being said, each language worker still needs to individually add support for arm64. Node.js, .NET and .NET Isolated are already working. Python, PowerShell, and Java are not - see issues linked in the original comment above for more information. You'll have to check those threads for any timelines or workarounds

[lks-hrsch]

I've fixed above by copying workers/python/3.9/OSX/X64 to workers/python/3.9/OSX/Arm64, updated workers/python/worker.config.json to include Arm64 and manually replaced existing libraries in my version of python (workers/python/3.8/OSX/Arm64) with locally installed through pip in some temp venv...

And so far so good! :D

It seems it'll hold water until official version does all of it. So far what I've seen it is not far off :)

I followed your steps along and I managed to build it.
Then I copied the worker to the Arm64 folder and updated the worker.config.json.
After that I got an Error that I could fix with replacing the grpc library's with my own ones from a .venv like you mentioned it before.

Now I got this error:
ImportError: dlopen(/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.3971/workers/python/3.8/OSX/Arm64/grpc/_cython/cygrpc.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '_CFRelease'

Do you can help me to show what I am missing?

[natdan]

No, I haven't done anything else. Maybe there's some mismatch of versions?

[kentongray]

I went through a lot of steps to try to get this working and figured I'd share what I did to fix this error

cygrpc.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))

I found that I had an ARM build in
/opt/homebrew/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so

copied it to

/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.4426/workers/python/3.9/OSX/X64/grpc/_cython/

and all worked.. this gives me hope that somehow this can be fixed in a new release but I can't say I'm familiar enough with brew/python to provide more context

[ahgraber]

brew install installs osx-64 version even using arm homebrew (see image/logs below)

[ejizba]

Just a quick update - we did an initial release of arm support through npm a few weeks ago, but ran into issues because Apple Silicon has stricter requirements for arm64 binaries than x64 binaries. I'm waiting to update brew to reference the arm64 bits until I can fix that issue. I've got it fixed in a local branch, so it should make it into the next v4 release.

Just to reiterate because a few people have commented about Python, that is tracked separately in this issue: Azure/azure-functions-python-worker#915

[mribrgr]

Thanks to the solution from @natdan I also got the build running locally.
Afterwards I had a problem with openssl, I fixed it adding the symlinks:

brew install [email protected]
sudo mkdir -p /usr/local/lib
sudo ln -sfn /opt/homebrew/Cellar/[email protected]/1.1.1n/lib/libssl.1.1.dylib /usr/local/lib
sudo ln -sfn /opt/homebrew/Cellar/[email protected]/1.1.1n/lib/libcrypto.1.1.dylib /usr/local/lib

[reubano]

@kentongray's solution worked for me.

brew install [email protected]
/opt/homebrew/bin/pip3 install grpcio
cp /opt/homebrew/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so /opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.4483/workers/python/3.9/OSX/X64/grpc/_cython/

[ejizba]

Closing this issue as an arm64 version of core tools can now be installed using either brew or npm! 🎉 Thanks for your patience everyone and sorry for the delay.

Please see the original comment at the top of this thread to track any remaining work for a particular language, which is handled by a separate feature team for that language.

[reubano]

FYI, if you are using Python don't upgrade. Now I'm running into Azure/azure-functions-python-worker#915 and the work-a-round doesn't appear to work anymore.

[reubano]

In case you've already updated, here's the new work-a-round...

brew install [email protected]
brew uninstall azure-functions-core-tools@4
/opt/homebrew/bin/pip3 install grpcio
npm i -g [email protected]
cp /opt/homebrew/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so ~/.nvm/versions/node/v14.17.6/lib/node_modules/azure-functions-core-tools/bin/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so

[kijewskimateusz]

@reubano How are you starting local functions server after uninstalling Azure Functions from brew? After doing so I've lost cli func app

Edit:
I've succeeded without removing azure-functions-core-tools@4, but had to install custom Node version of 14.17.6 using NVM

[reubano]

@kijewskimateusz npm i -g [email protected]. You also have to make sure you have node in your PATH. I have export PATH=~/.nvm/current/bin:$PATH in my dotfiles.

[kijewskimateusz]

@reubano Thank you. Indeed problem was on my side as Node was not being autostarted with new terminal instance. Once fixed (used NVM), everything started working.

[mk7exe]

I followed the instruction mentioned by @reubano. I don't get the initial error anymore but I still can't run functions locally. I get "connect ECONNREFUSED 127.0.01:9091" error. I believe this is because func command does not run properly. There are three run commands related to azure functions in my ~/.nvm/versions/node/v14.17.6/bin which are azfun, azurefunctions, and func. Tried to run them in terminal and non of them did anything. There is no output and no error. Am I missing something here?

EDIT: I figured out the issue. I'm Mac newbie and didn't realize that I suppose to run these command in Rosetta terminal. I installed node 14.17.6 and followed steps given by @reubeno in a Rosetta terminal and now I can run azure functions locally. I guess we have to wait for the ARM64 version of azure function core tool.

[eduedix]

so I am wondering whether this issue is open or closed. Is arm64 supported at the moment?

[ejizba]

@eduedix It's supported in core tools, but not every language yet. See the comment at the top to track each language (Python and Java are the only ones left with outstanding work): #2834 (comment)

If you have comments/questions, the best place to follow up is on the language-specific github issue

[KantiCodes]

Hi All I think I've encountered some bizarre error why pyodbc works on my machine after installing through brew etc. but when I try to use it inside an azure function it doesn't work.

Im running on python 3.9.10 on Latest Azure functions 3x I previously had a project running on 2x and just recently created a new one and installed core tools 3.x and 4.x to try it out.

When I run a import pyodbc in a regular file and execute it within that venv everything works fiine. When I try to locally func host start a simple function that just imports pydobc that uses the same venv I get flat namespace '_SQLAllocHandle' error?

Any hints? This hasn't happened before.

Python 3.9.10
Core Tools Version: 3.0.4585 Commit hash: N/A (64-bit)
Function Runtime Version: 3.7.1.0
MacOS Monterey 12.4

Stack trace
For detailed output, run func with --verbose flag. [2022-07-08T14:34:32.170Z] Traceback (most recent call last): [2022-07-08T14:34:32.176Z] File "/opt/homebrew/Cellar/azure-functions-core-tools@3/3.0.4585/workers/python/3.9/OSX/X64/azure_functions_worker/bindings/shared_memory_data_transfer/file_accessor_unix.py", line 127, in _get_valid_mem_map_dirs [2022-07-08T14:34:32.177Z] os.makedirs(dir_path) [2022-07-08T14:34:32.177Z] File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 215, in makedirs [2022-07-08T14:34:32.177Z] makedirs(head, exist_ok=exist_ok) [2022-07-08T14:34:32.177Z] File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 225, in makedirs [2022-07-08T14:34:32.177Z] mkdir(name, mode) [2022-07-08T14:34:32.178Z] PermissionError: [Errno 1] Operation not permitted: '/dev/shm' [2022-07-08T14:34:32.951Z] Worker process started and initialized. [2022-07-08T14:34:33.349Z] Worker failed to function id 699ff0c7-015f-48c5-ba47-086547a9ca97. [2022-07-08T14:34:33.352Z] Result: Failure [2022-07-08T14:34:33.353Z] Exception: ImportError: dlopen(/Users/bartoszlachowicz/Desktop/CodeProjects/customers-slack-app/.venv/lib/python3.9/site-packages/pyodbc.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_SQLAllocHandle'. Troubleshooting Guide: https://aka.ms/functions-modulenotfound [2022-07-08T14:34:33.353Z] Stack: File "/opt/homebrew/Cellar/azure-functions-core-tools@3/3.0.4585/workers/python/3.9/OSX/X64/azure_functions_worker/dispatcher.py", line 305, in _handle__function_load_request [2022-07-08T14:34:33.353Z] func = loader.load_function( [2022-07-08T14:34:33.353Z] File "/opt/homebrew/Cellar/azure-functions-core-tools@3/3.0.4585/workers/python/3.9/OSX/X64/azure_functions_worker/utils/wrappers.py", line 42, in call [2022-07-08T14:34:33.353Z] raise extend_exception_message(e, message) [2022-07-08T14:34:33.353Z] File "/opt/homebrew/Cellar/azure-functions-core-tools@3/3.0.4585/workers/python/3.9/OSX/X64/azure_functions_worker/utils/wrappers.py", line 40, in call [2022-07-08T14:34:33.353Z] return func(*args, **kwargs) [2022-07-08T14:34:33.353Z] File "/opt/homebrew/Cellar/azure-functions-core-tools@3/3.0.4585/workers/python/3.9/OSX/X64/azure_functions_worker/loader.py", line 85, in load_function [2022-07-08T14:34:33.353Z] mod = importlib.import_module(fullmodname) [2022-07-08T14:34:33.353Z] File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module [2022-07-08T14:34:33.353Z] return _bootstrap._gcd_import(name[level:], package, level) [2022-07-08T14:34:33.353Z] File "<frozen importlib._bootstrap>", line 1030, in _gcd_import [2022-07-08T14:34:33.353Z] File "<frozen importlib._bootstrap>", line 1007, in _find_and_load [2022-07-08T14:34:33.353Z] File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked [2022-07-08T14:34:33.353Z] File "<frozen importlib._bootstrap>", line 680, in _load_unlocked [2022-07-08T14:34:33.353Z] File "<frozen importlib._bootstrap_external>", line 850, in exec_module [2022-07-08T14:34:33.353Z] File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed [2022-07-08T14:34:33.353Z] File "/Users/bartoszlachowicz/Desktop/CodeProjects/customers-slack-app/__app__/overview/__init__.py", line 9, in <module> [2022-07-08T14:34:33.353Z] import pyodbc [2022-07-08T14:34:33.353Z] .

[shivasanthosh]

In case you've already updated, here's the new work-a-round...

brew install [email protected]
brew uninstall azure-functions-core-tools@4
/opt/homebrew/bin/pip3 install grpcio
npm i -g [email protected]
cp /opt/homebrew/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so ~/.nvm/versions/node/v14.17.6/lib/node_modules/azure-functions-core-tools/bin/workers/python/3.9/OSX/X64/grpc/_cython/cygrpc.cpython-39-darwin.so

[ormu5]

The above worked, upon running command:

Found Python version 3.9.13 (python3).
Azure Functions Core Tools
Core Tools Version:       4.0.4483 Commit hash: N/A  (64-bit)
Function Runtime Version: 4.1.3.17473

OS Monterey 12.3.1
Node 14.18.3
Python 3.9.13