Apple Silicon

[isuruf]

• Ask for support for uploading to anaconda.org
• Build LLVM 11.0.0.rc1 and the stack on osx-64 and upload to conda-forge/label/llvm_rc - osx-arm64 clang-compiler-activation-feedstock#29
  • Update to LLVM 11.0.0 llvmdev-feedstock#96
  • Update to LLVM 11.0.0.rc1 clangdev-feedstock#106
  • ENH 11 rc1 libcxx-feedstock#66
  • ENH 11 rc1 openmp-feedstock#39
  • osx-arm64 support compiler-rt-feedstock#23
  • ENH 11 rc1 lld-feedstock#19
• osx-arm64 target cctools-and-ld64-feedstock#23
• TAPI TBD-v4. Do one of,
  • Make cctools use the system linker from osx-64
  • Wait for Apple to open source new TAPI
  • Patch cctools to use libLLVMTextAPI
  • Use lld
• New conda release (Support for Apple silicon conda/conda#10128) - osx-arm64 support conda-feedstock#123
• New conda-build release (Set build_platform for aid in cross compiling conda/conda-build#4005, Add Apple silicon support conda/conda-build#4004, Fix relocation when cross compiling conda/conda-build#3995, Use host_platform instead of sys.platform conda/conda-build#3997) - osx-arm64 support conda-build-feedstock#146
• Update pinnings osx-arm64 support conda-forge-pinning-feedstock#727
• Update constructor
• Update automake macOS arm64 patches automake-feedstock#26
• Update libtool osx-arm64 support libtool-feedstock#19
• Update ci-setup MacOSX11.0.sdk support conda-forge-ci-setup-feedstock#117
• codesigning
  • update conda-feedstock with Run codesign on osx-arm64 conda/conda#10260
  • update conda-standalone-feedstock with Run codesign on osx-arm64 conda/conda#10260
  • update cctools-and-ld64-feedstock with codesign all arm64 binaries tpoechtrager/cctools-port#89
  • patch install_name_tool in cctools-port to run ldid

All of the above are independent tasks.

I could use some help in task 2 if anybody is up for it.

cc @conda-forge/core, @chenghlee

[beckermr]

We had another issue open for this @isuruf. Can you please close it or combine it here?

It was called arm fretting or some such thing.

[beckermr]

I could use some help in task 2 if anybody is up for it.

Which one is task 2? The new LLVM release?

[isuruf]

Yes

[jjhelmus]

• Ask for support for uploading to anaconda.org

I was able to upload and download osx-arm64 packages to anaconda.org without problem although I have not gotten to the point where I can do from directly from an Apple Silicon machine yet. An example: https://anaconda.org/jjhelmus/zlib/files?channel=osx-arm64

[jakirkham]

One question I've been wondering about, do we want to treat this as a new architecture or do we want to build packages that support both architectures (x86 and arm64)? It sounded like the latter is also possible. So wanted to check.

[jjhelmus]

A quick note on anaconda.org support. Channels backed by a CDN (e.g. conda-forge) will NOT have osx-arm64 packages in the CDN served repository. That will require a small change in the logic that syncs the packages with the CDN.

[isuruf]

One question I've been wondering about, do we want to treat this as a new architecture or do we want to build packages that support both architectures (x86 and arm64)?

Yes definitely as a new architecture.

It sounded like the latter is also possible.

Not without hacking build systems to use lipo.

[isuruf]

As to the linker issue, I tried several options and none of them work except using the system linker from xcode 11/12.

1. lld's old Mach-O backend is not maintained anymore.
2. lld's new Mach-O backend has less features than we need.
3. LLVM's TextAPI library and Apple's TAPI library have incompatible interfaces.

[beckermr]

good work!

[isuruf]

First C package PR: conda-forge/zlib-feedstock#38
First C++ package PR: conda-forge/libcxx-feedstock#67

@jjhelmus, can you check that the zlib package works fine?
@conda-forge/bot, can we start a migration with python and its dependencies as a start?

[CJ-Wright]

Sure, what does the migration need to encompass? Is this similar to the arch rebuild or more like the pypy rebuild?

[isuruf]

It's exactly like the arch rebuild, but with a couple of small parameter changes.

1. add the following in conda-forge.yml

build_platform:
  osx_arm64: osx_64
test_on_native_only: True

2. Use the set of packages from osx_arm64.txt (Not arch_rebuild.txt)

[CJ-Wright]

Let me see what I can do.

[jakirkham]

Would that build both the x86_64 packages and arm64 packages?

[CJ-Wright]

@isuruf is there a help team to contact?

[isuruf]

I can create one with me. conda-forge/help-osx-arm64. If anybody else would like to join, please let me know. (Or add yourself if you are in core)

[beckermr]

@isuruf i'd be best to add it as a subteam of staged-recipes

[isuruf]

I'm not sure how to do that. I already created one https://github.com/orgs/conda-forge/teams/help-osx-arm64. Can you fix it or delete it and recreate it?

[beckermr]

done!

[CJ-Wright]

Is there a list of packages to avoid?

[CJ-Wright]

@isuruf in zlib why was linux_armv71 removed from the conda-forge.yml?

[beckermr]

Is there a list of packages to avoid?

We can't build anything that needs fortran for starters.

[isuruf]

Is there a list of packages to avoid?

build and run requirements should be avoided. Only host. build is from osx-64 platform and we don't run tests.

in zlib why was linux_armv71 removed from the conda-forge.yml?

Because we've removed the docker image for linux_armv7l and the new conda-forge-ci-setup files are not updated for that platform.

[CJ-Wright]

@beckermr can you give me some context around why this clause is needed?

[isuruf]

We can't build anything that needs fortran for starters.

Yes. I don't know how the migrator would handle that node, but if it doesn't stop when it can't build gfortran, an explicit condition needs to be added.

[CJ-Wright]

I'm interested in if I need to duplicate this

[isuruf]

@CJ-Wright, those are all build dependencies and build dependencies should not be looked at for osx-arm64 migration.

[beckermr]

@CJ-Wright, those are all build dependencies and build dependencies should not be looked at for osx-arm64 migration.

Except if you need a fortran compiler in build. Then the package cannot be migrated.

@beckermr can you give me some context around why this clause is needed?

That is a from a change in smithy. We moved the names of the ci support files on linux from linux_* to linux_64_*. So for some repos that have not been rerendered, we only infer the 64 if it is not given.

[isuruf]

A quick note on anaconda.org support. Channels backed by a CDN (e.g. conda-forge) will NOT have osx-arm64 packages in the CDN served repository. That will require a small change in the logic that syncs the packages with the CDN.

Thanks @jjhelmus for letting us know. Saw this issue in conda-forge/tk-feedstock#43

[isuruf]

Currently blocked by conda/conda-build#3155 at libssh2.

[jjhelmus]

I pushed a few of the right buttons and the CDN now includes osx-arm64 packages. To verify:

wget https://conda-static.anaconda.org/conda-forge/osx-arm64/repodata.json

I believe that when the repodata patch package include instructions for patching this subdir (empty instructions are fine) then current_repodata.json will be availabled.

[beckermr]

thank you @jjhelmus!

[jjhelmus]

conda-forge/conda-forge-repodata-patches-feedstock/pull/86

[isuruf]

Could use some help in fixing some issues

1. In UpdateConfigSub mini-migrator don't add libtool if the recipe is libtool regro/cf-scripts#1222
2. Remove conda inspect linkages calls in a mini-migrator regro/cf-scripts#1220
3. In OSX Arm migrator selectively add test_on_native_only regro/cf-scripts#1221

cc @erykoff

[jakirkham]

If people are looking to try things out and provide feedback, have we reached where they could get started? If so, do we have some instructions for them to follow? How should they share feedback?

[bollwyvl]

Thanks for hustling so hard on all this, folks! We're now down to where most folk seem pretty happy. Had a couple folks asking on gitter/elsewhere on when will package X be available. Seems like something on knowledge_base.rst would give them someplace to land (and hopefully follow-up with testing on a native box). PR incoming!