Newer libcxx versions obsoleting old clang versions

[h-vetinari]

This is a long-standing problem. Upstream libcxx generally only supports the last two released clang versions (during development), meaning libcxx 18 only officially supports clang 16 & 17 (and of course 18 once released).

As libcxx is generally highly forward-compatible, we only ever have one libcxx version in conda-forge, which tends to be the newest. Strictly speaking, the C++ compilers on osx only do a run-export on their own version, so clangxx_ox-64 17.0.6 will produce packages that have a libcxx >=17 constraint. Once we update our compiler stack to a new LLVM version, this constraint spreads pretty quickly across the ecosystem, so that a requirement for an older libcxx is effectively equivalent to freezing the set of packages/versions that remain co-installable.

We do test our libcxx against older clangs, and for example, libcxx 17 passes basic testing all the way down to clang 11 (and even 10 still worked). That's for the most basic include tests, i.e. it needs to be said that there are almost certainly bugs in other parts of libcxx in that combination (as libcxx 17 only supports down to clang 15 officially, and usually removes all workarounds that are necessary for older clangs). However, it seems this setup has worked +/- acceptably for affected packages so far.

For libcxx 18, we ended up dropping support for clang<15, after checking the status of some known packages that are still relying on older clang:

Decide what to do (if anything) about losing support for clang<15 -> drop

• root moved on to clang 16
• cling is not maintained anymore (quoth @SylvainCorlay)
• llvmlite still at llvm 14, but that does not conflict with newer clang/libcxx

This is not yet an immediate concern because our default compilers aren't on clang 18 yet, but we'll run into the same situation once that happens, so this is to serve as an announcement & for discussion; c.f. also #309, where I commented:

This is OK by me. It should also be possible though to stay on clang 16 in cppyy-cling; you're not forced to follow the global pinning, and you have very good reason (currently) not to. One issue with that is of course that other packages being installed today pick up the libcxx>=17 requirement, so this PR is necessary for staying co-installable with packages currently being built.

Looking forward, root already has a 16.x version (and I see you've tried it in that PR, but cppyy doesn't support it yet), which is good, because when we move to LLVM 18, clang 13 won't work with libcxx 18 anymore (conda-forge/libcxx-feedstock@a09b2d6). We can maybe forestall the change for a while (or keep using libcxx 17 with the clang 18 compilers for a bit), but past some point, it'll always come down to either: switch to newer clang versions, or live with old packages in the environment.

We try to keep libcxx compatible with older clang versions much longer than upstream LLVM, but there are limits to how far we can push this, and ultimately the tide doesn't stop rising. 🥲

Originally posted by @h-vetinari in #309 (review)

CC @conda-forge/root @conda-forge/cling @conda-forge/cppyy

[isuruf]

Best option is to split libcxx (with headers and libraries) into libcxx (shared library only) and libcxx-devel (headers, static) and make clangxx depend on libcxx-devel.

[h-vetinari]

Best option is to [...]

I have a PR that does this: conda-forge/libcxx-feedstock#183. Once merged, I can backport this a few versions (libcxx 16/17 should be enough AFAIU, then we can also adopt that in the compiler-activation; if necessary we can backport further, e.g. until libcxx 13).

[h-vetinari]

PR to switch clangxx to using libcxx-devel: #311

If that's unoffensive I'll rebuild 17 & 16 + the compiler activation as well.

[isuruf]

How does libcxx.a work with cross-compiling?

[h-vetinari]

How does libcxx.a work with cross-compiling?

As before? For now (once we merge the open PR to update the compiler activation to use libcxx-devel) there should be no functional change to how things were. The content of a build environment with {{ compiler("cxx") }} stays the same, and so does the host environment (to the best of my understanding), because that activation PR uses a strong run-export on libcxx-devel for clangxx_{{ target }}.

Or am I misunderstanding your point? I guess we could also mirror the GCC setup and go for libcxx-devel_{{ target }} if you prefer? But mechanically I don't see right now where that would make a difference.

[h-vetinari]

and so does the host environment (to the best of my understanding), because that activation PR uses a strong run-export on libcxx-devel for clangxx_{{ target }}.

Hm. Never mind, I had misremembered that. So I think we're running into the "host-exports" situation again, because we'd want to distinguish what we host-export (libcxx-devel) and what we run-export ( libcxx).

[isuruf]

Since libc++.a appeared in libcxx, cross-compiling + static worked correctly. With the suggestion I made it wouldn't. Maybe we should keep the static archives in libcxx as previously.

[h-vetinari]

Maybe we should keep the static archives in libcxx as previously.

I think that's the easiest solution TBH.

[h-vetinari]

It'll also be easy to do with the work that happened so far (currently the actual content split hasn't happened yet to get things off the ground, c.f. conda-forge/libcxx-feedstock#187). In that vein, could I ask you to please review #311 & conda-forge/clang-compiler-activation-feedstock#138?

[h-vetinari]

cling is not maintained anymore

Tangent: cling is still alive after all 🥳 (just had a new release that's based on LLVM 16)

[h-vetinari]

Good news! After a bunch of effort, clang>=16 should now keep working far longer with newer libcxx than previous setups, because clang X will pull in the headers for libcxx X as well (now called libcxx-devel). That way only the library is newer, and we will not run into problems like the libcxx headers being incompatible with old clangs. More details:

If we're using the libcxx-devel 16.x headers to compile something, while having libcxx 20 in the environment, that does run the risk of running into minor ABI differences (and potentially missing symbols, if some got removed in later libcxx), but that's still essentially the same standpoint as we have already, i.e. "libcxx is essentially ABI-stable (and the enormous effort to properly handle any differences isn't worth it, given the relative rarity of breaks)"

Thanks for the idea & the reviews @isuruf!

---

For context, the following PRs were necessary to do this. As such, I'm not planning to backport this further than clang 16.

• Split off -devel package libcxx-feedstock#183
• [17.x] Split off -devel package libcxx-feedstock#184
• [16.x] Split off -devel package libcxx-feedstock#185
• Also pin and version by libcxx libcxx-testing-feedstock#9
• Also build for libcxx 17 libcxx-testing-feedstock#10
• Also build for libcxx 18; add an unconstrained build libcxx-testing-feedstock#11
• Bootstrap libcxx-devel as wrapper around libcxx libcxx-feedstock#187
• [17.x] Bootstrap libcxx-devel as wrapper around libcxx libcxx-feedstock#188
• [16.x] Re-enable downstream testing libcxx-feedstock#189
• [main] switch to using libcxx-devel #311
• [17.x] switch to using libcxx-devel #312
• [16.x] switch to using libcxx-devel #313
• depend on libcxx-devel where necessary libcxx-testing-feedstock#12
• [17.x] depend on libcxx-devel where necessary libcxx-testing-feedstock#13
• [16.x] depend on libcxx-devel where necessary libcxx-testing-feedstock#14
• Split off headers into libcxx-devel libcxx-feedstock#190
• [17.x] Split off headers into libcxx-devel libcxx-feedstock#191
• [16.x] Split off headers into libcxx-devel libcxx-feedstock#192
• add missing pin to libcxx-devel; fix libcxx run-export #315
• [17.x] add missing pin to libcxx-devel; fix libcxx run-export #316
• [16.x] add missing pin to libcxx-devel; fix libcxx run-export #317
• use libcxx-devel clang-compiler-activation-feedstock#138

[tadejsv]

@h-vetinari I think the name "libcxx-devel" is unfortunate. It makes it sound like this is some package only needed for development of libc++, and gives no hint that it contains the headers without which you can not use libcxx at all.

I just ran into this issue - I could not compile anything using just libcxx, and it took a while to figure out why this was the case

[h-vetinari]

This is a standard pattern in several distributions. We haven't adopted it universally in conda-forge (which is perhaps why it's a bit confusing), but we're definitely adding more and more -devel packages where this makes sense. If you feel this should be better documented, could you open a PR to https://github.com/conda-forge/conda-forge.github.io?

[tadejsv]

@h-vetinari I'm not sure how/where this could be better documented - for me the issue was there was no "migration guide" that would tell me that from 18.x I have to switch from libcxx to libcxx-devel. Anyways, for me this issue is solved, so I don't have any motivation to push for more general changes.