Prepare releases, including pre-built binaries

[andrewdavidmackenzie]

Fixes #207

Using cargo-dist to do the builds for selected targets and then create the release, upload sourc etarball as usual plus binary assets.

Adds a short RELEASING.md document to document the process.

Here is the first attempt at a release build in my own repo:

https://github.com/andrewdavidmackenzie/wild/actions/runs/12099771708

It's lacking something installed for musl so fails, which I'll try and fix.

[andrewdavidmackenzie]

warning: [email protected]: Compiler family detection failed due to error: ToolNotFound: Failed to find tool. Is musl-g++ installed?

on my machine I (seem to, never knew!!) have musl-gcc installed. I guess it needs that?

[davidlattimore]

warning: [email protected]: Compiler family detection failed due to error: ToolNotFound: Failed to find tool. Is musl-g++ installed?

on my machine I (seem to, never knew!!) have musl-gcc installed. I guess it needs that?

Is this for running tests? There are certainly some tests that check that static linking with musl works

[davidlattimore]

Do you want to try this out on your fork to see if it works as expected? You might need to temporarily enable actions on your fork, since I think they're disabled by default.

[andrewdavidmackenzie]

It's trying to do the release build in my forks actions. I'll try and get it passing and a release created in my fork, before removing draft/wip status

…

On Sun, Dec 1, 2024, 11:04 AM David Lattimore ***@***.***> wrote: Do you want to try this out on your fork to see if it works as expected? You might need to temporarily enable actions on your fork, since I think they're disabled by default. — Reply to this email directly, view it on GitHub <#209 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKF4LCRSFCXCZXRH66RYFD2DLNL3AVCNFSM6AAAAABSY6DK2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBZGY3DONZSGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[andrewdavidmackenzie]

cargo build --target x86_64-unknown-linux-musl

fails on my local machine with the same error.

error occurred: Failed to find tool. Is musl-g++ installed?

I guess you installed it manually, but I'm confused why I don't see it in ci.yml

[andrewdavidmackenzie]

I see symbolic-demangle is referenced in cackle.yml

[andrewdavidmackenzie]

Without understanding all this, it seems that ci.yml installs bubblewrap

      - run: sudo apt install bubblewrap lld

which is referenced from cackle.yml as the sandbox...

[andrewdavidmackenzie]

symbolic-demangle doesn't support building for musl. I asked them about that:

getsentry/symbolic#880

I see this issue against them:
rust-lang/cc-rs#105

[andrewdavidmackenzie]

A bit of a rat-hole...

We could dodge the whole issue and reduce the binaries we would build as part of a release...

[davidlattimore]

It looks like symbolic_demangle is only currently used by linker-diff. So if we're doing release builds of just the linker itself, then we shouldn't need it. Which I guess raises the question of which binaries we should include in the releases? My inclination is that we should probably just have the main wild binary - or if we do distribute linker-diff, that it should probably be a separate download. It's useful for diagnostics, but shouldn't be necessary for most users.

It's also fine to to not distribute musl binaries at this stage if that makes things easier.

[davidlattimore]

Another possible reason to not distribute a musl-based build is that it people might use it not realising that it doesn't give optimal performance. I just benchmarked wild built with musl against wild built with glibc and the musl version was 30% slower. I think this is probably a mix of a slower allocator and slower memcpy, memcmp etc. I then tried a musl build with a faster custom allocator (mimalloc) and it was then only 10% slower than glibc. This is in contrast to using mimalloc with glibc, where it seems to make no difference to performance. i.e. for the way wild uses the allocator, the default glibc allocator is fine performance wise, but not the musl allocator.

There's precedent for using a different allocator just for musl builds. I think I saw that ripgrep does that.

[andrewdavidmackenzie]

Dropping musl is simple and would avoid this problem, even if releasing all binaries.

Releasing only linker (gnu and musl) would also avoid it.

Apart from for musl based systems like alpine, what other reason to release musl binary?

[davidlattimore]

Apart from for musl based systems like alpine, what other reason to release musl binary?

One common reason for building a musl-based binary is to have a statically linked binary, which can then work on pretty much any Linux system regardless of libc version. However this isn't really an advantage of musl so much as an advantage of static linking. It's possible to statically link with glibc and to dynamically link with musl.

[andrewdavidmackenzie]

I tried reducing build to just wild, but it has a dev-dependency on linker-diff, and I think that's triggering the linker-diff build on musl, which fails.

So then I removed musl also.

If we want to re-add musl, even if just for wild, we'll need to resolve the build on linker diff, or find a way to avoid it in release builds of the wild binary (tests aren't run, so strictly speaking it's not needed but AFAIK, cargo doesn't distinguish between build dependencies and test dependencies in the dev-dependencies list...)

[andrewdavidmackenzie]

Successfull release build in GH Action in my repo: https://github.com/andrewdavidmackenzie/wild/actions/runs/12117386105

Here is the release it created: https://github.com/andrewdavidmackenzie/wild/releases

That can be manually edited by someone, and GH offers a "Generate release notes" button that will put in all the commits since the last release - and then you can edit that if you wish to clean it up.

Once you have a look and we merge this, I might delete that release to avoid confusion.

[davidlattimore]

Looks good to me, although there's merge conflicts

[andrewdavidmackenzie]

Fixed