build: Ensure binaries built on Fedora 33 run on Fedoras 32 & 31

[debarshiray]

The /usr/bin/toolbox binary is not only used to interact with toolbox
containers and images from the host. It's also used as the entry point
of the containers by bind mounting the binary from the host into the
container. This means that the /usr/bin/toolbox binary on the host must
also work inside the container, even if they have different operating
systems.

In the past, this worked perfectly well with the POSIX shell
implementation because it got intepreted by whichever /bin/sh was
available.

The Go implementation also mostly worked so far because it's largely
statically linked, with the notable exception of the standard C
library. However, recently glibc-2.32, which is used by Fedora 33
onwards, added a new version of the pthread_sigmask symbol [1] as part
of the libpthread removal project:

  $ objdump -T /usr/bin/toolbox | grep GLIBC_2.32
  0000000000000000      DO *UND*        0000000000000000  GLIBC_2.32
    pthread_sigmask

This means that /usr/bin/toolbox binaries built against glibc-2.32 on
newer Fedoras pick up the latest version of the symbol and fail to run
against older glibcs in older Fedoras.

One way to fix this is to disable the use of any C code from Go by
using the CGO_ENABLED environment variable [2]. However, this can
negatively impact packages like "os/user" [3] and "net" [4], where the
more featureful glibc APIs will be replaced by more limited
equivalents written only in Go.

Instead, since glibc uses symbol versioning, it's better to tell the
Go toolchain to avoid linking against any symbols from glibc-2.32.

This was accomplished by a few linker tricks:

• The GNU ld linker's --wrap flag was used when building the Go code
  to divert pthread_sigmask invocations from Go to another function
  called __wrap_pthread_sigmask.

• A static library was added to provide this __wrap_pthread_sigmask
  function, which forwards calls to the actual pthread_sigmask API in
  glibc. This library itself was not linked with --wrap, and
  specifies the latest permissible version of the pthread_sigmask
  symbol from glibc for each architecture. Currently, the list of
  architectures covers the ones that Fedora builds for.

• The Go cmd/link linker was switched to external mode [5]. This
  ensures that the final object file containing all the Go code gets
  linked to the standard C library and the wrapper static library by
  the GNU ld linker for the --wrap flag to kick in.

Based on ideas from Ondřej Míchal.

[1] glibc commit c6663fee4340291c
https://sourceware.org/git/?p=glibc.git;a=commit;h=c6663fee4340291c

[2] https://golang.org/cmd/cgo/

[3] https://golang.org/pkg/os/user/

[4] https://golang.org/pkg/net/

[5] https://golang.org/src/cmd/cgo/doc.go

#529

[softwarefactory-project-zuul]

Build failed.

• shellcheck : SUCCESS in 1m 55s
• system-test-fedora-31 : SUCCESS in 3m 48s
• system-test-fedora-32 : SUCCESS in 3m 09s
• system-test-fedora-rawhide : FAILURE in 3m 47s

[debarshiray]

Here's a scratch build for Fedora 34: https://koji.fedoraproject.org/koji/taskinfo?taskID=49760645

I extracted the /usr/bin/toolbox binary from the Fedora 34 build (against glibc master) and was able to use it on a Fedora 31 host to create and run containers. I also built the code on Fedora 31 (against glibc-2.31) and used it on the same host to create and run containers.

[debarshiray]

I merged this because it seems that Rawhide is pretty turbulent right now (there's also containers/podman#7389), so it's proving difficult to get people to test.

Let's see how it holds up in the wild.

[juanje]

I just tried on Fedora 34 and I can confirm that it works.

[vagrant@ci-node-34 toolbox]$ toolbox create -r 32
Image required to create toolbox container.
Download registry.fedoraproject.org/f32/fedora-toolbox:32 (500MB)? [y/N]: y
Created container: fedora-toolbox-32
Enter with: toolbox enter --release 32
[vagrant@ci-node-34 toolbox]$ toolbox enter --release 32
/usr/bin/id: cannot find name for group ID 1000
⬢[vagrant@toolbox toolbox]$

The error before entering the container (/usr/bin/id: cannot find name for group ID 1000) is already known (#523), so can be ignored, but now I can enter into an earlier version.

The toolbox's version is the current one from master.

Thanks for the fix 😄

[debarshiray]

Thanks for the feedback, @juanje !