[rawhide] ext.config.toolbox failing

[jlebon]

https://jenkins-fedora-coreos.apps.ocp.ci.centos.org/job/fedora-coreos/job/fedora-coreos-fedora-coreos-pipeline/2083/

systemctl status kola-runext.service:
�� kola-runext.service
     Loaded: loaded (/etc/systemd/system/kola-runext.service; static)
     Active: failed (Result: exit-code) since Wed 2021-07-07 05:11:39 UTC; 1s ago
    Process: 1996 ExecStart=/usr/local/bin/kola-runext-test.sh (code=exited, status=1/FAILURE)
   Main PID: 1996 (code=exited, status=1/FAILURE)
        CPU: 165ms

Jul 07 05:10:11 qemu0 systemd[1]: Started kola-runext.service.
Jul 07 05:10:11 qemu0 kola-runext-test.sh[1997]: ++ seq 1 5
Jul 07 05:10:11 qemu0 kola-runext-test.sh[1996]: + for i in $(seq 1 5)
Jul 07 05:10:11 qemu0 kola-runext-test.sh[1996]: + machinectl shell core@ /bin/toolbox create --assumeyes
Jul 07 05:10:11 qemu0 machinectl[1998]: Connected to the local host. Press ^] three times within 1s to exit session.
Jul 07 05:11:35 qemu0 machinectl[1998]: Connection to the local host terminated.
Jul 07 05:11:35 qemu0 kola-runext-test.sh[2426]: ++ machinectl shell core@ /bin/toolbox list --containers
Jul 07 05:11:35 qemu0 kola-runext-test.sh[2427]: ++ grep --count fedora-toolbox-
Jul 07 05:11:35 qemu0 machinectl[2426]: Connected to the local host. Press ^] three times within 1s to exit session.
Jul 07 05:11:36 qemu0 machinectl[2426]: Connection to the local host terminated.
Jul 07 05:11:36 qemu0 kola-runext-test.sh[1996]: + [[ 1 -ne 1 ]]
Jul 07 05:11:36 qemu0 kola-runext-test.sh[1996]: + break
Jul 07 05:11:36 qemu0 kola-runext-test.sh[2493]: ++ grep --count fedora-toolbox-
Jul 07 05:11:36 qemu0 kola-runext-test.sh[2492]: ++ machinectl shell core@ /bin/toolbox list --containers
Jul 07 05:11:36 qemu0 machinectl[2492]: Connected to the local host. Press ^] three times within 1s to exit session.
Jul 07 05:11:37 qemu0 machinectl[2492]: Connection to the local host terminated.
Jul 07 05:11:37 qemu0 kola-runext-test.sh[1996]: + [[ 1 -ne 1 ]]
Jul 07 05:11:37 qemu0 kola-runext-test.sh[1996]: + ok toolbox create
Jul 07 05:11:37 qemu0 kola-runext-test.sh[1996]: + echo ok toolbox create
Jul 07 05:11:37 qemu0 kola-runext-test.sh[1996]: ok toolbox create
Jul 07 05:11:37 qemu0 kola-runext-test.sh[1996]: + machinectl shell core@ /bin/toolbox run touch ok_toolbox
Jul 07 05:11:37 qemu0 machinectl[2557]: Connected to the local host. Press ^] three times within 1s to exit session.
Jul 07 05:11:39 qemu0 kola-runext-test.sh[2557]: Error: invalid entry point PID of container fedora-toolbox-35
Jul 07 05:11:39 qemu0 machinectl[2557]: Connection to the local host terminated.
Jul 07 05:11:39 qemu0 kola-runext-test.sh[1996]: + [[ ! -f /home/core/ok_toolbox ]]
Jul 07 05:11:39 qemu0 kola-runext-test.sh[1996]: + fatal 'Could not run a simple command inside a toolbox'
Jul 07 05:11:39 qemu0 kola-runext-test.sh[1996]: + echo 'Could not run a simple command inside a toolbox'
Jul 07 05:11:39 qemu0 kola-runext-test.sh[1996]: Could not run a simple command inside a toolbox
Jul 07 05:11:39 qemu0 kola-runext-test.sh[1996]: + exit 1
Jul 07 05:11:39 qemu0 systemd[1]: kola-runext.service: Main process exited, code=exited, status=1/FAILURE
Jul 07 05:11:39 qemu0 systemd[1]: kola-runext.service: Failed with result 'exit-code'.
--- FAIL: ext.config.toolbox (146.17s)
        harness.go:661: kolet failed: : kolet run-test-unit failed: Error: Unit kola-runext.service exited with code 1
2021-07-07T05:11:40Z cli: Unit kola-runext.service exited with code 1: Process exited with status 1

ext.config.toolbox.zip

[travier]

[core@cosa-devsh ~]$ podman start -a fedora-toolbox-35       
toolbox: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by toolbox)

[core@cosa-devsh ~]$ podman run --rm -ti registry.fedoraproject.org/fedora-toolbox:35     
sh-5.1#

[core@cosa-devsh ~]$ toolbox --version
toolbox version 0.0.99.2

[core@cosa-devsh ~]$ rpm -qa | grep glibc
...
glibc-2.33.9000-37.fc35.x86_64
...

Will have to check if it is reproduce-able on classic Rawhide.

[travier]

containers/toolbox#821

[travier]

We have to skip this one until it is fixed upstream

[dustymabe]

FWIW - I just ran the ext.config.toolbox against rawhide and it passed.

[travier]

It's now probably working as the Fedora toolbox (rawhide) container image has been updated with the new glibc.

[travier]

So this can probably be closed on our side as upstream is tracking it and I filed a request in containers/toolbox#832 for a long term fix.