Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker runner fails if package has a subpackage #1254

Open
smoser opened this issue Jun 5, 2024 · 4 comments
Open

docker runner fails if package has a subpackage #1254

smoser opened this issue Jun 5, 2024 · 4 comments

Comments

@smoser
Copy link
Contributor

smoser commented Jun 5, 2024

This is from wolfi-dev/os at 88a385e2a4c462cbf1a527df15ae99fe70ae9d48
2 things will make this work:

  1. not using --runner=docker
  2. removing the subpackages in the pkg-test.yaml
$ cat pkg-test.yaml
package:
  name: pkg-test
  version: 1
  epoch: 1
  description: pkg test
  copyright:
    - license: MIT

environment:
  contents:
    packages:
      - build-base
      - busybox

pipeline:
  - runs: |
      mkdir -p ${{targets.contextdir}}/usr/bin/
      ln -s true ${{targets.contextdir}}/usr/bin/pkg-test

subpackages:
  - name: "${{package.name}}-sub1"
    description: "subpackage"
    pipeline:
      - runs: |
          mkdir -p ${{targets.contextdir}}/usr/bin/
          ln -s true ${{targets.contextdir}}/usr/bin/pkg-test-sub1

update:
  enabled: false

$ make MELANGE_EXTRA_OPTS="--runner=docker" package/pkg-test
SOURCE_DATE_EPOCH= /usr/bin/melange build ./pkg-test.yaml
   --repository-append /home/user/src/wolfi-os/packages
   --keyring-append local-melange.rsa.pub --signing-key local-melange.rsa
   --arch x86_64 --env-file build-x86_64.env
   --namespace wolfi --generate-index false  --pipeline-dir ./pipelines/
   --runner=docker -k https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
   -r https://packages.wolfi.dev/os --source-dir ./pkg-test 
   --log-policy builtin:stderr,packages/x86_64/buildlogs/pkg-test-1-r1.log
2024/06/05 02:25:46 WARN SOURCE_DATE_EPOCH is specified but empty, setting it to 1970-01-01 00:00:00 +0000 UTC
2024/06/05 02:25:46 INFO melange is building:
2024/06/05 02:25:46 INFO   configuration file: ./pkg-test.yaml
2024/06/05 02:25:46 INFO   workspace dir: /home/user/tmp/melange-workspace-1156798952
2024/06/05 02:25:46 INFO evaluating pipelines for package requirements
2024/06/05 02:25:46 INFO populating workspace /home/user/tmp/melange-workspace-1156798952 from ./pkg-test
2024/06/05 02:25:46 INFO --cache-dir ./melange-cache/ not a dir; skipping
2024/06/05 02:25:46 INFO building workspace in '/home/user/tmp/melange-guest-4168734015' with apko
2024/06/05 02:25:46 INFO image configuration:
2024/06/05 02:25:46 INFO   contents:
2024/06/05 02:25:46 INFO     repositories: []
2024/06/05 02:25:46 INFO     keyring:      []
2024/06/05 02:25:46 INFO     packages:     [build-base busybox]
2024/06/05 02:25:46 INFO   accounts:
2024/06/05 02:25:46 INFO     runas:
2024/06/05 02:25:46 INFO     users:
2024/06/05 02:25:46 INFO       - uid=1000(build) gid=1000
2024/06/05 02:25:46 INFO     groups:
2024/06/05 02:25:46 INFO       - gid=1000(build) members=[build]
2024/06/05 02:25:46 INFO installing ca-certificates-bundle (20240315-r3)
...
2024/06/05 02:25:47 INFO installing busybox (1.36.1-r10)
2024/06/05 02:25:48 INFO built image layer tarball as /home/user/tmp/apko-temp-3010928227/apko-x86_64.tar.gz
2024/06/05 02:25:48 INFO using /home/user/tmp/apko-temp-3010928227/apko-x86_64.tar.gz for image layer
2024/06/05 02:25:48 INFO OCI layer digest: sha256:e6358f3ea0513327f7e5807da64c19d54007c92f7639309076eb5aed4487a524
2024/06/05 02:25:48 INFO OCI layer diffID: sha256:efc90902a86e357732e47f6a42170a38ee76994aebd60029b62a2411fe254
c2c
2024/06/05 02:25:48 INFO saving OCI image locally: apko.local/cache:5e0faeb92b7f757caa5c36f60190348a3dd1aa8dcaf93efa46ab8686352e0385
2024/06/05 02:25:48 WARN skipping local domain tagging 
   apko.local/cache:5e0faeb92b7f757caa5c36f60190348a3dd1aa8dcaf93efa46ab8686352e0385 as index.docker.io/library/melange:latest
2024/06/05 02:25:48 INFO ImgRef = apko.local/cache:5e0faeb92b7f757caa5c36f60190348a3dd1aa8dcaf93efa46ab8686352e0385
2024/06/05 02:25:50 INFO running pipeline for subpackage pkg-test-sub1
2024/06/05 02:25:50 INFO retrieving workspace from builder: c33223c3c7c8c109fba7c04d8efd2cec4cc0dca77f3f6dd7d51bd69ab881c05b
2024/06/05 02:25:50 INFO retrieved and wrote post-build workspace to: /home/user/tmp/melange-workspace-1156798952
2024/06/05 02:25:50 INFO running package linters for pkg-test
2024/06/05 02:25:50 INFO running package linters for pkg-test-sub1
2024/06/05 02:25:50 INFO generating SBOM for subpackage pkg-test-sub1
2024/06/05 02:25:50 INFO pod c33223c3c7c8c109fba7c04d8efd2cec4cc0dca77f3f6dd7d51bd69ab881c05b terminated
2024/06/05 02:25:50 ERRO ERROR: failed to build package. the build environment has been preserved:
2024/06/05 02:25:50 INFO   workspace dir: /home/user/tmp/melange-workspace-1156798952
2024/06/05 02:25:50 INFO   guest dir: /home/user/tmp/melange-guest-4168734015
2024/06/05 02:25:50 INFO error during command execution: failed to build package: writing SBOMs:
   writing sbom to disk: creating SBOM directory in apk filesystem:
     mkdir /home/user/tmp/melange-workspace-1156798952/melange-out/pkg-test-sub1/var: permission denied
make[1]: *** [Makefile:150: packages/x86_64/pkg-test-1-r1.apk] Error 1
make[1]: Leaving directory '/home/user/src/wolfi-os'
make: *** [Makefile:140: package/pkg-test] Error 2
@smoser
Copy link
Contributor Author

smoser commented Jun 5, 2024

This also fails on ubuntu 24.04 (did not test elsewhere).

@jonjohnsonjr
Copy link
Contributor

Doesn't fail for me on macos :/

@smoser
Copy link
Contributor Author

smoser commented Jul 1, 2024

I'm not sure if I reported this in user-error , old version of melange (why did I not put the melange version in the report?) or what.
Anyway, it looks fixed to me now.

$ which melange
/home/smoser/src/melange/melange

$ melange version
  __  __   _____   _          _      _   _    ____   _____
 |  \/  | | ____| | |        / \    | \ | |  / ___| | ____|
 | |\/| | |  _|   | |       / _ \   |  \| | | |  _  |  _|
 | |  | | | |___  | |___   / ___ \  | |\  | | |_| | | |___
 |_|  |_| |_____| |_____| /_/   \_\ |_| \_|  \____| |_____|
melange

GitVersion:    v0.10.0-10-gecbc334
GitCommit:     ecbc33415ecf3fb579687254beeafc0c9aecab94
GitTreeState:  clean
BuildDate:     '2024-06-30T21:21:55Z'
GoVersion:     go1.22.4
Compiler:      gc
Platform:      linux/amd64

$ make MELANGE_EXTRA_OPTS="--runner=docker" package/pkg-test
yamlfile is pkg-test.yaml
Building package pkg-test with version pkg-test-1-r1 from file pkg-test.yaml
make yamlfile=pkg-test.yaml pkgname=pkg-test packages/x86_64/pkg-test-1-r1.apk
make[1]: Entering directory '/home/smoser/src/wolfi/os'
make[1]: 'packages/x86_64/pkg-test-1-r1.apk' is up to date.
make[1]: Leaving directory '/home/smoser/src/wolfi/os'
$ rm -Rf packages/x86_64/
$ make MELANGE_EXTRA_OPTS="--runner=docker" package/pkg-test
yamlfile is pkg-test.yaml
Building package pkg-test with version pkg-test-1-r1 from file pkg-test.yaml
make yamlfile=pkg-test.yaml pkgname=pkg-test packages/x86_64/pkg-test-1-r1.apk
make[1]: Entering directory '/home/smoser/src/wolfi/os'
@SOURCE_DATE_EPOCH= /home/smoser/src/melange/melange build pkg-test.yaml --repository-append /home/smoser/src/wolfi/os/packages --keyring-append local-melange.rsa.pub --signing-key local-melange.rsa --arch x86_64 --env-file build-x86_64.env --namespace wolfi --generate-index false  --pipeline-dir ./pipelines/ --runner=docker -k https://packages.wolfi.dev/os/wolfi-signing.rsa.pub -r https://packages.wolfi.dev/os
2024/07/01 09:27:00 WARN SOURCE_DATE_EPOCH is specified but empty, setting it to 1969-12-31 19:00:00 -0500 EST
2024/07/01 09:27:00 INFO melange is building:
2024/07/01 09:27:00 INFO   configuration file: pkg-test.yaml
2024/07/01 09:27:00 INFO   workspace dir: /tmp/melange-workspace-4268561140
2024/07/01 09:27:00 INFO evaluating pipelines for package requirements
2024/07/01 09:27:00 INFO --cache-dir ./melange-cache/ not a dir; skipping
2024/07/01 09:27:00 INFO populating workspace /tmp/melange-workspace-4268561140 from ./pkg-test/
2024/07/01 09:27:00 INFO building workspace in '/tmp/melange-guest-503966388' with apko
2024/07/01 09:27:00 INFO setting apk repositories: [/home/smoser/src/wolfi/os/packages https://packages.wolfi.dev/os]
2024/07/01 09:27:00 INFO image configuration:
2024/07/01 09:27:00 INFO   contents:
2024/07/01 09:27:00 INFO     build repositories: []
2024/07/01 09:27:00 INFO     runtime repositories: []
2024/07/01 09:27:00 INFO     keyring:      []
2024/07/01 09:27:00 INFO     packages:     [build-base busybox]
2024/07/01 09:27:00 INFO   accounts:
2024/07/01 09:27:00 INFO     runas:  
2024/07/01 09:27:00 INFO     users:
2024/07/01 09:27:00 INFO       - uid=1000(build) gid=1000
2024/07/01 09:27:00 INFO     groups:
2024/07/01 09:27:00 INFO       - gid=1000(build) members=[build]
2024/07/01 09:27:00 INFO auth configured for: []
2024/07/01 09:27:01 INFO installing ca-certificates-bundle (20240315-r4)
2024/07/01 09:27:01 INFO installing wolfi-baselayout (20230201-r12)
2024/07/01 09:27:01 INFO installing ld-linux (2.39-r6)
2024/07/01 09:27:01 INFO installing glibc-locale-posix (2.39-r6)
2024/07/01 09:27:01 INFO installing glibc (2.39-r6)
2024/07/01 09:27:01 INFO installing make (4.4.1-r3)
2024/07/01 09:27:01 INFO installing libgcc (13.2.0-r7)
2024/07/01 09:27:01 INFO installing libstdc++ (13.2.0-r7)
2024/07/01 09:27:01 INFO installing binutils (2.42-r2)
2024/07/01 09:27:01 INFO installing pkgconf (2.2.0-r2)
2024/07/01 09:27:01 INFO installing posix-cc-wrappers (1-r3)
2024/07/01 09:27:01 INFO installing libgo (13.2.0-r7)
2024/07/01 09:27:01 INFO installing gmp (6.3.0-r2)
2024/07/01 09:27:01 INFO installing mpfr (4.2.1-r4)
2024/07/01 09:27:01 INFO installing mpc (1.3.1-r4)
2024/07/01 09:27:01 INFO installing isl (0.26-r4)
2024/07/01 09:27:01 INFO installing zlib (1.3.1-r3)
2024/07/01 09:27:01 INFO installing libstdc++-dev (13.2.0-r7)
2024/07/01 09:27:01 INFO installing libatomic (13.2.0-r7)
2024/07/01 09:27:01 INFO installing libgomp (13.2.0-r7)
2024/07/01 09:27:01 INFO installing gcc (13.2.0-r7)
2024/07/01 09:27:01 INFO installing libxcrypt (4.4.36-r7)
2024/07/01 09:27:01 INFO installing libxcrypt-dev (4.4.36-r7)
2024/07/01 09:27:01 INFO installing linux-headers (6.6.29-r1)
2024/07/01 09:27:02 INFO installing nss-hesiod (2.39-r6)
2024/07/01 09:27:02 INFO installing nss-db (2.39-r6)
2024/07/01 09:27:02 INFO installing glibc-dev (2.39-r6)
2024/07/01 09:27:02 INFO installing build-base (1-r8)
2024/07/01 09:27:02 INFO installing libcrypt1 (2.39-r6)
2024/07/01 09:27:02 INFO installing busybox (1.36.1-r10)
2024/07/01 09:27:03 INFO built image layer tarball as /tmp/apko-temp-2903258465/apko-x86_64.tar.gz
2024/07/01 09:27:03 INFO using /tmp/apko-temp-2903258465/apko-x86_64.tar.gz for image layer
2024/07/01 09:27:03 INFO OCI layer digest: sha256:f49dde21cfd83222aed91b2ff5431f5a41b52fe544233c99a994d3f549bae056
2024/07/01 09:27:03 INFO OCI layer diffID: sha256:c19fcfe5921b8facaa28677084188fddee598b533f3383b9b9f89117f8e272e3
2024/07/01 09:27:03 INFO saving OCI image locally: apko.local/cache:f43bdb6f6d89fbf3a0bdaa2afb031b85a74b692fe74a3d2e4e13e5c7b4d8de4c
2024/07/01 09:27:03 WARN skipping local domain tagging apko.local/cache:f43bdb6f6d89fbf3a0bdaa2afb031b85a74b692fe74a3d2e4e13e5c7b4d8de4c as index.docker.io/library/melange:latest
2024/07/01 09:27:03 INFO ImgRef = apko.local/cache:f43bdb6f6d89fbf3a0bdaa2afb031b85a74b692fe74a3d2e4e13e5c7b4d8de4c
2024/07/01 09:27:04 INFO running pipeline for subpackage pkg-test-sub1
2024/07/01 09:27:04 INFO retrieving workspace from builder: 24e3a73bfd68c3132475dee73a7f624bfcc27646c4c2f46c29133ad15c420231
2024/07/01 09:27:04 INFO retrieved and wrote post-build workspace to: /tmp/melange-workspace-4268561140
2024/07/01 09:27:04 INFO running package linters for pkg-test
2024/07/01 09:27:04 INFO linting apk: pkg-test
2024/07/01 09:27:04 INFO running package linters for pkg-test-sub1
2024/07/01 09:27:04 INFO linting apk: pkg-test-sub1
2024/07/01 09:27:04 INFO generating SBOM for subpackage pkg-test-sub1
2024/07/01 09:27:04 INFO generating SBOM for pkg-test
2024/07/01 09:27:04 INFO generating package pkg-test-1-r1
2024/07/01 09:27:04 INFO scanning for shared object dependencies...
2024/07/01 09:27:04 INFO scanning for commands...
2024/07/01 09:27:04 INFO scanning for pkg-config data...
2024/07/01 09:27:04 INFO scanning for python modules...
2024/07/01 09:27:04 INFO scanning for shbang deps...
2024/07/01 09:27:04 INFO Failed to open usr/bin/pkg-test: open usr/bin/pkg-test: no such file or directory
2024/07/01 09:27:04 INFO   installed-size: 29813
2024/07/01 09:27:04 INFO   data.tar.gz digest: 3f886e83760660bc82f1c0e5e6ca63555d78dd095555f9894a47f2711fc23359
2024/07/01 09:27:04 INFO wrote packages/x86_64/pkg-test-1-r1.apk
2024/07/01 09:27:04 INFO generating package pkg-test-sub1-1-r1
2024/07/01 09:27:04 INFO scanning for shared object dependencies...
2024/07/01 09:27:04 INFO scanning for commands...
2024/07/01 09:27:04 INFO scanning for pkg-config data...
2024/07/01 09:27:04 INFO scanning for python modules...
2024/07/01 09:27:04 INFO scanning for shbang deps...
2024/07/01 09:27:04 INFO Failed to open usr/bin/pkg-test-sub1: open usr/bin/pkg-test-sub1: no such file or directory
2024/07/01 09:27:04 INFO   installed-size: 29838
2024/07/01 09:27:04 INFO   data.tar.gz digest: f5cbb2daf93f7343a8989bb4f46eee1ee887e4193d4435b2df5f01a617d6e058
2024/07/01 09:27:04 INFO wrote packages/x86_64/pkg-test-sub1-1-r1.apk
2024/07/01 09:27:04 INFO generating apk index from packages in packages/x86_64
2024/07/01 09:27:04 INFO processing package packages/x86_64/pkg-test-sub1-1-r1.apk
2024/07/01 09:27:04 INFO processing package packages/x86_64/pkg-test-1-r1.apk
2024/07/01 09:27:04 INFO updating index at packages/x86_64/APKINDEX.tar.gz with new packages: [pkg-test-1-r1 pkg-test-sub1-1-r1]
2024/07/01 09:27:04 INFO signing apk index at packages/x86_64/APKINDEX.tar.gz
2024/07/01 09:27:04 INFO signing index packages/x86_64/APKINDEX.tar.gz with key local-melange.rsa
2024/07/01 09:27:04 INFO appending signature to index packages/x86_64/APKINDEX.tar.gz
2024/07/01 09:27:04 INFO writing signed index to packages/x86_64/APKINDEX.tar.gz
2024/07/01 09:27:04 INFO signed index packages/x86_64/APKINDEX.tar.gz with key local-melange.rsa
2024/07/01 09:27:14 INFO pod 24e3a73bfd68c3132475dee73a7f624bfcc27646c4c2f46c29133ad15c420231 terminated
make[1]: Leaving directory '/home/smoser/src/wolfi/os'

@smoser smoser closed this as completed Jul 1, 2024
@smoser smoser reopened this Nov 14, 2024
smoser added a commit to smoser/melange that referenced this issue Nov 14, 2024
Moving the creation of the subpackage dir before running the
pipelines means that the subpackage directory was created as
the user that ran melange rather than the user that is
doing the build.  Those uids can be different depending on
the runner.

SBOMs are written as the uid that invoked melange.  It assumes
that it can create Workspace/package-dir/var/lib/db/sbom . Previously,
the 'package-dir' portion of that would sometimes get created
by the uid inside the build (probably as a result of
'mkdir -p ${{targets.contextdir}}/usr/bin' or the like).  The result
was that the uid running melange could not create var/lib/db/sbom
because it did not have write perms to package-dir.

By creating package-dir first, we (mostly) ensure that we can later
create var/lib/db/sbom, and this will succeed more often.

There is still a problem in that we assume that we can write there.
Some part of the build might create var/lib/db and have all those
tokens as 755 with a different uid.  The right solution is probably
to do the population from inside the Runner.

This improves the situation for
chainguard-dev#1254,
but does not entirely fix it.

Signed-off-by: Scott Moser <[email protected]>
smoser added a commit to smoser/melange that referenced this issue Nov 14, 2024
Moving the creation of the subpackage dir before running the
pipelines means that the subpackage directory was created as
the user that ran melange rather than the user that is
doing the build.  Those uids can be different depending on
the runner.

SBOMs are written as the uid that invoked melange.  It assumes
that it can create Workspace/package-dir/var/lib/db/sbom . Previously,
the 'package-dir' portion of that would sometimes get created
by the uid inside the build (probably as a result of
'mkdir -p ${{targets.contextdir}}/usr/bin' or the like).  The result
was that the uid running melange could not create var/lib/db/sbom
because it did not have write perms to package-dir.

By creating package-dir first, we (mostly) ensure that we can later
create var/lib/db/sbom, and this will succeed more often.

There is still a problem in that we assume that we can write there.
Some part of the build might create var/lib/db and have all those
tokens as 755 with a different uid.  The right solution is probably
to do the population from inside the Runner.

This improves the situation for
chainguard-dev#1254,
but does not entirely fix it.

Signed-off-by: Scott Moser <[email protected]>
@maxgio92
Copy link
Contributor

I can confirm this is still applicable. Just reproduced it with Melange 0.19.1 on Linux where dockerd runs as 0, using the Docker Melange runner on openbao package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants