Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

failed to build package: writing SBOMs: #1504

Open
smoser opened this issue Sep 18, 2024 · 2 comments
Open

failed to build package: writing SBOMs: #1504

smoser opened this issue Sep 18, 2024 · 2 comments
Labels
bug Something isn't working needs-triage applied to all new customer/user issues. Removed after triage occurs

Comments

@smoser
Copy link
Contributor

smoser commented Sep 18, 2024

We're running into a problem using the docker runner on linux (Ubuntu 24.04).

$ dpkg -S `which docker`
docker.io: /usr/bin/docker
$ dpkg-query --show docker.io
docker.io	24.0.7-0ubuntu4.1

$ melange version | grep ^[A-Z]
GitVersion:    v0.12.0-17-gd2bf9f2
GitCommit:     d2bf9f276dd8a214002cc5c5526a7bea57a1d049
GitTreeState:  clean
BuildDate:     '2024-09-18T15:12:21Z'
GoVersion:     go1.23.1
Compiler:      gc
Platform:      linux/amd64

$ make MELANGE_EXTRA_OPTS=--runner=docker package/7zip
yamlfile is 7zip.yaml
Building package 7zip with version 7zip-2301-r2 from file 7zip.yaml
make yamlfile=7zip.yaml pkgname=7zip packages/x86_64/7zip-2301-r2.apk
make[1]: Entering directory '/home/smoser/src/wolfi/os'
@SOURCE_DATE_EPOCH=1706410629 /home/smoser/go/bin/melange build 7zip.yaml --repository-append /home/smoser/src/wolfi/os/packages --keyring-append local-melange.rsa.pub --signing-key local-melange.rsa --arch x86_64 --env-file build-x86_64.env --namespace wolfi --generate-index false  --pipeline-dir ./pipelines/ --runner=docker -k https://packages.wolfi.dev/os/wolfi-signing.rsa.pub -r https://packages.wolfi.dev/os
2024/09/18 15:09:28 INFO melange is building:
2024/09/18 15:09:28 INFO   configuration file: 7zip.yaml
2024/09/18 15:09:28 INFO   workspace dir: /tmp/melange-workspace-2526033556
...
2024/09/18 15:10:30 INFO running step "Split manpages"
2024/09/18 15:10:30 INFO retrieving workspace from builder: 10ad4d2a822da1b9d46de77b61144e46fc23bf852f9ad79fad1a894a905b8d5f
2024/09/18 15:10:30 INFO retrieved and wrote post-build workspace to: /tmp/melange-workspace-2526033556
2024/09/18 15:10:30 INFO running package linters for 7zip
2024/09/18 15:10:30 INFO linting apk: 7zip
2024/09/18 15:10:30 INFO running package linters for 7zip-doc
2024/09/18 15:10:30 INFO linting apk: 7zip-doc
2024/09/18 15:10:30 INFO generating SBOM for subpackage 7zip-doc
2024/09/18 15:10:40 INFO pod 10ad4d2a822da1b9d46de77b61144e46fc23bf852f9ad79fad1a894a905b8d5f terminated
2024/09/18 15:10:40 ERRO ERROR: failed to build package. the build environment has been preserved:
2024/09/18 15:10:40 INFO   workspace dir: /tmp/melange-workspace-2526033556
2024/09/18 15:10:40 INFO   guest dir: /tmp/melange-guest-2945227068
2024/09/18 15:10:40 ERRO failed to build package: writing SBOMs: writing sbom to disk: creating SBOM directory: mkdir /tmp/melange-workspace-2526033556/melange-out/7zip-doc/var: permission denied
make[1]: *** [Makefile:111: packages/x86_64/7zip-2301-r2.apk] Error 1
make[1]: Leaving directory '/home/smoser/src/wolfi/os'
make: *** [Makefile:101: package/7zip] Error 2
@smoser
Copy link
Contributor Author

smoser commented Sep 18, 2024 •
edited
Loading

Given mypkg.yaml:

package:
  name: mypkg
  version: 1.0.0
  epoch: 0
  copyright:
    - license: Apache-2.0

environment:
  contents:
    packages:
      - busybox

pipeline:
  - runs: |
      ( echo '#!/bin/sh'; echo 'echo howdy' ) > myctl
      install -Dm755 myctl "${{targets.destdir}}"/usr/bin/myctl

subpackages:
  - name: "mypkg-doc"
    description: "mypkg doc"
    pipeline:
      - runs: |
          p="${{targets.subpkgdir}}"/usr/share/doc/mypkg
          mkdir -p "$p"
          echo "hi world" > "$p/README.txt"

you can reproduce with:

( set -x; make MELANGE_EXTRA_OPTS="--runner=docker clean --debug" package/mypkg ; ) 2>&1 | tee out.log
+ make 'MELANGE_EXTRA_OPTS=--runner=docker clean --debug' package/mypkg
yamlfile is mypkg.yaml
Building package mypkg with version mypkg-1.0.0-r0 from file mypkg.yaml
make yamlfile=mypkg.yaml pkgname=mypkg packages/x86_64/mypkg-1.0.0-r0.apk
make[1]: Entering directory '/home/smoser/src/wolfi/os'
@SOURCE_DATE_EPOCH= /home/smoser/go/bin/melange build mypkg.yaml --repository-append /home/smoser/src/wolfi/os/packages --keyring-append local-melange.rsa.pub --signing-key local-melange.rsa --arch x86_64 --env-file build-x86_64.env --namespace wolfi --generate-index false  --pipeline-dir ./pipelines/ --runner=docker clean --debug -k https://packages.wolfi.dev/os/wolfi-signing.rsa.pub -r https://packages.wolfi.dev/os
2024/09/18 15:40:16 WARN SOURCE_DATE_EPOCH is specified but empty, setting it to 1969-12-31 19:00:00 -0500 EST
2024/09/18 15:40:16 INFO melange is building:
2024/09/18 15:40:16 INFO   configuration file: mypkg.yaml
2024/09/18 15:40:16 INFO   workspace dir: /tmp/melange-workspace-1699554886
2024/09/18 15:40:16 INFO evaluating pipelines for package requirements
2024/09/18 15:40:16 INFO --cache-dir ./melange-cache/ not a dir; skipping
2024/09/18 15:40:16 INFO populating workspace /tmp/melange-workspace-1699554886 from ./mypkg/
2024/09/18 15:40:16 INFO building workspace in '/tmp/melange-guest-2305750043' with apko
2024/09/18 15:40:16 ERRO request failed error="Get \"/home/smoser/src/wolfi/os/packages/apk-configuration\": unsupported protocol scheme \"\"" method=GET url=/home/smoser/src/wolfi/os/packages/apk-configuration
2024/09/18 15:40:16 WARN ignoring missing keys for /home/smoser/src/wolfi/os/packages: failed to perform key discovery: Get "/home/smoser/src/wolfi/os/packages/apk-configuration": GET /home/smoser/src/wolfi/os/packages/apk-configuration giving up after 1 attempt(s): Get "/home/smoser/src/wolfi/os/packages/apk-configuration": unsupported protocol scheme ""
2024/09/18 15:40:16 INFO setting apk repositories: [/home/smoser/src/wolfi/os/packages https://packages.wolfi.dev/os]
2024/09/18 15:40:16 INFO image configuration:
2024/09/18 15:40:16 INFO   contents:
2024/09/18 15:40:16 INFO     build repositories: []
2024/09/18 15:40:16 INFO     runtime repositories: []
2024/09/18 15:40:16 INFO     keyring:      []
2024/09/18 15:40:16 INFO     packages:     [busybox]
2024/09/18 15:40:16 INFO   accounts:
2024/09/18 15:40:16 INFO     runas:  
2024/09/18 15:40:16 INFO     users:
2024/09/18 15:40:16 INFO       - uid=1000(build) gid=1000
2024/09/18 15:40:16 INFO     groups:
2024/09/18 15:40:16 INFO       - gid=1000(build) members=[build]
2024/09/18 15:40:16 INFO auth configured for: []
2024/09/18 15:40:17 INFO installing ca-certificates-bundle (20240705-r0)
2024/09/18 15:40:17 INFO installing wolfi-baselayout (20230201-r15)
2024/09/18 15:40:17 INFO installing ld-linux (2.40-r2)
2024/09/18 15:40:17 INFO installing glibc-locale-posix (2.40-r2)
2024/09/18 15:40:17 INFO installing glibc (2.40-r2)
2024/09/18 15:40:17 INFO installing libxcrypt (4.4.36-r7)
2024/09/18 15:40:17 INFO installing libcrypt1 (2.40-r2)
2024/09/18 15:40:17 INFO installing busybox (1.36.1-r11)
2024/09/18 15:40:17 INFO built image layer tarball as /tmp/apko-temp-929460157/apko-x86_64.tar.gz
2024/09/18 15:40:17 INFO using /tmp/apko-temp-929460157/apko-x86_64.tar.gz for image layer
2024/09/18 15:40:17 INFO OCI layer digest: sha256:b463a08392552b2c6dbc529bf733690df194f1a6ab5c7fc2a49a343c0dbfa18a
2024/09/18 15:40:17 INFO OCI layer diffID: sha256:9f3cba49c3dd8fec88da8b59bc446585ff57ccae897c7ef998c466a092d770ad
2024/09/18 15:40:17 INFO saving OCI image locally: apko.local/cache:8616e648b7426e9a50306e6b743696d65e00e52aa88fb7c4b602897fb985c84e
2024/09/18 15:40:17 WARN skipping local domain tagging apko.local/cache:8616e648b7426e9a50306e6b743696d65e00e52aa88fb7c4b602897fb985c84e as index.docker.io/library/melange:latest
2024/09/18 15:40:17 INFO ImgRef = apko.local/cache:8616e648b7426e9a50306e6b743696d65e00e52aa88fb7c4b602897fb985c84e
2024/09/18 15:40:17 WARN + '[' -d /home/build ]
2024/09/18 15:40:17 WARN + cd /home/build
2024/09/18 15:40:17 WARN + mkdir DOC
2024/09/18 15:40:17 WARN + echo mydoc
2024/09/18 15:40:17 WARN + echo 'read this'
2024/09/18 15:40:17 WARN + echo '#!/bin/sh'
2024/09/18 15:40:17 WARN + echo 'echo howdy'
2024/09/18 15:40:17 WARN + install -Dm755 myctl /home/build/melange-out/mypkg/usr/bin/myctl
2024/09/18 15:40:17 WARN + exit 0
2024/09/18 15:40:17 INFO running pipeline for subpackage mypkg-doc
2024/09/18 15:40:17 WARN + '[' -d /home/build ]
2024/09/18 15:40:17 WARN + cd /home/build
2024/09/18 15:40:17 WARN + p=/home/build/melange-out/mypkg-doc/usr/share/doc/mypkg
2024/09/18 15:40:17 WARN + mkdir -p /home/build/melange-out/mypkg-doc/usr/share/doc/mypkg
2024/09/18 15:40:17 WARN + echo 'hi world'
2024/09/18 15:40:17 WARN + exit 0
2024/09/18 15:40:17 INFO retrieving workspace from builder: ceea88be6e9b6b61ad4fad9d03eb79957ed331f0d2d8ea498e2ac04e2587fede
2024/09/18 15:40:17 INFO retrieved and wrote post-build workspace to: /tmp/melange-workspace-1699554886
2024/09/18 15:40:17 INFO running package linters for mypkg
2024/09/18 15:40:17 INFO linting apk: mypkg
2024/09/18 15:40:17 INFO running package linters for mypkg-doc
2024/09/18 15:40:17 INFO linting apk: mypkg-doc
2024/09/18 15:40:17 INFO generating SBOM for subpackage mypkg-doc
2024/09/18 15:40:27 INFO pod ceea88be6e9b6b61ad4fad9d03eb79957ed331f0d2d8ea498e2ac04e2587fede terminated
2024/09/18 15:40:27 ERRO ERROR: failed to build package. the build environment has been preserved:
2024/09/18 15:40:27 INFO   workspace dir: /tmp/melange-workspace-1699554886
2024/09/18 15:40:27 INFO   guest dir: /tmp/melange-guest-2305750043
2024/09/18 15:40:27 ERRO failed to build package: writing SBOMs: writing sbom to disk: creating SBOM directory: mkdir /tmp/melange-workspace-1699554886/melange-out/mypkg-doc/var: permission denied
make[1]: *** [Makefile:111: packages/x86_64/mypkg-1.0.0-r0.apk] Error 1
make[1]: Leaving directory '/home/smoser/src/wolfi/os'
make: *** [Makefile:101: package/mypkg] Error 2

The same with bubblewrap works correctly

$ ( set -x; make MELANGE_EXTRA_OPTS="--runner=bubblewrap --debug" clean package/mypkg ; ) 2>&1 | tee out.log
+ make 'MELANGE_EXTRA_OPTS=--runner=bubblewrap --debug' clean package/mypkg
rm -rf packages/x86_64
yamlfile is mypkg.yaml
Building package mypkg with version mypkg-1.0.0-r0 from file mypkg.yaml
make yamlfile=mypkg.yaml pkgname=mypkg packages/x86_64/mypkg-1.0.0-r0.apk
make[1]: Entering directory '/home/smoser/src/wolfi/os'
@SOURCE_DATE_EPOCH= /home/smoser/go/bin/melange build mypkg.yaml --repository-append /home/smoser/src/wolfi/os/packages --keyring-append local-melange.rsa.pub --signing-key local-melange.rsa --arch x86_64 --env-file build-x86_64.env --namespace wolfi --generate-index false  --pipeline-dir ./pipelines/ --runner=bubblewrap --debug -k https://packages.wolfi.dev/os/wolfi-signing.rsa.pub -r https://packages.wolfi.dev/os
2024/09/18 15:44:42 WARN SOURCE_DATE_EPOCH is specified but empty, setting it to 1969-12-31 19:00:00 -0500 EST
2024/09/18 15:44:42 INFO melange is building:
2024/09/18 15:44:42 INFO   configuration file: mypkg.yaml
2024/09/18 15:44:42 INFO   workspace dir: /tmp/melange-workspace-4015051511
2024/09/18 15:44:42 INFO evaluating pipelines for package requirements
2024/09/18 15:44:42 INFO --cache-dir ./melange-cache/ not a dir; skipping
2024/09/18 15:44:42 INFO populating workspace /tmp/melange-workspace-4015051511 from ./mypkg/
2024/09/18 15:44:42 INFO building workspace in '/tmp/melange-guest-1507284756' with apko
2024/09/18 15:44:42 ERRO request failed error="Get \"/home/smoser/src/wolfi/os/packages/apk-configuration\": unsupported protocol scheme \"\"" method=GET url=/home/smoser/src/wolfi/os/packages/apk-configuration
2024/09/18 15:44:42 WARN ignoring missing keys for /home/smoser/src/wolfi/os/packages: failed to perform key discovery: Get "/home/smoser/src/wolfi/os/packages/apk-configuration": GET /home/smoser/src/wolfi/os/packages/apk-configuration giving up after 1 attempt(s): Get "/home/smoser/src/wolfi/os/packages/apk-configuration": unsupported protocol scheme ""
2024/09/18 15:44:42 INFO setting apk repositories: [/home/smoser/src/wolfi/os/packages https://packages.wolfi.dev/os]
2024/09/18 15:44:42 INFO image configuration:
2024/09/18 15:44:42 INFO   contents:
2024/09/18 15:44:42 INFO     build repositories: []
2024/09/18 15:44:42 INFO     runtime repositories: []
2024/09/18 15:44:42 INFO     keyring:      []
2024/09/18 15:44:42 INFO     packages:     [busybox]
2024/09/18 15:44:42 INFO   accounts:
2024/09/18 15:44:42 INFO     runas:  
2024/09/18 15:44:42 INFO     users:
2024/09/18 15:44:42 INFO       - uid=1000(build) gid=1000
2024/09/18 15:44:42 INFO     groups:
2024/09/18 15:44:42 INFO       - gid=1000(build) members=[build]
2024/09/18 15:44:42 INFO auth configured for: []
2024/09/18 15:44:42 INFO installing ca-certificates-bundle (20240705-r0)
2024/09/18 15:44:42 INFO installing wolfi-baselayout (20230201-r15)
2024/09/18 15:44:42 INFO installing ld-linux (2.40-r2)
2024/09/18 15:44:42 INFO installing glibc-locale-posix (2.40-r2)
2024/09/18 15:44:42 INFO installing glibc (2.40-r2)
2024/09/18 15:44:42 INFO installing libxcrypt (4.4.36-r7)
2024/09/18 15:44:42 INFO installing libcrypt1 (2.40-r2)
2024/09/18 15:44:42 INFO installing busybox (1.36.1-r11)
2024/09/18 15:44:42 INFO built image layer tarball as /tmp/apko-temp-50267896/apko-x86_64.tar.gz
2024/09/18 15:44:42 INFO using /tmp/apko-temp-50267896/apko-x86_64.tar.gz for image layer
2024/09/18 15:44:43 INFO ImgRef = /tmp/melange-guest-2606905586
2024/09/18 15:44:43 WARN + '[' -d /home/build ]
2024/09/18 15:44:43 WARN + cd /home/build
2024/09/18 15:44:43 WARN + echo '#!/bin/sh'
2024/09/18 15:44:43 WARN + echo 'echo howdy'
2024/09/18 15:44:43 WARN + install -Dm755 myctl /home/build/melange-out/mypkg/usr/bin/myctl
2024/09/18 15:44:43 WARN + exit 0
2024/09/18 15:44:43 INFO running pipeline for subpackage mypkg-doc
2024/09/18 15:44:43 WARN + '[' -d /home/build ]
2024/09/18 15:44:43 WARN + cd /home/build
2024/09/18 15:44:43 WARN + p=/home/build/melange-out/mypkg-doc/usr/share/doc/mypkg
2024/09/18 15:44:43 WARN + mkdir -p /home/build/melange-out/mypkg-doc/usr/share/doc/mypkg
2024/09/18 15:44:43 WARN + echo 'hi world'
2024/09/18 15:44:43 WARN + exit 0
2024/09/18 15:44:43 INFO retrieving workspace from builder: 
2024/09/18 15:44:43 INFO retrieved and wrote post-build workspace to: /tmp/melange-workspace-4015051511
2024/09/18 15:44:43 INFO running package linters for mypkg
2024/09/18 15:44:43 INFO linting apk: mypkg
2024/09/18 15:44:43 INFO running package linters for mypkg-doc
2024/09/18 15:44:43 INFO linting apk: mypkg-doc
2024/09/18 15:44:43 INFO generating SBOM for subpackage mypkg-doc
2024/09/18 15:44:43 INFO generating SBOM for mypkg
2024/09/18 15:44:43 INFO generating package mypkg-1.0.0-r0
2024/09/18 15:44:43 INFO scanning for shared object dependencies...
2024/09/18 15:44:43 INFO scanning for commands...
2024/09/18 15:44:43 INFO   found command usr/bin/myctl
2024/09/18 15:44:43 INFO scanning for pkg-config data...
2024/09/18 15:44:43 INFO scanning for python modules...
2024/09/18 15:44:43 INFO scanning for ruby gems...
2024/09/18 15:44:43 INFO scanning for shbang deps...
2024/09/18 15:44:43 INFO   provides:
2024/09/18 15:44:43 INFO     cmd:myctl=1.0.0-r0
2024/09/18 15:44:43 INFO   installed-size: 29842
2024/09/18 15:44:43 INFO   data.tar.gz digest: c6193ab897f4554b0ee44ccea3435b0762fd608f850e29692e9e2038acc71c30
2024/09/18 15:44:43 INFO wrote packages/x86_64/mypkg-1.0.0-r0.apk
2024/09/18 15:44:43 INFO generating package mypkg-doc-1.0.0-r0
2024/09/18 15:44:43 INFO scanning for shared object dependencies...
2024/09/18 15:44:43 INFO scanning for commands...
2024/09/18 15:44:43 INFO scanning for pkg-config data...
2024/09/18 15:44:43 INFO scanning for python modules...
2024/09/18 15:44:43 INFO scanning for ruby gems...
2024/09/18 15:44:43 INFO scanning for shbang deps...
2024/09/18 15:44:43 INFO   installed-size: 38042
2024/09/18 15:44:43 INFO   data.tar.gz digest: bc25004434be2cb0c9b80874d4bb9d7cb0dcdaa23efffa5db45cfc3c49b25fa3
2024/09/18 15:44:43 INFO wrote packages/x86_64/mypkg-doc-1.0.0-r0.apk
2024/09/18 15:44:43 INFO generating apk index from packages in packages/x86_64
2024/09/18 15:44:43 INFO processing package packages/x86_64/mypkg-doc-1.0.0-r0.apk
2024/09/18 15:44:43 INFO processing package packages/x86_64/mypkg-1.0.0-r0.apk
2024/09/18 15:44:43 INFO updating index at packages/x86_64/APKINDEX.tar.gz with new packages: [mypkg-1.0.0-r0 mypkg-doc-1.0.0-r0]
2024/09/18 15:44:43 INFO signing apk index at packages/x86_64/APKINDEX.tar.gz
2024/09/18 15:44:43 INFO signing index packages/x86_64/APKINDEX.tar.gz with key local-melange.rsa
2024/09/18 15:44:43 INFO appending signature RSA to index packages/x86_64/APKINDEX.tar.gz
2024/09/18 15:44:43 INFO writing signed index to packages/x86_64/APKINDEX.tar.gz
2024/09/18 15:44:43 INFO signed index packages/x86_64/APKINDEX.tar.gz with key local-melange.rsa
make[1]: Leaving directory '/home/smoser/src/wolfi/os'

@murraybd murraybd added bug Something isn't working needs-triage applied to all new customer/user issues. Removed after triage occurs labels Sep 20, 2024
@murraybd
Copy link
Contributor

murraybd commented Nov 5, 2024

It looks like the directories for the primary package and the subpackages are created by different users e.g.

 $ ls -lh /tmp/melange-workspace-1598736609/melange-out/
total 8.0K
drwxr-xr-x 3 brian-murray brian-murray 4.0K Nov  4 15:56 7zip
drwxr-xr-x 3 root         root         4.0K Nov  4 15:56 7zip-doc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working needs-triage applied to all new customer/user issues. Removed after triage occurs
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@smoser @murraybd