Can we expose the go modules in the BOM? #13

zmackie · 2020-01-22T18:06:15Z

What version of the buildpack you are using?

0.0.58

If you were attempting to accomplish a task, what was it you were attempting to do?

I'm trying to see the contents of my go.mod file attached to the image for OSL verification

What did you expect to happen?

The module information was somehow passed along

What was the actual behavior?

± zm |master U:1 ?:1 ✗| → inspect-bom gcr.io/cncf-buildpacks-ci/pm/github-actions-automate-projects:sandbox | jq
{
  "bom": [
    {
      "name": "go",
      "version": "1.13.4",
      "metadata": {
        "licenses": [],
        "name": "Go",
        "sha256": "692d17071736f74be04a72a06dab9cac1cd759377bd85316e52b2227604c004c",
        "stacks": [
          "io.buildpacks.stacks.bionic",
          "org.cloudfoundry.stacks.tiny"
        ],
        "uri": "https://dl.google.com/go/go1.13.4.linux-amd64.tar.gz"
      },
      "buildpack": {
        "id": "org.cloudfoundry.go-compiler",
        "version": "0.0.55"
      }
    },
    {
      "name": "",
      "version": "",
      "metadata": null,
      "buildpack": {
        "id": "org.cloudfoundry.go-compiler",
        "version": "0.0.55"
      }
    },
    {
      "name": "",
      "version": "",
      "metadata": null,
      "buildpack": {
        "id": "org.cloudfoundry.go-mod",
        "version": "0.0.58"
      }
    }
  ],
  "buildpacks": [
    {
      "id": "org.cloudfoundry.go-compiler",
      "version": "0.0.55"
    },
    {
      "id": "org.cloudfoundry.go-mod",
      "version": "0.0.58"
    }
  ],
  "launcher": {
    "version": "0.5.0",
    "source": {
      "git": {
        "repository": "https://github.com/buildpack/lifecycle",
        "commit": "f0a279f"
      }
    }
  }
}

inspect-bom is a function
inspect-bom ()
{
    image=$1;
    docker inspect "${image}" | jq '.[].Config.Labels | .["io.buildpacks.build.metadata"] ' | jq -r
}

Can you provide a sample app?

https://github.com/Zanadar/github-actions-automate-projects/tree/25b8a6c59f8e09344531d8363986b0c6b2d3355a

The text was updated successfully, but these errors were encountered:

cf-gitbot · 2020-01-22T18:06:18Z

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/170852414

The labels on this github issue will be updated when the story is started.

zmackie · 2020-01-22T18:15:27Z

One possible idea for a format:

{
+   "bom": [
+     {
+         "name": "go-mod",
+         "checksum//mdsum": "76c6ce5c948d771a6aa7ca671a9377cb",
+         "metadata": {
+           "go.sum": [
+             "cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=",
+             "github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=",
+             "github.com/google/go-github/v25 v25.0.4 h1:i/JXg8Et3dm4eD/u5VFB0tO6e9ICQ0zcUQavk5eSoSs=",
+             "github.com/google/go-github/v25 v25.0.4/go.mod h1:6z5pC69qHtrPJ0sXPsj4BLnd82b+r6sLB7qcBoRZqpw=",
+             "github.com/google/go-github/v28 v28.1.1 h1:kORf5ekX5qwXO2mGzXXOjMe/g6ap8ahVe0sBEulhSxo=",
+             "github.com/google/go-github/v28 v28.1.1/go.mod h1:bsqJWQX05omyWVmc00nEUql9mhQyv38lDZ8kPZcQVoM=",
+             "github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=",
+             "github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=",
+             "github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=",
+             "github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=",
+             "golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2 h1:VklqNMn3ovrHsnt90PveolxSbWFaJdECFbxSq0Mqo2M=",
+             "golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=",
+             "golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=",
+             "golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=",
+             "golang.org/x/net v0.0.0-20190311183353-d8887717615a h1:oWX7TPOiFAMXLq8o0ikBYfCJVlRHBcsciT5bXOrH628=",
+             "golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=",
+             "golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be h1:vEDujvNQGv4jgYKudGeI/+DAX4Jffq6hpD55MmoEvKs=",
+             "golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=",
+             "golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a h1:tImsplftrFpALCYumobsd0K86vlAs/eXGFms2txfJfA=",
+             "golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=",
+             "golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=",
+             "golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=",
+             "golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=",
+             "golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=",
+             "google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=",
+             "google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=",
+             "gopkg.in/go-playground/webhooks.v5 v5.9.0 h1:ROdLMPp0zBvsx9+8JsIxDX8et0x9XleXJi+CKV25STo=",
+             "gopkg.in/go-playground/webhooks.v5 v5.9.0/go.mod h1:LZbya/qLVdbqDR1aKrGuWV6qbia2zCYSR5dpom2SInQ="
+           ],
+           "go.mod": "module github.com\/takanabe\/github-actions-automate-projects\r\n\r\nrequire (\r\n\tgithub.com\/google\/go-github\/v25 v25.0.4\r\n\tgithub.com\/google\/go-github\/v28 v28.1.1\r\n\tgithub.com\/pkg\/errors v0.8.1\r\n\tgolang.org\/x\/oauth2 v0.0.0-20190402181905-9f3314589c9a\r\n\tgopkg.in\/go-playground\/webhooks.v5 v5.9.0\r\n)\r\n\r\ngo 1.13\r\n"
+         }
+     },
    {
      "name": "go",
      "version": "1.13.4",
      "metadata": {
        "licenses": [],
        "name": "Go",
        "sha256": "692d17071736f74be04a72a06dab9cac1cd759377bd85316e52b2227604c004c",
        "stacks": [
          "io.buildpacks.stacks.bionic",
          "org.cloudfoundry.stacks.tiny"
        ],
        "uri": "https://dl.google.com/go/go1.13.4.linux-amd64.tar.gz"
      },
      "buildpack": {
        "id": "org.cloudfoundry.go-compiler",
        "version": "0.0.55"
      }
    },
    {
      "name": "",
      "version": "",
      "metadata": null,
      "buildpack": {
        "id": "org.cloudfoundry.go-compiler",
        "version": "0.0.55"
      }
    },
    {
      "name": "",
      "version": "",
      "metadata": null,
      "buildpack": {
        "id": "org.cloudfoundry.go-mod",
        "version": "0.0.58"
      }
    }
  ],
  "buildpacks": [
    {
      "id": "org.cloudfoundry.go-compiler",
      "version": "0.0.55"
    },
    {
      "id": "org.cloudfoundry.go-mod",
      "version": "0.0.58"
    }
  ],
  "launcher": {
    "version": "0.5.0",
    "source": {
      "git": {
        "repository": "https://github.com/buildpack/lifecycle",
        "commit": "f0a279f"
      }
    }
  }
}

cc @pivotal/navcon thought you might be interested in this idea

zmackie · 2020-01-22T18:16:58Z

@pivotal/navcon

carlo-colombo · 2020-01-23T12:15:50Z

Hi @Zanadar , we agree on the idea of having the dependencies of the application on this metadata. Regarding the content I looked into go modules documentation and found that go.sum is not a lock file and it can contains dependencies not used anymore by the app [1].

I would suggest something along the lines of

{
  "name": "go-mod",
  "checksum//mdsum": "76c6ce5c948d771a6aa7ca671a9377cb",
  "metadata": {
    "modules": [
      "github.com/takanabe/github-actions-automate-projects",
      "cloud.google.com/go v0.34.0",
      "github.com/golang/protobuf v1.2.0",
      "github.com/google/go-github/v25 v25.0.4",
      "github.com/google/go-querystring v1.0.0",
      "github.com/pkg/errors v0.8.1",
      "golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2",
      "golang.org/x/net v0.0.0-20190311183353-d8887717615a",
      "golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a",
      "golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6",
      "golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a",
      "golang.org/x/text v0.3.0",
      "google.golang.org/appengine v1.4.0"
    ],
    "go.mod": "module github.com\/takanabe\/github-actions-automate-projects\r\n\r\nrequire (\r\n\tgithub.com\/google\/go-github\/v25 v25.0.4\r\n\tgithub.com\/google\/go-github\/v28 v28.1.1\r\n\tgithub.com\/pkg\/errors v0.8.1\r\n\tgolang.org\/x\/oauth2 v0.0.0-20190402181905-9f3314589c9a\r\n\tgopkg.in\/go-playground\/webhooks.v5 v5.9.0\r\n)\r\n\r\ngo 1.13\r\n"
  }
}

Where the json is generated running go list -m all | jq -R | jq -s . , go list generates the list of direct and undirect dependencies (including replacements). [2] go list -json -m all is also available with some more structured data (one json item for module)

[1] https://github.com/golang/go/wiki/Modules#is-gosum-a-lock-file-why-does-gosum-include-information-for-module-versions-i-am-no-longer-using
[2] https://github.com/golang/go/wiki/Modules#daily-workflow

zmackie · 2020-01-23T17:12:10Z

@carlo-colombo Love the idea of go list!

ForestEckhardt · 2020-08-03T15:08:03Z

@zmackie @carlo-colombo This is definitely something that we are interested in supporting, unfortunately it is not on our short list of priorities at the moment. We would be more than happy to field a PR from y'all if this is something that you would find to be useful in the shorter term!

fg-j · 2021-01-15T17:46:18Z

hey @carlo-colombo -- I took a bit of time to hash out a rough implementation of this feature. Feel free to take it for a test run and give feedback on the output (e.g. is the format useful, sufficiently parseable).

To test it out:

git clone [email protected]:paketo-buildpacks/go-mod-vendor && cd go-mod-vendor
git co modules-in-bom
./scripts/package.sh -v 9.9.9
pack build default-test -b gcr.io/paketo-buildpacks/go-dist:0.2.7 -b ./build/buildpack.tgz -b gcr.io/paketo-buildpacks/go-build:0.1.3 --path integration/testdata/default --clear-cache
docker inspect default-test | jq '.[].Config.Labels | .["io.buildpacks.build.metadata"] ' | jq -r | jq .

The BOM should end up looking something like:

  "bom": [
    {
      "name": "go",
      "metadata": {
        "licenses": [],
        "name": "Go",
        "sha256": "f93f67f73e7b00579363e32c70814755b66198533ccbb2b780ede19c04f11d55",
        "stacks": [
          "io.buildpacks.stacks.bionic",
          "io.paketo.stacks.tiny",
          "org.cloudfoundry.stacks.cflinuxfs3"
        ],
        "uri": "https://buildpacks.cloudfoundry.org/dependencies/go/go_1.14.13_linux_x64_cflinuxfs3_f93f67f7.tgz",
        "version": "1.14.13"
      },
      "buildpack": {
        "id": "paketo-buildpacks/go-dist",
        "version": "0.2.7"
      }
    },
    {
      "name": "go-mod",
      "metadata": {
        "go.mod sha256": "59a13f9bcb767e980fde66f3ec20effdca0ebd422e2b34adc77fb704fa626d55",
        "modules": [
          {
            "module": "github.com/BurntSushi/toml",
            "version": "v0.3.1"
          },
          {
            "module": "github.com/satori/go.uuid",
            "version": "v1.1.0"
          }
        ]
      },
      "buildpack": {
        "id": "paketo-buildpacks/go-mod-vendor",
        "version": "9.9.9"
      }
    }
  ]
}

The underlying implementation of the Bill of Materials will soon be changing for Paketo buildpacks (due to Cloud Native Buildpacks RFC#0053). So it's unlikely that I'll pull request my rough implementation into the buildpack right now.

But! Your feedback about BOM formatting/contents is still useful because it'll inform the BOM formatting for this buildpack regardless of its underlying implementation.

carlo-colombo · 2021-01-31T17:56:17Z

Hi @fg-j ,

Sorry for the late answer. I gave feedback on this issue because at the time I was part of a team in Pivotal (NavCon) that was looking into annotating container images to improve the traceability of software. Since then Pivotal got acquired and I moved on from the company and to different topics, so I don't know if I can provide valuable feedback.

I think the point from my comment at the time was to not use go.sum as it is not representative of the dependencies used, I think the provided format is ok. You said that your is just a rough implementation but I want to point out that is going to only show direct dependencies of the project, while using go tooling (as explained in my previous comment) would allow to report indirect dependencies too.

I remember we have some points about having the same (or a similar format) across buildpacks so common tooling could be built to consume a dependency list instead of different formats between java/node/go buildpacks.

Out of curiosity I tried to follow your instructions and had a couple of problems: jam is not executable (so I have to chmod +x it) and then I got a segmentation fault ./scripts/package.sh: line 86: 126671 Segmentation fault jam pack --buildpack "${ROOT_DIR}/buildpack.toml" --version "${version}" --output "${BUILD_DIR}/buildpack.tgz" , I have linux on my machine btw.

ForestEckhardt · 2021-11-15T19:10:52Z

Sorry for the radio silence on this. We have been heads done working on some very large SBOM changes across the whole Paketo project and within CNB itself. We have just put together an RFC that will hopefully address this concern product wide which you can check out here. Hopefully what is outlined in this RFC will meet the requirements of your use case! If not please reach out to us with more information an what is required to get y'all off the ground!

TisVictress · 2021-11-23T20:50:08Z

Hey. I'm currently taking a look at this.

fg-j · 2021-12-20T15:35:17Z

Implementing this feature is blocked on platform integration issues.

cf-gitbot added the unscheduled label Jan 22, 2020

zmackie changed the title ~~Can we expose the contents of go.mod in the BOM?~~ Can we expose the go modules in the BOM? Jan 22, 2020

ryanmoran removed the unscheduled label May 28, 2020

fg-j self-assigned this Jan 15, 2021

fg-j removed their assignment Mar 11, 2021

fg-j added the enhancement A new feature or request label Apr 2, 2021

fg-j assigned TisVictress Nov 29, 2021

fg-j mentioned this issue Dec 1, 2021

Implement Paketo RFC 0038: CycloneDX + Syft SBOM paketo-buildpacks/go#524

Closed

4 tasks

fg-j added the status/blocked This issue has been triaged and resolving it is blocked on some other issue label Dec 20, 2021

TisVictress mentioned this issue Jan 14, 2022

SBOM via packit v2 #296

Closed

fg-j mentioned this issue Jan 18, 2022

[RFC #0095] Exporter should export bom files for launch layers buildpacks/lifecycle#735

Closed

fg-j mentioned this issue Feb 28, 2022

Syft SBOM #311

Merged

5 tasks

ryanmoran closed this as completed in #311 Mar 2, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can we expose the go modules in the BOM? #13

Can we expose the go modules in the BOM? #13

zmackie commented Jan 22, 2020 •

edited

Loading

cf-gitbot commented Jan 22, 2020

zmackie commented Jan 22, 2020 •

edited

Loading

zmackie commented Jan 22, 2020

carlo-colombo commented Jan 23, 2020

zmackie commented Jan 23, 2020

ForestEckhardt commented Aug 3, 2020

fg-j commented Jan 15, 2021

carlo-colombo commented Jan 31, 2021

ForestEckhardt commented Nov 15, 2021

TisVictress commented Nov 23, 2021

fg-j commented Dec 20, 2021

Can we expose the go modules in the BOM? #13

Can we expose the go modules in the BOM? #13

Comments

zmackie commented Jan 22, 2020 • edited Loading

What version of the buildpack you are using?

If you were attempting to accomplish a task, what was it you were attempting to do?

What did you expect to happen?

What was the actual behavior?

Can you provide a sample app?

cf-gitbot commented Jan 22, 2020

zmackie commented Jan 22, 2020 • edited Loading

zmackie commented Jan 22, 2020

carlo-colombo commented Jan 23, 2020

zmackie commented Jan 23, 2020

ForestEckhardt commented Aug 3, 2020

fg-j commented Jan 15, 2021

carlo-colombo commented Jan 31, 2021

ForestEckhardt commented Nov 15, 2021

TisVictress commented Nov 23, 2021

fg-j commented Dec 20, 2021

zmackie commented Jan 22, 2020 •

edited

Loading

zmackie commented Jan 22, 2020 •

edited

Loading