Adding CI/CD security features and version CLI command #39

louison77 · 2024-08-09T09:35:02Z

I worked on the project on my own fork https://github.com/louison77/k8s-kms-plugin.

Indeed, I did the fork before renaming the original draft project from https://github.com/ThalesGroup/k8s-kms-plugin to https://github.com/ThalesGroup/k8s-kms-plugin-tmp, and before doing a clean move of the original k8s-kms-plugin project from its original repo on https://github.com/orgs/ThalesIgnite to its new repo https://github.com/ThalesGroup.

Therefore, the states of commit hashes was messed up, and the pull request from https://github.com/louison77/k8s-kms-plugin was harder to merged. So we did a clean fork on @Nicolas-Peiffer profile, and incorporated the contributions from @louison77.

Proposed Changes

Github workflow file using Goreleaser tool for generating a release.
Publish action file releasing artifacts with Goreleaser.
Automate the publication of container images with ko-build tool integrated into Goreleaser.
Signing of binary and container images artifacts.
Generation of SLSA level 3 attestations for both kind of artifacts.
Verification of SLSA images attestations.
Version CLI command for k8s-kms-plugin.
Gitlab-ci file equivalent for the Github workflow.
Detailed explanations of the changes in the Readme.

Types of Changes

These changes introduce a new feature with is a Github CI/CD workflow allowing the creation of release including artifacts signatures, provenance attestations.

Verification

These changes can be verified by tagging a commit and push the tag into Github. This tag will trigger the workflow which will creating release. For signing the artifacts, it is important to go on the Github project in the workflow previously triggered to sign manually the artifacts with OIDC authentication.
A verification of the provenance attestation is done in the pipeline but the detailed commands to verify the signatures are explained in the Readme file.

Testing

As it is CI/CD feature, there are no tests that covered it. But different examples are present on the fork project: https://github.com/louison77/k8s-kms-plugin.

Linked Issues

User-Facing Change

New command version:

k8s-kms-plugin version

{
  "major": 0,
  "minor": 6,
  "version": "0.6.0",
  "commitIdLong": "98026fbd433da5a81b5793db9d69f6b6bbddb895",
  "commitIdShort": "98026fbd",
  "goVersion": "go version go1.22.5 linux/amd64",
  "date": "2024-08-09T11:17:14+02:00",
  "plaform": "x86_64"
}

Further Comments

…-fork Signed-off-by: Nicolas-Peiffer <[email protected]> Add LDFlags in ko in goreleaser and change base image and repo env var Signed-off-by: Nicolas-Peiffer <[email protected]> Update Makefile Signed-off-by: Nicolas-Peiffer <[email protected]> Update Makefile LDFLAG Signed-off-by: Nicolas-Peiffer <[email protected]> Update ldflags field in goreleaser Makefile Signed-off-by: Nicolas-Peiffer <[email protected]> Update to ko action v0.7 Signed-off-by: Nicolas-Peiffer <[email protected]> Fix gitignore and add version CLI command Update README for container as an experimental feature Signed-off-by: Nicolas-Peiffer <[email protected]>

IceManGreen · 2024-11-25T15:56:09Z

Great contribution ! Thank you.
However, in case of no tag for a given build, a developer can end with this error :

./k8s-kms-plugin version
panic: 67dde6b is not in dotted-tri format
goroutine 1 [running]:
github.com/coreos/go-semver/semver.Must(...)
        /go/pkg/mod/github.com/coreos/[email protected]/semver/semver.go:65
github.com/coreos/go-semver/semver.New({0xbce8b8?, 0x0?})
        /go/pkg/mod/github.com/coreos/[email protected]/semver/semver.go:49 +0x34
github.com/ThalesGroup/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd.CreateJsonVersion()
        /go/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd/version.go:52 +0x25
github.com/ThalesGroup/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd.generateOutput()
        /go/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd/version.go:94 +0xaf
github.com/ThalesGroup/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd.init.func8(0xc0000df300?, {0xad76d4?, 0x4?, 0xad7670?})
        /go/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd/version.go:111 +0x78
github.com/spf13/cobra.(*Command).execute(0xfd7480, {0x1046e80, 0x0, 0x0})
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:987 +0xa91
github.com/spf13/cobra.(*Command).ExecuteC(0xfd6060)
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:1115 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:1039
github.com/ThalesGroup/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd.Execute()
        /go/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd/root.go:82 +0x10c
main.main()
        /go/k8s-kms-plugin/cmd/k8s-kms-plugin/main.go:29 +0xf

We should always have an output for the command version, even with no version number (or at least a 'snapshot' information in the version name), rather than a stack trace.

IceManGreen · 2024-11-25T16:03:40Z

Also check goreleaser DEPRECATED recipe :

# goreleaser release --clean --snapshot --skip validate,publish,sign

  • only configurations files on  version: 2  are supported, yours is  version: 0 , please update your configuration
  • skipping announce, publish, sign and validate...
  • cleaning distribution directory
  • loading environment variables
  • getting and validating git state
    • ignoring errors because this is a snapshot     error=git doesn't contain any tags. Either add a tag or use --snapshot
    • git state                                      commit=67dde6bcc03533e9d05f5a15aedcbfe7e1011ed1 branch=master current_tag=v0.0.0 previous_tag=<unknown> dirty=false
    • pipe skipped                                   reason=disabled during snapshot mode
  • parsing tag
  • setting defaults
    • DEPRECATED:  snapshot.name_template  should not be used anymore, check https://goreleaser.com/deprecations#snapshotnametemplate for more info
  ⨯ release failed after 0s                          error=template: failed to apply "{{ .Env.GITHUB_REPOSITORY_OWNER }}": template: failed to apply "{{ .Env.GITHUB_REPOSITORY_OWNER }}": map has no entry for key "GITHUB_REPOSITORY_OWNER"

…s issue ThalesGroup#40 Signed-off-by: Nicolas-Peiffer <[email protected]>

https://goreleaser.com/deprecations/#snapshotname_template Signed-off-by: Nicolas-Peiffer <[email protected]>

Signed-off-by: Nicolas-Peiffer <[email protected]>

Nicolas-Peiffer · 2024-12-16T16:10:04Z

Great contribution ! Thank you. However, in case of no tag for a given build, a developer can end with this error :

./k8s-kms-plugin version
panic: 67dde6b is not in dotted-tri format
goroutine 1 [running]:
github.com/coreos/go-semver/semver.Must(...)
        /go/pkg/mod/github.com/coreos/[email protected]/semver/semver.go:65
github.com/coreos/go-semver/semver.New({0xbce8b8?, 0x0?})
        /go/pkg/mod/github.com/coreos/[email protected]/semver/semver.go:49 +0x34
github.com/ThalesGroup/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd.CreateJsonVersion()
        /go/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd/version.go:52 +0x25
github.com/ThalesGroup/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd.generateOutput()
        /go/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd/version.go:94 +0xaf
github.com/ThalesGroup/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd.init.func8(0xc0000df300?, {0xad76d4?, 0x4?, 0xad7670?})
        /go/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd/version.go:111 +0x78
github.com/spf13/cobra.(*Command).execute(0xfd7480, {0x1046e80, 0x0, 0x0})
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:987 +0xa91
github.com/spf13/cobra.(*Command).ExecuteC(0xfd6060)
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:1115 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:1039
github.com/ThalesGroup/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd.Execute()
        /go/k8s-kms-plugin/cmd/k8s-kms-plugin/cmd/root.go:82 +0x10c
main.main()
        /go/k8s-kms-plugin/cmd/k8s-kms-plugin/main.go:29 +0xf

We should always have an output for the command version, even with no version number (or at least a 'snapshot' information in the version name), rather than a stack trace.

This has been fixed by 2a9cff0

Now the k8s-kms-plugin version command should be able to work even in when goreleaser is ran in --snapshot mode during developement. Now the k8s-kms-plugin version will be able to handle when the result of command git describe --tags --always does not match a semantic version tag.

git describe --tags --always
1096c2e

Notice 1096c2e does not follow semantic versioning.

$LDFLAGS should look like this:

echo $LDFLAGS
-X 'github.com/ThalesGroup/k8s-kms-plugin/pkg/version.RawGitDescribe=1096c2e' -X 'github.com/ThalesGroup/k8s-kms-plugin/pkg/version.GitCommitIdLong=1096c2ea3777b389583b658fe6919645334bcdf6' -X 'github.com/ThalesGroup/k8s-kms-plugin/pkg/version.GitCommitIdShort=1096c2ea' -X 'github.com/ThalesGroup/k8s-kms-plugin/pkg/version.GoVersion=go version go1.23.4 linux/amd64' -X 'github.com/ThalesGroup/k8s-kms-plugin/pkg/version.BuildPlatform=x86_64' -X 'github.com/ThalesGroup/k8s-kms-plugin/pkg/version.BuildDate=2024-12-16T16:46:15+01:00' -X 'github.com/ThalesGroup/k8s-kms-plugin/pkg/version.GitCommitTimestamp=2024-12-16T16:04:01+01:00'

Note: This particular git describe --tags --always example starts with a figure 1096, which makes the "github.com/hashicorp/go-version" module thinks this 1096 is a Major version. It is actually not a semantic version string. This is an edge case of the "github.com/hashicorp/go-version".

JSON pretty print

./dist/k8s-kms-plugin-linux-amd64_SNAPSHOT-1096c2e version -o json

{
  "k8s-kms-plugin-cli": {
    "major": 1096,
    "minor": 0,
    "patch": 0,
    "version": "1096c2e",
    "gitCommitIdLong": "1096c2ea3777b389583b658fe6919645334bcdf6",
    "gitCommitIdShort": "1096c2ea",
    "gitCommitTimestamp": "2024-12-16T16:04:01+01:00",
    "goVersion": "go version go1.23.4 linux/amd64",
    "buildDate": "2024-12-16T16:46:15+01:00",
    "buildPlatform": "x86_64"
  }
}

JSON without pretty print

./dist/k8s-kms-plugin-linux-amd64_SNAPSHOT-1096c2e version -o json --pretty=false

{"k8s-kms-plugin-cli":{"major":1096,"minor":0,"patch":0,"version":"1096c2e","gitCommitIdLong":"1096c2ea3777b389583b658fe6919645334bcdf6","gitCommitIdShort":"1096c2ea","gitCommitTimestamp":"2024-12-16T16:04:01+01:00","goVersion":"go version go1.23.4 linux/amd64","buildDate":"2024-12-16T16:46:15+01:00","buildPlatform":"x86_64"}}

YAML

./dist/k8s-kms-plugin-linux-amd64_SNAPSHOT-1096c2e version -o yaml

k8s-kms-plugin-cli:
  buildDate: "2024-12-16T16:46:15+01:00"
  buildPlatform: x86_64
  gitCommitIdLong: 1096c2ea3777b389583b658fe6919645334bcdf6
  gitCommitIdShort: 1096c2ea
  gitCommitTimestamp: "2024-12-16T16:04:01+01:00"
  goVersion: go version go1.23.4 linux/amd64
  major: 1096
  minor: 0
  patch: 0
  version: 1096c2e

Simple version STD OUT

 ./dist/k8s-kms-plugin-linux-amd64_SNAPSHOT-1096c2e version                       
k8s-kms-plugin: v1096.0.0

pretty print flag only applies to JSON YAML

./dist/k8s-kms-plugin-linux-amd64_SNAPSHOT-1096c2e version -o yaml --pretty=false
2024/12/16 15:57:01 ERROR Flag --pretty must NOT be used when output format is yaml. --pretty is only used when output format is json

only support json and yaml as version output format

./dist/k8s-kms-plugin-linux-amd64_SNAPSHOT-1096c2e version -o xml                
2024/12/16 15:58:37 ERROR Invalid output format. Must be json or yaml. output=xml
k8s-kms-plugin: v1096.0.0

Here is another example:

git describe --tags --always
a289339

a289339 is not matching semantic versioning. "github.com/hashicorp/go-version" will parse this and detects it is not valid semver. So the output of major.minor.patch is arbitrary set to 0.0.0.

./dist/k8s-kms-plugin-linux-amd64_SNAPSHOT-a289339 version 
2024/12/16 16:03:59 ERROR Raw git describe --tags --always version is not parsable as semantic versioning. Set major, minor and patch to 0  raw_git_describe=a289339 error="Malformed version: a289339"
k8s-kms-plugin: v0.0.0

./dist/k8s-kms-plugin-linux-amd64_SNAPSHOT-a289339 version -o json
2024/12/16 16:04:33 ERROR Raw git describe --tags --always version is not parsable as semantic versioning. Set major, minor and patch to 0  raw_git_describe=a289339 error="Malformed version: a289339"

{
  "k8s-kms-plugin-cli": {
    "major": 0,
    "minor": 0,
    "patch": 0,
    "version": "a289339",
    "gitCommitIdLong": "a289339fd186df0b6f00a69681ce0ab9710b941c",
    "gitCommitIdShort": "a289339f",
    "gitCommitTimestamp": "2024-12-16T17:01:45+01:00",
    "goVersion": "go version go1.23.4 linux/amd64",
    "buildDate": "2024-12-16T17:02:02+01:00",
    "buildPlatform": "x86_64"
  }
}

If the git describe --tags --always follows semver, then

Example

git describe --tags --always
0.1.6-56-g888baf5

k8s-kms-plugin version -o json
{
  "k8s-kms-plugin-cli": {
    "major": 0,
    "minor": 1,
    "patch": 6,
    "version": "0.1.6-55-g91fe0ba",
    "gitCommitIdLong": "91fe0bacef5b6df3532f9e0d7e7b33fc7a69e7b0",
    "gitCommitIdShort": "91fe0bac",
    "gitCommitTimestamp": "2024-12-16T15:47:10+01:00",
    "goVersion": "go version go1.23.4 linux/amd64",
    "buildDate": "2024-12-16T16:06:45+01:00",
    "buildPlatform": "x86_64"
  }
}

Nicolas-Peiffer · 2024-12-16T16:13:05Z

Also check goreleaser DEPRECATED recipe :

# goreleaser release --clean --snapshot --skip validate,publish,sign

  • only configurations files on  version: 2  are supported, yours is  version: 0 , please update your configuration
  • skipping announce, publish, sign and validate...
  • cleaning distribution directory
  • loading environment variables
  • getting and validating git state
    • ignoring errors because this is a snapshot     error=git doesn't contain any tags. Either add a tag or use --snapshot
    • git state                                      commit=67dde6bcc03533e9d05f5a15aedcbfe7e1011ed1 branch=master current_tag=v0.0.0 previous_tag=<unknown> dirty=false
    • pipe skipped                                   reason=disabled during snapshot mode
  • parsing tag
  • setting defaults
    • DEPRECATED:  snapshot.name_template  should not be used anymore, check https://goreleaser.com/deprecations#snapshotnametemplate for more info
  ⨯ release failed after 0s                          error=template: failed to apply "{{ .Env.GITHUB_REPOSITORY_OWNER }}": template: failed to apply "{{ .Env.GITHUB_REPOSITORY_OWNER }}": map has no entry for key "GITHUB_REPOSITORY_OWNER"

DEPRECATED: snapshot.name_template is fixed by Nicolas-Peiffer@61b77d2.

And you need to provide a GITHUB_REPOSITORY_OWNER environment variable is you want to run goreleaser locally, since this value is actually set by Github Actions.

Signed-off-by: Nicolas-Peiffer <[email protected]>

Nicolas-Peiffer force-pushed the master branch from 98026fb to 28d0de2 Compare August 9, 2024 09:43

Nicolas-Peiffer force-pushed the master branch from 28d0de2 to 67dde6b Compare August 9, 2024 10:01

Nicolas-Peiffer added 8 commits November 27, 2024 18:09

Update k8s kms dependency location following the information from thi…

5adeb4e

…s issue ThalesGroup#40 Signed-off-by: Nicolas-Peiffer <[email protected]>

Fix deprecation of snapshot.name_template

61b77d2

https://goreleaser.com/deprecations/#snapshotname_template Signed-off-by: Nicolas-Peiffer <[email protected]>

comment gitlab ci part

ca4b69c

Signed-off-by: Nicolas-Peiffer <[email protected]>

Update k8s-kms-plugin version command

2a9cff0

Signed-off-by: Nicolas-Peiffer <[email protected]>

remove github.com/blang/semver/v4

1096c2e

Signed-off-by: Nicolas-Peiffer <[email protected]>

Merge branch 'ThalesGroup:master' into master

139b97e

Merge branch 'update-version-method'

e12f0de

add dist

a289339

Signed-off-by: Nicolas-Peiffer <[email protected]>

Nicolas-Peiffer added 2 commits December 16, 2024 18:01

Update README with local goreleaser build informations

3c5579c

Signed-off-by: Nicolas-Peiffer <[email protected]>

Merge branch 'update-version-method'

7a339d6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding CI/CD security features and version CLI command #39

Adding CI/CD security features and version CLI command #39

louison77 commented Aug 9, 2024

IceManGreen commented Nov 25, 2024

IceManGreen commented Nov 25, 2024

Nicolas-Peiffer commented Dec 16, 2024

Nicolas-Peiffer commented Dec 16, 2024

Adding CI/CD security features and version CLI command #39

Are you sure you want to change the base?

Adding CI/CD security features and version CLI command #39

Conversation

louison77 commented Aug 9, 2024

Proposed Changes

Types of Changes

Verification

Testing

Linked Issues

User-Facing Change

Further Comments

IceManGreen commented Nov 25, 2024

IceManGreen commented Nov 25, 2024

Nicolas-Peiffer commented Dec 16, 2024

JSON pretty print

JSON without pretty print

YAML

Simple version STD OUT

pretty print flag only applies to JSON YAML

only support json and yaml as version output format

Nicolas-Peiffer commented Dec 16, 2024