Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FIPS env var to build crossbuild images for FIPS compliance #495

Merged
merged 1 commit into from
Feb 7, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 26 additions & 4 deletions .buildkite/pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,14 @@ steps:
key: "staging"

steps:
- label: ":linux: Staging / Ubuntu X86_64 - {{matrix.makefile}}"
- label: ":linux: Staging / Ubuntu X86_64 - {{matrix.makefile}} - fips: {{matrix.fips}}"
key: "build-ubuntu-x86"
command:
- ".buildkite/scripts/build.sh {{matrix.makefile}}"
- ".buildkite/scripts/publish.sh {{matrix.makefile}}"
env:
REPOSITORY: "${STAGING_IMAGE}"
FIPS: "{{matrix.fips}}"
if: build.env("BUILDKITE_PULL_REQUEST") != "false"
notify:
- github_commit_status:
Expand All @@ -37,14 +38,18 @@ steps:
- "Makefile.debian10"
- "Makefile.debian11"
- "Makefile.debian12"
fips:
- "false"
- "true"

- label: ":linux: Staging / Ubuntu ARM - Makefile.debian9"
- label: ":linux: Staging / Ubuntu ARM - Makefile.debian9 - fips: {{matrix.fips}}"
key: "build-ubuntu-arm"
command:
- ".buildkite/scripts/build.sh Makefile.debian9"
- ".buildkite/scripts/publish.sh Makefile.debian9"
env:
REPOSITORY: "${STAGING_IMAGE}"
FIPS: "{{matrix.fips}}"
if: build.env("BUILDKITE_PULL_REQUEST") != "false"
notify:
- github_commit_status:
Expand All @@ -53,16 +58,23 @@ steps:
provider: "aws"
imagePrefix: "${IMAGE_UBUNTU_ARM_64}"
instanceType: "t4g.large"
matrix:
setup:
fips:
- "false"
- "true"

- group: "Release"
key: "release"

steps:
- label: ":linux: Release / Ubuntu X86_64 - {{matrix.makefile}}"
- label: ":linux: Release / Ubuntu X86_64 - {{matrix.makefile}} - fips: {{matrix.fips}}"
key: "release-ubuntu-x86"
command:
- ".buildkite/scripts/build.sh {{matrix.makefile}}"
- ".buildkite/scripts/publish.sh {{matrix.makefile}}"
env:
FIPS: "{{matrix.fips}}"
if: build.branch == "main" || build.branch =~ /^[0-9]+\.[0-9]+$$/
agents:
provider: "gcp"
Expand All @@ -77,20 +89,30 @@ steps:
- "Makefile.debian10"
- "Makefile.debian11"
- "Makefile.debian12"
fips:
- "false"
- "true"
notify:
- github_commit_status:
context: "Release / Ubuntu X86_64"

- label: ":linux: Release / Ubuntu ARM - Makefile.debian9"
- label: ":linux: Release / Ubuntu ARM - Makefile.debian9 - fips: {{matrix.fips}}"
key: "release-ubuntu-arm"
command:
- ".buildkite/scripts/build.sh Makefile.debian9"
- ".buildkite/scripts/publish.sh Makefile.debian9"
env:
FIPS: "{{matrix.fips}}"
if: build.branch == "main" || build.branch =~ /^[0-9]+\.[0-9]+$$/
agents:
provider: "aws"
imagePrefix: "${IMAGE_UBUNTU_ARM_64}"
instanceType: "t4g.large"
matrix:
setup:
fips:
- "false"
- "true"
notify:
- github_commit_status:
context: "Release / Ubuntu ARM"
Expand Down
7 changes: 7 additions & 0 deletions .github/updatecli.d/bump-go-release-version.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@ MAJOR_MINOR_VERSION=$(echo "$GO_RELEASE_VERSION" | sed -E -e "s#([0-9]+\.[0-9]+)
GOLANG_DOWNLOAD_SHA256_ARM=$(curl -s -L https://golang.org/dl/\?mode\=json | jq -r ".[] | select( .version | contains(\"go${GO_RELEASE_VERSION}\")) | .files[] | select (.filename | contains(\"go${GO_RELEASE_VERSION}.linux-arm64.tar.gz\")) | .sha256")
GOLANG_DOWNLOAD_SHA256_AMD=$(curl -s -L https://golang.org/dl/\?mode\=json | jq -r ".[] | select( .version | contains(\"go${GO_RELEASE_VERSION}\")) | .files[] | select (.filename | contains(\"go${GO_RELEASE_VERSION}.linux-amd64.tar.gz\")) | .sha256")

# Gather microsoft/go sha256 values
MSFT_DOWNLOAD_METADATA=$(curl -s -L https://aka.ms/golang/release/latest/go${GO_RELEASE_VERSION}.assets.json)
MSFT_DOWNLOAD_SHA256_ARM=$(echo $MSFT_DOWNLOAD_METADATA | jq -r ".arches[] | select( .env.GOOS == \"linux\") | select( .env.GOARCH == \"arm64\") | .sha256")
MSFT_DOWNLOAD_SHA256_AMD=$(echo $MSFT_DOWNLOAD_METADATA | jq -r ".arches[] | select( .env.GOOS == \"linux\") | select( .env.GOARCH == \"amd64\") | .sha256")

## As long as https://golang.org/dl/?mode=json supports only 2 major versions
## and there is a new major release, then it's required to parse https://golang.org/dl
## see https://github.com/elastic/golang-crossbuild/pull/389/commits/d0af04f97a2381630ea5e8da5a99f50cf27856a0
Expand All @@ -41,7 +46,9 @@ find "go" -type f -name Dockerfile.tmpl -print0 |
${SED} -E -e "s#(ARG GOLANG_VERSION)=[0-9]+\.[0-9]+(\.[0-9]+)?#\1=${GO_RELEASE_VERSION}#g" "$line"
if echo "$line" | grep -q 'arm' ; then
${SED} -E -e "s#(ARG GOLANG_DOWNLOAD_SHA256)=.+#\1=${GOLANG_DOWNLOAD_SHA256_ARM}#g" "$line"
${SED} -E -e "s#(ARG MSFT_DOWNLOAD_SHA256)=.+#\1=${MSFT_DOWNLOAD_SHA256_ARM}#g" "$line"
else
${SED} -E -e "s#(ARG GOLANG_DOWNLOAD_SHA256)=.+#\1=${GOLANG_DOWNLOAD_SHA256_AMD}#g" "$line"
${SED} -E -e "s#(ARG MSFT_DOWNLOAD_SHA256)=.+#\1=${MSFT_DOWNLOAD_SHA256_AMD}#g" "$line"
fi
done
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ stateDiagram-v2

## Releasing images for a new Go version

With every new version of `go` we made a new branch with the name of the previous version to allow us to continue building the Docker images for the previous version of `go`. So if we are in go `1.21` and go `1.22` is released, we create a new branch `1.21`, then we update the `main` branch to install go `1.22` as explained in the below steps:
With every new version of `go` we made a new branch with the name of the previous version to allow us to continue building the Docker images for the previous version of `go`. So if we are in go `1.21` and go `1.22` is released, we create a new branch `1.21`, then we update the `main` branch to install go `1.22` as explained in the below steps:

1. Update the Go version in [.go-version](https://github.com/elastic/golang-crossbuild/blob/main/.go-version).
1. Update the Docker tag in
Expand All @@ -230,6 +230,16 @@ With every new version of `go` we made a new branch with the name of the previou

**NOTE**: Due to the changes in the Debian packages repositories, there are no guarantees that the Docker images for the previous version of `go` will continue to work after some time.

## FIPS Build

[Makefile.common](./go/Makefile.common) has an env var that can be used to create a FIPS compliant golang-crossbuild image:
```make
FIPS ?=
```

When this var is set to `"true"` the [microsoft/go](https://github.com/microsoft/go) will be used instead of the regular upstream go release.
Additionally the docker image will have the env vars `CGO_ENABLED=1` and `GOEXPERIMENT=systemcrypto` set to ensure binaries built within the crossbuild image are FIPS compliant by default.

## Packaging MacOS SDK

The osxcross repository used to cross compile for MacOSX has [instructions for packaging the SDK](https://github.com/tpoechtrager/osxcross#packaging-the-sdk).
Expand Down
6 changes: 6 additions & 0 deletions go/Makefile.common
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@ VERSION := 1.23.6
DEBIAN_VERSION ?= 9
SUFFIX := -$(shell basename $(CURDIR))
TAG_EXTENSION ?=
FIPS ?=
ifeq "${FIPS}" "true"
ifeq ($(findstring -fips,$(TAG_EXTENSION)),)
TAG_EXTENSION:=$(TAG_EXTENSION)-fips
endif
endif

export DEBIAN_VERSION TAG_EXTENSION NPCAP_FILE

Expand Down
14 changes: 13 additions & 1 deletion go/base-arm/Dockerfile.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,19 @@ RUN \
&& rm -rf /var/lib/apt/lists/*

ARG GOLANG_VERSION=1.23.6
{{- if eq .FIPS "true"}}
ARG GOLANG_DOWNLOAD_URL=https://aka.ms/golang/release/latest/go$GOLANG_VERSION-1.linux-arm64.tar.gz
# Use a different arg name for microsoft/go sha so it can be handled seperately from the regular golang sha
ARG MSFT_DOWNLOAD_SHA256=959c3bed26f259f0af600fcba6c49643abde8823a775dec99fc9c68b2391bb3a
ARG DOWNLOAD_SHA256=$MSFT_DOWNLOAD_SHA256
{{- else}}
ARG GOLANG_DOWNLOAD_URL=https://golang.org/dl/go$GOLANG_VERSION.linux-arm64.tar.gz
ARG GOLANG_DOWNLOAD_SHA256=561c780e8f4a8955d32bf72e46af0b5ee5e0debe1e4633df9a03781878219202
ARG DOWNLOAD_SHA256=$GOLANG_DOWNLOAD_SHA256
{{- end}}

RUN curl -fsSL "$GOLANG_DOWNLOAD_URL" -o golang.tar.gz \
&& echo "$GOLANG_DOWNLOAD_SHA256 golang.tar.gz" | sha256sum -c - \
&& echo "$DOWNLOAD_SHA256 golang.tar.gz" | sha256sum -c - \
&& rm -rf /usr/local/go \
&& tar -C /usr/local -xzf golang.tar.gz \
&& rm golang.tar.gz
Expand Down Expand Up @@ -75,6 +83,10 @@ RUN mkdir /libpcap \
&& ./configure --enable-usb=no --enable-bluetooth=no --enable-dbus=no --host=aarch64-unknown-linux-gnu --with-pcap=linux \
&& make

{{- if eq .FIPS "true"}}
ENV GOEXPERIMENT systemcrypto
ENV CGO_ENABLED 1
{{- end}}
ENV GOLANG_CROSSBUILD=1
VOLUME /app
WORKDIR /app
Expand Down
14 changes: 13 additions & 1 deletion go/base/Dockerfile.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,19 @@ RUN ln -s /usr/bin/pip3 /usr/bin/pip
{{- end }}

ARG GOLANG_VERSION=1.23.6
{{- if eq .FIPS "true"}}
ARG GOLANG_DOWNLOAD_URL=https://aka.ms/golang/release/latest/go$GOLANG_VERSION-1.linux-amd64.tar.gz
# Use a different arg name for microsoft/go sha so it can be handled seperately from the regular golang sha
ARG MSFT_DOWNLOAD_SHA256=6b6c3cea5a0469f10ba90c568c298d7f83717c6feb240005bbc41041a659400c
ARG DOWNLOAD_SHA256=$MSFT_DOWNLOAD_SHA256
{{- else}}
ARG GOLANG_DOWNLOAD_URL=https://golang.org/dl/go$GOLANG_VERSION.linux-amd64.tar.gz
ARG GOLANG_DOWNLOAD_SHA256=9379441ea310de000f33a4dc767bd966e72ab2826270e038e78b2c53c2e7802d
ARG DOWNLOAD_SHA256=$GOLANG_DOWNLOAD_SHA256
{{- end }}

RUN curl -fsSL "$GOLANG_DOWNLOAD_URL" -o golang.tar.gz \
&& echo "$GOLANG_DOWNLOAD_SHA256 golang.tar.gz" | sha256sum -c - \
&& echo "$DOWNLOAD_SHA256 golang.tar.gz" | sha256sum -c - \
&& rm -rf /usr/local/go \
&& tar -C /usr/local -xzf golang.tar.gz \
&& rm golang.tar.gz
Expand Down Expand Up @@ -67,6 +75,10 @@ RUN mkdir /libpcap \
&& rm libpcap-1.8.1.tar.gz


{{- if eq .FIPS "true"}}
ENV GOEXPERIMENT systemcrypto
ENV CGO_ENABLED 1
{{- end}}
ENV GOLANG_CROSSBUILD=1
VOLUME /app
WORKDIR /app
Expand Down