Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dependencies image as artifact #1333

Merged
merged 7 commits into from
Apr 18, 2022
Merged

dependencies image as artifact #1333

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
21d35f4
dependencies image as artifact
esigo Apr 17, 2022
9cc771c
Update docker/Dockerfile comment
esigo Apr 18, 2022
696781a
update retention
esigo Apr 18, 2022
fc613db
rename docker file
esigo Apr 18, 2022
49f83ce
lib files only
esigo Apr 18, 2022
8b98217
change deps image name
esigo Apr 18, 2022
554cb81
no cache
esigo Apr 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/dependencies_image.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: 'OpenTelemetry-cpp dependencies image'
on:
schedule:
- cron: "0 3 * * 6"

jobs:
docker_image:
name: Docker Image
runs-on: ubuntu-latest
timeout-minutes: 300
steps:
-
name: checkout
uses: actions/checkout@v2
-
name: Set up QEMU
uses: docker/setup-qemu-action@v1
-
name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
-
name: Build Image
uses: docker/build-push-action@v2
with:
builder: ${{ steps.buildx.outputs.name }}
context: ci/
file: ./docker/Dockerfile
build-args: BASE_IMAGE=ubuntu:latest
platforms: linux/amd64
# platforms: linux/amd64,linux/arm64
push: false
tags: otel-cpp
load: true
cache-from: type=gha
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this effective if cache-to is not used ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no it's not, I cleaned it.

# cache-to: type=gha,mode=max
-
name: Save Image
run: |
docker images
docker save -o /opt/otel-cpp-deps.tar otel-cpp
lalitb marked this conversation as resolved.
Show resolved Hide resolved
-
name: Upload Image
uses: actions/upload-artifact@v3
with:
name: otel-cpp-deps
path: /opt/otel-cpp-deps.tar
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have the estimated size of these tar-ed docker images, as GitHub cache is limited to 10GB, and we are already using some of it for Bazel builds? We can later also see if these artifacts can be stored separately in azure blob -
https://github.com/marketplace/actions/upload-a-build-artifact-to-azure-blob-storage

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't find the size limit in the documentation. I think the same rule as logs should apply here. I'm going to reduce the retention period for the artifact to two weeks.

The size of the artifact is about 1GB.

We can reduce this size (probably to 200MB) by storing gRPC and thrift only in the image.

Copy link
Member

@lalitb lalitb Apr 18, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok interesting. I also don't see it documented anywhere. I was thinking it to be part of Cache size limit.
What all we store right now apart from gRPC and thrift ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the image had everything for building the library git, cmake,...
I changed the docker file. now it has only gRPC and thrift.

35 changes: 35 additions & 0 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
ARG BASE_IMAGE=ubuntu:latest
ARG CORES=${nproc}

FROM ${BASE_IMAGE} as base

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y build-essential autoconf \
lalitb marked this conversation as resolved.
Show resolved Hide resolved
libtool pkg-config cmake git libssl-dev curl \
libcurl4-openssl-dev libgtest-dev libgmock-dev libbenchmark-dev

WORKDIR /work

RUN apt-get update && apt-get install -y autoconf libtool pkg-config cmake git libssl-dev
RUN if echo ${BASE_IMAGE} | grep -q "ubuntu"; then apt-get install -y build-essential; fi

FROM base as grpc
# install grpc and abseil
esigo marked this conversation as resolved.
Show resolved Hide resolved
ARG GRPC_VERSION=1.43.2

ADD setup_grpc.sh .
RUN ./setup_grpc.sh -v GRPC_VERSION

FROM base as thrift
RUN apt-get install -y --no-install-recommends wget

# install thrift
ARG THRIFT_VERSION=0.14.1
ADD setup_thrift.sh .
RUN ./setup_thrift.sh

lalitb marked this conversation as resolved.
Show resolved Hide resolved
FROM base as final
ARG CORES
COPY --from=grpc /usr /usr
COPY --from=thrift /usr /usr