Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce terraform deployer build verbosity #665

Merged
merged 10 commits into from
Feb 8, 2022
Merged

Reduce terraform deployer build verbosity #665

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
5ee5cfd
terraform deployer: silent gcloud SDK installation
endorama Jan 26, 2022
04a69dd
try disable-prompts flag
endorama Jan 26, 2022
40b2d3e
Revert "try disable-prompts flag"
endorama Jan 26, 2022
95b3fa4
use ubuntu:20.04 base image
endorama Jan 31, 2022
5f5d9c3
install in single RUN
endorama Jan 31, 2022
ce34806
split apt requisites installation
endorama Feb 2, 2022
bf68394
install terraform using deb
endorama Feb 2, 2022
cc8fa55
suppress terraform colored output
endorama Feb 2, 2022
ca029f6
run terraform deployer script in BASH
endorama Feb 4, 2022
af5bbcc
remove -x
endorama Feb 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 15 additions & 14 deletions internal/install/_static/Dockerfile.terraform_deployer
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,25 @@
FROM hashicorp/terraform:light as terraform
FROM ubuntu:20.04
ENV GCLOUD_SDK_VERSION 370.0.0-0
ENV TERRAFORM_VERSION 1.1.4

FROM python:3-alpine
RUN apt-get -qq update \
&& apt-get install -yq curl apt-transport-https ca-certificates gnupg

# required by gcloud SDK
RUN apk add --no-cache git openssh curl
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you mixed all commands a bit.

  1. Install GPG key and cloud-sdk repo
  2. apt-get update -yq && apt-get install -yq curl apt-transport-https.... - google-cloud-sdk=${GCLOUD_SDK_VERSION} && apt-get clean

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curl apt-transport-https gnupg are required to add the deb repo and install the repo key, so two steps are required unfortunately.
Would you prefer to split GPG key and repo addition from the actual package installation?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can try something like this:

apt-get update -yq && \ 
apt-get install apt-transport-https gnupg && \
deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
apt-get update -yq && \
apt-get install -yq google-cloud-sdk=${GCLOUD_SDK_VERSION} && \
apt-get clean

The aim is to consider this as an atomic piece to prevent any APT repo update from breaking the container build.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated it with the suggestion. Just consider that if in the future another repo will be required I think would be best to extract the dependency installation so there is no implicit dependency.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mtojek I extracted it again, as Hashicorp has it's own repo that has the same requirements. It happened sooner than expected :D

&& curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - \
&& apt-get update -qq \
&& apt-get install google-cloud-sdk=${GCLOUD_SDK_VERSION} -yq
mtojek marked this conversation as resolved.
Show resolved Hide resolved

ENV GCLOUD_SDK_VERSION 369.0.0
ENV PATH $PATH:/usr/local/gcloud/google-cloud-sdk/bin
RUN curl "https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-$GCLOUD_SDK_VERSION-linux-x86_64.tar.gz" > /tmp/google-cloud-sdk.tar.gz \
&& mkdir -p /usr/local/gcloud \
&& tar -C /usr/local/gcloud -xvf /tmp/google-cloud-sdk.tar.gz \
&& /usr/local/gcloud/google-cloud-sdk/install.sh -q --override-components="bq" \
&& rm /tmp/google-cloud-sdk.tar.gz
RUN echo "deb [arch=amd64] https://apt.releases.hashicorp.com focal main" | tee -a /etc/apt/sources.list.d/hashicorp.list \
&& curl -fsSL https://apt.releases.hashicorp.com/gpg | apt-key add - \
&& apt-get -qq update \
&& apt-get install -yq terraform=${TERRAFORM_VERSION}

HEALTHCHECK --timeout=3s CMD sh -c "[ -f /tmp/tf-applied ]"

COPY --from=terraform /bin/terraform /usr/bin/terraform

ENV TF_IN_AUTOMATION=true
ENV TF_CLI_ARGS="-no-color"
ADD run.sh /
WORKDIR /workspace

ENTRYPOINT sh /run.sh
ENTRYPOINT bash /run.sh
4 changes: 2 additions & 2 deletions internal/install/_static/terraform_deployer_run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!sh
#!/usr/bin/env bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason why the CI is failing is because of the Dockerfile ENTRYPOINT set to sh: here.

Let's quickly refactor it to /bin/bash instead and verify.


set -euxo pipefail
set -euo pipefail

# Terraform code may rely on content from other files than .tf files (es json, zip, html, text), so we copy all the content over
# See more: https://github.com/elastic/elastic-package/pull/603
Expand Down