Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(docker-build): simplify the docker build process and reduce docker image size #9447

Merged
merged 5 commits into from
Feb 29, 2024
Merged

chore(docker-build): simplify the docker build process and reduce docker image size #9447

merged 5 commits into from
Feb 29, 2024

Conversation

rafaelromcar-parabol
Copy link
Contributor

@rafaelromcar-parabol rafaelromcar-parabol commented Feb 22, 2024 •
edited
Loading

Description

  • Reduces size of the image. Down from 795 MB to 445 MB.
  • Reduces build time. Down form more than a minute to around 3.5s + the time to download node:VERSION-bookworm-slim (around 201 MB).
  • Simplifies the build-env files to have only what is required.
  • Reduce the number of Dockefiles to only one.

@rafaelromcar-parabol
Dockerfile basic created. Improvements added to reduce build time and…
140cccd 
… size (down from 795MB to 445MB, depending on systemtap). Readme reduced, removing the old process used to build the image.
@rafaelromcar-parabol rafaelromcar-parabol self-assigned this Feb 22, 2024
rafaelromcar-parabol
rafaelromcar-parabol commented Feb 22, 2024
View reviewed changes
docker/parabol-ubi/docker-build/environments/basic-env
POSTGRES_PORT='5432'
REDIS_URL='redis://localhost:6379'
RETHINKDB_SSL=''
RETHINKDB_URL='rethinkdb://localhost:28015/buildDB'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the existing files we have RETHINKDB_URL='rethinkdb://localhost:28015/actionProduction'.

Is there any problem if we just use buildDB instead of actionProduction?

@mattkrick

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's probably ok

rafaelromcar-parabol
rafaelromcar-parabol commented Feb 22, 2024
View reviewed changes
docker/parabol-ubi/docker-build/dockerfiles/basic.dockerfile Outdated
ENV NPM_CONFIG_PREFIX=/home/node/.npm-global
ENV PORT=3000

# RUN apt update -y && \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adaniels-parabol you added systemtap in a PR, but there are no details about why it was added. Could you elaborate a bit more and tell us if we can remove it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was used for some of the security analysis (inspection scripts) and for some of the security scripts.

@rafaelromcar-parabol
Copy link
Contributor Author

rafaelromcar-parabol commented Feb 22, 2024 •
edited
Loading

One side effect of this PR: as we are using only one base image and just adding our files, a big part of the image will be cached from now on, making our deployment go faster as well 😸

@rafaelromcar-parabol
Copy link
Contributor Author

We cannot reduce the number of dockerfiles to one and we cannot remove all the security scripts until all our instances (PPMIs included) are using the new version. Thus, I'm going to merge this PR with the improvements we have added so far and let the rest for the, I hope, not so far future.

@rafaelromcar-parabol rafaelromcar-parabol merged commit 5e356c2 into master Feb 29, 2024
4 checks passed
@rafaelromcar-parabol rafaelromcar-parabol deleted the chore/simplify-parabol-ubi branch February 29, 2024 16:02
@parabol-release-bot parabol-release-bot bot mentioned this pull request Feb 29, 2024
Merged
@github-actions github-actions bot mentioned this pull request Feb 29, 2024
Closed
24 tasks
rafaelromcar-parabol added a commit that referenced this pull request Feb 29, 2024
@github-actions @rafaelromcar-parabol
@Dschoordsch @nickoferrall @dependabot @parabol-release-bot @mattkrick @jordanh @adaniels-parabol @snyk-bot @dbumblis-parabol
chore(release): Test v7.19.7 (#9484)
94be730 
* chore(env vars): Stripe vars moved to the Integrations section (#9427)

* chore: fix misleading `isLead` field name on `Team` (#9413)

* chore: fix misleading `isLead` field name on `Team`

The field indicates whether the viewer is the lead, but when used in a
query for a different user, the result could be read wrong.

* Fix Team.isLead dependencies

* feat: remove team template limit (#9424)

* update error message and increase template limit

* remove max team template limits

* remove canClone prop from CloneTemplate

* remove unused threshold

* remove unused threshold

* feat: Add Google calendar meeting series for recurrence (#9380)

* feat: Add recurrence to GCal events

* Fun with timezones

* fix: Increase the number of projects fetched per request from Atlassian (#9435)

We ran into timeouts in `getAllProjects`, presumably because we're doing
too many roundtrips. As a quick fix, increse the number of projects
fetched per request from 50 to 500.

* chore(deps): bump ip from 1.1.8 to 1.1.9 (#9442)

Bumps [ip](https://github.com/indutny/node-ip) from 1.1.8 to 1.1.9.
- [Commits](indutny/node-ip@v1.1.8...v1.1.9)

---
updated-dependencies:
- dependency-name: ip
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(release): release v7.17.0 (#9428)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* feat(standalone-deployment): Standalone host deployment improved and documented (#9445)

* Docker compose stack improved

* Remove unused containers from docker-compse and add useful comment on .env.example about PGSSLMODE

* Docker compose profiles added. Documentation extended on how to use the profiles to manage the stack.

* README fixed as docker compose up and down commands were not working

* Typo fixed and docker-compose command replaced by docker compose

* feat: support env-defined saml issuer for PPMIs (#9455)

* feat: support env-defined saml issuer for PPMIs

Signed-off-by: Matt Krick <[email protected]>

* feat: support single SAML for entire tenant

Signed-off-by: Matt Krick <[email protected]>

---------

Signed-off-by: Matt Krick <[email protected]>

* chore: Associate logs with traces (#9444)

* chore: Associate logs with traces

Add trace information to log output for server side log statements. This
does not include logging from code exclusively used for debugging,
deploying or development.

* Actually add the logger

* Fix DD_LOGS_INJECTION check

* chore(release): release v7.18.0 (#9450)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* chore: no force-push to prod (#9401)

Signed-off-by: Matt Krick <[email protected]>

* chore(release): release v7.18.1 (#9459)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* feat: embedder service (#9417)

* feat: add embedder service

---------

Signed-off-by: Matt Krick <[email protected]>
Co-authored-by: Matt Krick <[email protected]>

* merge production to avoid force push (#9461)

Signed-off-by: Matt Krick <[email protected]>

* chore(release): release v7.19.0 (#9460)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: checkout prod before merging it (#9463)

Signed-off-by: Matt Krick <[email protected]>

* chore(release): release v7.19.1 (#9464)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: mrege origin/production strategy (#9465)

Signed-off-by: Matt Krick <[email protected]>

* chore(release): release v7.19.2 (#9466)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: force push 5 (#9467)

Signed-off-by: Matt Krick <[email protected]>

* chore(release): release v7.19.3 (#9468)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: limit invites from spammers (#9416)

* fix: limit invites from spammers

* update where we check pending emails

* check total plus pending invites

* use invitees instead of pending

* fix: Fetch Jira projects in parallel (#9456)

Previously we tried to fetch more projects per page, but Jira only ever
returns 50 max. Instead once we know how many projects there are after
fetching the first page, we fetch all remaining pages in parallel.

* fix: replace lone surrogates in draft-js content (#9415)

* fix: replace lone surrogates in draft-js content

Signed-off-by: Matt Krick <[email protected]>

* fix typo

Signed-off-by: Matt Krick <[email protected]>

* fix: eslint errors

Signed-off-by: Matt Krick <[email protected]>

---------

Signed-off-by: Matt Krick <[email protected]>

* chore(deps): bump es5-ext from 0.10.62 to 0.10.64 (#9457)

Bumps [es5-ext](https://github.com/medikoo/es5-ext) from 0.10.62 to 0.10.64.
- [Release notes](https://github.com/medikoo/es5-ext/releases)
- [Changelog](https://github.com/medikoo/es5-ext/blob/main/CHANGELOG.md)
- [Commits](medikoo/es5-ext@v0.10.62...v0.10.64)

---
updated-dependencies:
- dependency-name: es5-ext
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: packages/server/package.json to reduce vulnerabilities (#9434)

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-UNDICI-6252336

Co-authored-by: snyk-bot <[email protected]>

* fix: packages/server/package.json to reduce vulnerabilities (#9392)

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-NODEMAILER-6219989

Co-authored-by: snyk-bot <[email protected]>

* fix: packages/server/package.json to reduce vulnerabilities (#9298)

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-FOLLOWREDIRECTS-6141137

Co-authored-by: snyk-bot <[email protected]>

* chore(deps): bump follow-redirects from 1.14.8 to 1.15.4 (#9312)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.14.8 to 1.15.4.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.14.8...v1.15.4)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Matt Krick <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matt Krick <[email protected]>

* chore: add upload to GCS step in ironbank (#9471)

* add upload to GCS step in ironbank

* update workflow name

* chore(release): release v7.19.4 (#9470)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: Fix seasonal templates for leap years (#9476)

* fix: Fix seasonal templates for leap years

It would produce invalid dates on February 29th.

* Master was not clean

* chore(release): release v7.19.5 (#9477)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: After parameter for meetingCount was ignored (#9479)

* chore(docker-build): simplify the docker build process and reduce docker image size (#9447)

* Dockerfile basic created. Improvements added to reduce build time and size (down from 795MB to 445MB, depending on systemtap). Readme reduced, removing the old process used to build the image.

* basic-env file using a RethinkDB database name that is clearly dedicated to the building proces.

* Readme improved to run all three components

* Unused dockerfiles removed. Docker entrypoint renamed. Docker Readme adapted

* Legacy build kept in both dockerfile and env file. Readme adapted to use the new basic image. Build GH workflow adapted to use the new basic.dockerfile.

* chore(release): release v7.19.6 (#9480)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix(docker-build): home folder is /home/node now (#9482)

* chore(release): release v7.19.7 (#9483)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Matt Krick <[email protected]>
Co-authored-by: Rafa <[email protected]>
Co-authored-by: Georg Bremer <[email protected]>
Co-authored-by: Nick O'Ferrall <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>
Co-authored-by: Matt Krick <[email protected]>
Co-authored-by: Jordan Husney <[email protected]>
Co-authored-by: adaniels-parabol <[email protected]>
Co-authored-by: snyk-bot <[email protected]>
Co-authored-by: Dale Bumblis <[email protected]>
Co-authored-by: github-actions <[email protected]>
mattkrick added a commit that referenced this pull request Mar 1, 2024
@github-actions @rafaelromcar-parabol
@Dschoordsch @nickoferrall @dependabot @parabol-release-bot @mattkrick @jordanh @adaniels-parabol @snyk-bot @dbumblis-parabol
chore(release): Test v7.20.0 (#9488)
75a5676 
* chore(env vars): Stripe vars moved to the Integrations section (#9427)

* chore: fix misleading `isLead` field name on `Team` (#9413)

* chore: fix misleading `isLead` field name on `Team`

The field indicates whether the viewer is the lead, but when used in a
query for a different user, the result could be read wrong.

* Fix Team.isLead dependencies

* feat: remove team template limit (#9424)

* update error message and increase template limit

* remove max team template limits

* remove canClone prop from CloneTemplate

* remove unused threshold

* remove unused threshold

* feat: Add Google calendar meeting series for recurrence (#9380)

* feat: Add recurrence to GCal events

* Fun with timezones

* fix: Increase the number of projects fetched per request from Atlassian (#9435)

We ran into timeouts in `getAllProjects`, presumably because we're doing
too many roundtrips. As a quick fix, increse the number of projects
fetched per request from 50 to 500.

* chore(deps): bump ip from 1.1.8 to 1.1.9 (#9442)

Bumps [ip](https://github.com/indutny/node-ip) from 1.1.8 to 1.1.9.
- [Commits](indutny/node-ip@v1.1.8...v1.1.9)

---
updated-dependencies:
- dependency-name: ip
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(release): release v7.17.0 (#9428)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* feat(standalone-deployment): Standalone host deployment improved and documented (#9445)

* Docker compose stack improved

* Remove unused containers from docker-compse and add useful comment on .env.example about PGSSLMODE

* Docker compose profiles added. Documentation extended on how to use the profiles to manage the stack.

* README fixed as docker compose up and down commands were not working

* Typo fixed and docker-compose command replaced by docker compose

* feat: support env-defined saml issuer for PPMIs (#9455)

* feat: support env-defined saml issuer for PPMIs

Signed-off-by: Matt Krick <[email protected]>

* feat: support single SAML for entire tenant

Signed-off-by: Matt Krick <[email protected]>

---------

Signed-off-by: Matt Krick <[email protected]>

* chore: Associate logs with traces (#9444)

* chore: Associate logs with traces

Add trace information to log output for server side log statements. This
does not include logging from code exclusively used for debugging,
deploying or development.

* Actually add the logger

* Fix DD_LOGS_INJECTION check

* chore(release): release v7.18.0 (#9450)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* chore: no force-push to prod (#9401)

Signed-off-by: Matt Krick <[email protected]>

* chore(release): release v7.18.1 (#9459)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* feat: embedder service (#9417)

* feat: add embedder service

---------

Signed-off-by: Matt Krick <[email protected]>
Co-authored-by: Matt Krick <[email protected]>

* merge production to avoid force push (#9461)

Signed-off-by: Matt Krick <[email protected]>

* chore(release): release v7.19.0 (#9460)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: checkout prod before merging it (#9463)

Signed-off-by: Matt Krick <[email protected]>

* chore(release): release v7.19.1 (#9464)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: mrege origin/production strategy (#9465)

Signed-off-by: Matt Krick <[email protected]>

* chore(release): release v7.19.2 (#9466)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: force push 5 (#9467)

Signed-off-by: Matt Krick <[email protected]>

* chore(release): release v7.19.3 (#9468)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: limit invites from spammers (#9416)

* fix: limit invites from spammers

* update where we check pending emails

* check total plus pending invites

* use invitees instead of pending

* fix: Fetch Jira projects in parallel (#9456)

Previously we tried to fetch more projects per page, but Jira only ever
returns 50 max. Instead once we know how many projects there are after
fetching the first page, we fetch all remaining pages in parallel.

* fix: replace lone surrogates in draft-js content (#9415)

* fix: replace lone surrogates in draft-js content

Signed-off-by: Matt Krick <[email protected]>

* fix typo

Signed-off-by: Matt Krick <[email protected]>

* fix: eslint errors

Signed-off-by: Matt Krick <[email protected]>

---------

Signed-off-by: Matt Krick <[email protected]>

* chore(deps): bump es5-ext from 0.10.62 to 0.10.64 (#9457)

Bumps [es5-ext](https://github.com/medikoo/es5-ext) from 0.10.62 to 0.10.64.
- [Release notes](https://github.com/medikoo/es5-ext/releases)
- [Changelog](https://github.com/medikoo/es5-ext/blob/main/CHANGELOG.md)
- [Commits](medikoo/es5-ext@v0.10.62...v0.10.64)

---
updated-dependencies:
- dependency-name: es5-ext
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: packages/server/package.json to reduce vulnerabilities (#9434)

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-UNDICI-6252336

Co-authored-by: snyk-bot <[email protected]>

* fix: packages/server/package.json to reduce vulnerabilities (#9392)

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-NODEMAILER-6219989

Co-authored-by: snyk-bot <[email protected]>

* fix: packages/server/package.json to reduce vulnerabilities (#9298)

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-FOLLOWREDIRECTS-6141137

Co-authored-by: snyk-bot <[email protected]>

* chore(deps): bump follow-redirects from 1.14.8 to 1.15.4 (#9312)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.14.8 to 1.15.4.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.14.8...v1.15.4)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Matt Krick <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matt Krick <[email protected]>

* chore: add upload to GCS step in ironbank (#9471)

* add upload to GCS step in ironbank

* update workflow name

* chore(release): release v7.19.4 (#9470)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: Fix seasonal templates for leap years (#9476)

* fix: Fix seasonal templates for leap years

It would produce invalid dates on February 29th.

* Master was not clean

* chore(release): release v7.19.5 (#9477)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix: After parameter for meetingCount was ignored (#9479)

* chore(docker-build): simplify the docker build process and reduce docker image size (#9447)

* Dockerfile basic created. Improvements added to reduce build time and size (down from 795MB to 445MB, depending on systemtap). Readme reduced, removing the old process used to build the image.

* basic-env file using a RethinkDB database name that is clearly dedicated to the building proces.

* Readme improved to run all three components

* Unused dockerfiles removed. Docker entrypoint renamed. Docker Readme adapted

* Legacy build kept in both dockerfile and env file. Readme adapted to use the new basic image. Build GH workflow adapted to use the new basic.dockerfile.

* chore(release): release v7.19.6 (#9480)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* fix(docker-build): home folder is /home/node now (#9482)

* chore(release): release v7.19.7 (#9483)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

* feat: OpenAIGeneration model for embedder (#9474)

* fix: support single-tenant saml record (#9486)

Signed-off-by: Matt Krick <[email protected]>

* chore(release): release v7.20.0 (#9485)

Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Matt Krick <[email protected]>
Co-authored-by: Rafa <[email protected]>
Co-authored-by: Georg Bremer <[email protected]>
Co-authored-by: Nick O'Ferrall <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: parabol-release-bot[bot] <150284312+parabol-release-bot[bot]@users.noreply.github.com>
Co-authored-by: Matt Krick <[email protected]>
Co-authored-by: Jordan Husney <[email protected]>
Co-authored-by: adaniels-parabol <[email protected]>
Co-authored-by: snyk-bot <[email protected]>
Co-authored-by: Dale Bumblis <[email protected]>
Co-authored-by: github-actions <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adaniels-parabol adaniels-parabol adaniels-parabol left review comments

@mattkrick mattkrick mattkrick approved these changes

@dbumblis-parabol dbumblis-parabol Awaiting requested review from dbumblis-parabol

Assignees

@rafaelromcar-parabol rafaelromcar-parabol

Labels
size/m
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rafaelromcar-parabol @mattkrick @adaniels-parabol