Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce size of local Docker images by fixing .dockerignore patterns #2360

Merged
merged 3 commits into from
Jan 30, 2023

Conversation

NevilleS
Copy link
Contributor

@NevilleS NevilleS commented Jan 25, 2023

Code Changes

  • Fix .dockerignore paths to ignore subdirectories

Steps to Confirm

  • Run nox -s test_env to confirm everything builds as expected and runs nicely

Pre-Merge Checklist

Description Of Changes

We've all noticed that our local Docker development is slow, resource-intensive, and requires a lot of rebuilding (and often lots of cleaning out Docker files!).

In my own local testing I inspected the size of my local dev image and was pretty shocked that it was 2.3 GB! After some tinkering and troubleshooting I realized that all the local build artifacts from both Python (e.g. *.pyc files) and NextJS (e.g. .next/) were getting copied into the local image, which has at least two painful consequences:

  1. Bloats the size of the dev images locally, which increases RAM utilization, eats up Docker disk space, and generally slows down performance
  2. Busts the cache of the COPY . fides/ command in the webserver build, forcing unnecessary rebuilds

The culprit here is just that the .dockerignore patterns are relative to the Dockerfile... so .next or *.pyc only ignore those folders/files in the root of the repo. Changing these to **/.next correctly ignores these from all the subdirectories.

Here's the before/after of docker image ls ethyca/fides:local on my machine:

Before (Neville's Machine) After
2.31 GB 1.31 GB

I don't think this'll have a massive benefit on CI builds, since those won't have those types of local build artifacts lying around, but I suspect this'll be a subtle boost to local development. Hard to quantify exactly how much it'll help yet...

@NevilleS NevilleS changed the title Reduce size of local Docker images and improve cacheability Reduce size of local Docker images by fixing .dockerignore patterns Jan 25, 2023
@NevilleS NevilleS added the run unsafe ci checks Runs fides-related CI checks that require sensitive credentials label Jan 25, 2023
@NevilleS NevilleS merged commit 56d9e56 into main Jan 30, 2023
@NevilleS NevilleS deleted the ns-docker-optimization branch January 30, 2023 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
run unsafe ci checks Runs fides-related CI checks that require sensitive credentials
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants