Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: optimize image layers in base images #2212

Merged
merged 1 commit into from
Apr 17, 2024
Merged

Conversation

VietND96
Copy link
Member

@VietND96 VietND96 commented Apr 17, 2024

User description

Thanks for contributing to the Docker-Selenium project!
A PR well described will help maintainers to quickly review and merge it

Before submitting your PR, please check our contributing guidelines, applied for this repository.
Avoid large PRs, help reviewers by making them as simple and short as possible.

Description

Motivation and Context

Update for #2210

With enough CI tests, I think there is no breakage.
Batch 1 update for Base and NodeBase images. Diff can be seen as below

Before

docker history selenium/base:trunk-20240417 -q | wc -l
48
docker history selenium/hub:trunk-20240417 -q | wc -l
60
docker history selenium/distributor:trunk-20240417 -q | wc -l
57
docker history selenium/router:trunk-20240417 -q | wc -l
56
docker history selenium/sessions:trunk-20240417 -q | wc -l
54
docker history selenium/session-queue:trunk-20240417 -q | wc -l
56
docker history selenium/event-bus:trunk-20240417 -q | wc -l
56
docker history selenium/node-base:trunk-20240417 -q | wc -l
101
docker history selenium/node-chrome:trunk-20240417 -q | wc -l
116
docker history selenium/node-edge:trunk-20240417 -q | wc -l
116
docker history selenium/node-firefox:trunk-20240417 -q | wc -l
114
docker history selenium/node-docker:trunk-20240417 -q | wc -l
56
docker history selenium/standalone-chrome:trunk-20240417 -q | wc -l
127
docker history selenium/standalone-edge:trunk-20240417 -q | wc -l
127
docker history selenium/standalone-firefox:trunk-20240417 -q | wc -l
125
docker history selenium/standalone-docker:trunk-20240417 -q | wc -l
64
docker history selenium/video:ffmpeg-7.0-20240417 -q | wc -l
53

After

docker history selenium/base:trunk-20240417 -q | wc -l
29
docker history selenium/hub:trunk-20240417 -q | wc -l
41
docker history selenium/distributor:trunk-20240417 -q | wc -l
38
docker history selenium/router:trunk-20240417 -q | wc -l
37
docker history selenium/sessions:trunk-20240417 -q | wc -l
35
docker history selenium/session-queue:trunk-20240417 -q | wc -l
37
docker history selenium/event-bus:trunk-20240417 -q | wc -l
37
docker history selenium/node-base:trunk-20240417 -q | wc -l
45
docker history selenium/node-chrome:trunk-20240417 -q | wc -l
60
docker history selenium/node-edge:trunk-20240417 -q | wc -l
60
docker history selenium/node-firefox:trunk-20240417 -q | wc -l
58
docker history selenium/node-docker:trunk-20240417 -q | wc -l
37
docker history selenium/standalone-chrome:trunk-20240417 -q | wc -l
71
docker history selenium/standalone-edge:trunk-20240417 -q | wc -l
71
docker history selenium/standalone-firefox:trunk-20240417 -q | wc -l
69
docker history selenium/standalone-docker:trunk-20240417 -q | wc -l
45
docker history selenium/video:ffmpeg-7.0-20240417 -q | wc -l
53

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • I have read the contributing document.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Type

enhancement, configuration changes


Description

  • Updated GitHub Actions workflows to use AUTHORS instead of GH_ORG environment variable.
  • Optimized Dockerfile for Base and NodeBase images by introducing new environment variables, including locale and timezone settings, and Selenium configuration options.
  • Improved Docker layer caching and reduced the number of layers in Docker images.
  • Aligned Makefile with Dockerfile changes by replacing GH_ORG with AUTHORS.

Changes walkthrough

Relevant files
Configuration changes
build-test.yml
Update GitHub Actions Workflow for Build and Test               

.github/workflows/build-test.yml

  • Replaced GH_ORG environment variable with AUTHORS in the GitHub
    Actions workflow for build and test.
  • +2/-2     
    deploy.yml
    Modify Deploy Workflow Environment Variables                         

    .github/workflows/deploy.yml

  • Updated environment variable from GH_ORG to AUTHORS in the deploy
    workflow.
  • +1/-1     
    helm-chart-test.yml
    Adjust Helm Chart Test Workflow Variables                               

    .github/workflows/helm-chart-test.yml

  • Switched GH_ORG to AUTHORS in the environment variables for Helm chart
    testing.
  • +2/-2     
    nightly.yaml
    Update Nightly Build Workflow Environment Settings             

    .github/workflows/nightly.yaml

  • Changed GH_ORG to AUTHORS in the nightly build workflow environment
    settings.
  • +2/-2     
    test-video.yml
    Update Video Test Workflow Environment Variables                 

    .github/workflows/test-video.yml

  • Updated GH_ORG to AUTHORS in the video test workflow environment
    variables.
  • +2/-2     
    Makefile
    Align Makefile with Dockerfile Environment Variable Changes

    Makefile

  • Replaced GH_ORG with AUTHORS in the Makefile to align with Dockerfile
    changes.
  • +2/-3     
    Enhancement
    Dockerfile
    Optimize Base Image and Update Environment Variables         

    Base/Dockerfile

  • Replaced GH_ORG with AUTHORS and added several new environment
    variables and configurations.
  • Introduced timezone and locale settings, and optimized layer
    instructions for better caching.
  • +58/-82 
    Dockerfile
    Enhance NodeBase Image with Locale Settings and Selenium
    Configurations

    NodeBase/Dockerfile

  • Introduced locale and encoding settings, and various Selenium
    configuration environment variables.
  • Optimized installation and cleanup steps for better Docker layer
    caching.
  • +83/-151

    PR-Agent usage:
    Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    Copy link

    PR Description updated to latest commit (b70efcd)

    Copy link

    PR Review

    ⏱️ Estimated effort to review [1-5]

    4, because the PR involves multiple files across different workflows and Dockerfiles, indicating a broad impact on the project's build and deployment processes. The changes include modifications to environment variables, Docker build arguments, and the introduction of new configurations, which require a thorough review to ensure compatibility and correctness.

    🧪 Relevant tests

    No

    🔍 Possible issues

    Possible Issue: Environment Variable Changes: The replacement of GH_ORG with AUTHORS in GitHub Actions workflows and Dockerfiles might affect scripts or processes that rely on the old variable. It's crucial to verify that all references to GH_ORG have been updated accordingly.

    Possible Issue: Dockerfile Modifications: The extensive changes in Dockerfile instructions, including the addition of new environment variables and changes in the base image setup, require careful review to ensure they don't introduce any regressions or compatibility issues with existing deployments.

    Possible Issue: Locale and Encoding Settings: The introduction of locale and encoding settings in the NodeBase/Dockerfile could potentially affect the behavior of the containers, especially in terms of language support and character encoding. It's important to validate these changes across different environments.

    🔒 Security concerns

    No


    ✨ Review tool usage guide:

    Overview:
    The review tool scans the PR code changes, and generates a PR review which includes several types of feedbacks, such as possible PR issues, security threats and relevant test in the PR. More feedbacks can be added by configuring the tool.

    The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.

    • When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:
    /review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...
    
    [pr_reviewer]
    some_config1=...
    some_config2=...
    

    See the review usage page for a comprehensive guide on using this tool.

    Copy link

    PR Code Suggestions

    CategorySuggestions                                                                                                                                                       
    Enhancement
    Conditionally run apt-get upgrade to optimize build times and layer changes.

    Instead of using apt-get upgrade -yq directly, consider adding a condition to check if
    upgrades are necessary. This can help reduce build times and avoid unnecessary layer
    changes when no packages need upgrading. You can use apt list --upgradable to check if any
    packages need upgrading and then run apt-get upgrade only if necessary.

    Base/Dockerfile [43]

    -&& apt-get upgrade -yq \
    +&& apt list --upgradable | grep -q 'upgradable from' && apt-get upgrade -yq || echo "No packages need upgrading." \
     
    Use multi-stage builds to optimize Docker image size and security.

    Consider using multi-stage builds to optimize the size and security of your Docker images.
    This can help you separate the build environment from the runtime environment, reducing
    the final image size and minimizing the attack surface by including only the necessary
    artifacts and dependencies in the final image.

    Base/Dockerfile [1]

    +# Build stage
    +FROM ubuntu as builder
    +# Build instructions here
    +
    +# Final stage
     FROM ubuntu
    +COPY --from=builder /path/to/artifact /path/to/target
     
    Combine multiple ENV instructions into a single instruction to reduce image layers.

    Combine ENV instructions to reduce the number of layers created in the Docker image.
    Multiple ENV instructions can be combined into a single instruction to make the Dockerfile
    more efficient and reduce the overall size of the image.

    NodeBase/Dockerfile [24-28]

     ENV LANG_WHICH=${LANG_WHICH} \
         LANG_WHERE=${LANG_WHERE} \
         ENCODING=${ENCODING} \
         LANGUAGE=${LANG_WHICH}_${LANG_WHERE}.${ENCODING} \
    -    LANG=${LANGUAGE} \
    +    LANG=${LANGUAGE}
     
    Best practice
    Pin package versions in apt-get install to ensure reproducible builds.

    Consider pinning the versions of the packages you are installing with apt-get install to
    ensure reproducible builds and avoid unexpected changes when the packages are updated in
    the repositories. This can help make your Docker builds more predictable and stable over
    time.

    Base/Dockerfile [44-45]

     && apt-get -qqy --no-install-recommends install \
    -  acl \
    +  acl=2.2.53-6 \
     
    Use specific base image tags to ensure predictable builds.

    Use a more specific base image tag instead of relying on the default latest tag. This can
    help ensure that your builds are more predictable and that updates to the base image do
    not unexpectedly break your Dockerfile.

    Base/Dockerfile [1]

    -FROM ubuntu
    +FROM ubuntu:20.04
     
    Use specific version tags instead of "master" for ARGs to ensure reproducible builds.

    Use a more specific tag instead of "master" for the ARG NOVNC_VERSION and
    WEBSOCKIFY_VERSION to ensure reproducible builds and avoid potential breaks when the
    master branch is updated.

    NodeBase/Dockerfile [12-14]

    -ARG NOVNC_VERSION="master"
    -ARG WEBSOCKIFY_VERSION="master"
    +ARG NOVNC_VERSION="specific_version_tag"
    +ARG WEBSOCKIFY_VERSION="specific_version_tag"
     
    Add a cleanup step after installations to reduce the Docker image size.

    Consider adding a cleanup step after package installations to reduce the Docker image
    size. This can be done by removing the apt cache and unnecessary files after installation.

    NodeBase/Dockerfile [68-76]

     RUN apt-get update -qqy \
       && apt-get -qqy --no-install-recommends install \
    -  xvfb \
    -  xauth \
    -  pulseaudio \
    +  xvfb xauth pulseaudio \
    +  && apt-get clean \
    +  && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
     
    Use COPY with --chown to directly set file ownership, reducing layers.

    Use COPY with the --chown flag to set the ownership directly instead of using chown
    command separately. This reduces layers and improves build performance.

    NodeBase/Dockerfile [137-141]

    -COPY --chown="${SEL_UID}:${SEL_GID}" start-selenium-node.sh \
    -    start-xvfb.sh \
    -    start-vnc.sh \
    -    start-novnc.sh \
    -    generate_config /opt/bin/
    +COPY --chown="${SEL_UID}:${SEL_GID}" start-selenium-node.sh start-xvfb.sh start-vnc.sh start-novnc.sh generate_config /opt/bin/
     
    Security
    Set a non-root user for running the container for enhanced security.

    For better security, consider explicitly setting a non-root user for running your
    container. This can help mitigate potential risks and is a best practice for Docker
    container security.

    Base/Dockerfile [23]

    -USER root
    +USER ${SEL_UID}:${SEL_GID}
     
    Avoid making /dev/shm executable to reduce security risks.

    Avoid using chmod +x /dev/shm as it can introduce security vulnerabilities by making
    shared memory executable. Consider alternative approaches to address the underlying issue.

    NodeBase/Dockerfile [123]

    -&& chmod +x /dev/shm \
    +# Consider alternative security measures or configurations
     

    ✨ Improve tool usage guide:

    Overview:
    The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.

    • When commenting, to edit configurations related to the improve tool (pr_code_suggestions section), use the following template:
    /improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...
    
    [pr_code_suggestions]
    some_config1=...
    some_config2=...
    

    See the improve usage page for a comprehensive guide on using this tool.

    Copy link
    Member

    @diemol diemol left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    It might get hard to debug when one of the commands fails, but let's see how it goes.

    Thank you, @VietND96!

    @VietND96 VietND96 merged commit 55e05b1 into SeleniumHQ:trunk Apr 17, 2024
    11 of 12 checks passed
    @VietND96 VietND96 linked an issue Apr 17, 2024 that may be closed by this pull request
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    [🚀 Feature]: Decreasing container layers
    2 participants