Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove unnecessary packages and make image compact #1209

Closed
axot opened this issue Oct 11, 2021 · 4 comments
Closed

Remove unnecessary packages and make image compact #1209

axot opened this issue Oct 11, 2021 · 4 comments

Comments

@axot
Copy link

axot commented Oct 11, 2021 •
edited
Loading

The PHPIZE_DEPS packages didn't be removed and we can also make a more compact image by merging all apt-get installed layers to a single layer.

gcc can be executed in current image

$ docker run --rm -ti php:8.0.11-fpm-bullseye gcc --version
gcc (Debian 10.2.1-6) 10.2.1 20210110
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Current layers

$ docker history php:8.0.11-fpm-bullseye
IMAGE          CREATED       CREATED BY                                      SIZE      COMMENT
d30aedb54bc6   13 days ago   /bin/sh -c #(nop)  CMD ["php-fpm"]              0B
<missing>      13 days ago   /bin/sh -c #(nop)  EXPOSE 9000                  0B
<missing>      13 days ago   /bin/sh -c #(nop)  STOPSIGNAL SIGQUIT           0B
<missing>      13 days ago   /bin/sh -c set -eux;  cd /usr/local/etc;  if…   26.3kB
<missing>      13 days ago   /bin/sh -c #(nop) WORKDIR /var/www/html         0B
<missing>      13 days ago   /bin/sh -c #(nop)  ENTRYPOINT ["docker-php-e…   0B
<missing>      13 days ago   /bin/sh -c docker-php-ext-enable sodium         17B
<missing>      13 days ago   /bin/sh -c #(nop) COPY multi:6dfba8f7e64bd54…   6.75kB
<missing>      13 days ago   /bin/sh -c set -eux;   savedAptMark="$(apt-m…   95.4MB
<missing>      13 days ago   /bin/sh -c #(nop) COPY file:ce57c04b70896f77…   587B
<missing>      13 days ago   /bin/sh -c set -eux;   savedAptMark="$(apt-m…   12MB
<missing>      13 days ago   /bin/sh -c #(nop)  ENV PHP_SHA256=e3e5f764ae…   0B
<missing>      13 days ago   /bin/sh -c #(nop)  ENV PHP_URL=https://www.p…   0B
<missing>      13 days ago   /bin/sh -c #(nop)  ENV PHP_VERSION=8.0.11       0B
<missing>      13 days ago   /bin/sh -c #(nop)  ENV GPG_KEYS=1729F83938DA…   0B
<missing>      13 days ago   /bin/sh -c #(nop)  ENV PHP_LDFLAGS=-Wl,-O1 -…   0B
<missing>      13 days ago   /bin/sh -c #(nop)  ENV PHP_CPPFLAGS=-fstack-…   0B
<missing>      13 days ago   /bin/sh -c #(nop)  ENV PHP_CFLAGS=-fstack-pr…   0B
<missing>      13 days ago   /bin/sh -c #(nop)  ENV PHP_EXTRA_CONFIGURE_A…   0B
<missing>      13 days ago   /bin/sh -c set -eux;  mkdir -p "$PHP_INI_DIR…   0B
<missing>      13 days ago   /bin/sh -c #(nop)  ENV PHP_INI_DIR=/usr/loca…   0B
<missing>      13 days ago   /bin/sh -c set -eux;  apt-get update;  apt-g…   249MB
<missing>      13 days ago   /bin/sh -c #(nop)  ENV PHPIZE_DEPS=autoconf …   0B
<missing>      13 days ago   /bin/sh -c set -eux;  {   echo 'Package: php…   46B
<missing>      13 days ago   /bin/sh -c #(nop)  CMD ["bash"]                 0B
<missing>      13 days ago   /bin/sh -c #(nop) ADD file:6472ab63529e68873…   74.3MB

A new test Dockerfile

diff --git i/8.0/bullseye/fpm/Dockerfile w/8.0/bullseye/fpm/Dockerfile
index f10af51..dbe11c9 100644
--- i/8.0/bullseye/fpm/Dockerfile
+++ w/8.0/bullseye/fpm/Dockerfile
@@ -28,17 +28,6 @@ ENV PHPIZE_DEPS \
        pkg-config \
        re2c

-# persistent / runtime deps
-RUN set -eux; \
-   apt-get update; \
-   apt-get install -y --no-install-recommends \
-       $PHPIZE_DEPS \
-       ca-certificates \
-       curl \
-       xz-utils \
-   ; \
-   rm -rf /var/lib/apt/lists/*
-
 ENV PHP_INI_DIR /usr/local/etc/php
 RUN set -eux; \
    mkdir -p "$PHP_INI_DIR/conf.d"; \
@@ -66,11 +55,17 @@ ENV PHP_VERSION 8.0.11
 ENV PHP_URL="https://www.php.net/distributions/php-8.0.11.tar.xz" PHP_ASC_URL="https://www.php.net/distributions/php-8.0.11.tar.xz.asc"
 ENV PHP_SHA256="e3e5f764ae57b31eb65244a45512f0b22d7bef05f2052b23989c053901552e16"

+COPY docker-php-source /usr/local/bin/
+
 RUN set -eux; \
    \
    savedAptMark="$(apt-mark showmanual)"; \
    apt-get update; \
-   apt-get install -y --no-install-recommends gnupg dirmngr; \
+   apt-get install -y --no-install-recommends gnupg dirmngr \
+       ca-certificates \
+       curl \
+       xz-utils \
+   ; \
    rm -rf /var/lib/apt/lists/*; \
    \
    mkdir -p /usr/src; \
@@ -94,16 +89,12 @@ RUN set -eux; \
    fi; \
    \
    apt-mark auto '.*' > /dev/null; \
-   apt-mark manual $savedAptMark > /dev/null; \
-   apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false
-
-COPY docker-php-source /usr/local/bin/
-
-RUN set -eux; \
-   \
+   [ -z "$savedAptMark" ] || apt-mark manual $savedAptMark > /dev/null; \
+   apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false ; \
    savedAptMark="$(apt-mark showmanual)"; \
    apt-get update; \
    apt-get install -y --no-install-recommends \
+       $PHPIZE_DEPS \
        ${PHP_EXTRA_BUILD_DEPS:-} \
        libargon2-dev \
        libcurl4-openssl-dev \

New overlay layers

IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT
74ab7cf3e4c7   13 minutes ago   CMD ["php-fpm"]                                 0B        buildkit.dockerfile.v0
<missing>      13 minutes ago   EXPOSE map[9000/tcp:{}]                         0B        buildkit.dockerfile.v0
<missing>      13 minutes ago   STOPSIGNAL SIGQUIT                              0B        buildkit.dockerfile.v0
<missing>      13 minutes ago   RUN /bin/sh -c set -eux;  cd /usr/local/etc;…   26.3kB    buildkit.dockerfile.v0
<missing>      13 minutes ago   WORKDIR /var/www/html                           0B        buildkit.dockerfile.v0
<missing>      13 minutes ago   ENTRYPOINT ["docker-php-entrypoint"]            0B        buildkit.dockerfile.v0
<missing>      13 minutes ago   RUN /bin/sh -c docker-php-ext-enable sodium …   17B       buildkit.dockerfile.v0
<missing>      13 minutes ago   COPY docker-php-ext-* docker-php-entrypoint …   6.75kB    buildkit.dockerfile.v0
<missing>      13 minutes ago   RUN /bin/sh -c set -eux;   savedAptMark="$(a…   127MB     buildkit.dockerfile.v0
<missing>      27 minutes ago   COPY docker-php-source /usr/local/bin/ # bui…   587B      buildkit.dockerfile.v0
<missing>      27 minutes ago   ENV PHP_SHA256=e3e5f764ae57b31eb65244a45512f…   0B        buildkit.dockerfile.v0
<missing>      27 minutes ago   ENV PHP_URL=https://www.php.net/distribution…   0B        buildkit.dockerfile.v0
<missing>      27 minutes ago   ENV PHP_VERSION=8.0.11                          0B        buildkit.dockerfile.v0
<missing>      27 minutes ago   ENV GPG_KEYS=1729F83938DA44E27BA0F4D3DBDB397…   0B        buildkit.dockerfile.v0
<missing>      27 minutes ago   ENV PHP_LDFLAGS=-Wl,-O1 -pie                    0B        buildkit.dockerfile.v0
<missing>      27 minutes ago   ENV PHP_CPPFLAGS=-fstack-protector-strong -f…   0B        buildkit.dockerfile.v0
<missing>      27 minutes ago   ENV PHP_CFLAGS=-fstack-protector-strong -fpi…   0B        buildkit.dockerfile.v0
<missing>      27 minutes ago   ENV PHP_EXTRA_CONFIGURE_ARGS=--enable-fpm --…   0B        buildkit.dockerfile.v0
<missing>      27 minutes ago   RUN /bin/sh -c set -eux;  mkdir -p "$PHP_INI…   0B        buildkit.dockerfile.v0
<missing>      27 minutes ago   ENV PHP_INI_DIR=/usr/local/etc/php              0B        buildkit.dockerfile.v0
<missing>      27 minutes ago   ENV PHPIZE_DEPS=autoconf   dpkg-dev   file  …   0B        buildkit.dockerfile.v0
<missing>      32 minutes ago   RUN /bin/sh -c set -eux;  {   echo 'Package:…   46B       buildkit.dockerfile.v0
<missing>      13 days ago      /bin/sh -c #(nop)  CMD ["bash"]                 0B
<missing>      13 days ago      /bin/sh -c #(nop) ADD file:6472ab63529e68873…   74.3MB

Image size comparison

REPOSITORY                    TAG                   IMAGE ID       CREATED          SIZE
php-reduced                   8.0.11-fpm-bullseye   74ab7cf3e4c7   12 minutes ago   201MB
php                           8.0.11-fpm-bullseye   d30aedb54bc6   13 days ago      431MB
@yosifkit
Copy link
Member

Duplicate of #769 (see also #513 #438 #557 #751 #716 #778)

gcc/g++ and all of the PHPIZE_DEPS packages are left in the Debian based images on purpose:

php/8.0/bullseye/apache/Dockerfile

Line 18 in 1e6fd3b

# dependencies required for running "phpize"

make is used directly in the docker-php-ext-install script (and thus needs all the other packages in the PHPIZE_DEPS list). For the alpine based image we can quickly add and remove packages without changing what was manually added by the user in the docker-php-ext* scripts, but apt is much to slow and not idempotent enough for that.

There are no packages or large files that are deleted in a later layer (and if there is, that is a bug we can fix), so there is no need to combine the layers.

@mvorisek
Copy link
Contributor

Hi, we are affected by this issue as well and the 130 MB for us is substancial as we test 5 different PHP versions and the numbers sum up.

For the alpine based image we can quickly add and remove packages without changing what was manually added by the user in the docker-php-ext* scripts, but apt is much to slow and not idempotent enough for that.

can you please help me to understand this sentence? why on Alpine, it is fine, but not on Debian?

@yosifkit
Copy link
Member

apt is much to slow and not idempotent enough for that

  1. apt update + install is very slow in comparison to apk add --no-cache
  2. apt purge --auto-remove [packages] does not always remove the same set of packages that was added with apt install [packages] even when using the exact same list of packages
    • apk add also has --virtual so we also can track if the scripts installed them (php-ext-configure vs php-ext-install) and where they should be uninstalled.

the 130 MB for us is substancial as we test 5 different PHP versions and the numbers sum up.

If you are using the same Debian base for each of the php versions, they should not be taking extra space. See that the layer Already exists when you pull the second image. We've designed the Dockerfiles and build system to share layers where possible.

$ docker pull php:8.0-bullseye
8.0-bullseye: Pulling from library/php
5eb5b503b376: Pull complete 
8b1ad84cf101: Pull complete 
38c937dadeb7: Pull complete 
6a2f1dc96e59: Pull complete 
f7edb6b97031: Pull complete 
0a4e724cb836: Pull complete 
594d2dab6131: Pull complete 
2f995ce336de: Pull complete 
e3614e4d9c0c: Pull complete 
Digest: sha256:2bbd3f211f54808c9dad828370ea991e3d286b609e9cd5898616bec6dfc809b1
Status: Downloaded newer image for php:8.0-bullseye
docker.io/library/php:8.0-bullseye
$ docker pull php:8.1-bullseye
8.1-bullseye: Pulling from library/php
5eb5b503b376: Already exists 
8b1ad84cf101: Already exists 
38c937dadeb7: Already exists 
6a2f1dc96e59: Already exists 
e24b294127d9: Pull complete 
fd1d71e39585: Pull complete 
809a0df88a45: Pull complete 
f6d19fea490e: Pull complete 
b40e1e711ca8: Pull complete 
Digest: sha256:df1c92275635d52c5b5f460bfa2fd33726e9d9baf61505836ca937cea8e38a85
Status: Downloaded newer image for php:8.1-bullseye
docker.io/library/php:8.1-bullseye

Which corresponds to the four layers without #(nop) in the history (starting from the bottom).

$ docker history php:8.1-bullseye
IMAGE          CREATED       CREATED BY                                      SIZE      COMMENT
e2aa0e000dd1   9 days ago    /bin/sh -c #(nop)  CMD ["php" "-a"]             0B        
<missing>      9 days ago    /bin/sh -c #(nop)  ENTRYPOINT ["docker-php-e…   0B        
<missing>      9 days ago    /bin/sh -c docker-php-ext-enable sodium         17B       
<missing>      9 days ago    /bin/sh -c #(nop) COPY multi:a00980ff863125d…   6.86kB    
<missing>      9 days ago    /bin/sh -c set -eux;   savedAptMark="$(apt-m…   121MB     
<missing>      9 days ago    /bin/sh -c #(nop) COPY file:ce57c04b70896f77…   587B      
<missing>      9 days ago    /bin/sh -c set -eux;   savedAptMark="$(apt-m…   13MB      
<missing>      9 days ago    /bin/sh -c #(nop)  ENV PHP_SHA256=5d65a11071…   0B        
<missing>      9 days ago    /bin/sh -c #(nop)  ENV PHP_URL=https://www.p…   0B        
<missing>      9 days ago    /bin/sh -c #(nop)  ENV PHP_VERSION=8.1.3        0B        
<missing>      4 weeks ago   /bin/sh -c #(nop)  ENV GPG_KEYS=528995BFEDFB…   0B        
<missing>      4 weeks ago   /bin/sh -c #(nop)  ENV PHP_LDFLAGS=-Wl,-O1 -…   0B        
<missing>      4 weeks ago   /bin/sh -c #(nop)  ENV PHP_CPPFLAGS=-fstack-…   0B        
<missing>      4 weeks ago   /bin/sh -c #(nop)  ENV PHP_CFLAGS=-fstack-pr…   0B        
<missing>      4 weeks ago   /bin/sh -c set -eux;  mkdir -p "$PHP_INI_DIR…   0B        
<missing>      4 weeks ago   /bin/sh -c #(nop)  ENV PHP_INI_DIR=/usr/loca…   0B        
<missing>      4 weeks ago   /bin/sh -c set -eux;  apt-get update;  apt-g…   269MB     
<missing>      4 weeks ago   /bin/sh -c #(nop)  ENV PHPIZE_DEPS=autoconf …   0B        
<missing>      4 weeks ago   /bin/sh -c set -eux;  {   echo 'Package: php…   46B       
<missing>      4 weeks ago   /bin/sh -c #(nop)  CMD ["bash"]                 0B        
<missing>      4 weeks ago   /bin/sh -c #(nop) ADD file:90495c24c897ec479…   80.4MB

@mvorisek
Copy link
Contributor

If you are using the same Debian base for each of the php versions, they should not be taking extra space

thank you for pointing this out, yes, at the end, the decision seems very logical and helpful

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@axot @yosifkit @mvorisek