Improve our Docker game

[Robinlovelace]

Currently we use

geocompy/.devcontainer.json

Line 2 in 2f0c38a

"image": "ghcr.io/geocompx/docker:python",

in the devcontainer

geocompy/.github/workflows/pr.yml

Line 8 in 2f0c38a

container: ghcr.io/geocompx/docker:python

in the workflow

These are not optimal in some ways, as already pointed out in other issues.

With apologies for out-of-the blue tagging @benz0li, is this something you could advise us on? Two options that jump to mind are:

• Update Python containers in https://github.com/geocompx/docker to build on your images (I like this option because it means we can control exactly the dependencies, maximising flexibility while keeping image sizes small)
• Use one of your images directly

I'm thinking the great images you created, especially https://github.com/b-data/jupyterlab-python-docker-stack look really good in this space, happy to contribute upstream + to collaborate.

[benz0li]

I'm thinking the great images you created, especially https://github.com/b-data/jupyterlab-python-docker-stack look really good in this space, happy to contribute upstream + to collaborate.

@Robinlovelace For Dev Containers you should build on my/b-data's language docker stacks. That is what I/b-data is using for the [CUDA-enabled] Data Science Dev Containers.

E.g. Python scipy: devcontainer.json, Python.Dockerfile

[benz0li]

But I think R qgisprocess is the one you are looking for: devcontainer.json, R.Dockerfile.
ℹ️ R and Julia images have Python (currently v3.11.5) installed, too.

[Robinlovelace]

OK to chat over this at some point @benz0li ? I sense a few minutes of chat could save many minutes if not hours of faff 🙏

[benz0li]

You can test the Data Science Dev Containers with GitHub Codespaces at https://github.com/b-data/data-science-devcontainers. (New with options...)

See also https://docs.github.com/en/codespaces/developing-in-codespaces/creating-a-codespace-for-a-repository#creating-a-codespace-for-a-repository

[benz0li]

OK to chat over this at some point @benz0li ? I sense a few minutes of chat could save many minutes if not hours of faff 🙏

@Robinlovelace Yes, of course. See my GitHub profile for my email address.

[benz0li]

Cross references:

• Update .devcontainer.json #175 (comment)
• Codespaces Support #173 (comment)

[Robinlovelace]

Great stuff Olivier. Looking forward to talking next week.

[Robinlovelace]

One option: use 'slim' version to reduce size.

[Robinlovelace]

From docker build python it installs 2GB+ uncompressed:

docker image ls | rg geop
geopython-test                                                                            latest     877683b000ac   About a minute ago   2.33GB

[Robinlovelace]

FROM glcr.b-data.ch/python/base:3.11.5
RUN wget https://github.com/quarto-dev/quarto-cli/releases/download/v1.3.433/quarto-1.3.433-linux-amd64.deb
RUN dpkg -i quarto*
RUN rm quarto*
RUN wget https://raw.githubusercontent.com/geocompx/py/main/requirements.txt
RUN pip install -r requirements.txt

Results in an image that's around 600 MB bigger...

[Robinlovelace]

docker build python
[+] Building 179.1s (10/10) FINISHED                                                                               docker:default
 => [internal] load .dockerignore                                                                                            0.0s
 => => transferring context: 2B                                                                                              0.0s
 => [internal] load build definition from Dockerfile                                                                         0.1s
 => => transferring dockerfile: 333B                                                                                         0.0s
 => [internal] load metadata for glcr.b-data.ch/python/base:3.11.5                                                           0.8s
 => [1/6] FROM glcr.b-data.ch/python/base:3.11.5@sha256:20a182fc6675854c13496335de45a192eff309ead41530ab9ae7c30563d320fa    63.2s
 => => resolve glcr.b-data.ch/python/base:3.11.5@sha256:20a182fc6675854c13496335de45a192eff309ead41530ab9ae7c30563d320fa     0.0s
 => => sha256:a1f150f13ed7cb29f5dee29e77f61232a3a7cfe3f70dfa16d3c974a7e7db52c7 1.59kB / 1.59kB                               0.0s
 => => sha256:ebee21dc25c6ffac608c92f460949589624b5a2977d6204430f5c7a604d81a1e 7.93kB / 7.93kB                               0.0s
 => => sha256:f30ef1ae4a71870f6b98644d8c7c6a57fa85898bc3e72462c4a0e46d17db5f9a 313.57MB / 313.57MB                          48.9s
 => => sha256:20a182fc6675854c13496335de45a192eff309ead41530ab9ae7c30563d320fa 772B / 772B                                   0.0s
 => => sha256:9776ffde2df0b9a5a03cccbb0abbe6cb6d529870410fc5ea840d1d70ee20020d 33.69MB / 33.69MB                             7.3s
 => => sha256:878637ecdc5facdf7e31f08004095c4e4523c26bb8d89797c9700fda9d1a1804 4.69MB / 4.69MB                               2.1s
 => => extracting sha256:9776ffde2df0b9a5a03cccbb0abbe6cb6d529870410fc5ea840d1d70ee20020d                                    1.2s
 => => extracting sha256:878637ecdc5facdf7e31f08004095c4e4523c26bb8d89797c9700fda9d1a1804                                    0.1s
 => => extracting sha256:f30ef1ae4a71870f6b98644d8c7c6a57fa85898bc3e72462c4a0e46d17db5f9a                                   13.8s
 => [2/6] RUN wget https://github.com/quarto-dev/quarto-cli/releases/download/v1.3.433/quarto-1.3.433-linux-amd64.deb        9.0s
 => [3/6] RUN dpkg -i quarto*                                                                                                3.4s
 => [4/6] RUN rm quarto*                                                                                                     0.5s
 => [5/6] RUN wget https://raw.githubusercontent.com/geocompx/py/main/requirements.txt                                       4.1s
 => [6/6] RUN pip install -r requirements.txt                                                                               88.0s
 => exporting to image                                                                                                       9.9s
 => => exporting layers                                                                                                      9.9s
 => => writing image sha256:d8a7143c4d9fd0fa5b15d6cae9c590d02a23a2b624919f20666b3c91bf20ece8                                 0.0s
robin@robin-Proteus23:~/github/geocompx/docker$ docker build python -t geopy-test2
[+] Building 0.7s (10/10) FINISHED                                                                                 docker:default
 => [internal] load build definition from Dockerfile                                                                         0.0s
 => => transferring dockerfile: 333B                                                                                         0.0s
 => [internal] load .dockerignore                                                                                            0.0s
 => => transferring context: 2B                                                                                              0.0s
 => [internal] load metadata for glcr.b-data.ch/python/base:3.11.5                                                           0.5s
 => [1/6] FROM glcr.b-data.ch/python/base:3.11.5@sha256:20a182fc6675854c13496335de45a192eff309ead41530ab9ae7c30563d320fa     0.0s
 => CACHED [2/6] RUN wget https://github.com/quarto-dev/quarto-cli/releases/download/v1.3.433/quarto-1.3.433-linux-amd64.de  0.0s
 => CACHED [3/6] RUN dpkg -i quarto*                                                                                         0.0s
 => CACHED [4/6] RUN rm quarto*                                                                                              0.0s
 => CACHED [5/6] RUN wget https://raw.githubusercontent.com/geocompx/py/main/requirements.txt                                0.0s
 => CACHED [6/6] RUN pip install -r requirements.txt                                                                         0.0s
 => exporting to image                                                                                                       0.0s
 => => exporting layers                                                                                                      0.0s
 => => writing image sha256:d8a7143c4d9fd0fa5b15d6cae9c590d02a23a2b624919f20666b3c91bf20ece8                                 0.0s
 => => naming to docker.io/library/geopy-test2                                                                               0.0s
robin@robin-Proteus23:~/github/geocompx/docker$ docker image ls | rg test2
geopy-test2                                                                               latest     d8a7143c4d9f   52 seconds ago   2.91GB

[Robinlovelace]

Good news on this, thanks to mini pair programming session with @benz0li, the main Python image is ~30% smaller than it was, previously 900MB, this is impressive given the number of pkgs in there, thanks for the tip about using the slim images Olivier!

There are other improvements we can make, including making all images start from the same base, for R and Python, we can continue the conversation here: geocompx/docker#40

I think I'm happy with the updated

docker pull ghcr.io/geocompx/docker:python

image but will leave this issue open for now in case there are any other quick wins.

[Robinlovelace]

I think this can be closed for now. If anyone knows of any other quick wins let us know. Thanks @benz0li for speeding up our Docker game!