Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: library upgrade #251

Merged
merged 12 commits into from
Jul 27, 2022
32 changes: 16 additions & 16 deletions .github/workflows/build_and_push_to_docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
- docker/qgis/Dockerfile
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v2
- uses: hadolint/[email protected]
with:
dockerfile: ${{ matrix.DOCKERFILE_PATH }}
Expand All @@ -38,8 +38,8 @@ jobs:
fail-fast: true
matrix:
BASE_IMAGE_TAG:
- lab-3.2.1
- python-3.9.7
- lab-3.4.0
- python-3.9.12
- python-3.8.8
- python-3.7.6

Expand All @@ -62,7 +62,7 @@ jobs:

# This ensures the same image tags as before are built, in addition to new ones
if [[ "${{ matrix.BASE_IMAGE_TAG }}" == lab-* ]]; then
export LABEL_PREFIX=3.9
export LABEL_PREFIX=3.10
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rokroskar I thought none of the images are using python 3.10. And from Ralf's comment it seems we are not ready for 3.10 yet.

So this should then probably be rolled back to 3.9?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
export LABEL_PREFIX=3.10
export LABEL_PREFIX=3.9

else
export LABEL_PREFIX=${{ matrix.BASE_IMAGE_TAG }}
fi
Expand Down Expand Up @@ -114,7 +114,7 @@ jobs:
fi

docker build docker/$EXTENSION \
--build-arg BASE_IMAGE="$DOCKER_NAME-py:3.9-$LABEL" \
--build-arg BASE_IMAGE="$DOCKER_NAME-py:3.10-$LABEL" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--build-arg BASE_IMAGE="$DOCKER_NAME-py:3.10-$LABEL" \
--build-arg BASE_IMAGE="$DOCKER_NAME-py:3.9-$LABEL" \

--tag $DOCKER_NAME-$EXTENSION:$LABEL
docker push $DOCKER_NAME-$EXTENSION:$LABEL
env:
Expand Down Expand Up @@ -142,25 +142,25 @@ jobs:
CUDA_COMPAT_PACKAGE: "cuda-compat-11-1"
LIBCUDNN_PACKAGE: "libcudnn8=8.0.5.39-1+cuda11.1"
- CUDA_VERSION: "11.2"
PYTHON_VERSION: "3.9.7"
PYTHON_VERSION: "3.9.12"
EXTRA_LIBRARIES: ""
CUDA_CUDART_PACKAGE: "cuda-cudart-11-2=11.2.152-1"
CUDA_COMPAT_PACKAGE: "cuda-compat-11-2"
LIBCUDNN_PACKAGE: "libcudnn8=8.1.1.33-1+cuda11.2"
- CUDA_VERSION: "11.3"
PYTHON_VERSION: "3.9.7"
PYTHON_VERSION: "3.9.12"
EXTRA_LIBRARIES: ""
CUDA_CUDART_PACKAGE: "cuda-cudart-11-3=11.3.109-1"
CUDA_COMPAT_PACKAGE: "cuda-compat-11-3"
LIBCUDNN_PACKAGE: "libcudnn8=8.2.1.32-1+cuda11.3"
- CUDA_VERSION: "11.4"
PYTHON_VERSION: "3.9.7"
PYTHON_VERSION: "3.9.12"
EXTRA_LIBRARIES: ""
CUDA_CUDART_PACKAGE: "cuda-cudart-11-4=11.4.148-1"
CUDA_COMPAT_PACKAGE: "cuda-compat-11-4"
LIBCUDNN_PACKAGE: "libcudnn8=8.2.4.15-1+cuda11.4"
- CUDA_VERSION: "11.5"
PYTHON_VERSION: "3.9.7"
PYTHON_VERSION: "3.9.12"
EXTRA_LIBRARIES: ""
CUDA_CUDART_PACKAGE: "cuda-cudart-11-5=11.5.117-1"
CUDA_COMPAT_PACKAGE: "cuda-compat-11-5"
Expand Down Expand Up @@ -298,7 +298,7 @@ jobs:
env:
EXTENSION: ${{ matrix.VNC-EXTENSIONS }}
DOCKER_NAME: "renku/renkulab"

build-py-batch:
needs: build-py
runs-on: ubuntu-latest
Expand All @@ -320,8 +320,8 @@ jobs:
fi

docker build docker/batch \
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.9-$LABEL" \
--build-arg BASE_IMAGE="python:3.9-slim-buster" \
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.10-$LABEL" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.10-$LABEL" \
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.9-$LABEL" \

--build-arg BASE_IMAGE="python:3.10-slim-buster" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--build-arg BASE_IMAGE="python:3.10-slim-buster" \
--build-arg BASE_IMAGE="python:3.9-slim-buster" \

--tag $DOCKER_NAME-batch:$LABEL
docker push $DOCKER_NAME-batch:$LABEL
env:
Expand Down Expand Up @@ -356,7 +356,7 @@ jobs:

export DOCKER_TAG="${{ matrix.JULIAVERSIONS }}-$LABEL"
docker build docker/julia \
--build-arg BASE_IMAGE="$DOCKER_NAME-py:3.9-$LABEL" \
--build-arg BASE_IMAGE="$DOCKER_NAME-py:3.10-$LABEL" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--build-arg BASE_IMAGE="$DOCKER_NAME-py:3.10-$LABEL" \
--build-arg BASE_IMAGE="$DOCKER_NAME-py:3.9-$LABEL" \

--tag $DOCKER_NAME-julia:$DOCKER_TAG
echo "::set-output name=IMAGE_NAME::$DOCKER_NAME-julia:$DOCKER_TAG"
- name: Image Acceptance Tests
Expand Down Expand Up @@ -408,11 +408,11 @@ jobs:

export DOCKER_TAG="${{ matrix.RVERSION }}-$LABEL"
docker build docker/r \
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.9-$LABEL" \
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.10-$LABEL" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.10-$LABEL" \
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.9-$LABEL" \

--build-arg BASE_IMAGE="rocker/verse:${{ matrix.RVERSION }}" \
--build-arg RSTUDIO_VERSION_OVERRIDE="${{ matrix.RSTUDIO_VERSION }}" \
--tag $DOCKER_NAME-r:$DOCKER_TAG

echo "::set-output name=IMAGE_NAME::$DOCKER_NAME-r:$DOCKER_TAG"
- name: Image Acceptance Tests
uses: cypress-io/github-action@v2
Expand Down Expand Up @@ -462,7 +462,7 @@ jobs:

export DOCKER_TAG="${{ matrix.RELEASE }}-$LABEL"
docker build docker/r \
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.9-$LABEL" \
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.10-$LABEL" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.10-$LABEL" \
--build-arg RENKU_BASE="$DOCKER_NAME-py:3.9-$LABEL" \

--build-arg BASE_IMAGE=bioconductor/bioconductor_docker:${{ matrix.RELEASE }} \
--tag $DOCKER_NAME-bioc:$DOCKER_TAG
docker push $DOCKER_NAME-bioc:$DOCKER_TAG
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ pipx install --force renku==<version>
| [renku/renkulab-cuda-tf](https://hub.docker.com/r/renku/renkulab-cuda-tf/tags) | Cuda image with Python and Tensorflow | [renku/renkulab-cuda](https://hub.docker.com/r/renku/renkulab-cuda/tags) |
| [renku/renkulab-vnc](https://hub.docker.com/r/renku/renkulab-vnc/tags) | VNC Image with Python | [renku/renkulab-py](https://hub.docker.com/r/renku/renkulab-py/tags) |

Please refer to the [release notes](https://github.com/SwissDataScienceCenter/renkulab-docker/releases)
Please refer to the [release notes](https://github.com/SwissDataScienceCenter/renkulab-docker/releases)
for more detailed lists of released images and specific links to Dockerhub.

### py
Expand Down Expand Up @@ -148,7 +148,9 @@ https://hub.docker.com/r/renku/renkulab-vnc/tags
## Development

Build with Docker by running `docker build -t <name:tag> .` in the directory
of the image you would like to build.
of the image you would like to build. Note that on arm-based systems (e.g.
Apple M1/M2) you may need to use the flag `--platform=linux/amd64` for the
build because `git-lfs 3.2.0` is not available for the arm architecture.

## Adding renku to your own images

Expand Down
4 changes: 2 additions & 2 deletions docker/py/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
ARG BASE_IMAGE=jupyter/base-notebook:lab-3.2.1
ARG RENKU_VERSION=1.4.0
ARG BASE_IMAGE=jupyter/base-notebook:lab-3.4.0
ARG RENKU_VERSION=1.6.0
FROM $BASE_IMAGE as base

LABEL maintainer="Swiss Data Science Center <[email protected]>"
Expand Down
6 changes: 2 additions & 4 deletions docker/py/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
jupyter-server-proxy~=3.2.1
jupyterlab-git==0.30.1
jupyter-server-proxy==3.2.1
jupyterlab-git==0.37.1
jupyterlab-system-monitor~=0.8.0
jupyterlab~=3.0.0
papermill~=2.3.0
powerline-shell~=0.7.0
requests>=2.20.0
setuptools==57.5.0
virtualenv>=20.7.2
mistune>=2.0.1 # not directly required, pinned by Snyk to avoid a vulnerability