feat: run all services as unprivileged containers

With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.

To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.

We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.

With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.

Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.

We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.

In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.

Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.

Close #323.

CHANGELOG-nightly.md

@@ -2,7 +2,10 @@
 
 Note: Breaking changes between versions are indicated by "💥".
 
+- 💥[Improvement] Run all services as unprivileged containers, for better security. This has multiple consequences:
+  - The "openedx-dev" image is now built with `tutor dev dc build lms`.
+  - The "smtp" service now runs the "devture/exim-relay" Docker image, which is unprivileged. Also, the default SMTP port is now 8025.
 - 💥[Feature] Get rid of the nginx container and service, which is now replaced by Caddy. this has the following consequences:
     - Patches "nginx-cms", "nginx-lms", "nginx-extra", "local-docker-compose-nginx-aliases" are replaced by "caddyfile-cms", "caddyfile-lms", "caddyfile", " local-docker-compose-caddy-aliases".
     - Patches "k8s-deployments-nginx-volume-mounts", "k8s-deployments-nginx-volumes" were obsolete and are removed.
-    - The `NGINX_HTTP_PORT` setting is renamed to `CADDY_HTTP_PORT`.
+    - The `NGINX_HTTP_PORT` setting is renamed to `CADDY_HTTP_PORT`.

docs/configuration.rst

@@ -352,7 +352,7 @@ And djangojs.po::
 
 Then you will have to re-build the openedx Docker image::
 
-    tutor images build openedx openedx-dev
+    tutor images build openedx
 
 Beware that this will take a long time! Unfortunately it's difficult to accelerate this process, as translation files need to be compiled prior to collecting the assets. In development it's possible to accelerate the iteration loop -- but that exercise is left to the reader.
 

docs/dev.rst

@@ -19,7 +19,7 @@ Once the local platform has been configured, you should stop it so that it does
 
 Finally, you should build the ``openedx-dev`` docker image::
 
-    tutor images build openedx-dev
+    tutor dev dc build lms
 
 This ``openedx-dev`` development image differs from the ``openedx`` production image:
 

docs/tutorials/theming.rst

@@ -50,10 +50,6 @@ The LMS can then be accessed at http://local.overhang.io:8000. You will then hav
 
     tutor dev settheme mythemename
 
-Re-build development docker image (and compile assets)::
-
-    tutor images build openedx-dev
-
 Watch the themes folders for changes (in a different terminal)::
 
     tutor dev run watchthemes

tutor/bindmounts.py

@@ -34,6 +34,7 @@ def create(
         "run",
         "--rm",
         "--no-deps",
+        "--user=0",
         "--volume",
         "{}:{}".format(volumes_root_path, container_volumes_root_path),
         service,

tutor/commands/images.py

@@ -8,11 +8,9 @@
 from .. import images
 from .. import plugins
 from ..types import Config
-from .. import utils
 from .context import Context
 
-BASE_IMAGE_NAMES = ["openedx", "forum"]
-DEV_IMAGE_NAMES = ["openedx-dev"]
+BASE_IMAGE_NAMES = ["openedx", "forum", "permissions"]
 VENDOR_IMAGES = [
     "caddy",
     "elasticsearch",
@@ -136,13 +134,6 @@ def build_image(root: str, config: Config, image: str, *args: str) -> None:
             (tutor_env.pathjoin(root, "plugins", plugin, "build", img), tag, args)
         )
 
-    # Build dev images with user id argument
-    dev_build_arg = ("--build-arg", "USERID={}".format(utils.get_user_id()))
-    for img, tag in iter_images(config, image, DEV_IMAGE_NAMES):
-        to_build.append(
-            (tutor_env.pathjoin(root, "build", img), tag, dev_build_arg + args)
-        )
-
     if not to_build:
         raise ImageNotFoundError(image)
 

tutor/env.py

@@ -59,6 +59,7 @@ def __init__(
         environment.globals["rsa_import_key"] = utils.rsa_import_key
         environment.filters["rsa_private_key"] = utils.rsa_private_key
         environment.filters["walk_templates"] = self.walk_templates
+        environment.globals["HOST_USER_ID"] = utils.get_user_id()
         environment.globals["TUTOR_APP"] = __app__.replace("-", "_")
         environment.globals["TUTOR_VERSION"] = __version__
         self.environment = environment

tutor/jobs.py

@@ -50,6 +50,7 @@ def docker_compose(self, *command: str) -> int:
 def initialise(runner: BaseJobRunner, limit_to: Optional[str] = None) -> None:
     fmt.echo_info("Initialising all services...")
     if limit_to is None or limit_to == "mysql":
+        fmt.echo_info("Initialising mysql...")
         runner.run_job_from_template("mysql", "hooks", "mysql", "init")
     for plugin_name, hook in runner.iter_plugin_hooks("pre-init"):
         if limit_to is None or limit_to == plugin_name:

tutor/templates/build/forum/Dockerfile

@@ -1,4 +1,4 @@
-FROM docker.io/ubuntu:20.04
+FROM docker.io/ruby:2.5.7-slim-stretch
 MAINTAINER Overhang.io <[email protected]>
 
 ENV DEBIAN_FRONTEND=noninteractive
@@ -12,32 +12,27 @@ RUN wget -O /tmp/dockerize.tar.gz https://github.com/jwilder/dockerize/releases/
     && tar -C /usr/local/bin -xzvf /tmp/dockerize.tar.gz \
     && rm /tmp/dockerize.tar.gz
 
-RUN mkdir /openedx
+# Create unprivileged "app" user
+RUN useradd --home-dir /app --create-home --shell /bin/bash --uid 1000 app
 
-# Install ruby-build for building specific version of ruby
-# The ruby-build version should be periodically updated to reflect the latest release
-ARG RUBY_BUILD_VERSION=v20200401
-RUN git clone https://github.com/rbenv/ruby-build.git --branch $RUBY_BUILD_VERSION /openedx/ruby-build
-WORKDIR /openedx/ruby-build
-RUN PREFIX=/usr/local ./install.sh
+# Copy custom scripts
+COPY ./bin /app/bin
+RUN chmod a+x /app/bin/*
+ENV PATH :${PATH}
 
-# Install ruby and some specific dependencies
-ARG RUBY_VERSION=2.5.7
-ARG BUNDLER_VERSION=1.17.3
-ARG RAKE_VERSION=13.0.1
-RUN ruby-build $RUBY_VERSION /openedx/ruby
-ENV PATH "/openedx/ruby/bin:$PATH"
-RUN gem install bundler -v $BUNDLER_VERSION
-RUN gem install rake -v $RAKE_VERSION
+# From then on, run as unprivileged app user
+USER app
+
+# Install rake and bundler
+ENV PATH "/app/bin:/app/.gem/ruby/2.5.0/bin:$PATH"
+RUN gem install --user-install bundler --version 1.17.3
+RUN gem install --user-install rake --version 13.0.1
 
 # Install forum
-RUN git clone https://github.com/edx/cs_comments_service.git --branch {{ OPENEDX_COMMON_VERSION }} --depth 1 /openedx/cs_comments_service
-WORKDIR /openedx/cs_comments_service
+RUN git clone https://github.com/edx/cs_comments_service.git --branch {{ OPENEDX_COMMON_VERSION }} --depth 1 /app/cs_comments_service
+WORKDIR /app/cs_comments_service
 RUN bundle install --deployment
 
-COPY ./bin /openedx/bin
-RUN chmod a+x /openedx/bin/*
-ENV PATH /openedx/bin:${PATH}
 ENTRYPOINT ["docker-entrypoint.sh"]
 
 ENV SINATRA_ENV staging

tutor/templates/build/openedx/Dockerfile

@@ -127,14 +127,19 @@ RUN apt update && \
     apt install -y gettext gfortran graphviz graphviz-dev libffi-dev libfreetype6-dev libgeos-dev libjpeg8-dev liblapack-dev libmysqlclient-dev libpng-dev libsqlite3-dev libxmlsec1-dev lynx ntp pkg-config rdfind && \
     rm -rf /var/lib/apt/lists/*
 
-COPY --from=dockerize /usr/local/bin/dockerize /usr/local/bin/dockerize
-COPY --from=code /openedx/edx-platform /openedx/edx-platform
-COPY --from=locales /openedx/locale/contrib/locale /openedx/locale/contrib/locale
-COPY --from=python /opt/pyenv /opt/pyenv
-COPY --from=python-requirements /openedx/venv /openedx/venv
-COPY --from=python-requirements /openedx/requirements /openedx/requirements
-COPY --from=nodejs-requirements /openedx/nodeenv /openedx/nodeenv
-COPY --from=nodejs-requirements /openedx/edx-platform/node_modules /openedx/edx-platform/node_modules
+# From then on, run as unprivileged "app" user
+ARG APP_USER_ID=1000
+RUN useradd --home-dir /openedx --create-home --shell /bin/bash --uid ${APP_USER_ID} app
+USER ${APP_USER_ID}
+
+COPY --chown=app:app --from=dockerize /usr/local/bin/dockerize /usr/local/bin/dockerize
+COPY --chown=app:app --from=code /openedx/edx-platform /openedx/edx-platform
+COPY --chown=app:app --from=locales /openedx/locale /openedx/locale
+COPY --chown=app:app --from=python /opt/pyenv /opt/pyenv
+COPY --chown=app:app --from=python-requirements /openedx/venv /openedx/venv
+COPY --chown=app:app --from=python-requirements /openedx/requirements /openedx/requirements
+COPY --chown=app:app --from=nodejs-requirements /openedx/nodeenv /openedx/nodeenv
+COPY --chown=app:app --from=nodejs-requirements /openedx/edx-platform/node_modules /openedx/edx-platform/node_modules
 
 ENV PATH /openedx/venv/bin:./node_modules/.bin:/openedx/nodeenv/bin:${PATH}
 ENV VIRTUAL_ENV /openedx/venv/
@@ -146,16 +151,16 @@ RUN pip install -r requirements/edx/local.in
 # Create folder that will store lms/cms.env.json files, as well as
 # the tutor-specific settings files.
 RUN mkdir -p /openedx/config ./lms/envs/tutor ./cms/envs/tutor
-COPY revisions.yml /openedx/config/
+COPY --chown=app:app revisions.yml /openedx/config/
 ENV LMS_CFG /openedx/config/lms.env.json
 ENV STUDIO_CFG /openedx/config/cms.env.json
 ENV REVISION_CFG /openedx/config/revisions.yml
-COPY settings/lms/*.py ./lms/envs/tutor/
-COPY settings/cms/*.py ./cms/envs/tutor/
+COPY --chown=app:app settings/lms/*.py ./lms/envs/tutor/
+COPY --chown=app:app settings/cms/*.py ./cms/envs/tutor/
 
 # Copy user-specific locales to /openedx/locale/user/locale and compile them
-RUN mkdir -p /openedx/locale/user
-COPY ./locale/ /openedx/locale/user/locale/
+RUN mkdir /openedx/locale/user
+COPY --chown=app:app ./locale/ /openedx/locale/user/locale/
 RUN cd /openedx/locale/user && \
     django-admin.py compilemessages -v1
 
@@ -166,7 +171,7 @@ RUN ./manage.py lms --settings=tutor.i18n compilejsi18n
 RUN ./manage.py cms --settings=tutor.i18n compilejsi18n
 
 # Copy scripts
-COPY ./bin /openedx/bin
+COPY --chown=app:app ./bin /openedx/bin
 RUN chmod a+x /openedx/bin/*
 ENV PATH /openedx/bin:${PATH}
 
@@ -188,7 +193,7 @@ RUN openedx-assets xmodule \
     && openedx-assets npm \
     && openedx-assets webpack --env=prod \
     && openedx-assets common
-COPY ./themes/ /openedx/themes/
+COPY --chown=app:app ./themes/ /openedx/themes/
 RUN openedx-assets themes \
     && openedx-assets collect --settings=tutor.assets \
     # De-duplicate static assets with symlinks
@@ -205,9 +210,40 @@ ENV SETTINGS tutor.production
 
 # Entrypoint will set right environment variables
 ENTRYPOINT ["docker-entrypoint.sh"]
+EXPOSE 8000
+
+###### Intermediate image with dev/test dependencies
+FROM production as development
+
+# Install useful system requirements (as root)
+USER root
+RUN apt update && \
+    apt install -y vim iputils-ping dnsutils telnet \
+    && rm -rf /var/lib/apt/lists/*
+USER app
+
+# Install dev python requirements
+RUN pip install -r requirements/edx/development.txt
+RUN pip install ipdb==0.13.4 ipython==7.27.0
+
+# Recompile static assets: in development mode all static assets are stored in edx-platform,
+# and the location of these files is stored in webpack-stats.json. If we don't recompile
+# static assets, then production assets will be served instead.
+RUN rm -r /openedx/staticfiles && \
+    mkdir /openedx/staticfiles && \
+    openedx-assets webpack --env=dev
+
+{{ patch("openedx-dev-dockerfile-post-python-requirements") }}
+
+# Default django settings
+ENV SETTINGS tutor.development
+
+CMD ./manage.py $SERVICE_VARIANT runserver 0.0.0.0:8000
+
+###### Final image with production cmd
+FROM production as final
 
 # Run server
-EXPOSE 8000
 CMD uwsgi \
     --static-map /static=/openedx/staticfiles/ \
     --static-map /media=/openedx/media/ \

tutor/templates/build/permissions/Dockerfile

@@ -0,0 +1,7 @@
+from docker.io/alpine:3.13.6
+MAINTAINER Overhang.io <[email protected]>
+
+COPY ./setowner.sh /usr/local/bin/setowner
+RUN chmod a+x /usr/local/bin/setowner
+
+ENTRYPOINT ["setowner"]

tutor/templates/build/permissions/setowner.sh

@@ -0,0 +1,14 @@
+#! /bin/sh
+set -e
+user_id="$1"
+shift
+for path in $@; do
+  path_user_id="$(stat -c '%u' $path)"
+  if [ "$path_user_id" != "$user_id" ]
+  then
+    echo "$path changing UID from $path_user_id to $user_id..."
+    chown --recursive $user_id $path
+  else
+    echo "$path already owned by $user_id"
+  fi
+done

tutor/templates/config.yml

@@ -26,16 +26,17 @@ OPENEDX_AWS_SECRET_ACCESS_KEY: ""
 DEV_PROJECT_NAME: "tutor_dev"
 DOCKER_REGISTRY: "docker.io/"
 DOCKER_IMAGE_OPENEDX: "{{ DOCKER_REGISTRY }}overhangio/openedx:{{ TUTOR_VERSION }}"
-DOCKER_IMAGE_OPENEDX_DEV: "{{ DOCKER_REGISTRY }}overhangio/openedx-dev:{{ TUTOR_VERSION }}"
+DOCKER_IMAGE_OPENEDX_DEV: "openedx-dev"
 DOCKER_IMAGE_CADDY: "{{ DOCKER_REGISTRY }}caddy:2.3.0"
 DOCKER_IMAGE_ELASTICSEARCH: "{{ DOCKER_REGISTRY }}elasticsearch:7.10.1"
 DOCKER_IMAGE_FORUM: "{{ DOCKER_REGISTRY }}overhangio/openedx-forum:{{ TUTOR_VERSION }}"
 DOCKER_IMAGE_MONGODB: "{{ DOCKER_REGISTRY }}mongo:4.2.17"
 DOCKER_IMAGE_MYSQL: "{{ DOCKER_REGISTRY }}mysql:5.7.35"
 DOCKER_IMAGE_ELASTICSEARCH: "{{ DOCKER_REGISTRY }}elasticsearch:7.10.1"
 DOCKER_IMAGE_NGINX: "{{ DOCKER_REGISTRY }}nginx:1.21.1"
+DOCKER_IMAGE_PERMISSIONS: "{{ DOCKER_REGISTRY }}alpine:3.13.6"
 DOCKER_IMAGE_REDIS: "{{ DOCKER_REGISTRY }}redis:6.2.6"
-DOCKER_IMAGE_SMTP: "{{ DOCKER_REGISTRY }}namshi/smtp:latest"
+DOCKER_IMAGE_SMTP: "{{ DOCKER_REGISTRY }}devture/exim-relay:4.94.2-r0-4"
 LOCAL_PROJECT_NAME: "{{ TUTOR_APP }}_local"
 ELASTICSEARCH_HOST: "elasticsearch"
 ELASTICSEARCH_PORT: 9200
@@ -77,7 +78,7 @@ REDIS_PORT: 6379
 REDIS_USERNAME: ""
 REDIS_PASSWORD: ""
 SMTP_HOST: "smtp"
-SMTP_PORT: 25
+SMTP_PORT: 8025
 SMTP_USERNAME: ""
 SMTP_PASSWORD: ""
 SMTP_USE_TLS: false