feat: upgrade to Maple

- A shared cookie domain between lms and cms is no longer recommended:
https://github.com/edx/edx-platform/blob/master/docs/guides/studio_oauth.rst
- refactor: clean mounted data folder in lms/cms. In Lilac, the
bind-mounted lms/data and cms/data folders are a mess because new
folders are created there for every new course organisation.  These
folders are empty. As far as we know they are useless... With this
change we move these folders to a dedicated "modulestore" subdirectory;
which corresponds better to the initial intent of the fs_root setting.
- fix: frontend failure during login to the lms. See:
openedx/wg-build-test-release#104
- feat: move all forum-related code to a dedicated plugin. Forum is an
optional feature, and as such it deserves its own plugin. Starting from
Maple, users will be able to install the forum from
https://github.com/overhangio/tutor-forum/
- migrate from DCS_* session cookie settings to SESSION_*. That's
because edx-platform no longer depends on django-cookies-samesite. Close
openedx/wg-build-test-release#110
- get rid of tons of deprecation warnings in the lms/cms
- feat: make it possible to point to themed assets. Cherry-picking this
change makes it possible to point to themed assets with a theme-agnostic
url, notably from MFEs.
- Install all official plugins as part of the `tutor[full]` package.
- Don't print error messages about loading plugins during autocompletion.

Close #450.

CHANGELOG-nightly.md

@@ -1,26 +1,3 @@
 # Changelog (nightly branch)
 
 Note: Breaking changes between versions are indicated by "💥".
-
-- [Feature] Better support of Caddy as a load balancer in Kubernetes:
-  - Make it possible to start/stop a selection of resources with ``tutor k8s start/stop [names...]``.
-  - Make it easy to deploy an independent LoadBalancer by converting the caddy service to a ClusterIP when ``ENABLE_WEB_PROXY=false``.
-  - Add a ``app.kubernetes.io/component: loadbalancer`` label to the LoadBalancer service.
-  - Add ``app.kubernetes.io/name`` labels to all services.
-  - Preserve the LoadBalancer service in ``tutor k8s stop`` commands.
-  - Wait for the caddy deployment to be ready before running initialisation jobs.
-- [Security] On Kubernetes, convert all NodePort services to ClusterIP to guarantee network isolation from outside the cluster.
-- 💥[Improvement] Drop Python 3.5 compatibility.
-- [Bugfix] Fix docker-compose project name in development on nightly branch.
-- 💥[Bugfix] No longer track the Tutor version number in resource labels (and label selectors, which breaks the update of Deployment resources), but instead do so in resource annotations.
-- [Bugfix] Make it possible for plugins to implement the "caddyfile" patch without relying on the "port" local variable.
-- 💥[Improvement] Move the Open edX forum to a [dedicated plugin](https://github.com/overhangio/tutor-forum/) (#450).
-- 💥[Improvement] Get rid of the "tutor-openedx" package, which is no longer supported.
-- [Bugfix] Fix running Caddy container in k8s, which should always be the case even if `ENABLE_WEB_PROXY` is false.
-- 💥[Improvement] Run all services as unprivileged containers, for better security. This has multiple consequences:
-  - The "openedx-dev" image is now built with `tutor dev dc build lms`.
-  - The "smtp" service now runs the "devture/exim-relay" Docker image, which is unprivileged. Also, the default SMTP port is now 8025.
-- 💥[Feature] Get rid of the nginx container and service, which is now replaced by Caddy. this has the following consequences:
-    - Patches "nginx-cms", "nginx-lms", "nginx-extra", "local-docker-compose-nginx-aliases" are replaced by "caddyfile-cms", "caddyfile-lms", "caddyfile", " local-docker-compose-caddy-aliases".
-    - Patches "k8s-deployments-nginx-volume-mounts", "k8s-deployments-nginx-volumes" were obsolete and are removed.
-    - The `NGINX_HTTP_PORT` setting is renamed to `CADDY_HTTP_PORT`.

CHANGELOG.md

@@ -4,6 +4,29 @@ Note: Breaking changes between versions are indicated by "💥".
 
 ## Unreleased
 
+- 💥[Improvement] Upgrade to Maple
+  - Install all official plugins as part of the `tutor[full]` package.
+  - Don't print error messages about loading plugins during autocompletion.
+- [Feature] Better support of Caddy as a load balancer in Kubernetes:
+  - Make it possible to start/stop a selection of resources with ``tutor k8s start/stop [names...]``.
+  - Make it easy to deploy an independent LoadBalancer by converting the caddy service to a ClusterIP when ``ENABLE_WEB_PROXY=false``.
+  - Add a ``app.kubernetes.io/component: loadbalancer`` label to the LoadBalancer service.
+  - Add ``app.kubernetes.io/name`` labels to all services.
+  - Preserve the LoadBalancer service in ``tutor k8s stop`` commands.
+  - Wait for the caddy deployment to be ready before running initialisation jobs.
+  - Fix running Caddy container in k8s, which should always be the case even if `ENABLE_WEB_PROXY` is false.
+- [Security] On Kubernetes, convert all NodePort services to ClusterIP to guarantee network isolation from outside the cluster.
+- 💥[Improvement] Move the Open edX forum to a [dedicated plugin](https://github.com/overhangio/tutor-forum/) (#450).
+- 💥[Improvement] Drop Python 3.5 compatibility.
+- 💥[Bugfix] No longer track the Tutor version number in resource labels (and label selectors, which breaks the update of Deployment resources), but instead do so in resource annotations.
+- 💥[Improvement] Get rid of the "tutor-openedx" package, which is no longer supported.
+- 💥[Improvement] Run all services as unprivileged containers, for better security. This has multiple consequences:
+  - The "openedx-dev" image is now built with `tutor dev dc build lms`.
+  - The "smtp" service now runs the "devture/exim-relay" Docker image, which is unprivileged. Also, the default SMTP port is now 8025.
+- 💥[Feature] Get rid of the nginx container and service, which is now replaced by Caddy. this has the following consequences:
+    - Patches "nginx-cms", "nginx-lms", "nginx-extra", "local-docker-compose-nginx-aliases" are replaced by "caddyfile-cms", "caddyfile-lms", "caddyfile", " local-docker-compose-caddy-aliases".
+    - Patches "k8s-deployments-nginx-volume-mounts", "k8s-deployments-nginx-volumes" were obsolete and are removed.
+    - The `NGINX_HTTP_PORT` setting is renamed to `CADDY_HTTP_PORT`.
 - [Bugfix] Fix building of the `openedx` image on ARM64 due to missing `libgeos-dev`
 
 ## v12.2.0 (2021-12-08)

docs/configuration.rst

@@ -79,7 +79,7 @@ You may want to pull/push images from/to a custom docker registry. For instance,
 Open edX customisation
 ~~~~~~~~~~~~~~~~~~~~~~
 
-- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/lilac.2"``)
+- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/maple.beta2"``)
 
 This defines the default version that will be pulled from all Open edX git repositories.
 
@@ -281,16 +281,16 @@ Note that your edx-platform version must be a fork of the latest release **tag**
 
 If you don't create your fork from this tag, you *will* have important compatibility issues with other services. In particular:
 
-- Do not try to run a fork from an older (pre-Lilac) version of edx-platform: this will simply not work.
+- Do not try to run a fork from an older (pre-Maple) version of edx-platform: this will simply not work.
 - Do not try to run a fork from the edx-platform master branch: there is a 99% probability that it will fail.
-- Do not try to run a fork from the open-release/lilac.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/lilac.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/lilac.master branch.
+- Do not try to run a fork from the open-release/maple.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/maple.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/maple.master branch.
 
 .. _i18n:
 
 Adding custom translations
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-If you are not running Open edX in English, chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX in your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform <https://github.com/edx/edx-platform/tree/open-release/lilac.master/conf/locale>`__ as well as those from `openedx-i18n <https://github.com/openedx/openedx-i18n/tree/master/edx-platform/locale>`__.
+If you are not running Open edX in English, chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX in your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform <https://github.com/edx/edx-platform/tree/open-release/maple.master/conf/locale>`__ as well as those from `openedx-i18n <https://github.com/openedx/openedx-i18n/tree/master/edx-platform/locale>`__.
 
 Tutor offers a relatively simple mechanism to add custom translations to the openedx Docker image. You should create a folder that corresponds to your language code in the "build/openedx/locale" folder of the Tutor environment. This folder should contain a "LC_MESSAGES" folder. For instance::
 
@@ -311,9 +311,9 @@ Then, add a "django.po" file there that will contain your custom translations::
 .. warning::
     Don't forget to specify the file ``Content-Type`` when adding message strings with non-ASCII characters; otherwise a ``UnicodeDecodeError`` will be raised during compilation.
 
-The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language <https://github.com/edx/edx-platform/blob/open-release/lilac.master/conf/locale/en/LC_MESSAGES/django.po>`__.
+The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language <https://github.com/edx/edx-platform/blob/open-release/maple.master/conf/locale/en/LC_MESSAGES/django.po>`__.
 
-If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well <https://github.com/edx/edx-platform/blob/open-release/lilac.master/conf/locale/en/LC_MESSAGES/djangojs.po>`__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory.
+If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well <https://github.com/edx/edx-platform/blob/open-release/maple.master/conf/locale/en/LC_MESSAGES/djangojs.po>`__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory.
 
 To recap, here is an example. To translate a few strings in French, both from django.po and djangojs.po, we would have the following file hierarchy::
 

docs/dev.rst

@@ -25,7 +25,7 @@ This ``openedx-dev`` development image differs from the ``openedx`` production i
 
 - The user that runs inside the container has the same UID as the user on the host, in order to avoid permission problems inside mounted volumes (and in particular in the edx-platform repository).
 - Additional python and system requirements are installed for convenient debugging: `ipython <https://ipython.org/>`__, `ipdb <https://pypi.org/project/ipdb/>`__, vim, telnet.
-- The edx-platform `development requirements <https://github.com/edx/edx-platform/blob/open-release/lilac.master/requirements/edx/development.in>`__ are installed.
+- The edx-platform `development requirements <https://github.com/edx/edx-platform/blob/open-release/maple.master/requirements/edx/development.in>`__ are installed.
 
 Since the ``openedx-dev`` is based upon the ``openedx`` docker image, it should be re-built every time the ``openedx`` docker image is modified.
 
@@ -137,7 +137,7 @@ Following the instructions :ref:`above <bind_mounts>` on how to bind-mount direc
 
 If you choose any but the first solution above, you will have to make sure that your fork works with Tutor.
 
-First of all, you should make sure that you are working off the ``open-release/lilac.2`` tag. See the :ref:`fork edx-platform section <edx_platform_fork>` for more information.
+First of all, you should make sure that you are working off the ``open-release/maple.beta2`` tag. See the :ref:`fork edx-platform section <edx_platform_fork>` for more information.
 
 Then, you should run the following commands::
 

docs/download/pip.rst

@@ -1,3 +1,3 @@
 .. parsed-literal::
 
-    pip install tutor
+    pip install tutor[full]

docs/faq.rst

@@ -38,7 +38,7 @@ The `devstack <https://github.com/edx/devstack>`_ is meant for development only,
 Is Tutor officially supported by edX?
 -------------------------------------
 
-As of the Open edX Lilac release (June 9th 2021), Tutor is one of the two officially supported installation methods for Open edX: see the `official installation instructions <https://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/open-release-lilac.master/installation/index.html>`__. We expect that by Maple (December 9th 2021) the native installation will be deprecated and Tutor will become the only officially recommended installation method, unless major issues are discovered. However, Tutor remains developed independently from edX, both by its parent company Overhang.IO and the :ref:`project maintainers <maintainers>`.
+Yes: as of the Open edX Maple release (December 9th 2021), Tutor is the only officially supported installation methods for Open edX: see the `official installation instructions <https://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/open-release-maple.master/installation/index.html>`__. 
 
 What features are missing from Tutor?
 -------------------------------------

docs/gettingstarted.rst

@@ -6,7 +6,7 @@ Getting started
 .. toctree::
    :maxdepth: 2
 
+   install
    intro
    quickstart
-   install
-   whatnext
+   whatnext

docs/install.rst

@@ -1,14 +1,15 @@
 .. _install:
 
-Install Tutor
-=============
+Installing Tutor
+================
 
 .. _requirements:
 
 Requirements
 ------------
 
-* Supported OS: Tutor runs on any 64-bit, UNIX-based system. It was also reported to work on Windows.
+* Supported OS: Tutor runs on any 64-bit, UNIX-based OS. It was also reported to work on Windows (with `WSL 2 <https://docs.microsoft.com/en-us/windows/wsl/install>`__).
+* Architecture: support for ARM64 is a work-in-progress. See `this issue <https://github.com/overhangio/tutor/issues/510>`__.
 * Required software:
 
     - `Docker <https://docs.docker.com/engine/installation/>`__: v18.06.0+
@@ -26,54 +27,56 @@ Requirements
 .. note::
     On Mac OS, by default, containers are allocated 2 GB of RAM, which is not enough. You should follow `these instructions from the official Docker documentation <https://docs.docker.com/docker-for-mac/#advanced>`__ to allocate at least 4-5 GB to the Docker daemon. If the deployment fails because of insufficient memory during database migrations, check the :ref:`relevant section in the troubleshooting guide <migrations_killed>`.
 
-.. _install_binary:
+Download
+--------
 
-Direct binary download
-----------------------
+Choose **one** of the installation methods below. If you install Tutor in different ways, you will end up with multiple ``tutor`` executables, which is going to be very confusing. At any time, you can check the path to your ``tutor`` executable by running ``which tutor``.
 
-The latest binaries can be downloaded from https://github.com/overhangio/tutor/releases. From the command line:
+Python package
+~~~~~~~~~~~~~~
 
-.. include:: download/binary.rst
+.. include:: download/pip.rst
 
-This is the simplest and recommended installation method for most people. Note however that you will not be able to use custom plugins with this pre-compiled binary. The only plugins you can use with this approach are those that are already bundled with the binary: see the :ref:`existing plugins <existing_plugins>`.
+Check the "tutor" package on Pypi: https://pypi.org/project/tutor. You will python >= 3.6 with pip and the libyaml development headers. On Ubuntu, these requirements can be installed by running::
 
-.. _install_source:
+    sudo apt install python3 python3-pip libyaml-dev
+
+.. _install_binary:
 
-Alternative installation methods
---------------------------------
+Binary release
+~~~~~~~~~~~~~~
 
-If you would like to inspect the Tutor source code, you are most welcome to install Tutor from `Pypi <https://pypi.org/project/tutor/>`_ or directly from `the Github repository <https://github.com/overhangio/tutor>`_. You will need python >= 3.6 with pip and the libyaml development headers. On Ubuntu, these requirements can be installed by running::
+The latest binaries can be downloaded from https://github.com/overhangio/tutor/releases. From the command line:
 
-    sudo apt install python3 python3-pip libyaml-dev
+.. include:: download/binary.rst
 
-Installing from pypi
-~~~~~~~~~~~~~~~~~~~~
+This is the simplest and recommended installation method for most people who do not have Python 3 on their machine. Note however that **you will not be able to use custom plugins** with this pre-compiled binary. The only plugins you can use with this approach are those that are already bundled with the binary: see the :ref:`existing plugins <existing_plugins>`.
 
-.. include:: download/pip.rst
+.. _install_source:
 
 Installing from source
 ~~~~~~~~~~~~~~~~~~~~~~
 
-::
+To inspect the Tutor source code, install Tutor from `the Github repository <https://github.com/overhangio/tutor>`__::
 
     git clone https://github.com/overhangio/tutor
     cd tutor
     pip install -e .
 
-DNS records
------------
+Configuring DNS records
+-----------------------
 
 When running a server in production, it is necessary to define `DNS records <https://en.wikipedia.org/wiki/Domain_Name_System#Resource_records>`__ which will make it possible to access your Open edX platform by name in your browser. The precise procedure to create DNS records vary from one provider to the next and is beyond the scope of these docs. You should create a record of type A with a name equal to your LMS hostname (given by ``tutor config printvalue LMS_HOST``) and a value that indicates the IP address of your server. Applications other than the LMS, such as the studio, ecommerce, etc. typically reside in subdomains of the LMS. Thus, you should also create a CNAME record to point all subdomains of the LMS to the LMS_HOST.
 
-For instance, the demo Open edX server that runs at http://demo.openedx.overhang.io has the following DNS records::
+For instance, the demo Open edX server that runs at https://demo.openedx.overhang.io has the following DNS records::
 
     demo.openedx 1800 IN A 172.105.89.208
     *.demo.openedx 1800 IN CNAME demo.openedx.overhang.io.
 
 .. _cloud_install:
 
 Zero-click AWS installation
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
+---------------------------
 
 Tutor can be launched on Amazon Web Services very quickly with the `official Tutor AMI <https://aws.amazon.com/marketplace/pp/B07PV3TB8X>`__. Shell access is not required, as all configuration will happen through the Tutor web user interface. For detailed installation instructions, we recommend watching the following video:
 
@@ -86,14 +89,14 @@ Upgrading
 
 With Tutor, it is very easy to upgrade to a more recent Open edX or Tutor release. Just install the latest ``tutor`` version (using either methods above) and run the ``quickstart`` command again. If you have :ref:`customised <configuration_customisation>` your docker images, you will have to re-build them prior to running ``quickstart``.
 
-``quickstart`` should take care of automatically running the upgrade process. If for some reason you need to *manually* upgrade from an Open edX release to the next, you should run ``tutor local upgrade``. For instance, to upgrade from Koa to Lilac, run::
+``quickstart`` should take care of automatically running the upgrade process. If for some reason you need to *manually* upgrade from an Open edX release to the next, you should run ``tutor local upgrade``. For instance, to upgrade from Lilac to Maple, run::
 
-    tutor local upgrade --from=koa
+    tutor local upgrade --from=lilac
 
 .. _autocomplete:
 
-Autocomplete
-------------
+Shell autocompletion
+--------------------
 
 Tutor is built on top of `Click <https://click.palletsprojects.com>`_, which is a great library for building command line interface (CLI) tools. As such, Tutor benefits from all Click features, including `auto-completion <https://click.palletsprojects.com/en/8.x/bashcomplete/>`_. After installing Tutor, auto-completion can be enabled in bash by running::