Skip to content

Commit

Permalink
feat: upgrade to quince
Browse files Browse the repository at this point in the history
  • Loading branch information
regisb committed Dec 11, 2023
1 parent 71e4691 commit 0ccf486
Show file tree
Hide file tree
Showing 36 changed files with 172 additions and 189 deletions.
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,18 @@ instructions, because git commits are used to generate release notes:

<!-- scriv-insert-here -->

<a id='changelog-17.0.0'></a>
## v17.0.0 (2023-12-09)

- 💥[Feature] Upgrade to Quince. (by @regisb)
- 💥[Feature] Replace "*.local.overhang.io" hostnames by "*.local.edly.io". (by @regisb)
- 💥[Feature] Enable the Indigo theme by default, if no other theme is set. (by @regisb)
- 💥[Deprecation] Tutor no longer supports the legacy Docker builder, which was previously available by setting `DOCKER_BUILDKIT=0` in the host environment. Going forward, Tutor will always use BuildKit (a.k.a. `docker buildx` in Docker v19-v22, or just `docker build` in Docker v23). This transition will improve build performance and should be seamless for Tutor users who are running a supported Docker version (by @kdmccormick).
- 💥[Deprecation] The template variable `is_buildkit_enabled`, which now always returns True, is deprecated. Plugin authors should assume BuildKit is enabled and remove the variable from their templates (by @kdmccormick).
- 💥[Deprecation] Adding Python packages to edx-platform via `private.txt` is no longer supported. Instead, users should bind-mount their requirement directories with `tutor mounts add ...`. (by @regisb)
- [Bugfix] Updated how the Tutor setting `JWT_RSA_PRIVATE_KEY` is rendered into the LMS Django setting `JWT_AUTH['JWT_PRIVATE_SIGNING_JWK']` as required by a recent breaking upstream change. The new representation of the `JWT_PRIVATE_SIGNING_JWK` simply adds the `dq`, `dp`, and `qi` parameters. Without this fix, LMS would encounter an `InvalidKeyError` on all logins. (by @kdmccormick)
- [Improvement] You don't have to run `tutor config save` every time you enable or disable a plugin anymore. (by @CodeWithEmad)

<a id='changelog-16.1.8'></a>
## v16.1.8 (2023-12-10)

Expand Down
2 changes: 0 additions & 2 deletions changelog.d/20230818_112124_kyle_buildkit.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/20231011_000434_codewithemad_save_configs.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/20231020_143112_kyle_jwk_full_complement.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/20231205_113430_regis_local_edly_io.md

This file was deleted.

38 changes: 9 additions & 29 deletions docs/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ Open edX customisation

This defines the git repository from which you install Open edX platform code. If you run an Open edX fork with custom patches, set this to your own git repository. You may also override this configuration parameter at build time, by providing a ``--build-arg`` option.

- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/palm.4"``, or ``master`` in :ref:`nightly <nightly>`)
- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/quince.1"``, or ``master`` in :ref:`nightly <nightly>`)

This defines the default version that will be pulled from all Open edX git repositories.

Expand All @@ -156,9 +156,9 @@ These two configuration parameters define which Redis database to use for Open e

.. _openedx_extra_pip_requirements:

- ``OPENEDX_EXTRA_PIP_REQUIREMENTS`` (default: ``["openedx-scorm-xblock>=16.0.0,<17.0.0"]``)
- ``OPENEDX_EXTRA_PIP_REQUIREMENTS`` (default: ``[]``)

This defines extra pip packages that are going to be installed for Open edX.
Define extra pip packages that are going to be installed for edx-platform.

- ``NPM_REGISTRY`` (default: ``"https://registry.npmjs.org/"``)

Expand Down Expand Up @@ -354,36 +354,16 @@ See :ref:`the corresponding tutorial <theming>`.

.. _custom_extra_xblocks:

Installing extra xblocks and requirements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Would you like to include custom xblocks, or extra requirements to your Open edX platform? Additional requirements can be added to the ``OPENEDX_EXTRA_PIP_REQUIREMENTS`` parameter in the :ref:`config file <configuration>` or to the ``env/build/openedx/requirements/private.txt`` file. The difference between them, is that ``private.txt`` file, even though it could be used for both, :ref:`should be used for installing extra xblocks or requirements from private repositories <extra_private_xblocks>`. For instance, to include the `polling xblock from Opencraft <https://github.com/open-craft/xblock-poll/>`_:
Would you like to include custom xblocks, or extra requirements to your Open edX platform? Additional requirements can be added to the ``OPENEDX_EXTRA_PIP_REQUIREMENTS`` parameter in the :ref:`config file <configuration>`. For instance, to include the `polling xblock from Opencraft <https://github.com/open-craft/xblock-poll/>`_:

tutor config save --append OPENEDX_EXTRA_PIP_REQUIREMENTS=git+https://github.com/open-craft/xblock-poll.git

Alternatively, add the dependency to ``private.txt``::

echo "git+https://github.com/open-craft/xblock-poll.git" >> "$(tutor config printroot)/env/build/openedx/requirements/private.txt"

Then, the ``openedx`` docker image must be rebuilt::

tutor images build openedx

.. _extra_private_xblocks:

Installing extra requirements from private repositories
*******************************************************

When installing extra xblock or requirements from private repositories, ``private.txt`` file should be used, because it allows installing dependencies without adding git credentials to the Docker image. By adding your git credentials to the Docker image, you're risking leaking your git credentials, if you were to publish (intentionally or unintentionally) the Docker image in a public place.

To install xblocks from a private repository that requires authentication, you must first clone the repository inside the ``openedx/requirements`` folder on the host::

git clone [email protected]:me/myprivaterepo.git "$(tutor config printroot)/env/build/openedx/requirements/myprivaterepo"

Then, declare your extra requirements with the ``-e`` flag in ``openedx/requirements/private.txt``::

echo "-e ./myprivaterepo" >> "$(tutor config printroot)/env/build/openedx/requirements/private.txt"

.. _edx_platform_fork:

Running a fork of ``edx-platform``
Expand All @@ -401,16 +381,16 @@ Note that your edx-platform version must be a fork of the latest release **tag**

If you don't create your fork from this tag, you *will* have important compatibility issues with other services. In particular:

- Do not try to run a fork from an older (pre-Palm) version of edx-platform: this will simply not work.
- Do not try to run a fork from an older (pre-Quince) version of edx-platform: this will simply not work.
- Do not try to run a fork from the edx-platform master branch: there is a 99% probability that it will fail.
- Do not try to run a fork from the open-release/palm.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/palm.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/palm.master branch.
- Do not try to run a fork from the open-release/quince.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/quince.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/quince.master branch.

.. _i18n:

Adding custom translations
~~~~~~~~~~~~~~~~~~~~~~~~~~

If you are not running Open edX in English (``LANGUAGE_CODE`` default: ``"en"``), chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX into your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform <https://github.com/openedx/edx-platform/tree/open-release/palm.master/conf/locale>`__ as well as those from `openedx-i18n <https://github.com/openedx/openedx-i18n/tree/master/edx-platform/locale>`__.
If you are not running Open edX in English (``LANGUAGE_CODE`` default: ``"en"``), chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX into your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform <https://github.com/openedx/edx-platform/tree/open-release/quince.master/conf/locale>`__ as well as those from `openedx-i18n <https://github.com/openedx/openedx-i18n/tree/master/edx-platform/locale>`__.

Tutor offers a relatively simple mechanism to add custom translations to the openedx Docker image. You should create a folder that corresponds to your language code in the "build/openedx/locale" folder of the Tutor environment. This folder should contain a "LC_MESSAGES" folder. For instance::

Expand All @@ -431,9 +411,9 @@ Then, add a "django.po" file there that will contain your custom translations::
.. warning::
Don't forget to specify the file ``Content-Type`` when adding message strings with non-ASCII characters; otherwise a ``UnicodeDecodeError`` will be raised during compilation.

The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language <https://github.com/openedx/edx-platform/blob/open-release/palm.master/conf/locale/en/LC_MESSAGES/django.po>`__.
The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language <https://github.com/openedx/edx-platform/blob/open-release/quince.master/conf/locale/en/LC_MESSAGES/django.po>`__.

If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well <https://github.com/openedx/edx-platform/blob/open-release/palm.master/conf/locale/en/LC_MESSAGES/djangojs.po>`__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory.
If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well <https://github.com/openedx/edx-platform/blob/open-release/quince.master/conf/locale/en/LC_MESSAGES/djangojs.po>`__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory.

To recap, here is an example. To translate a few strings in French, both from django.po and djangojs.po, we would have the following file hierarchy::

Expand Down
2 changes: 1 addition & 1 deletion docs/dev.rst
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ The ``openedx-dev`` Docker image is based on the same ``openedx`` image used by

- The user that runs inside the container has the same UID as the user on the host, to avoid permission problems inside mounted volumes (and in particular in the edx-platform repository).
- Additional Python and system requirements are installed for convenient debugging: `ipython <https://ipython.org/>`__, `ipdb <https://pypi.org/project/ipdb/>`__, vim, telnet.
- The edx-platform `development requirements <https://github.com/openedx/edx-platform/blob/open-release/palm.master/requirements/edx/development.in>`__ are installed.
- The edx-platform `development requirements <https://github.com/openedx/edx-platform/blob/open-release/quince.master/requirements/edx/development.in>`__ are installed.


If you are using a custom ``openedx`` image, then you will need to rebuild ``openedx-dev`` every time you modify ``openedx``. To so, run::
Expand Down
2 changes: 1 addition & 1 deletion docs/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ The `devstack <https://github.com/openedx/devstack>`_ is meant for development o
Is Tutor officially supported by edX?
-------------------------------------

Yes: as of the Open edX Maple release (December 9th 2021), Tutor is the only officially supported installation method for Open edX: see the `official installation instructions <https://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/open-release-palm.master/installation/index.html>`__.
Yes: as of the Open edX Maple release (December 9th 2021), Tutor is the only officially supported installation method for Open edX: see the `official installation instructions <https://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/open-release-quince.master/installation/index.html>`__.

What features are missing from Tutor?
-------------------------------------
Expand Down
6 changes: 3 additions & 3 deletions docs/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Requirements
* Architecture: Both AMD64 and ARM64 are supported.
* Required software:

- `Docker <https://docs.docker.com/engine/installation/>`__: v20.10.15+
- `Docker <https://docs.docker.com/engine/installation/>`__: v24.0.5+ (with BuildKit 0.11+)
- `Docker Compose <https://docs.docker.com/compose/install/>`__: v2.0.0+

.. warning::
Expand Down Expand Up @@ -123,11 +123,11 @@ Major Open edX releases are published twice a year, in June and December, by the
4. Test the new release in a sandboxed environment.
5. If you are running edx-platform, or some other repository from a custom branch, then you should rebase (and test) your changes on top of the latest release tag (see :ref:`edx_platform_fork`).

The process for upgrading from one major release to the next works similarly to any other upgrade, with the ``launch`` command (see above). The single difference is that if the ``launch`` command detects that your tutor environment was generated with an older release, it will perform a few release-specific upgrade steps. These extra upgrade steps will be performed just once. But they will be ignored if you updated your local environment (for instance: with ``tutor config save``) before running ``launch``. This situation typically occurs if you need to re-build some Docker images (see above). In such a case, you should make use of the ``upgrade`` command. For instance, to upgrade a local installation from Olive to Palm and rebuild some Docker images, run::
The process for upgrading from one major release to the next works similarly to any other upgrade, with the ``launch`` command (see above). The single difference is that if the ``launch`` command detects that your tutor environment was generated with an older release, it will perform a few release-specific upgrade steps. These extra upgrade steps will be performed just once. But they will be ignored if you updated your local environment (for instance: with ``tutor config save``) before running ``launch``. This situation typically occurs if you need to re-build some Docker images (see above). In such a case, you should make use of the ``upgrade`` command. For instance, to upgrade a local installation from Palm to Quince and rebuild some Docker images, run::

tutor config save
tutor images build all # list the images that should be rebuilt here
tutor local upgrade --from=olive
tutor local upgrade --from=palm
tutor local launch

.. _autocomplete:
Expand Down
2 changes: 1 addition & 1 deletion docs/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Yes :) This is what happens when you run ``tutor local launch``:
2. Configuration files are generated from templates.
3. Docker images are downloaded.
4. Docker containers are provisioned.
5. A full, production-ready Open edX platform (`Palm <https://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/open-release-palm.master/platform_releases/palm.html>`__ release) is run with docker-compose.
5. A full, production-ready Open edX platform (`Quince <https://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/open-release-quince.master/platform_releases/quince.html>`__ release) is run with docker-compose.

The whole procedure should require less than 10 minutes, on a server with good bandwidth. Note that your host environment will not be affected in any way, since everything runs inside docker containers. Root access is not even necessary.

Expand Down
10 changes: 5 additions & 5 deletions docs/reference/indexes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ Plugin indexes are a great way to have your plugins discovered by other users. P
Index file paths
================

A plugin index is a yaml-formatted file. It can be stored on the web or on your computer. In both cases, the index file location must end with "<current release name>/plugins.yml". For instance, the following are valid index locations if you run the Open edX "Palm" release:
A plugin index is a yaml-formatted file. It can be stored on the web or on your computer. In both cases, the index file location must end with "<current release name>/plugins.yml". For instance, the following are valid index locations if you run the Open edX "Quince" release:

- https://overhang.io/tutor/main/palm/plugins.yml
- ``/path/to/your/local/index/palm/plugins.yml``
- https://overhang.io/tutor/main/quince/plugins.yml
- ``/path/to/your/local/index/quince/plugins.yml``

To add either indexes, run the ``tutor plugins index add`` command without the suffix. For instance::

Expand Down Expand Up @@ -106,9 +106,9 @@ Manage plugins in development
Plugin developers and maintainers often want to install local versions of their plugins. They usually achieve this with ``pip install -e /path/to/tutor-plugin``. We can improve that workflow by creating an index for local plugins::

# Create the plugin index directory
mkdir -p ~/localindex/palm/
mkdir -p ~/localindex/quince/
# Edit the index
vim ~/localindex/palm/plugins.yml
vim ~/localindex/quince/plugins.yml

Add the following to the index::

Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/edx-platform.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Quite often, developers don't want to work on edx-platform directly, but on a de
cd /my/workspace/edx-ora2
git clone https://github.com/openedx/edx-ora2 .

Then, check out the right version of the package. This is the version that is indicated in the `edx-platform/requirements/edx/base.txt <https://github.com/openedx/edx-platform/blob/open-release/palm.master/requirements/edx/base.txt>`__. Be careful that the version that is currently in use in your version of edx-platform is **not necessarily the head of the master branch**::
Then, check out the right version of the package. This is the version that is indicated in the `edx-platform/requirements/edx/base.txt <https://github.com/openedx/edx-platform/blob/open-release/quince.master/requirements/edx/base.txt>`__. Be careful that the version that is currently in use in your version of edx-platform is **not necessarily the head of the master branch**::

git checkout <my-version-tag-or-branch>

Expand Down
Loading

0 comments on commit 0ccf486

Please sign in to comment.