v14.0.0: upgrade to Nutmeg

- 💥 [Feature] Upgrade to Nutmeg: (by @regisb)
    - 💥 [Feature] Persistent grades are now enabled by default.
    - [Bugfix] Remove edX references from bulk emails ([issue](openedx/wg-build-test-release#100)).
    - [Improvement] For Tutor Nightly (and only Nightly), official plugins are now installed from their nightly branches on GitHub instead of a version range on PyPI. This will allow Nightly users to install all official plugins by running ``pip install -e ".[full]"``.
    - [Bugfix] Start MongoDB when running migrations, because a new data migration fails if MongoDB is not running

CHANGELOG.md

@@ -18,6 +18,16 @@ Every user-facing change should have an entry in this changelog. Please respect
 
 ## Unreleased
 
+## v14.0.0 (2022-06-09)
+
+- 💥[Feature] Upgrade to Nutmeg: (by @regisb)
+  - 💥[Feature] Hide a course from the `/course` search page in the LMS when the course visibility is set to "none" in the Studio. (thanks @ghassanmas!)
+  - 💥[Improvement] The `lms.env.json` and `cms.env.json` files are moved to `lms.env.yml` and `cms.env.yml`. As a consequence, plugin developers must reformat the following plugins to use YAML format, and not JSON: 
+  - 💥[Feature] Persistent grades are now enabled by default.
+  - [Bugfix] Remove edX references from bulk emails ([issue](https://github.com/openedx/build-test-release-wg/issues/100)).
+  - [Improvement] For Tutor Nightly (and only Nightly), official plugins are now installed from their nightly branches on GitHub instead of a version range on PyPI. This will allow Nightly users to install all official plugins by running ``pip install -e ".[full]"``.
+  - [Bugfix] Start MongoDB when running migrations, because a new data migration fails if MongoDB is not running
+
 ## v13.3.1 (2022-06-06)
 
 - [Fix] Crashing celery workers in development (#681). (by @regisb)

docs/configuration.rst

@@ -8,7 +8,7 @@ Tutor offers plenty of possibilities for platform customisation out of the box.
 a. Modifying the Tutor :ref:`configuration parameters <configuration>`.
 b. Modifying the :ref:`Open edX docker image <customise>` that runs the Open edX platform.
 
-This section does not cover :ref:`plugin development <plugins>`. For simple changes, such as modifying the ``*.env.json`` files or the edx-platform settings, *you should not fork edx-platform or tutor*! Instead, you should create a simple :ref:`plugin for Tutor <plugins_yaml>`.
+This section does not cover :ref:`plugin development <plugins>`. For simple changes, such as modifying the ``*.env.yml`` files or the edx-platform settings, *you should not fork edx-platform or tutor*! Instead, you should create a simple :ref:`plugin for Tutor <plugins_yaml>`.
 
 .. _configuration:
 
@@ -31,7 +31,7 @@ Or from the system environment::
 
     export TUTOR_PARAM1=VALUE1
 
-Once the base configuration is created or updated, the environment is automatically re-generated. The environment is the set of all files required to manage an Open edX platform: Dockerfile, ``lms.env.json``, settings files, etc. You can view the environment files in the ``env`` folder::
+Once the base configuration is created or updated, the environment is automatically re-generated. The environment is the set of all files required to manage an Open edX platform: Dockerfile, ``lms.env.yml``, settings files, etc. You can view the environment files in the ``env`` folder::
 
     ls "$(tutor config printroot)/env"
 

docs/install.rst

@@ -121,11 +121,11 @@ Major Open edX releases are published twice a year, in June and December, by the
 4. Test the new release in a sandboxed environment.
 5. If you are running edx-platform, or some other repository from a custom branch, then you should rebase (and test) your changes on top of the latest release tag (see :ref:`edx_platform_fork`).
 
-The process for upgrading from one major release to the next works similarly to any other upgrade, with the ``quickstart`` command (see above). The single difference is that if the ``quickstart`` command detects that your tutor environment was generated with an older release, it will perform a few release-specific upgrade steps. These extra upgrade steps will be performed just once. But they will be ignored if you updated your local environment (for instance: with ``tutor config save``) before running ``quickstart``. This situation typically occurs if you need to re-build some Docker images (see above). In such a case, you should make use of the ``upgrade`` command. For instance, to upgrade a local installation from Lilac to Maple and rebuild some Docker images, run::
+The process for upgrading from one major release to the next works similarly to any other upgrade, with the ``quickstart`` command (see above). The single difference is that if the ``quickstart`` command detects that your tutor environment was generated with an older release, it will perform a few release-specific upgrade steps. These extra upgrade steps will be performed just once. But they will be ignored if you updated your local environment (for instance: with ``tutor config save``) before running ``quickstart``. This situation typically occurs if you need to re-build some Docker images (see above). In such a case, you should make use of the ``upgrade`` command. For instance, to upgrade a local installation from Maple to Nutmeg and rebuild some Docker images, run::
 
     tutor config save
     tutor images build all # list the images that should be rebuilt here
-    tutor local upgrade --from=lilac
+    tutor local upgrade --from=maple
     tutor local quickstart
 
 .. _autocomplete:

docs/plugins/intro.rst

@@ -4,7 +4,7 @@
 Introduction
 ============
 
-Tutor comes with a plugin system that allows anyone to customise the deployment of an Open edX platform very easily. The vision behind this plugin system is that users should not have to fork the Tutor repository to customise their deployments. For instance, if you have created a new application that integrates with Open edX, you should not have to describe how to manually patch the platform settings, ``urls.py`` or ``*.env.json`` files. Instead, you can create a "tutor-myapp" plugin for Tutor. Then, users will start using your application in three simple steps::
+Tutor comes with a plugin system that allows anyone to customise the deployment of an Open edX platform very easily. The vision behind this plugin system is that users should not have to fork the Tutor repository to customise their deployments. For instance, if you have created a new application that integrates with Open edX, you should not have to describe how to manually patch the platform settings, ``urls.py`` or ``*.env.yml`` files. Instead, you can create a "tutor-myapp" plugin for Tutor. Then, users will start using your application in three simple steps::
 
     # 1) Install the plugin
     pip install tutor-myapp

docs/reference/patches.rst

@@ -53,21 +53,21 @@ File: ``apps/caddy/Caddyfile``
 ``cms-env``
 ===========
 
-File: ``apps/openedx/config/cms.env.json``
+File: ``apps/openedx/config/cms.env.yml``
 
 .. patch:: cms-env-features
 
 ``cms-env-features``
 ====================
 
-File: ``apps/openedx/config/cms.env.json``
+File: ``apps/openedx/config/cms.env.yml``
 
 .. patch:: common-env-features
 
 ``common-env-features``
 =======================
 
-Files: ``apps/openedx/config/cms.env.json``, ``apps/openedx/config/lms.env.json``
+Files: ``apps/openedx/config/cms.env.yml``, ``apps/openedx/config/lms.env.yml``
 
 .. patch:: dev-docker-compose-jobs-services
 
@@ -137,14 +137,14 @@ File: ``kustomization.yml``
 ``lms-env``
 ===========
 
-File: ``apps/openedx/config/lms.env.json``
+File: ``apps/openedx/config/lms.env.yml``
 
 .. patch:: lms-env-features
 
 ``lms-env-features``
 ====================
 
-File: ``apps/openedx/config/lms.env.json``
+File: ``apps/openedx/config/lms.env.yml``
 
 .. patch:: local-docker-compose-caddy-aliases
 
@@ -200,7 +200,7 @@ File: ``local/docker-compose.yml``
 ``openedx-auth``
 ================
 
-File: ``apps/openedx/config/partials/auth.json``
+File: ``apps/openedx/config/partials/auth.yml``
 
 .. patch:: openedx-cms-common-settings
 

requirements/plugins.txt

@@ -1,12 +1,12 @@
-# change version ranges when upgrading from maple
-tutor-android>=13.0.0,<14.0.0
-tutor-discovery>=13.0.0,<14.0.0
-tutor-ecommerce>=13.0.0,<14.0.0
-tutor-forum>=13.0.0,<14.0.0
-tutor-license>=13.0.0,<14.0.0
-tutor-mfe>=13.0.0,<14.0.0
-tutor-minio>=13.0.0,<14.0.0
-tutor-notes>=13.0.0,<14.0.0
-tutor-richie>=13.0.0,<14.0.0
-tutor-webui>=13.0.0,<14.0.0
-tutor-xqueue>=13.0.0,<14.0.0
+# change version ranges when upgrading from nutmeg
+tutor-android>=14.0.0,<15.0.0
+tutor-discovery>=14.0.0,<15.0.0
+tutor-ecommerce>=14.0.0,<15.0.0
+tutor-forum>=14.0.0,<15.0.0
+tutor-license>=14.0.0,<15.0.0
+tutor-mfe>=14.0.0,<15.0.0
+tutor-minio>=14.0.0,<15.0.0
+tutor-notes>=14.0.0,<15.0.0
+tutor-richie>=14.0.0,<15.0.0
+tutor-webui>=14.0.0,<15.0.0
+tutor-xqueue>=14.0.0,<15.0.0

tests/test_env.py

@@ -258,6 +258,6 @@ def test_current_version_in_latest_env(self) -> None:
             ) as f:
                 f.write(__version__)
             self.assertEqual(__version__, env.current_version(root))
-            self.assertEqual("maple", env.get_env_release(root))
+            self.assertEqual("nutmeg", env.get_env_release(root))
             self.assertIsNone(env.should_upgrade_from_release(root))
             self.assertTrue(env.is_up_to_date(root))

tutor/__about__.py

@@ -2,7 +2,7 @@
 
 # Increment this version number to trigger a new release. See
 # docs/tutor.html#versioning for information on the versioning scheme.
-__version__ = "13.3.1"
+__version__ = "14.0.0"
 
 # The version suffix will be appended to the actual version, separated by a
 # dash. Use this suffix to differentiate between the actual released version and

tutor/commands/k8s.py

@@ -468,7 +468,7 @@ def wait(context: K8sContext, name: str) -> None:
 @click.option(
     "--from",
     "from_release",
-    type=click.Choice(["ironwood", "juniper", "koa", "lilac"]),
+    type=click.Choice(["ironwood", "juniper", "koa", "lilac", "maple"]),
 )
 @click.pass_context
 def upgrade(context: click.Context, from_release: Optional[str]) -> None:

tutor/commands/local.py

@@ -154,7 +154,7 @@ def quickstart(
 @click.option(
     "--from",
     "from_release",
-    type=click.Choice(["ironwood", "juniper", "koa", "lilac"]),
+    type=click.Choice(["ironwood", "juniper", "koa", "lilac", "maple"]),
 )
 @click.pass_context
 def upgrade(context: click.Context, from_release: t.Optional[str]) -> None:

tutor/commands/upgrade/k8s.py

@@ -27,6 +27,9 @@ def upgrade_from(context: Context, from_release: str) -> None:
         upgrade_from_lilac(config)
         running_release = "maple"
 
+    if running_release == "maple":
+        running_release = "nutmeg"
+
 
 def upgrade_from_ironwood(config: Config) -> None:
     if not config["RUN_MONGODB"]:

tutor/commands/upgrade/local.py

@@ -31,6 +31,9 @@ def upgrade_from(context: click.Context, from_release: str) -> None:
         common_upgrade.upgrade_from_lilac(config)
         running_release = "maple"
 
+    if running_release == "maple":
+        running_release = "nutmeg"
+
 
 def upgrade_from_ironwood(context: click.Context, config: Config) -> None:
     click.echo(fmt.title("Upgrading from Ironwood"))

tutor/env.py

@@ -388,6 +388,7 @@ def get_release(version: str) -> str:
         "11": "koa",
         "12": "lilac",
         "13": "maple",
+        "14": "nutmeg",
     }[version.split(".", maxsplit=1)[0]]
 
 

tutor/templates/apps/openedx/config/cms.env.yml

@@ -0,0 +1,73 @@
+SITE_NAME: "{{ CMS_HOST }}"
+BOOK_URL: ""
+LOG_DIR: "/openedx/data/logs"
+LOGGING_ENV: "sandbox"
+OAUTH_OIDC_ISSUER: "{{ JWT_COMMON_ISSUER }}"
+PLATFORM_NAME: "{{ PLATFORM_NAME }}"
+FEATURES:
+  {{ patch("common-env-features") }}
+  {{ patch("cms-env-features") }}
+  CERTIFICATES_HTML_VIEW: true
+  PREVIEW_LMS_BASE: "{{ PREVIEW_LMS_HOST }}"
+  ENABLE_COURSEWARE_INDEX: true
+  ENABLE_CSMH_EXTENDED: false
+  ENABLE_LEARNER_RECORDS: false
+  ENABLE_LIBRARY_INDEX: true
+  MILESTONES_APP: true
+  ENABLE_PREREQUISITE_COURSES: true
+LMS_ROOT_URL: "{{ "https" if ENABLE_HTTPS else "http" }}://{{ LMS_HOST }}"
+CMS_ROOT_URL: "{{ "https" if ENABLE_HTTPS else "http" }}://{{ CMS_HOST }}"
+CMS_BASE: "{{ CMS_HOST }}"
+LMS_BASE: "{{ LMS_HOST }}"
+CONTACT_EMAIL: "{{ CONTACT_EMAIL }}"
+CELERY_BROKER_TRANSPORT: "redis"
+CELERY_BROKER_HOSTNAME: "{{ REDIS_HOST }}:{{ REDIS_PORT }}"
+CELERY_BROKER_VHOST: "{{ OPENEDX_CELERY_REDIS_DB }}"
+CELERY_BROKER_USER: "{{ REDIS_USERNAME }}"
+CELERY_BROKER_PASSWORD: "{{ REDIS_PASSWORD }}"
+ALTERNATE_WORKER_QUEUES: "lms"
+ENABLE_COMPREHENSIVE_THEMING: true
+COMPREHENSIVE_THEME_DIRS: ["/openedx/themes"]
+STATIC_ROOT_BASE: "/openedx/staticfiles"
+EMAIL_BACKEND: "django.core.mail.backends.smtp.EmailBackend"
+EMAIL_HOST: "{{ SMTP_HOST }}"
+EMAIL_PORT: {{ SMTP_PORT }},
+EMAIL_USE_TLS: {{ "true" if SMTP_USE_TLS else "false" }}
+HTTPS: "{{ "on" if ENABLE_HTTPS else "off" }}"
+LANGUAGE_CODE: "{{ LANGUAGE_CODE }}"
+SESSION_COOKIE_DOMAIN: "{{ CMS_HOST }}"
+{{ patch("cms-env") }}
+CACHES:
+  default:
+    KEY_PREFIX: "default"
+    VERSION: "1"
+    BACKEND: "django_redis.cache.RedisCache"
+    LOCATION: "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/{{ OPENEDX_CACHE_REDIS_DB }}"
+  general:
+    KEY_PREFIX: "general"
+    BACKEND: "django_redis.cache.RedisCache"
+    LOCATION: "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/{{ OPENEDX_CACHE_REDIS_DB }}"
+  mongo_metadata_inheritance:
+    KEY_PREFIX: "mongo_metadata_inheritance"
+    TIMEOUT: 300,
+    BACKEND: "django_redis.cache.RedisCache"
+    LOCATION: "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/{{ OPENEDX_CACHE_REDIS_DB }}"
+  staticfiles:
+    KEY_PREFIX: "staticfiles_cms"
+    BACKEND: "django.core.cache.backends.locmem.LocMemCache"
+    LOCATION: "staticfiles_cms"
+  configuration:
+    KEY_PREFIX: "configuration"
+    BACKEND: "django_redis.cache.RedisCache"
+    LOCATION: "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/{{ OPENEDX_CACHE_REDIS_DB }}"
+  celery:
+    KEY_PREFIX: "celery"
+    TIMEOUT: "7200"
+    BACKEND: "django_redis.cache.RedisCache"
+    LOCATION: "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/{{ OPENEDX_CACHE_REDIS_DB }}"
+  course_structure_cache:
+    KEY_PREFIX: "course_structure"
+    TIMEOUT: "7200"
+    BACKEND: "django_redis.cache.RedisCache"
+    LOCATION: "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/{{ OPENEDX_CACHE_REDIS_DB }}"
+{% include "apps/openedx/config/partials/auth.yml" %}