feature(views-with-tablets): enable testing materialized views with t…

…ablets

We plan to disallow materialized views in tablet keyspaces, until the remaining issues with them
are resolved. For that, we've introduced the "views-with-tablets" experimental feature, so that
they can be enabled regardless, for example for testing. Currently, the feature has no effect,
but we want to prepare the cluster tests to use it, so that they won't start failing when it
starts actually having an effect.

While improving views with tablets, we should make sure that the existing tests keep passing,
so we should run materialized view (and index) tests with tablets using the experimental feature.

The feature doesn't affect tests ran on vnodes. For tablets, the tests would fail without the feature
due to unsupported configuration.

Considering that, we should enable the feature by default for all tests running on 2025.1 or later.

The change includes:
* adjusting experimental_features based on scylla version in sct_config, setting the new feature
when running on 2025.1 or later
* adding a new parameter "enable_views_with_tablets_on_upgrade". When it's set, we enable the new
feature in upgrade_test.py after upgrading a node. Currently, there are no upgrade tests to 2025.1,
so it isn't used
* adding a config file configurations/enable_views_with_tablets_on_upgrade.yaml that enables the
mentioned parameter without setting an env variable

configurations/enable_views_with_tablets_on_upgrade.yaml

@@ -0,0 +1 @@
+enable_views_with_tablets_on_upgrade: true

defaults/test_default.yaml

@@ -142,6 +142,7 @@ test_upgrade_from_installed_3_1_0: false
 target_upgrade_version: ''
 disable_raft: true
 enable_tablets_on_upgrade: false
+enable_views_with_tablets_on_upgrade: false
 upgrade_node_system: true
 
 stress_cdclog_reader_cmd: "cdc-stressor -stream-query-round-duration 30s"

docs/configuration_options.md

@@ -739,6 +739,7 @@ Set type of ssh library to use. Could be 'fabric' (default) or 'libssh2'
 
 unlock specified experimental features
 
+
 **default:** N/A
 
 **type:** list
@@ -2642,6 +2643,13 @@ By default, the tablets feature is disabled. With this parameter, created for th
 
 **type:** boolean
 
+## **enable_views_with_tablets_on_upgrade** / SCT_ENABLE_VIEWS_WITH_TABLETS_ON_UPGRADE
+
+Enables creating materialized views in keyspaces using tablets by adding an experimental feature. It should not be used when upgrading to versions before 2025.1 and it should be used for upgrades where we create such views. In particular, in 6.2 tablets are enabled by default, so for the 6.2 -> 2025.1 upgrade, it should also be enabled when using defaults.
+
+**default:** False
+
+**type:** boolean
 
 ## **upgrade_node_packages** / SCT_UPGRADE_NODE_PACKAGES
 

sdcm/sct_config.py

@@ -58,6 +58,7 @@
     get_specific_tag_of_docker_image,
     find_scylla_repo,
     is_enterprise,
+    ComparableScyllaVersion,
 )
 from sdcm.sct_events.base import add_severity_limit_rules, print_critical_events
 from sdcm.utils.gce_utils import (
@@ -1319,6 +1320,11 @@ class SCTConfiguration(dict):
              help="By default, the tablets feature is disabled. With this parameter, created for the upgrade test,"
                   "the tablets feature will only be enabled after the upgrade"),
 
+        dict(name="enable_views_with_tablets_on_upgrade", env="SCT_ENABLE_VIEWS_WITH_TABLETS_ON_UPGRADE", type=boolean,
+             help="Enables creating materialized views in keyspaces using tablets by adding an experimental feature."
+                  "It should not be used when upgrading to versions before 2025.1 and it should be used for upgrades"
+               "where we create such views."),
+
         dict(name="upgrade_node_packages", env="SCT_UPGRADE_NODE_PACKAGES", type=str,
              help=""),
 
@@ -2770,6 +2776,10 @@ def update_argus_with_version(self, scylla_version: str, package_name: str):
         except Exception as exc:  # pylint: disable=broad-except
             self.log.exception("Failed to save target Scylla version in Argus", exc_info=exc)
 
+    def update_config_based_on_version(self):
+        if self.is_enterprise and ComparableScyllaVersion(self.scylla_version) >= "2025.1.0~dev":
+            self['experimental_features'] = ['views-with-tablets']
+
     def dict(self):
         out = deepcopy(self)
 
@@ -2890,5 +2900,6 @@ def init_and_verify_sct_config() -> SCTConfiguration:
     sct_config.verify_configuration()
     sct_config.verify_configuration_urls_validity()
     sct_config.get_version_based_on_conf()
+    sct_config.update_config_based_on_version()
     sct_config.check_required_files()
     return sct_config

upgrade_test.py

@@ -221,7 +221,8 @@ def _upgrade_node(self, node, upgrade_sstables=True, new_scylla_repo=None, new_v
 
         if self.params.get("enable_tablets_on_upgrade"):
             scylla_yaml_updates.update({"enable_tablets": True})
-
+        if self.params.get("enable_views_with_tablets_on_upgrade"):
+            scylla_yaml_updates.update({"experimental_features": ["views-with-tablets"]})
         if self.params.get('test_sst3'):
             scylla_yaml_updates.update({"enable_sstables_mc_format": True})