Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate mapper-related modules to internal-*-rest-test #117298

Merged
merged 5 commits into from
Nov 23, 2024

Conversation

lkts
Copy link
Contributor

@lkts lkts commented Nov 21, 2024

Legacy yaml test runner does not support supplying test cluster with test features. Due to recently introduced mapper.source.mode_from_index_setting test feature, a lot of tests using it were silently ignored. This PR fixes this by migrating all projects using legacy runner and mapper.source.mode_from_index_setting to a "proper" test plugin.

Related to #117284.
Related to #116072.

@lkts lkts added >non-issue auto-backport Automatically create backport pull requests when merged :StorageEngine/Mapping The storage related side of mappings v8.17.0 v8.18.0 labels Nov 21, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

Copy link
Contributor

@kkrik-es kkrik-es left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The direction looks good, a couple of tests still failing.. Approving to get your going.


clusterModules project(':modules:lang-painless')
clusterModules project(':modules:parent-join')
clusterModules(project(":modules:rest-root"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: just for consistency you can omit the parens on this one

// Whitelist reindexing from the local node so we can test reindex-from-remote.
setting 'reindex.remote.whitelist', '127.0.0.1:*'
requiresFeature 'es.index_mode_feature_flag_registered', Version.fromString("8.0.0")
if (buildParams.isSnapshotBuild() == false) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this bit required? It wasn't in the old build script. Was this perhaps just copy/pasted from elsewhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my attempt to replicate index_mode_feature_flag_registered that you asked about below. I realize now that it is a noop since it does not actually apply to a cluster. I have no clue what index_mode_feature_flag_registered does but it seems to not impact anything indeed. I'll research a bit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct, you need to add the feature flag to the cluster. Feature flags are only required when running the tests in "release" mode. Maybe this flag isn't required anymore and is enabled by default in current versions so indeed may be unnecessary now. Probably worth confirming.

Copy link
Contributor Author

@lkts lkts Nov 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last usage of index_mode_feature_flag_registered was removed in #95220. It's a complete noop and should be removed everywhere.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome. This can be done separately in a follow up.

module ':modules:rest-root'
// Whitelist reindexing from the local node so we can test reindex-from-remote.
setting 'reindex.remote.whitelist', '127.0.0.1:*'
requiresFeature 'es.index_mode_feature_flag_registered', Version.fromString("8.0.0")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this feature flag no longer required? I don't see it added to any of the migrated test classes.

@@ -42,7 +42,3 @@ if (buildParams.isSnapshotBuild() == false) {
systemProperty 'es.index_mode_feature_flag_registered', 'true'
}
}

testClusters.configureEach {
requiresFeature 'es.index_mode_feature_flag_registered', Version.fromString("8.0.0")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as before, do we not need this feature flag?

@lkts lkts added the test-full-bwc Trigger full BWC version matrix tests label Nov 22, 2024
Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch.

@lkts lkts enabled auto-merge (squash) November 22, 2024 19:26
@lkts lkts merged commit 2b8e4e7 into elastic:main Nov 23, 2024
16 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.17 Commit could not be cherrypicked due to conflicts
8.x Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 117298

lkts added a commit to lkts/elasticsearch that referenced this pull request Nov 23, 2024
(cherry picked from commit 2b8e4e7)

# Conflicts:
#	modules/mapper-extras/build.gradle
#	plugins/mapper-annotated-text/build.gradle
#	plugins/mapper-murmur3/build.gradle
#	x-pack/plugin/mapper-unsigned-long/build.gradle
#	x-pack/plugin/mapper-version/build.gradle
#	x-pack/plugin/wildcard/build.gradle
@lkts
Copy link
Contributor Author

lkts commented Nov 23, 2024

💚 All backports created successfully

Status Branch Result
8.x
8.17

Questions ?

Please refer to the Backport tool documentation

lkts added a commit to lkts/elasticsearch that referenced this pull request Nov 23, 2024
(cherry picked from commit 2b8e4e7)

# Conflicts:
#	modules/mapper-extras/build.gradle
#	plugins/mapper-annotated-text/build.gradle
#	plugins/mapper-murmur3/build.gradle
#	x-pack/plugin/mapper-unsigned-long/build.gradle
#	x-pack/plugin/mapper-version/build.gradle
#	x-pack/plugin/wildcard/build.gradle
lkts added a commit that referenced this pull request Nov 25, 2024
…7407)

(cherry picked from commit 2b8e4e7)

# Conflicts:
#	modules/mapper-extras/build.gradle
#	plugins/mapper-annotated-text/build.gradle
#	plugins/mapper-murmur3/build.gradle
#	x-pack/plugin/mapper-unsigned-long/build.gradle
#	x-pack/plugin/mapper-version/build.gradle
#	x-pack/plugin/wildcard/build.gradle
lkts added a commit that referenced this pull request Nov 25, 2024
…7406)

(cherry picked from commit 2b8e4e7)

# Conflicts:
#	modules/mapper-extras/build.gradle
#	plugins/mapper-annotated-text/build.gradle
#	plugins/mapper-murmur3/build.gradle
#	x-pack/plugin/mapper-unsigned-long/build.gradle
#	x-pack/plugin/mapper-version/build.gradle
#	x-pack/plugin/wildcard/build.gradle
@lkts lkts deleted the migrate-yaml-tests branch November 25, 2024 16:25
alexey-ivanov-es pushed a commit to alexey-ivanov-es/elasticsearch that referenced this pull request Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged backport pending >non-issue :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine test-full-bwc Trigger full BWC version matrix tests v8.17.0 v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants