Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert modules to use yamlRestTest #59089

Merged
merged 11 commits into from
Jul 13, 2020
3 changes: 3 additions & 0 deletions modules/aggs-matrix-stats/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't yaml-rest-test apply the rest-resources plugin?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does, wasn't sure if we wanted to keep it explicit since most projects are configured in some way or if we should just let it be implicit via the yaml-rest-test.

I don't have a strong opinion either way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to be explicit. We are saying this is a "yaml test project", we should assume that plugin will apply any dependencies.


esplugin {
description 'Adds aggregations whose input are a list of numeric fields and output includes a matrix.'
Expand All @@ -28,3 +29,5 @@ restResources {
includeCore '_common', 'indices', 'cluster', 'index', 'search', 'nodes'
}
}

integTest.enabled = false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's creating this task? Can we structure things somehow such that projects only with yaml tests simply don't have this task at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is the PluginBuildPlugin :( I can't really remove it or conditionally apply it until we get javaYamlTest (and all internalClusterTest are correctly moved too).

I should be able to get to that after all of the yamlRestTests are converted.

1 change: 1 addition & 0 deletions modules/analysis-common/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'Adds "built in" analyzers to Elasticsearch.'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertHitCount;

public class QueryStringWithAnalyzersTests extends ESIntegTestCase {
public class QueryStringWithAnalyzersIT extends ESIntegTestCase {
@Override
protected Collection<Class<? extends Plugin>> nodePlugins() {
return Arrays.asList(CommonAnalysisPlugin.class);
Expand Down
11 changes: 10 additions & 1 deletion modules/geo/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,21 @@
* under the License.
*/
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'Placeholder plugin for geospatial features in ES. only registers geo_shape field mapper for now'
classname 'org.elasticsearch.geo.GeoPlugin'
}

restResources {
restApi {
includeCore '_common', 'indices', 'index', 'search'
}
}
artifacts {
restTests(new File(projectDir, "src/test/resources/rest-api-spec/test"))
restTests(new File(projectDir, "src/yamlRestTest/resources/rest-api-spec/test"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use file() instead of new File().

}

integTest.enabled = false
test.enabled = false
28 changes: 0 additions & 28 deletions modules/geo/src/test/java/org/elasticsearch/geo/GeoTests.java

This file was deleted.

3 changes: 2 additions & 1 deletion modules/ingest-common/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'Module for ingest processors that do not require additional security permissions or have large dependencies and resources'
Expand All @@ -36,7 +37,7 @@ restResources {
}
}

testClusters.integTest {
testClusters.all {
// Needed in order to test ingest pipeline templating:
// (this is because the integTest node is not using default distribution, but only the minimal number of required modules)
module project(':modules:lang-mustache').tasks.bundlePlugin.archiveFile
Expand Down
4 changes: 4 additions & 0 deletions modules/ingest-geoip/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
import org.apache.tools.ant.taskdefs.condition.Os

apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'
apply plugin: 'elasticsearch.internal-cluster-test'

esplugin {
description 'Ingest processor that uses lookup geo data based on IP adresses using the MaxMind geo database'
Expand All @@ -42,6 +44,8 @@ restResources {
}
}

integTest.enabled = false

task copyDefaultGeoIp2DatabaseFiles(type: Copy) {
from { zipTree(configurations.testCompileClasspath.files.find { it.name.contains('geolite2-databases') }) }
into "${project.buildDir}/ingest-geoip"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked;
import static org.hamcrest.Matchers.equalTo;

public class GeoIpProcessorNonIngestNodeTests extends ESIntegTestCase {
public class GeoIpProcessorNonIngestNodeIT extends ESIntegTestCase {

public static class IngestGeoIpSettingsPlugin extends Plugin {

Expand Down
5 changes: 4 additions & 1 deletion modules/ingest-user-agent/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'Ingest processor that extracts information from a user agent'
Expand All @@ -29,6 +30,8 @@ restResources {
}
}

testClusters.integTest {
testClusters.all {
extraConfigFile 'ingest-user-agent/test-regexes.yml', file('src/test/test-regexes.yml')
}

integTest.enabled = false
1 change: 1 addition & 0 deletions modules/lang-expression/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'Lucene expressions integration for Elasticsearch'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@
import static org.hamcrest.Matchers.notNullValue;

// TODO: please convert to unit tests!
public class MoreExpressionTests extends ESIntegTestCase {
public class MoreExpressionIT extends ESIntegTestCase {

@Override
protected Collection<Class<? extends Plugin>> nodePlugins() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
import static org.hamcrest.Matchers.containsString;

//TODO: please convert to unit tests!
public class StoredExpressionTests extends ESIntegTestCase {
public class StoredExpressionIT extends ESIntegTestCase {
@Override
protected Settings nodeSettings(int nodeOrdinal) {
Settings.Builder builder = Settings.builder().put(super.nodeSettings(nodeOrdinal));
Expand Down
1 change: 1 addition & 0 deletions modules/lang-mustache/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'Mustache scripting integration for Elasticsearch'
Expand Down
5 changes: 4 additions & 1 deletion modules/lang-painless/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,14 @@
import org.elasticsearch.gradle.testclusters.DefaultTestClustersTask;
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.validate-rest-spec'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'An easy, safe and fast scripting language for Elasticsearch'
classname 'org.elasticsearch.painless.PainlessPlugin'
}

testClusters.integTest {
testClusters.all {
module project(':modules:mapper-extras').tasks.bundlePlugin.archiveFile
systemProperty 'es.scripting.update.ctx_in_params', 'false'
// TODO: remove this once cname is prepended to transport.publish_address by default in 8.0
Expand Down Expand Up @@ -54,6 +55,8 @@ restResources {
}
}

integTest.enabled = false

test {
// in WhenThingsGoWrongTests we intentionally generate an out of memory error, this prevents the heap from being dumped to disk
jvmArgs '-XX:-OmitStackTraceInFastThrow', '-XX:-HeapDumpOnOutOfMemoryError'
Expand Down
1 change: 1 addition & 0 deletions modules/mapper-extras/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'Adds advanced field mappers'
Expand Down
1 change: 1 addition & 0 deletions modules/parent-join/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'This module adds the support parent-child queries and aggregations'
Expand Down
3 changes: 3 additions & 0 deletions modules/percolator/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'Percolator module adds capability to index queries and query these queries by specifying documents'
Expand Down Expand Up @@ -45,3 +46,5 @@ restResources {
includeCore '_common', 'indices', 'index', 'search', 'msearch'
}
}

integTest.enabled = false
3 changes: 2 additions & 1 deletion modules/rank-eval/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'The Rank Eval module adds APIs to evaluate ranking quality.'
Expand All @@ -29,7 +30,7 @@ restResources {
}
}

testClusters.integTest {
testClusters.all {
// Modules who's integration is explicitly tested in integration tests
module project(':modules:lang-mustache').tasks.bundlePlugin.archiveFile
}
3 changes: 2 additions & 1 deletion modules/reindex/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,14 @@ import org.elasticsearch.gradle.info.BuildParams
apply plugin: 'elasticsearch.test-with-dependencies'
apply plugin: 'elasticsearch.jdk-download'
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'The Reindex module adds APIs to reindex from one index to another or update documents in place.'
classname 'org.elasticsearch.index.reindex.ReindexPlugin'
}

testClusters.integTest {
testClusters.all {
// Modules who's integration is explicitly tested in integration tests
module project(':modules:parent-join').tasks.bundlePlugin.archiveFile
module project(':modules:lang-painless').tasks.bundlePlugin.archiveFile
Expand Down
7 changes: 6 additions & 1 deletion modules/repository-url/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import org.elasticsearch.gradle.info.BuildParams
import org.elasticsearch.gradle.test.AntFixture

apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

esplugin {
description 'Module for URL repository'
Expand All @@ -48,14 +49,18 @@ task urlFixture(type: AntFixture) {
args 'org.elasticsearch.repositories.url.URLFixture', baseDir, "${repositoryDir.absolutePath}"
}

yamlRestTest {
dependsOn urlFixture
}
integTest {
dependsOn urlFixture
}
testClusters.integTest {
testClusters.all {
// repositoryDir is used by a FS repository to create snapshots
setting 'path.repo', "${repositoryDir.absolutePath}", PropertyNormalization.IGNORE_VALUE
// repositoryDir is used by two URL repositories to restore snapshots
setting 'repositories.url.allowed_urls', {
"http://snapshot.test*,http://${urlFixture.addressAndPort}"
}, PropertyNormalization.IGNORE_VALUE
}

Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
import static org.hamcrest.Matchers.notNullValue;

@ESIntegTestCase.ClusterScope(scope = ESIntegTestCase.Scope.TEST)
public class URLSnapshotRestoreTests extends ESIntegTestCase {
public class URLSnapshotRestoreIT extends ESIntegTestCase {

@Override
protected Collection<Class<? extends Plugin>> nodePlugins() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ public void registerRepositories() throws IOException {

// Create a URL repository using the file://{path.repo} URL
Request createFileRepositoryRequest = new Request("PUT", "/_snapshot/repository-file");
createFileRepositoryRequest.setEntity(buildRepositorySettings(URLRepository.TYPE,
createFileRepositoryRequest.setEntity(buildRepositorySettings("url",
Settings.builder().put("url", pathRepoUri.toString()).build()));
Response createFileRepositoryResponse = client().performRequest(createFileRepositoryRequest);
assertThat(createFileRepositoryResponse.getStatusLine().getStatusCode(), equalTo(RestStatus.OK.getStatus()));
Expand All @@ -106,7 +106,7 @@ public void registerRepositories() throws IOException {
InetAddress inetAddress = InetAddress.getByName(new URL(allowedUrl).getHost());
if (inetAddress.isAnyLocalAddress() || inetAddress.isLoopbackAddress()) {
Request createUrlRepositoryRequest = new Request("PUT", "/_snapshot/repository-url");
createUrlRepositoryRequest.setEntity(buildRepositorySettings(URLRepository.TYPE,
createUrlRepositoryRequest.setEntity(buildRepositorySettings("url",
Settings.builder().put("url", allowedUrl).build()));
Response createUrlRepositoryResponse = client().performRequest(createUrlRepositoryRequest);
assertThat(createUrlRepositoryResponse.getStatusLine().getStatusCode(), equalTo(RestStatus.OK.getStatus()));
Expand Down
1 change: 1 addition & 0 deletions modules/transport-netty4/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import org.elasticsearch.gradle.info.BuildParams
import org.elasticsearch.gradle.test.RestIntegTestTask

apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'elasticsearch.yaml-rest-test'

/*
TODOs:
Expand Down