Skip to content

Commit

Permalink
Merge pull request #1 from elastic/master
Browse files Browse the repository at this point in the history
merge the lastest commits from author
  • Loading branch information
dengweisysu authored Aug 21, 2019
2 parents 30a9a76 + b0d346f commit 624729e
Show file tree
Hide file tree
Showing 1,426 changed files with 32,250 additions and 19,395 deletions.
19 changes: 8 additions & 11 deletions LICENSE.txt
Original file line number Diff line number Diff line change
@@ -1,13 +1,10 @@
Source code in this repository is variously licensed under the Apache License
Version 2.0, an Apache compatible license, or the Elastic License. Outside of
the "x-pack" folder, source code in a given file is licensed under the Apache
License Version 2.0, unless otherwise noted at the beginning of the file or a
LICENSE file present in the directory subtree declares a separate license.
Within the "x-pack" folder, source code in a given file is licensed under the
Elastic License, unless otherwise noted at the beginning of the file or a
LICENSE file present in the directory subtree declares a separate license.
Source code in this repository is covered by one of three licenses: (i) the
Apache License 2.0 (ii) an Apache License 2.0 compatible license (iii) the
Elastic License. The default license throughout the repository is Apache License
2.0 unless the header specifies another license. Elastic Licensed code is found
only in the x-pack directory.

The build produces two sets of binaries - one set that falls under the Elastic
License and another set that falls under Apache License Version 2.0. The
binaries that contain `-oss` in the artifact name are licensed under the Apache
License Version 2.0.
License and another set that falls under Apache License 2.0. The binaries that
contain `-oss` in the artifact name are licensed under Apache License 2.0 and
these binaries do not package any code from the x-pack directory.
37 changes: 15 additions & 22 deletions TESTING.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -76,28 +76,21 @@ In order to set an Elasticsearch setting, provide a setting with the following p

=== Test case filtering.

- `tests.class` is a class-filtering shell-like glob pattern,
- `tests.method` is a method-filtering glob pattern.
You can run a single test, provided that you specify the Gradle project. See the documentation on
https://docs.gradle.org/current/userguide/userguide_single.html#simple_name_pattern[simple name pattern filtering].

Run a single test case (variants)
Run a single test case in the `server` project:

----------------------------------------------------------
./gradlew test -Dtests.class=org.elasticsearch.package.ClassName
./gradlew test "-Dtests.class=*.ClassName"
./gradlew :server:test --tests org.elasticsearch.package.ClassName
----------------------------------------------------------

Run all tests in a package and its sub-packages
Run all tests in a package and its sub-packages:

----------------------------------------------------
./gradlew test "-Dtests.class=org.elasticsearch.package.*"
./gradlew :server:test --tests 'org.elasticsearch.package.*'
----------------------------------------------------

Run any test methods that contain 'esi' (like: ...r*esi*ze...)

-------------------------------
./gradlew test "-Dtests.method=*esi*"
-------------------------------

Run all tests that are waiting for a bugfix (disabled by default)

------------------------------------------------
Expand All @@ -118,7 +111,7 @@ Every test repetition will have a different method seed
(derived from a single random master seed).

--------------------------------------------------
./gradlew test -Dtests.iters=N -Dtests.class=*.ClassName
./gradlew :server:test -Dtests.iters=N --tests org.elasticsearch.package.ClassName
--------------------------------------------------

=== Repeats _all_ tests of ClassName N times.
Expand All @@ -127,7 +120,7 @@ Every test repetition will have exactly the same master (0xdead) and
method-level (0xbeef) seed.

------------------------------------------------------------------------
./gradlew test -Dtests.iters=N -Dtests.class=*.ClassName -Dtests.seed=DEAD:BEEF
./gradlew :server:test -Dtests.iters=N -Dtests.seed=DEAD:BEEF --tests org.elasticsearch.package.ClassName
------------------------------------------------------------------------

=== Repeats a given test N times
Expand All @@ -137,14 +130,14 @@ ie: testFoo[0], testFoo[1], etc... so using testmethod or tests.method
ending in a glob is necessary to ensure iterations are run).

-------------------------------------------------------------------------
./gradlew test -Dtests.iters=N -Dtests.class=*.ClassName -Dtests.method=mytest*
./gradlew :server:test -Dtests.iters=N --tests org.elasticsearch.package.ClassName.methodName
-------------------------------------------------------------------------

Repeats N times but skips any tests after the first failure or M initial failures.

-------------------------------------------------------------
./gradlew test -Dtests.iters=N -Dtests.failfast=true -Dtestcase=...
./gradlew test -Dtests.iters=N -Dtests.maxfailures=M -Dtestcase=...
./gradlew test -Dtests.iters=N -Dtests.failfast=true ...
./gradlew test -Dtests.iters=N -Dtests.maxfailures=M ...
-------------------------------------------------------------

=== Test groups.
Expand Down Expand Up @@ -175,7 +168,7 @@ systemProp.tests.jvms=8
----------------------------

Its difficult to pick the "right" number here. Hypercores don't count for CPU
intensive tests and you should leave some slack for JVM-interal threads like
intensive tests and you should leave some slack for JVM-internal threads like
the garbage collector. And you have to have enough RAM to handle each JVM.

=== Test compatibility.
Expand Down Expand Up @@ -553,10 +546,10 @@ When running `./gradlew check`, minimal bwc checks are also run against compatib
Sometimes a backward compatibility change spans two versions. A common case is a new functionality
that needs a BWC bridge in an unreleased versioned of a release branch (for example, 5.x).
To test the changes, you can instruct Gradle to build the BWC version from a another remote/branch combination instead of
pulling the release branch from GitHub. You do so using the `tests.bwc.remote` and `tests.bwc.refspec.BRANCH` system properties:
pulling the release branch from GitHub. You do so using the `bwc.remote` and `bwc.refspec.BRANCH` system properties:

-------------------------------------------------
./gradlew check -Dtests.bwc.remote=${remote} -Dtests.bwc.refspec.5.x=index_req_bwc_5.x
./gradlew check -Dbwc.remote=${remote} -Dbwc.refspec.5.x=index_req_bwc_5.x
-------------------------------------------------

The branch needs to be available on the remote that the BWC makes of the
Expand All @@ -571,7 +564,7 @@ will need to:
will contain your change.
. Create a branch called `index_req_bwc_5.x` off `5.x`. This will contain your bwc layer.
. Push both branches to your remote repository.
. Run the tests with `./gradlew check -Dtests.bwc.remote=${remote} -Dtests.bwc.refspec.5.x=index_req_bwc_5.x`.
. Run the tests with `./gradlew check -Dbwc.remote=${remote} -Dbwc.refspec.5.x=index_req_bwc_5.x`.

==== Skip fetching latest

Expand Down
6 changes: 3 additions & 3 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ import org.gradle.plugins.ide.eclipse.model.SourceFolder
import static org.elasticsearch.gradle.tool.Boilerplate.maybeConfigure

plugins {
id 'com.gradle.build-scan' version '2.3'
id 'com.gradle.build-scan' version '2.4'
id 'base'
id 'elasticsearch.global-build-info'
}
Expand Down Expand Up @@ -524,9 +524,9 @@ allprojects {
def checkPart2 = tasks.register('checkPart2')
plugins.withId('lifecycle-base') {
if (project.path.startsWith(":x-pack:")) {
checkPart1.configure { dependsOn 'check' }
} else {
checkPart2.configure { dependsOn 'check' }
} else {
checkPart1.configure { dependsOn 'check' }
}
}
}
20 changes: 5 additions & 15 deletions buildSrc/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -77,27 +77,13 @@ sourceSets {
minimumRuntime { }
}

configurations {
reaper
}

compileMinimumRuntimeJava {
targetCompatibility = 8
sourceCompatibility = 8
}

normalization {
runtimeClasspath {
// Ignore the embedded JAR as we track this separately below as a runtime dependency
ignore 'META-INF/*.jar'
}
}

jar {
from sourceSets.minimumRuntime.output
into('META-INF') {
from configurations.reaper
}
}

javadoc {
Expand Down Expand Up @@ -135,7 +121,6 @@ dependencies {
testCompile "junit:junit:${props.getProperty('junit')}"
testCompile "com.carrotsearch.randomizedtesting:randomizedtesting-runner:${props.getProperty('randomizedrunner')}"
testCompile 'com.github.tomakehurst:wiremock-jre8-standalone:2.23.2'
reaper project('reaper')
minimumRuntimeCompile "junit:junit:${props.getProperty('junit')}"
minimumRuntimeCompile localGroovy()
minimumRuntimeCompile gradleApi()
Expand Down Expand Up @@ -184,9 +169,11 @@ if (project != rootProject) {

configurations {
distribution
reaper
}

dependencies {
reaper project('reaper')
distribution project(':distribution:archives:windows-zip')
distribution project(':distribution:archives:oss-windows-zip')
distribution project(':distribution:archives:darwin-tar')
Expand All @@ -198,6 +185,9 @@ if (project != rootProject) {
// for external projects we want to remove the marker file indicating we are running the Elasticsearch project
processResources {
exclude 'buildSrc.marker'
into('META-INF') {
from configurations.reaper
}
}

String localDownloads = "${rootProject.buildDir}/local-downloads"
Expand Down
1 change: 1 addition & 0 deletions buildSrc/reaper/build.gradle
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
jar {
archiveName = "${project.name}.jar"
manifest {
attributes 'Main-Class': 'org.elasticsearch.gradle.reaper.Reaper'
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ import org.gradle.authentication.http.HttpHeaderAuthentication
import org.gradle.external.javadoc.CoreJavadocOptions
import org.gradle.internal.jvm.Jvm
import org.gradle.language.base.plugins.LifecycleBasePlugin
import org.gradle.process.CommandLineArgumentProvider
import org.gradle.process.ExecResult
import org.gradle.process.ExecSpec
import org.gradle.util.GradleVersion
Expand Down Expand Up @@ -834,10 +835,9 @@ class BuildPlugin implements Plugin<Project> {

test.jvmArgs "-Xmx${System.getProperty('tests.heap.size', '512m')}",
"-Xms${System.getProperty('tests.heap.size', '512m')}",
'-XX:+HeapDumpOnOutOfMemoryError',
"-XX:HeapDumpPath=$heapdumpDir",
'--illegal-access=warn'

test.jvmArgumentProviders.add({ ['-XX:+HeapDumpOnOutOfMemoryError', "-XX:HeapDumpPath=$heapdumpDir"] } as CommandLineArgumentProvider)

if (System.getProperty('tests.jvm.argline')) {
test.jvmArgs System.getProperty('tests.jvm.argline').split(" ")
Expand All @@ -848,8 +848,7 @@ class BuildPlugin implements Plugin<Project> {
}

// we use './temp' since this is per JVM and tests are forbidden from writing to CWD
test.systemProperties 'gradle.dist.lib': new File(project.class.location.toURI()).parent,
'java.io.tmpdir': './temp',
test.systemProperties 'java.io.tmpdir': './temp',
'java.awt.headless': 'true',
'tests.gradle': 'true',
'tests.artifact': project.name,
Expand All @@ -865,6 +864,7 @@ class BuildPlugin implements Plugin<Project> {
}

// don't track these as inputs since they contain absolute paths and break cache relocatability
nonInputProperties.systemProperty('gradle.dist.lib', new File(project.class.location.toURI()).parent)
nonInputProperties.systemProperty('gradle.worker.jar', "${project.gradle.getGradleUserHomeDir()}/caches/${project.gradle.gradleVersion}/workerMain/gradle-worker.jar")
nonInputProperties.systemProperty('gradle.user.home', project.gradle.getGradleUserHomeDir())

Expand All @@ -881,6 +881,9 @@ class BuildPlugin implements Plugin<Project> {
// TODO: remove this once ctx isn't added to update script params in 7.0
test.systemProperty 'es.scripting.update.ctx_in_params', 'false'

// TODO: remove this once cname is prepended to transport.publish_address by default in 8.0
test.systemProperty 'es.transport.cname_in_publish_address', 'true'

test.testLogging { TestLoggingContainer logging ->
logging.showExceptions = true
logging.showCauses = true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -237,16 +237,12 @@ class PrecommitTasks {
// on them. We also want `precommit` to depend on `checkstyle`.
project.pluginManager.apply('checkstyle')
project.checkstyle {
config = project.resources.text.fromFile(checkstyleConf, 'UTF-8')
configProperties = [
suppressions: checkstyleSuppressions
]
configDir = checkstyleDir
toolVersion = CHECKSTYLE_VERSION
}

project.tasks.withType(Checkstyle).configureEach { task ->
task.dependsOn(copyCheckstyleConf)
task.inputs.file(checkstyleSuppressions)
task.reports {
html.enabled false
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ package org.elasticsearch.gradle.test
import org.apache.tools.ant.DefaultLogger
import org.apache.tools.ant.taskdefs.condition.Os
import org.elasticsearch.gradle.BuildPlugin
import org.elasticsearch.gradle.BwcVersions
import org.elasticsearch.gradle.LoggedExec
import org.elasticsearch.gradle.Version
import org.elasticsearch.gradle.BwcVersions
import org.elasticsearch.gradle.VersionProperties
import org.elasticsearch.gradle.plugin.PluginBuildPlugin
import org.elasticsearch.gradle.plugin.PluginPropertiesExtension
Expand All @@ -39,11 +39,13 @@ import org.gradle.api.logging.Logger
import org.gradle.api.tasks.Copy
import org.gradle.api.tasks.Delete
import org.gradle.api.tasks.Exec
import org.gradle.internal.jvm.Jvm

import java.nio.charset.StandardCharsets
import java.nio.file.Paths
import java.util.concurrent.TimeUnit
import java.util.stream.Collectors

/**
* A helper for creating tasks to build a cluster that is used by a task, and tear down the cluster when the task is finished.
*/
Expand Down Expand Up @@ -887,15 +889,7 @@ class ClusterFormationTasks {
onlyIf { node.pidFile.exists() }
// the pid file won't actually be read until execution time, since the read is wrapped within an inner closure of the GString
ext.pid = "${ -> node.pidFile.getText('UTF-8').trim()}"
File jps
if (Os.isFamily(Os.FAMILY_WINDOWS)) {
jps = getJpsExecutableByName(project, "jps.exe")
} else {
jps = getJpsExecutableByName(project, "jps")
}
if (!jps.exists()) {
throw new GradleException("jps executable not found; ensure that you're running Gradle with the JDK rather than the JRE")
}
final File jps = Jvm.forHome(project.runtimeJavaHome).getExecutable('jps')
commandLine jps, '-l'
standardOutput = new ByteArrayOutputStream()
doLast {
Expand All @@ -914,10 +908,6 @@ class ClusterFormationTasks {
}
}

private static File getJpsExecutableByName(Project project, String jpsExecutableName) {
return Paths.get(project.runtimeJavaHome.toString(), "bin/" + jpsExecutableName).toFile()
}

/** Adds a task to kill an elasticsearch node with the given pidfile */
static Task configureStopTask(String name, Project project, Object depends, NodeInfo node) {
return project.tasks.create(name: name, type: LoggedExec, dependsOn: depends) {
Expand Down
Loading

0 comments on commit 624729e

Please sign in to comment.