Derive JDK version at runtime

With this commit we derive the appropriate JDK version at benchmark
runtime instead of requiring that it is preconfigured. This requires two
new variables in the rally-teams repo: `build.jdk` and `runtime.jdk` to
resolve the appropriate (major) JDK version depending on the
Elasticsearch version.

When we know the required JDK version, we check the environment
variables `JAVAx_HOME` (where `x` is the JDK version) and `JAVA_HOME` to
find the correct JAVA_HOME path. We also allow the user to override the
runtime JDK with a new command line parameter `--runtime-jdk` as Rally
will default to the highest available and supported JDK version on the
target system.

Closes #452
Relates #518

docs/command_line_reference.rst

@@ -284,6 +284,20 @@ Example::
 
 This enables the Java flight recorder telemetry device and sets the ``recording-template`` parameter to "profile".
 
+``runtime-jdk``
+~~~~~~~~~~~~~~~
+
+By default, Rally will derive the appropriate runtime JDK versions automatically per version of Elasticsearch. For example, it will choose JDK 8 or 7 for Elasticsearch 2.x but only JDK 8 for Elasticsearch 5.0.0. It will choose the highest available version.
+
+This command line parameter sets the major version of the JDK that Rally should use to run Elasticsearch. It is required that either ``JAVA_HOME`` or ``JAVAx_HOME`` (where ``x`` is the major version, e.g. ``JAVA8_HOME`` for a JDK 8) points to the appropriate JDK.
+
+Example::
+
+   # Run a benchmark with defaults (i.e. JDK 8)
+   esrally --distribution-version=2.4.0
+   # Force to run with JDK 7
+   esrally --distribution-version=2.4.0 --runtime-jdk=7
+
 .. _clr_revision:
 
 ``revision``

docs/configuration.rst

@@ -15,7 +15,6 @@ Rally can build Elasticsearch either from sources or use an `official binary dis
 Let's go through an example step by step: First run ``esrally``::
 
     dm@io:~ $ esrally
-
         ____        ____
        / __ \____ _/ / /_  __
       / /_/ / __ `/ / / / / /
@@ -27,42 +26,11 @@ Let's go through an example step by step: First run ``esrally``::
 
       esrally configure --advanced-config
 
-    * Autodetecting available third-party software
-      git    : [OK]
-      JDK    : [OK]
-
-    * Setting up benchmark data directory in /Users/dm/.rally/benchmarks
-    Enter the JDK 10 root directory (Press Enter to skip):
-
-As you can see above, Rally autodetects if git and a JDK are installed. It also searches for a JDK 10; if you don't have it, that's no problem, you are just not able to build Elasticsearch from sources. Let's assume you press Enter and don't specify a path for JDK 10::
-
-    ********************************************************************************
-    You don't have a valid JDK 10 installation and cannot benchmark source builds.
-
-    You can still benchmark binary distributions with e.g.:
-
-      esrally --distribution-version=6.0.0
-    ********************************************************************************
-
-As you can see, Rally tells you that you cannot build Elasticsearch from sources but you can still benchmark official binary distributions.
-
-It's also possible that Rally cannot automatically your JDK home directory. In that case, it will ask you later in the configuration process. If you do not provide a JDK home directory, Rally cannot start Elasticsearch on this machine but you can still use it as a load generator to :doc:`benchmark remote clusters </recipes>`.
-
-If you specify a valid path for JDK 10, Rally will try to autodetect your Elasticsearch project directory (either in the current directory or in ``../elasticsearch``) or will choose a default directory::
-
     * Setting up benchmark data directory in /Users/dm/.rally/benchmarks
     * Setting up benchmark source directory in /Users/dm/.rally/benchmarks/src/elasticsearch
 
-If a valid path for JDK 10 was not found (or entered), it will not ask you for a source directory and just go on.
-
-Now Rally is done::
-
     Configuration successfully written to /Users/dm/.rally/rally.ini. Happy benchmarking!
 
-    To benchmark Elasticsearch 6.0.0 with the default benchmark, run:
-
-      esrally --distribution-version=6.0.0
-
     More info about Rally:
 
     * Type esrally --help
@@ -102,7 +70,6 @@ Rally will ask you a few more things in the advanced setup:
 
 * **Benchmark data directory**: Rally stores all benchmark related data in this directory which can take up to several tens of GB. If you want to use a dedicated partition, you can specify a different data directory here.
 * **Elasticsearch project directory**: This is the directory where the Elasticsearch sources are located. If you don't actively develop on Elasticsearch you can just leave the default but if you want to benchmark local changes you should point Rally to your project directory. Note that Rally will run builds with the Gradle Wrapper in this directory (it runs ``./gradlew clean`` and ``./gradlew :distribution:tar:assemble``).
-* **JDK root directory**: Rally will only ask this if it could not autodetect the JDK home by itself. Just enter the root directory of the JDK you want to use. By default, Rally will choose Java 8 if available and fallback to Java 10.
 * **Metrics store type**: You can choose between ``in-memory`` which requires no additional setup or ``elasticsearch`` which requires that you start a dedicated Elasticsearch instance to store metrics but gives you much more flexibility to analyse results.
 * **Metrics store settings** (only for metrics store type ``elasticsearch``): Provide the connection details to the Elasticsearch metrics store. This should be an instance that you use just for Rally but it can be a rather small one. A single node cluster with default setting should do it. When using self-signed certificates on the Elasticsearch metrics store, certificate verification can be turned off by setting the ``datastore.ssl.verification_mode`` setting to ``none``. Alternatively you can enter the path to the certificate authority's signing certificate in ``datastore.ssl.certificate_authorities``. Both settings are optional.
 * **Name for this benchmark environment** (only for metrics store type ``elasticsearch``): You can use the same metrics store for multiple environments (e.g. local, continuous integration etc.) so you can separate metrics from different environments by choosing a different name.

docs/install.rst

@@ -101,10 +101,12 @@ In all other cases, Rally requires ``git 1.9`` or better. Verify with ``git --ve
 JDK
 ~~~
 
-A JDK is required on all machines where you want to launch Elasticsearch. If you use Rally just as a load generator, no JDK is required.
+A JDK is required on all machines where you want to launch Elasticsearch. If you use Rally just as a load generator to :doc:`benchmark remote clusters </recipes>`, no JDK is required.
 
 We recommend to use Oracle JDK but you are free to use OpenJDK as well. For details on how to install a JDK check your operating system's documentation pages.
 
+To find the JDK, Rally expects the environment variable ``JAVA_HOME`` to be set on all targeted machines. To have more specific control, for example when you want to benchmark across a wide range of Elasticsearch releases, you can also set ``JAVAx_HOME`` where ``x``  is the major version of a JDK (e.g. ``JAVA8_HOME`` would point to a JDK 8 installation). Rally will then choose the highest supported JDK per version of Elasticsearch that is available.
+
 
 .. note::
 

docs/migrate.rst

@@ -1,6 +1,23 @@
 Migration Guide
 ===============
 
+Migrating to Rally 1.0.0
+------------------------
+
+Handling of JDK versions
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+Previously the path to the JDK needed to be configured in Rally's configuration file (``~/.rally/rally.ini``) but this is too inflexible given the increased JDK release cadence. In order to keep up, we define now the allowed runtime JDKs in `rally-teams <https://github.com/elastic/rally-teams/blob/master/cars/v1/vanilla/config.ini>`_ per Elasticsearch version.
+
+To resolve the path to the appropriate JDK you need to define the environment variable ``JAVA_HOME`` on each targeted machine.
+
+You can also set version-specific environment variables, e.g. ``JAVA7_HOME``, ``JAVA8_HOME`` or ``JAVA10_HOME`` which will take precedence over ``JAVA_HOME``.
+
+.. note::
+
+    Rally will choose the highest appropriate JDK per Elasticsearch version. You can use ``--runtime-jdk`` to force a specific JDK version but the path will still be resolved according to the logic above.
+
+
 Migrating to Rally 0.11.0
 -------------------------
 

docs/quickstart.rst

@@ -4,7 +4,7 @@ Quickstart
 Install
 -------
 
-Install Python 3.4+ including ``pip3``, git 1.9+ and at least JDK 8. If you want to benchmark source builds of Elasticsearch you also need JDK 10. Then run the following command, optionally prefixed by ``sudo`` if necessary::
+Install Python 3.4+ including ``pip3``, git 1.9+ and an `appropriate JDK to run Elasticsearch <https://www.elastic.co/support/matrix#matrix_jvm>`_ Be sure that ``JAVA_HOME`` points to that JDK. Then run the following command, optionally prefixed by ``sudo`` if necessary::
 
     pip3 install esrally
 

esrally/config.py

@@ -106,7 +106,7 @@ def auto_load_local_config(base_config, additional_sections=None, config_file_cl
 class Config:
     EARLIEST_SUPPORTED_VERSION = 12
 
-    CURRENT_CONFIG_VERSION = 16
+    CURRENT_CONFIG_VERSION = 17
 
     """
     Config is the main entry point to retrieve and set benchmark properties. It provides multiple scopes to allow overriding of values on
@@ -270,7 +270,7 @@ def __init__(self, i=input, sec_i=getpass.getpass, o=console.println):
         self.prompter = None
         self.logger = logging.getLogger(__name__)
 
-    def create_config(self, config_file, advanced_config=False, assume_defaults=False, java_home=None, runtime_java_home=None):
+    def create_config(self, config_file, advanced_config=False, assume_defaults=False):
         """
         Either creates a new configuration file or overwrites an existing one. Will ask the user for input on configurable properties
         and writes them to the configuration file in ~/.rally/rally.ini.
@@ -280,7 +280,6 @@ def create_config(self, config_file, advanced_config=False, assume_defaults=Fals
         :param assume_defaults: If True, assume the user accepted all values for which defaults are provided. Mainly intended for automatic
         configuration in CI run. Default: False.
         """
-        benchmark_from_sources = True
         self.prompter = Prompter(self.i, self.sec_i, self.o, assume_defaults)
 
         if advanced_config:
@@ -300,48 +299,12 @@ def create_config(self, config_file, advanced_config=False, assume_defaults=Fals
         else:
             self.logger.debug("Did not detect a configuration file at [%s]. Running initial configuration routine.", config_file.location)
 
-        # Autodetect settings
-        self.o("* Autodetecting available third-party software")
-        git_path = io.guess_install_location("git")
-
-        java_8_home = runtime_java_home if runtime_java_home else io.guess_java_home(major_version=8)
-        java_10_home = java_home if java_home else io.guess_java_home(major_version=10)
-        from esrally.utils import jvm
-        if java_8_home:
-            auto_detected_java_home = java_8_home
-        # Don't auto-detect an EA release and bring trouble to the user later on. They can still configure it manually if they want to.
-        elif java_10_home and not jvm.is_early_access_release(java_10_home):
-            auto_detected_java_home = java_10_home
-        else:
-            auto_detected_java_home = None
-
-        self.print_detection_result("git    ", git_path)
-        self.print_detection_result("JDK    ", auto_detected_java_home,
-                                    warn_if_missing=True,
-                                    additional_message="You cannot benchmark Elasticsearch on this machine without a JDK.")
-        self.o("")
-
         root_dir = io.normalize_path(os.path.abspath(os.path.join(config_file.config_dir, "benchmarks")))
         if advanced_config:
             root_dir = io.normalize_path(self._ask_property("Enter the benchmark data directory", default_value=root_dir))
         else:
             self.o("* Setting up benchmark data directory in %s" % root_dir)
 
-        if not java_10_home or jvm.is_early_access_release(java_10_home):
-            raw_java_10_home = self._ask_property("Enter the JDK 10 root directory", check_path_exists=True, mandatory=False)
-            if raw_java_10_home and jvm.major_version(raw_java_10_home) == 10 and not jvm.is_early_access_release(raw_java_10_home):
-                java_10_home = io.normalize_path(raw_java_10_home) if raw_java_10_home else None
-            else:
-                benchmark_from_sources = False
-                self.o("********************************************************************************")
-                self.o("You don't have a valid JDK 10 installation and cannot benchmark source builds.")
-                self.o("")
-                self.o("You can still benchmark binary distributions with e.g.:")
-                self.o("")
-                self.o("  %s --distribution-version=6.0.0" % PROGRAM_NAME)
-                self.o("********************************************************************************")
-                self.o("")
-
         # We try to autodetect an existing ES source directory
         guess = self._guess_es_src_dir()
         if guess:
@@ -362,29 +325,6 @@ def create_config(self, config_file, advanced_config=False, assume_defaults=Fals
         # Not everybody might have SSH access. Play safe with the default. It may be slower but this will work for everybody.
         repo_url = "https://github.com/elastic/elasticsearch.git"
 
-        if auto_detected_java_home:
-            java_home = auto_detected_java_home
-            local_benchmarks = True
-        else:
-            raw_java_home = self._ask_property("Enter the JDK root directory (version 8 or later)", check_path_exists=True, mandatory=False)
-            java_home = io.normalize_path(raw_java_home) if raw_java_home else None
-            if not java_home:
-                local_benchmarks = False
-                self.o("")
-                self.o("********************************************************************************")
-                self.o("You don't have a JDK installed but Elasticsearch requires one to run. This means")
-                self.o("that you cannot benchmark Elasticsearch on this machine.")
-                self.o("")
-                self.o("You can still benchmark against remote machines e.g.:")
-                self.o("")
-                self.o("  %s --pipeline=benchmark-only --target-host=\"NODE_IP:9200\"" % PROGRAM_NAME)
-                self.o("")
-                self.o("See %s for further info." % console.format.link("%srecipes.html" % DOC_LINK))
-                self.o("********************************************************************************")
-                self.o("")
-            else:
-                local_benchmarks = True
-
         if advanced_config:
             data_store_choice = self._ask_property("Where should metrics be kept?"
                                                    "\n\n"
@@ -429,12 +369,6 @@ def create_config(self, config_file, advanced_config=False, assume_defaults=Fals
         # the Elasticsearch directory is just the last path component (relative to the source root directory)
         config["source"]["elasticsearch.src.subdir"] = io.basename(source_dir)
 
-        config["runtime"] = {}
-        if java_home:
-            config["runtime"]["java.home"] = java_home
-        if java_10_home:
-            config["runtime"]["java10.home"] = java_10_home
-
         config["benchmarks"] = {}
         config["benchmarks"]["local.dataset.cache"] = "${node:root.dir}/data"
 
@@ -462,20 +396,6 @@ def create_config(self, config_file, advanced_config=False, assume_defaults=Fals
 
         self.o("Configuration successfully written to %s. Happy benchmarking!" % config_file.location)
         self.o("")
-        if local_benchmarks and benchmark_from_sources:
-            self.o("To benchmark Elasticsearch with the default benchmark, run:")
-            self.o("")
-            self.o("  %s" % PROGRAM_NAME)
-            self.o("")
-        elif local_benchmarks:
-            self.o("To benchmark Elasticsearch 6.0.0 with the default benchmark, run:")
-            self.o("")
-            self.o("  %s --distribution-version=6.0.0" % PROGRAM_NAME)
-            self.o("")
-        else:
-            # we've already printed an info for the user. No need to repeat that.
-            pass
-
         self.o("More info about Rally:")
         self.o("")
         self.o("* Type %s --help" % PROGRAM_NAME)
@@ -712,6 +632,11 @@ def warn_if_plugin_build_task_is_in_use(config):
         current_version = 16
         config["meta"]["config.version"] = str(current_version)
 
+    if current_version == 16 and target_version > current_version:
+        config.pop("runtime", None)
+        current_version = 17
+        config["meta"]["config.version"] = str(current_version)
+
     # all migrations done
     config_file.store(config)
     logger.info("Successfully self-upgraded configuration to version [%s]", target_version)