From 13a93c092e35e9372a6feaf4a186b2170a690890 Mon Sep 17 00:00:00 2001 From: Dimitrios Liappis Date: Wed, 12 May 2021 12:53:52 +0300 Subject: [PATCH 1/2] Consolidate and improve track/team repo revision logic docs Currently the repository version matching logic for Rally tracks/teams is duplicated in Rally docs as well as README files in in our standard tracks/teams repositories. Update docs consolidating existing documentation from elastic/rally-tracks and elastic/rally-teams Relates: - https://github.com/elastic/rally-tracks/issues/162 - https://github.com/elastic/rally-teams/issues/62 --- docs/car.rst | 2 +- docs/elasticsearch_plugins.rst | 2 +- docs/track.rst | 34 ++++++++++++++++++++++++---------- 3 files changed, 26 insertions(+), 12 deletions(-) diff --git a/docs/car.rst b/docs/car.rst index 7cc3a4d9c..243f181f5 100644 --- a/docs/car.rst +++ b/docs/car.rst @@ -121,7 +121,7 @@ Rally provides a default team repository that is hosted on `Github `. +Rally implements a branch matching logic similar to the one used for :ref:`track-repositories `. Creating a new team repository """""""""""""""""""""""""""""" diff --git a/docs/elasticsearch_plugins.rst b/docs/elasticsearch_plugins.rst index 871caf3ed..c4ecfddc5 100644 --- a/docs/elasticsearch_plugins.rst +++ b/docs/elasticsearch_plugins.rst @@ -228,4 +228,4 @@ Now you can run benchmarks with the custom Elasticsearch plugin, e.g. with ``esr For this to work you need ensure two things: 1. The plugin needs to be available for the version that you want to benchmark (7.12.0 in the example above). -2. Rally will choose the most appropriate branch in the team repository before starting the benchmark. See the documentation on :ref:`how branches are mapped to Elasticsearch versions `. +2. Rally will choose the most appropriate branch in the team repository before starting the benchmark. See the documentation on :ref:`how branches are mapped to Elasticsearch versions `. diff --git a/docs/track.rst b/docs/track.rst index 28c02bf52..2ff16702b 100644 --- a/docs/track.rst +++ b/docs/track.rst @@ -35,23 +35,37 @@ Custom Track Repositories Alternatively, you can store Rally tracks also in a dedicated git repository which we call a "track repository". Rally provides a default track repository that is hosted on `Github `_. You can also add your own track repositories although this requires a bit of additional work. First of all, track repositories need to be managed by git. The reason is that Rally can benchmark multiple versions of Elasticsearch and we use git branches in the track repository to determine the best match for each track (based on the command line parameter ``--distribution-version``). The versioning scheme is as follows: -* The `master` branch needs to work with the latest `master` branch of Elasticsearch. +* The ``master`` branch needs to work with the latest ``master`` branch of Elasticsearch. * All other branches need to match the version scheme of Elasticsearch, i.e. ``MAJOR.MINOR.PATCH-SUFFIX`` where all parts except ``MAJOR`` are optional. -.. _track-repositories-fall-back-logic: +.. _track-repositories-branch-logic: -Rally implements a fallback logic in order of specificity up to the minor version level, so you don't need to define a branch for each patch release of Elasticsearch. +When a track repository has several branches, Rally will pick the most appropriate branch, depending on the Elasticsearch version to be benchmarked, using a match logic in the following order: -Assuming the track repository has several branches, the order is: +#. *Exact match major.minor.patch-extension_label* (e.g. ``7.0.0-beta1``) +#. *Exact match major.minor.patch* (e.g. ``7.10.2``, ``6.7.0``) +#. *Exact match major.minor* (e.g. ``7.10``) +#. *Nearest prior minor branch* -1. Exact branch matches; e.g. if the repo contains branches `7`, `7.1` and `7.10.2` and Elasticsearch version is `7.10.2`, `7.10.2` will be checked out. -2. Nearest prior minor matches; e.g. if the repo contains branches `7`, `7.1` and `7.10` and Elasticsearch version is `7.10.2`, `7.10` will be checked out. Alternatively if version is `7.9`, `7.1` will be checked out. -3. Major branch matches; e.g. if the repo contains branches `7` and `7.10` and Elasticsearch version is `7.1`, `7` will be checked out. -4. Failing everything, `master` will be elected, e.g. if the repo contains branches `6`, `7` and `master` and Elasticsearch version is `8.1.0`, `master` will be checked out. + e.g. if available branches are ``master``, ``7``, ``7.2`` and ``7.11`` attempting to benchmarking ES ``7.10.2`` will pick ``7.2``, whereas benchmarking ES ``7.12.1`` will pick branch ``7.11`` +#. *Nearest major branch* -In general, Rally tries to use the branch with the best match to the benchmarked version of Elasticsearch. + e.g. if available branches are ``master``, ``5``, ``6`` and ``7``, benchmarking ES ``7.11.0`` will pick branch ``7`` -Rally will also search for related files like mappings or custom runners or parameter sources in the track repository. However, Rally will use a separate directory to look for data files (``~/.rally/benchmarks/data/$TRACK_NAME/``). The reason is simply that we do not want to check multi-GB data files into git. +The following table explains in more detail the ES version compatibility scheme using a scenario where a track repository contains four branches ``master``, ``7.0.0-beta1``, ``7.3`` and ``6``: + +.. tabularcolumns:: |l|l| + +================ =========================================================================================================== +track branch ES version compatibility +================ =========================================================================================================== +``master`` compatible only with the latest development version of Elasticsearch +``7.0.0-beta1`` compatible only with the released version ``7.0.0-beta1`` +``7.3`` compatible with all ES ``7`` releases equal or greater than ``7.3`` (e.g. ``7.3.0``, ``7.10.2``) +``6`` compatible with all Elasticsearch releases with the major release number ``6`` (e.g. ``6.4.0``, ``6.8.13``) +================ =========================================================================================================== + +Rally will also search for related files like mappings or custom runners or parameter sources in the track repository. However, Rally will use a separate directory to look for data files (``~/.rally/benchmarks/data/$TRACK_NAME/``). The reason is simply that we do not want to check-in multi-GB data files into git. Creating a new track repository ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 3a4ca085b156c804ba042fac6583f2c8c2c98571 Mon Sep 17 00:00:00 2001 From: Dimitrios Liappis Date: Wed, 12 May 2021 14:18:30 +0300 Subject: [PATCH 2/2] Address comment. --- docs/track.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/track.rst b/docs/track.rst index 2ff16702b..970da2246 100644 --- a/docs/track.rst +++ b/docs/track.rst @@ -42,7 +42,7 @@ Alternatively, you can store Rally tracks also in a dedicated git repository whi When a track repository has several branches, Rally will pick the most appropriate branch, depending on the Elasticsearch version to be benchmarked, using a match logic in the following order: -#. *Exact match major.minor.patch-extension_label* (e.g. ``7.0.0-beta1``) +#. *Exact match major.minor.patch-SUFFIX* (e.g. ``7.0.0-beta1``) #. *Exact match major.minor.patch* (e.g. ``7.10.2``, ``6.7.0``) #. *Exact match major.minor* (e.g. ``7.10``) #. *Nearest prior minor branch*