Skip to content

Commit

Permalink
Consolidate and improve track/team repo revision logic docs (#1265)
Browse files Browse the repository at this point in the history
Currently the repository version matching logic for Rally tracks/teams
is duplicated in Rally docs as well as README files in in our standard
tracks/teams repositories.

Update docs consolidating existing documentation from
elastic/rally-tracks and elastic/rally-teams

Relates:

- elastic/rally-tracks#162
- elastic/rally-teams#62
  • Loading branch information
dliappis authored May 12, 2021
1 parent 9608870 commit c1bba05
Show file tree
Hide file tree
Showing 3 changed files with 26 additions and 12 deletions.
2 changes: 1 addition & 1 deletion docs/car.rst
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ Rally provides a default team repository that is hosted on `Github <https://gith
* The `master` branch needs to work with the latest `master` branch of Elasticsearch.
* All other branches need to match the version scheme of Elasticsearch, i.e. ``MAJOR.MINOR.PATCH-SUFFIX`` where all parts except ``MAJOR`` are optional.

Rally implements a fallback logic similar to the one used for :ref:`track-repositories <track-repositories-fall-back-logic>`.
Rally implements a branch matching logic similar to the one used for :ref:`track-repositories <track-repositories-branch-logic>`.

Creating a new team repository
""""""""""""""""""""""""""""""
Expand Down
2 changes: 1 addition & 1 deletion docs/elasticsearch_plugins.rst
Original file line number Diff line number Diff line change
Expand Up @@ -228,4 +228,4 @@ Now you can run benchmarks with the custom Elasticsearch plugin, e.g. with ``esr
For this to work you need ensure two things:

1. The plugin needs to be available for the version that you want to benchmark (7.12.0 in the example above).
2. Rally will choose the most appropriate branch in the team repository before starting the benchmark. See the documentation on :ref:`how branches are mapped to Elasticsearch versions <track-repositories-fall-back-logic>`.
2. Rally will choose the most appropriate branch in the team repository before starting the benchmark. See the documentation on :ref:`how branches are mapped to Elasticsearch versions <track-repositories-branch-logic>`.
34 changes: 24 additions & 10 deletions docs/track.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,23 +35,37 @@ Custom Track Repositories

Alternatively, you can store Rally tracks also in a dedicated git repository which we call a "track repository". Rally provides a default track repository that is hosted on `Github <https://github.com/elastic/rally-tracks>`_. You can also add your own track repositories although this requires a bit of additional work. First of all, track repositories need to be managed by git. The reason is that Rally can benchmark multiple versions of Elasticsearch and we use git branches in the track repository to determine the best match for each track (based on the command line parameter ``--distribution-version``). The versioning scheme is as follows:

* The `master` branch needs to work with the latest `master` branch of Elasticsearch.
* The ``master`` branch needs to work with the latest ``master`` branch of Elasticsearch.
* All other branches need to match the version scheme of Elasticsearch, i.e. ``MAJOR.MINOR.PATCH-SUFFIX`` where all parts except ``MAJOR`` are optional.

.. _track-repositories-fall-back-logic:
.. _track-repositories-branch-logic:

Rally implements a fallback logic in order of specificity up to the minor version level, so you don't need to define a branch for each patch release of Elasticsearch.
When a track repository has several branches, Rally will pick the most appropriate branch, depending on the Elasticsearch version to be benchmarked, using a match logic in the following order:

Assuming the track repository has several branches, the order is:
#. *Exact match major.minor.patch-SUFFIX* (e.g. ``7.0.0-beta1``)
#. *Exact match major.minor.patch* (e.g. ``7.10.2``, ``6.7.0``)
#. *Exact match major.minor* (e.g. ``7.10``)
#. *Nearest prior minor branch*

1. Exact branch matches; e.g. if the repo contains branches `7`, `7.1` and `7.10.2` and Elasticsearch version is `7.10.2`, `7.10.2` will be checked out.
2. Nearest prior minor matches; e.g. if the repo contains branches `7`, `7.1` and `7.10` and Elasticsearch version is `7.10.2`, `7.10` will be checked out. Alternatively if version is `7.9`, `7.1` will be checked out.
3. Major branch matches; e.g. if the repo contains branches `7` and `7.10` and Elasticsearch version is `7.1`, `7` will be checked out.
4. Failing everything, `master` will be elected, e.g. if the repo contains branches `6`, `7` and `master` and Elasticsearch version is `8.1.0`, `master` will be checked out.
e.g. if available branches are ``master``, ``7``, ``7.2`` and ``7.11`` attempting to benchmarking ES ``7.10.2`` will pick ``7.2``, whereas benchmarking ES ``7.12.1`` will pick branch ``7.11``
#. *Nearest major branch*

In general, Rally tries to use the branch with the best match to the benchmarked version of Elasticsearch.
e.g. if available branches are ``master``, ``5``, ``6`` and ``7``, benchmarking ES ``7.11.0`` will pick branch ``7``

Rally will also search for related files like mappings or custom runners or parameter sources in the track repository. However, Rally will use a separate directory to look for data files (``~/.rally/benchmarks/data/$TRACK_NAME/``). The reason is simply that we do not want to check multi-GB data files into git.
The following table explains in more detail the ES version compatibility scheme using a scenario where a track repository contains four branches ``master``, ``7.0.0-beta1``, ``7.3`` and ``6``:

.. tabularcolumns:: |l|l|

================ ===========================================================================================================
track branch ES version compatibility
================ ===========================================================================================================
``master`` compatible only with the latest development version of Elasticsearch
``7.0.0-beta1`` compatible only with the released version ``7.0.0-beta1``
``7.3`` compatible with all ES ``7`` releases equal or greater than ``7.3`` (e.g. ``7.3.0``, ``7.10.2``)
``6`` compatible with all Elasticsearch releases with the major release number ``6`` (e.g. ``6.4.0``, ``6.8.13``)
================ ===========================================================================================================

Rally will also search for related files like mappings or custom runners or parameter sources in the track repository. However, Rally will use a separate directory to look for data files (``~/.rally/benchmarks/data/$TRACK_NAME/``). The reason is simply that we do not want to check-in multi-GB data files into git.

Creating a new track repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down

0 comments on commit c1bba05

Please sign in to comment.