Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeout parameter to bulk operation #1362

Merged
merged 5 commits into from
Oct 19, 2021
Merged

Add timeout parameter to bulk operation #1362

merged 5 commits into from
Oct 19, 2021

Conversation

probakowski
Copy link
Contributor

This change adds timeout parameter that can be used in bulk operation to define server-side timeout (for applying mappings updates, automatic index creation etc)

@probakowski probakowski added enhancement Improves the status quo :Track Management New operations, changes in the track format, track download changes and the like labels Oct 13, 2021
@probakowski probakowski self-assigned this Oct 13, 2021
@danielmitterdorfer danielmitterdorfer added this to the 2.3.1 milestone Oct 14, 2021
Copy link
Member

@danielmitterdorfer danielmitterdorfer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your PR! The change itself looks fine but can you please add a unit-test that the timeout parameter is used by the client (see BulkIndexRunnerTests)? We should also add user-level docs for that parameter and it would be good to link to the respective section of the Elasticsearch reference docs as well there.

@probakowski
Copy link
Contributor Author

@danielmitterdorfer I added test and docs

Copy link
Member

@danielmitterdorfer danielmitterdorfer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left one suggestion for the documentation but otherwise it looks good.

docs/track.rst Outdated
@@ -736,6 +736,7 @@ Properties
* ``on-conflict`` (optional, defaults to ``index``): Determines whether Rally should use the action ``index`` or ``update`` on id conflicts.
* ``recency`` (optional, defaults to 0): A number between [0,1] indicating whether to bias conflicting ids towards more recent ids (``recency`` towards 1) or whether to consider all ids for id conflicts (``recency`` towards 0). See the diagram below for details.
* ``detailed-results`` (optional, defaults to ``false``): Records more detailed meta-data for bulk requests. As it analyzes the corresponding bulk response in more detail, this might incur additional overhead which can skew measurement results. See the section below for the meta-data that are returned.
* ``timeout`` (optional, defaults to ``1m``): `Query parameter <https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#docs-bulk-api-query-params>`_ defining period each action waits for the following operations: Automatic index creation, Dynamic mapping updates, Waiting for active shards
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about slightly rewording this to:

defines the `time period that Elasticsearch will wait per action <https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#docs-bulk-api-query-params>`_ until it has finished processing the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's better, I've updated docs per your suggestion

Copy link
Member

@danielmitterdorfer danielmitterdorfer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for iterating. LGTM

@probakowski probakowski merged commit 6a16ec6 into master Oct 19, 2021
@probakowski probakowski deleted the daily-indices branch October 19, 2021 07:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improves the status quo :Track Management New operations, changes in the track format, track download changes and the like
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants