-
Notifications
You must be signed in to change notification settings - Fork 313
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new wait-for-current-snapshots-create operation #1542
Conversation
This commit adds a new runner `WaitForCurrentSnapshotsCreate`, with the corresponding operation name `wait-for-current-snapshots-create` that waits until all current snapshots for a given repository have completed. Closes elastic#1537
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! LGTM. Left two nits.
esrally/driver/runner.py
Outdated
headers["Content-Type"] = "application/json" | ||
|
||
# significantly reduce response size when lots of snapshots have been taken | ||
# https://github.com/elastic/elasticsearch/pull/86269 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure the link to the pull request helps that much now that 8.3 is out? Maybe we can put in the TODO instead. So when the TODO gets removed, this gets removed too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reasoning behind putting the PR link here is to clarify that this feature (optionally excluding the large number of indices from the response) is only available from ES 8.3.0 onwards.
The TODO, later on, is about the support in elasticsearch-py since currently the index_names
parameter in the version that we use is not supported.
I have improved and moved the comment to the beginning of the conditional in e482d72 to make this clearer. The TODO remained as is.
esrally/driver/runner.py
Outdated
if int(response.get("total")) > 0: | ||
await asyncio.sleep(wait_period) | ||
continue | ||
break |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's a potential simplification:
if int(response.get("total")) > 0: | |
await asyncio.sleep(wait_period) | |
continue | |
break | |
if int(response.get("total")) == 0: | |
break | |
await asyncio.sleep(wait_period) |
Also this logic is untested, should we in tests first have a response with total > 0?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks added the simplification in 87b7040.
Also this logic is untested, should we in tests first have a response with total > 0?
Not sure I understood this comment, we do test with total > 0
e.g. in
rally/tests/driver/runner_test.py
Line 4029 in ac80c61
"total": 4, |
rally/tests/driver/runner_test.py
Line 4047 in ac80c61
assert es.snapshot.get.await_count == 2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, sorry I missed this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left a minor suggestion, but otherwise LGTM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Latest commits also LGTM. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! LGTM.
Thanks everyone for the great feedback! |
This commit adds a new runner
WaitForCurrentSnapshotsCreate
, with thecorresponding operation name
wait-for-current-snapshots-create
thatwaits until all current snapshots for a given repository have completed.
Closes #1537