Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Log results of cluster health check #1150

Closed
danielmitterdorfer opened this issue Jan 13, 2021 · 0 comments · Fixed by #1203
Closed

Log results of cluster health check #1150

danielmitterdorfer opened this issue Jan 13, 2021 · 0 comments · Fixed by #1203
Labels
enhancement Improves the status quo good first issue Small, contained changes that are good for newcomers help wanted We'd be happy about a community contribution :Usability Makes Rally easier to use
Milestone
2.1.0

Comments

@danielmitterdorfer
Copy link
Member

Usually the cluster health operation is run in a loop until it reaches the expected status. This is problematic for cases when cluster health green is expected and a the benchmark creates an index with one replica on a single node cluster because the benchmark will hang indefinitely without giving an indication in the logs. In order to help users diagnose this case, we should add a log statement every time cluster health is checked, indicating the expected and the actual status.
@danielmitterdorfer danielmitterdorfer added enhancement Improves the status quo help wanted We'd be happy about a community contribution :Usability Makes Rally easier to use good first issue Small, contained changes that are good for newcomers labels Jan 13, 2021
@danielmitterdorfer danielmitterdorfer added this to the 2.x milestone Jan 13, 2021
@danielmitterdorfer danielmitterdorfer changed the title Log results of for cluster health check Log results of cluster health check Jan 13, 2021
danielmitterdorfer added a commit to danielmitterdorfer/rally that referenced this issue Mar 9, 2021
@danielmitterdorfer
Log results of cluster health check
b723fd6 
When the cluster health check fails e.g. due to invalid setup where we
check for a green cluster on a single node but indices have been
configured with replicas, the benchmark appears to hang indefinitely.
The reason is that the cluster health check is retried by default but
there is no indication in the logs.

With this commit we log the result of the cluster health check. We also
raise the log level of retry log messages so users can analyze the
behavior by inspecting logs.

Closes elastic#1150
@danielmitterdorfer danielmitterdorfer modified the milestones: 2.x, 2.1.0 Mar 9, 2021
@danielmitterdorfer danielmitterdorfer mentioned this issue Mar 9, 2021
Merged
danielmitterdorfer added a commit that referenced this issue Mar 9, 2021
@danielmitterdorfer
Log results of cluster health check (#1203)
5d93d2b 
When the cluster health check fails e.g. due to invalid setup where we
check for a green cluster on a single node but indices have been
configured with replicas, the benchmark appears to hang indefinitely.
The reason is that the cluster health check is retried by default but
there is no indication in the logs.

With this commit we log the result of the cluster health check. We also
raise the log level of retry log messages so users can analyze the
behavior by inspecting logs.

Closes #1150
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement Improves the status quo good first issue Small, contained changes that are good for newcomers help wanted We'd be happy about a community contribution :Usability Makes Rally easier to use
Projects
None yet
Milestone
2.1.0
Development

Successfully merging a pull request may close this issue.

Log results of cluster health check danielmitterdorfer/rally
1 participant
@danielmitterdorfer