Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable cluster and replication package debug logging for testSingleIndexShardAllocation test #11878

Merged
merged 1 commit into from
Jan 30, 2024
Merged

Enable cluster and replication package debug logging for testSingleIndexShardAllocation test #11878

merged 1 commit into from
Jan 30, 2024

Conversation

dreamer-89
Copy link
Member

@dreamer-89 dreamer-89 commented Jan 12, 2024 •
edited by kotwanikunal
Loading

Description

Coming from #11299, need to enable logs at cluster and replication level for debugging the cause of cluster heapth request timeout.

Related Issues

Related #11299

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 12, 2024 •
edited
Loading

Compatibility status:

Checks if related components are compatible with change ebaa216

Incompatible components

Incompatible components: [https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/sql.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security.git]

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 2686943: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for ebaa216: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

✅ Gradle check result for ebaa216: SUCCESS

@codecov Codecov
Copy link

codecov bot commented Jan 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (5c82ab8) 71.43% compared to head (ebaa216) 71.39%.
Report is 1 commits behind head on main.

Additional details and impacted files 
@@             Coverage Diff              @@
##               main   #11878      +/-   ##
============================================
- Coverage     71.43%   71.39%   -0.04%     
+ Complexity    59407    59385      -22     
============================================
  Files          4921     4921              
  Lines        278989   278989              
  Branches      40543    40543              
============================================
- Hits         199287   199176     -111     
- Misses        63086    63212     +126     
+ Partials      16616    16601      -15

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

deshsidd
deshsidd reviewed Jan 16, 2024
View reviewed changes
Copy link
Contributor

@deshsidd deshsidd left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a config to enable this in a particular cluster rather than globally enabling? Will this flood all the logs for all clusters? (Resolved)

@deshsidd
Copy link
Contributor

Looks like a build is failing please fix

@dreamer-89
Copy link
Member Author

Can we have a config to enable this in a particular cluster rather than globally enabling? Will this flood all the logs for all clusters?

Thanks @deshsidd for the review. The added TestLogging annotation should only impact the test which is annotated.

@deshsidd
Copy link
Contributor

@msfroh @peternied @dblock Maintainers approval required here

peternied
peternied previously approved these changes Jan 16, 2024
View reviewed changes
@peternied peternied dismissed their stale review January 16, 2024 23:52

Didn't see the result in the logs

peternied
peternied reviewed Jan 16, 2024
View reviewed changes
Copy link
Member

@peternied peternied left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dreamer-89 Thanks for making this change - can you help me see the new log messages that have been logged with this annotation? I don't see them gradle check log associated with the test, [link]

@dreamer-89
Copy link
Member Author

@dreamer-89 Thanks for making this change - can you help me see the new log messages that have been logged with this annotation? I don't see them gradle check log associated with the test, [link]

Thanks @peternied for the review. I think the log response was truncated in this case(not sure why). Though post failure, the test logs are added to gradle check console output.

[2024-01-13T01:49:02,920][INFO ][o.o.i.r.SegmentReplicationAllocationIT] [testAllocationWithDisruption] before test
[2024-01-13T01:49:02,923][INFO ][o.o.i.r.SegmentReplicationAllocationIT] [testAllocationWithDisruption] [SegmentReplicationAllocationIT#testAllocationWithDisruption]: setting up test
[2024-01-13T01:49:02,930][INFO ][o.o.t.InternalTestCluster] [testAllocationWithDisruption] Setup InternalTestCluster [TEST-TEST_WORKER_VM=[662]-CLUSTER_SEED=[5564473573564607375]-HASH=[1FA7A3A0CE9]-clu
...[truncated 2582630 chars]...
3T01:50:06,762][INFO ][o.o.n.Node               ] [testPerIndexPrimaryAllocation] closing ...
[2024-01-13T01:50:06,769][INFO ][o.o.n.Node               ] [testPerIndexPrimaryAllocation] closed
[2024-01-13T01:50:06,769][INFO ][o.o.i.r.SegmentReplicationAllocationIT] [testPerIndexPrimaryAllocation] [SegmentReplicationAllocationIT#testPerIndexPrimaryAllocation]: cleaned up after test
[2024-01-13T01:50:06,769][INFO ][o.o.i.r.SegmentReplicationAllocationIT] [testPerIndexPrimaryAllocation] after test

@dreamer-89 dreamer-89 requested a review from peternied January 30, 2024 19:42
@dreamer-89
Copy link
Member Author

@peternied @kotwanikunal : Request for review.

@dreamer-89 dreamer-89 merged commit f9ab801 into opensearch-project:main Jan 30, 2024
38 of 46 checks passed
@dreamer-89 dreamer-89 added the backport 2.x Backport to 2.x branch label Jan 30, 2024
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 30, 2024
@github-actions
Enable cluster and replication package debug logging for testSingleIn…
1919aff 
…dexShardAllocation test (#11878)

Signed-off-by: Suraj Singh <[email protected]>
(cherry picked from commit f9ab801)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Jan 30, 2024
Merged
dreamer-89 pushed a commit that referenced this pull request Jan 30, 2024
@opensearch-trigger-bot @github-actions
Enable cluster and replication package debug logging for testSingleIn…
810ceb6 
…dexShardAllocation test (#11878) (#12091)

(cherry picked from commit f9ab801)

Signed-off-by: Suraj Singh <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
peteralfonsi pushed a commit to peteralfonsi/OpenSearch that referenced this pull request Mar 1, 2024
@dreamer-89
Enable cluster and replication package debug logging for testSingleIn…
3a126a9 
…dexShardAllocation test (opensearch-project#11878)

Signed-off-by: Suraj Singh <[email protected]>
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
@dreamer-89 @rayshrey
Enable cluster and replication package debug logging for testSingleIn…
994f6e7 
…dexShardAllocation test (opensearch-project#11878)

Signed-off-by: Suraj Singh <[email protected]>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
@dreamer-89 @shiv0408
Enable cluster and replication package debug logging for testSingleIn…
571c0b7 
…dexShardAllocation test (opensearch-project#11878)

Signed-off-by: Suraj Singh <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Poojita-Raj Poojita-Raj Poojita-Raj approved these changes

@kotwanikunal kotwanikunal kotwanikunal approved these changes

@deshsidd deshsidd deshsidd approved these changes

@abbashus abbashus Awaiting requested review from abbashus

@adnapibar adnapibar Awaiting requested review from adnapibar

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@andrross andrross Awaiting requested review from andrross andrross is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dblock dblock Awaiting requested review from dblock dblock is a code owner

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@kartg kartg Awaiting requested review from kartg

@mch2 mch2 Awaiting requested review from mch2 mch2 is a code owner

@msfroh msfroh Awaiting requested review from msfroh msfroh is a code owner

@nknize nknize Awaiting requested review from nknize nknize is a code owner

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@reta reta Awaiting requested review from reta reta is a code owner

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@ryanbogan ryanbogan Awaiting requested review from ryanbogan

@sachinpkale sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@shwetathareja shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

@tlfeng tlfeng Awaiting requested review from tlfeng

@VachaShah VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

@peternied peternied Awaiting requested review from peternied

Assignees
No one assigned
Labels
backport 2.x Backport to 2.x branch skip-changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@dreamer-89 @deshsidd @peternied @kotwanikunal @Poojita-Raj