Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] CrossClusterIT testCancel failing #108061

Open
andreidan opened this issue Apr 30, 2024 · 8 comments · Fixed by #117750
Open

[CI] CrossClusterIT testCancel failing #108061

andreidan opened this issue Apr 30, 2024 · 8 comments · Fixed by #117750
Assignees
@quux00 @drempapis
Labels
low-risk An open issue or test failure that is a low risk to future releases :Search Foundations/CCS Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch >test-failure Triaged test failures from CI

Comments

@andreidan
Copy link
Contributor

Build scan:
https://gradle-enterprise.elastic.co/s/upvcurkqfrxs4/tests/:server:internalClusterTest/org.elasticsearch.search.ccs.CrossClusterIT/testCancel

Reproduction line:

./gradlew ':server:internalClusterTest' --tests "org.elasticsearch.search.ccs.CrossClusterIT.testCancel" -Dtests.seed=FC7B5561729BC147 -Dtests.locale=en -Dtests.timezone=Europe/London -Druntime.java=21

Applicable branches:
main

Reproduces locally?:
Didn't try

Failure history:
Failure dashboard for org.elasticsearch.search.ccs.CrossClusterIT#testCancel

Failure excerpt:

junit.framework.AssertionFailedError: Expected exception RuntimeException but no exception was thrown

  at __randomizedtesting.SeedInfo.seed([FC7B5561729BC147:9FE0A56254F7AAD2]:0)
  at org.apache.lucene.tests.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2882)
  at org.apache.lucene.tests.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2868)
  at org.elasticsearch.search.ccs.CrossClusterIT.testCancel(CrossClusterIT.java:308)
  at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
  at java.lang.reflect.Method.invoke(Method.java:580)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  at java.lang.Thread.run(Thread.java:1583)
@andreidan andreidan added :Search/Search Search-related issues that do not fall into other categories >test-failure Triaged test failures from CI labels Apr 30, 2024
@elasticsearchmachine elasticsearchmachine added Team:Search Meta label for search team needs:risk Requires assignment of a risk label (low, medium, blocker) labels Apr 30, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

DaveCTurner added a commit that referenced this issue Apr 30, 2024
@DaveCTurner
AwaitsFix for #108061
fd6c267
@quux00 quux00 added low-risk An open issue or test failure that is a low risk to future releases and removed needs:risk Requires assignment of a risk label (low, medium, blocker) labels Apr 30, 2024
@javanna javanna removed the :Search/Search Search-related issues that do not fall into other categories label Jul 17, 2024
@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label and removed Team:Search Meta label for search team labels Jul 17, 2024
@javanna javanna added the :Search Foundations/Search Catch all for Search Foundations label Jul 17, 2024
@elasticsearchmachine elasticsearchmachine added Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch and removed needs:triage Requires assignment of a team area label labels Jul 17, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-foundations (Team:Search Foundations)

@javanna javanna added :Search Foundations/CCS and removed :Search Foundations/Search Catch all for Search Foundations labels Jul 17, 2024
@elasticsearchmachine elasticsearchmachine closed this as not planned Won't fix, can't repro, duplicate, stale Nov 5, 2024
@elasticsearchmachine
Copy link
Collaborator

This issue has been closed because it has been open for too long with no activity.

Any muted tests that were associated with this issue have been unmuted.

If the tests begin failing again, a new issue will be opened, and they may be muted again.

@elasticsearchmachine
Copy link
Collaborator

This issue is getting re-opened because there are still AwaitsFix mutes for the given test. It will likely be closed again in the future.

@drempapis drempapis self-assigned this Nov 28, 2024
@drempapis drempapis mentioned this issue Nov 29, 2024
Merged
elasticsearchmachine added a commit that referenced this issue Nov 29, 2024
@elasticsearchmachine
Mute org.elasticsearch.search.ccs.CrossClusterIT testCancel #108061
0b764ad
@elasticsearchmachine
Copy link
Collaborator

This has been muted on branch main

Mute Reasons:

  • [main] 2 consecutive failures in test testCancel
  • [main] 10 failures in test testCancel (45.5% fail rate in 22 executions)
  • [main] 7 failures in step part-1 (38.9% fail rate in 18 executions)
  • [main] 5 failures in pipeline elasticsearch-pull-request (31.3% fail rate in 16 executions)

Build Scans:

@elasticsearchmachine
Copy link
Collaborator

This has been muted on branch 8.x

Mute Reasons:

  • [8.x] 17 failures in test testCancel (42.5% fail rate in 40 executions)
  • [8.x] 4 failures in step part-1 (80.0% fail rate in 5 executions)
  • [8.x] 2 failures in pipeline elasticsearch-periodic-platform-support (100.0% fail rate in 2 executions)
  • [8.x] 4 failures in pipeline elasticsearch-pull-request (80.0% fail rate in 5 executions)

Build Scans:

elasticsearchmachine added a commit that referenced this issue Nov 30, 2024
@elasticsearchmachine
Mute org.elasticsearch.search.ccs.CrossClusterIT testCancel #108061
6c44b5e
@elasticsearchmachine
Copy link
Collaborator

This has been muted on branch 8.x

Mute Reasons:

  • [8.x] 5 consecutive failures in test testCancel
  • [8.x] 2 consecutive failures in step rocky-8_platform-support-unix
  • [8.x] 2 consecutive failures in step sles-15_platform-support-unix
  • [8.x] 2 consecutive failures in step almalinux-8-aarch64_checkpart1_platform-support-arm
  • [8.x] 2 consecutive failures in step openjdk22_checkpart1_java-matrix
  • [8.x] 2 consecutive failures in step openjdk23_checkpart1_java-matrix
  • [8.x] 3 consecutive failures in step amazonlinux-2023_platform-support-aws
  • [8.x] 2 consecutive failures in step concurrent-search-tests
  • [8.x] 2 consecutive failures in step oraclelinux-7_platform-support-unix
  • [8.x] 2 consecutive failures in step oraclelinux-8_platform-support-unix
  • [8.x] 2 consecutive failures in step ubuntu-1804_platform-support-unix
  • [8.x] 2 consecutive failures in step debian-11_platform-support-unix
  • [8.x] 2 consecutive failures in step ubuntu-2004-aarch64_checkpart1_platform-support-arm
  • [8.x] 58 failures in test testCancel (52.7% fail rate in 110 executions)
  • [8.x] 3 failures in step rocky-8_platform-support-unix (75.0% fail rate in 4 executions)

Build Scans:

elasticsearchmachine added a commit that referenced this issue Dec 1, 2024
@elasticsearchmachine
Mute org.elasticsearch.search.ccs.CrossClusterIT testCancel #108061
8386a2c
@drempapis
Copy link
Contributor

@quux00, can you please have a look and advice for failing the test? I changed the Exception type and merged it successfully (#117750), but it seems it still fails in main and 8.x with another failure,

ExecutionException, but no exception was thrown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@quux00 quux00

@drempapis drempapis

Labels
low-risk An open issue or test failure that is a low risk to future releases :Search Foundations/CCS Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

CrossClusterIT testCancel failure drempapis/elasticsearch
5 participants
@quux00 @javanna @andreidan @drempapis @elasticsearchmachine