[BUG] org.opensearch.action.admin.indices.create.SplitIndexIT.testCreateSplitIndex flaky #5409

dblock · 2022-11-30T15:41:34Z

Describe the bug

https://build.ci.opensearch.org/job/gradle-check/7415/
#5402 (comment)

Not reproducible.

./gradlew ':server:internalClusterTest' --tests "org.opensearch.action.admin.indices.create.SplitIndexIT.testCreateSplitIndex" -Dtests.seed=2E50FB3665C10C67 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ru-RU -Dtests.timezone=America/Argentina/Ushuaia -Druntime.java=17

The text was updated successfully, but these errors were encountered:

vikasvb90 · 2024-02-14T05:44:56Z

Suspect seems to be pending recovery even after test completion due to ongoing uploads in remote store flow. @gbbafna fixed the same in PR #11720 . Please reopen if it re-occurs.

reta · 2024-05-27T20:19:57Z

The issue is not gone, org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex: https://build.ci.opensearch.org/job/gradle-check/39309/testReport/junit/org.opensearch.action.admin.indices.create/RemoteSplitIndexIT/testCreateSplitIndex/

UncategorizedExecutionException[Failed execution]; nested: TranslogUploadFailedException[Failed to upload 1 files during transfer];
	at __randomizedtesting.SeedInfo.seed([EF41E70EC8C7B6E:76B4C8CAE00CB16C]:0)
	at app//org.opensearch.action.support.AdapterActionFuture.unwrapEsException(AdapterActionFuture.java:102)
	at app//org.opensearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:57)
	at app//org.opensearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:73)
	at app//org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex(RemoteSplitIndexIT.java:414)
	at [email protected]/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at [email protected]/java.lang.reflect.Method.invoke(Method.java:580)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at app//org.opensearch.test.OpenSearchTestClusterRule$1.evaluate(OpenSearchTestClusterRule.java:369)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at [email protected]/java.lang.Thread.run(Thread.java:1583)
Caused by: org.opensearch.index.translog.transfer.TranslogUploadFailedException: Failed to upload 1 files during transfer
	at app//org.opensearch.index.translog.transfer.TranslogTransferManager.transferSnapshot(TranslogTransferManager.java:199)
	at app//org.opensearch.index.translog.RemoteFsTranslog.upload(RemoteFsTranslog.java:426)
	at app//org.opensearch.index.translog.RemoteFsTranslog.prepareAndUpload(RemoteFsTranslog.java:409)
	at app//org.opensearch.index.translog.RemoteFsTranslog.ensureSynced(RemoteFsTranslog.java:341)
	at app//org.opensearch.index.translog.Translog.ensureSynced(Translog.java:837)
	at app//org.opensearch.index.translog.InternalTranslogManager.ensureTranslogSynced(InternalTranslogManager.java:184)
	at app//org.opensearch.index.engine.InternalEngine.ensureTranslogSynced(InternalEngine.java:605)
	at app//org.opensearch.index.shard.IndexShard.lambda$createTranslogSyncProcessor$44(IndexShard.java:4442)
	at app//org.opensearch.index.shard.IndexShard$6.write(IndexShard.java:4456)
	at app//org.opensearch.common.util.concurrent.AsyncIOProcessor.processList(AsyncIOProcessor.java:131)
	at app//org.opensearch.common.util.concurrent.AsyncIOProcessor.drainAndProcessAndRelease(AsyncIOProcessor.java:119)
	at app//org.opensearch.common.util.concurrent.BufferedAsyncIOProcessor.process(BufferedAsyncIOProcessor.java:80)
	at app//org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:854)
	at [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	... 1 more
	Suppressed: org.opensearch.index.translog.transfer.FileTransferException: java.lang.NullPointerException
		at app//org.opensearch.index.translog.transfer.BlobStoreTransferService.uploadBlob(BlobStoreTransferService.java:187)
		at app//org.opensearch.index.translog.transfer.BlobStoreTransferService.lambda$uploadBlobs$2(BlobStoreTransferService.java:105)
		at [email protected]/java.lang.Iterable.forEach(Iterable.java:75)
		at app//org.opensearch.index.translog.transfer.BlobStoreTransferService.uploadBlobs(BlobStoreTransferService.java:100)
		at app//org.opensearch.index.translog.transfer.TranslogTransferManager.transferSnapshot(TranslogTransferManager.java:163)
		... 15 more
	Caused by: java.lang.NullPointerException
		at java.base/java.util.Objects.requireNonNull(Objects.java:233)
		at org.opensearch.index.translog.transfer.BlobStoreTransferService.uploadBlob(BlobStoreTransferService.java:165)
		... 19 more

peternied · 2024-05-29T15:24:42Z

[Triage - attendees 1 2 3 4 5 6
@dblock Thanks for creating this issue

dblock added bug Something isn't working untriaged flaky-test Random test failure that succeeds on second run labels Nov 30, 2022

owaiskazi19 removed the untriaged label Nov 30, 2022

RamakrishnaChilaka assigned nknize and owaiskazi19 Oct 24, 2023

shwetathareja added the Cluster Manager label Oct 26, 2023

reta mentioned this issue Oct 30, 2023

Fix passing wrong parameter when calling newConfigurationException() in DotExpanderProcessor #10737

Merged

7 tasks

mch2 mentioned this issue Nov 14, 2023

Muting flaky tests in SegmentReplicationSuiteIT #11159

Merged

8 tasks

minalsha assigned gauravruhela and unassigned owaiskazi19 Nov 15, 2023

ankitkala assigned vikasvb90 and unassigned nknize and gauravruhela Dec 7, 2023

owaiskazi19 mentioned this issue Feb 6, 2024

[Backport 2.x] Added spotless for json files #12133

Merged

vikasvb90 closed this as completed Feb 14, 2024

reta reopened this May 27, 2024

github-actions bot added the untriaged label May 27, 2024

github-project-automation bot added this to Cluster Manager Project Board May 27, 2024

github-project-automation bot moved this to 🆕 New in Cluster Manager Project Board May 27, 2024

reta mentioned this issue May 27, 2024

[Backport] [2.x] Bump com.google.code.gson:gson in /test/fixtures/hdfs-fixture (#13842) #13847

Merged

peternied removed the untriaged label May 29, 2024

shiv0408 mentioned this issue Jun 8, 2024

[Backport 2.x] Optimize remote state stale file deletion (#13995) #14025

Merged

3 tasks

bowenlan-amzn mentioned this issue Jun 12, 2024

[AUTOCUT] Gradle Check Failure on push to 2.x #14210

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] org.opensearch.action.admin.indices.create.SplitIndexIT.testCreateSplitIndex flaky #5409

[BUG] org.opensearch.action.admin.indices.create.SplitIndexIT.testCreateSplitIndex flaky #5409

dblock commented Nov 30, 2022

vikasvb90 commented Feb 14, 2024 •

edited

Loading

reta commented May 27, 2024

peternied commented May 29, 2024

[BUG] org.opensearch.action.admin.indices.create.SplitIndexIT.testCreateSplitIndex flaky #5409

[BUG] org.opensearch.action.admin.indices.create.SplitIndexIT.testCreateSplitIndex flaky #5409

Comments

dblock commented Nov 30, 2022

vikasvb90 commented Feb 14, 2024 • edited Loading

reta commented May 27, 2024

peternied commented May 29, 2024

vikasvb90 commented Feb 14, 2024 •

edited

Loading