Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AutoFollowIT failures #41483

Closed
jkakavas opened this issue Apr 24, 2019 · 3 comments
Closed

AutoFollowIT failures #41483

jkakavas opened this issue Apr 24, 2019 · 3 comments
Labels
:Distributed Indexing/CCR Issues around the Cross Cluster State Replication features >test-failure Triaged test failures from CI

Comments

@jkakavas
Copy link
Member

This has only failed once today, and it is not clear if there was a recent change that caused it, I will keep an eye and not mute for now

REPRODUCE WITH: ./gradlew :x-pack:plugin:ccr:internalClusterTest --tests "org.elasticsearch.xpack.ccr.AutoFollowIT.null" -Dtests.seed=9FDCF6F09052CFE -Dtests.security.manager=true -Dtests.locale=en-US -Dtests.timezone=America/Los_Angeles -Dcompiler.java=12 -Druntime.java=8

2> java.lang.AssertionError
  2> junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?) and null them, your test still has references to classes of which the sizes cannot be measured due to security restrictions or Java 9 module encapsulation:
      - private static org.elasticsearch.xpack.CcrIntegTestCase$ClusterGroup org.elasticsearch.xpack.CcrIntegTestCase.clusterGroup

        Caused by:
        java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.fs")
  2> com.carrotsearch.randomizedtesting.ThreadLeakError: 54 threads leaked from SUITE scope at org.elasticsearch.xpack.ccr.AutoFollowIT: 
       1) Thread[id=932, name=elasticsearch[followerm2][masterService#updateTask][T#1], state=WAITING, group=TGRP-AutoFollowIT]
            at sun.misc.Unsafe.park(Native Method)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
            at java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:550)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
       2) Thread[id=949, name=elasticsearch[followerm2][management][T#4], state=TIMED_WAITING, group=TGRP-AutoFollowIT]
            at sun.misc.Unsafe.park(Native Method)
            at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:734)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.poll(LinkedTransferQueue.java:1277)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
       3) Thread[id=921, name=elasticsearch[followerm1][clusterApplierService#updateTask][T#1], state=WAITING, group=TGRP-AutoFollowIT]
            at sun.misc.Unsafe.park(Native Method)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
            at java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:550)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
       4) Thread[id=976, name=elasticsearch[followerm2][ccr][T#6], state=WAITING, group=TGRP-AutoFollowIT]
            at sun.misc.Unsafe.park(Native Method)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at org.elasticsearch.common.util.concurrent.SizeBlockingQueue.take(SizeBlockingQueue.java:165)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
       5) Thread[id=882, name=elasticsearch[followerm2][transport_worker][T#1], state=RUNNABLE, group=TGRP-AutoFollowIT]
            at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
            at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
            at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
            at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
            at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
            at org.elasticsearch.nio.NioSelector.singleLoop(NioSelector.java:162)
            at org.elasticsearch.nio.NioSelector.runLoop(NioSelector.java:129)
            at org.elasticsearch.nio.NioSelectorGroup$$Lambda$1741/1999126013.run(Unknown Source)
            at java.lang.Thread.run(Thread.java:748)
       6) Thread[id=900, name=elasticsearch[followerm2][generic][T#2], state=WAITING, group=TGRP-AutoFollowIT]
            at sun.misc.Unsafe.park(Native Method)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
       7) Thread[id=935, name=elasticsearch[followerm2][management][T#2], state=TIMED_WAITING, group=TGRP-AutoFollowIT]
            at sun.misc.Unsafe.park(Native Method)
            at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:734)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.poll(LinkedTransferQueue.java:1277)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
       8) Thread[id=981, name=elasticsearch[followerm2][ccr][T#7], state=WAITING, group=TGRP-AutoFollowIT]
            at sun.misc.Unsafe.park(Native Method)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at org.elasticsearch.common.util.concurrent.SizeBlockingQueue.take(SizeBlockingQueue.java:165)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
       9) Thread[id=901, name=elasticsearch[followerm2][clusterApplierService#updateTask][T#1], state=WAITING, group=TGRP-AutoFollowIT]
            at sun.misc.Unsafe.park(Native Method)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
            at java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:550)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
      10) Thread[id=904, name=elasticsearch[followerm2][[unicast_configured_hosts_resolver]][T#2], state=TIMED_WAITING, group=TGRP-AutoFollowIT]
            at sun.misc.Unsafe.park(Native Method)
            at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:734)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.poll(LinkedTransferQueue.java:1277)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)

.....
@jkakavas jkakavas added >test-failure Triaged test failures from CI :Distributed Indexing/CCR Issues around the Cross Cluster State Replication features labels Apr 24, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@albertzaharovits
Copy link
Contributor

@jkakavas I think it's better to track this failure under #41526

@ywelsch
Copy link
Contributor

ywelsch commented May 7, 2019

Tracked in #41526

@ywelsch ywelsch closed this as completed May 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:Distributed Indexing/CCR Issues around the Cross Cluster State Replication features >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ywelsch @albertzaharovits @jkakavas @elasticmachine