Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Actuator health endpoint for neo4j throws NoSuchElementException and always returns Status.DOWN #33428

Closed
x-wing opened this issue Nov 30, 2022 · 2 comments
Closed

Actuator health endpoint for neo4j throws NoSuchElementException and always returns Status.DOWN #33428

x-wing opened this issue Nov 30, 2022 · 2 comments
Assignees
@philwebb
Labels
type: regression A regression from a previous release
Milestone
3.0.1

Comments

@x-wing
Copy link

x-wing commented Nov 30, 2022 •
edited
Loading

This happens with Spring boot 3.0 and the default neo4j-java-driver version 5.2.0 or the latest 5.3.0.
The JDK used is: OpenJDK 64-Bit Server VM Corretto-19.0.0.36.1

I have created a demo project with Initializr which can trigger the issue through http://localhost:8080/actuator/health

demo.zip

Here is the docker command to set up a local neo4j instance:

docker run \
    --name neo4j \
    -p7474:7474 -p7687:7687 \
    -d \
    -v $HOME/neo4j/data:/data \
    -v $HOME/neo4j/logs:/logs \
    -v $HOME/neo4j/import:/var/lib/neo4j/import \
    -v $HOME/neo4j/plugins:/plugins \
    --env NEO4J_AUTH=neo4j/test \
    [neo4j:latest]

Here is the full health endpoint json response

{
  "status": "DOWN",
  "components": {
    "diskSpace": {
      "status": "UP",
      "details": {
        "total": 994662584320,
        "free": 564777406464,
        "threshold": 10485760,
        "path": "/Users/texas/docker/neo4j-issue/demo/.",
        "exists": true
      }
    },
    "neo4j": {
      "status": "DOWN",
      "details": {
        "error": "java.util.NoSuchElementException: Source was empty"
      }
    },
    "ping": {
      "status": "UP"
    }
  }
}

Here is the full stack trace:
java.util.NoSuchElementException: Source was empty
        at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:174) ~[reactor-core-3.5.0.jar:3.5.0]
        at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:144) ~[reactor-core-3.5.0.jar:3.5.0]
        at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260) ~[reactor-core-3.5.0.jar:3.5.0]
        at reactor.core.publisher.FluxZip$ZipCoordinator.drain(FluxZip.java:890) ~[reactor-core-3.5.0.jar:3.5.0]
        at reactor.core.publisher.FluxZip$ZipInner.onComplete(FluxZip.java:1142) ~[reactor-core-3.5.0.jar:3.5.0]
        at reactor.core.publisher.FluxCreate$BaseSink.complete(FluxCreate.java:460) ~[reactor-core-3.5.0.jar:3.5.0]
        at reactor.core.publisher.FluxCreate$SerializedFluxSink.drainLoop(FluxCreate.java:247) ~[reactor-core-3.5.0.jar:3.5.0]
        at reactor.core.publisher.FluxCreate$SerializedFluxSink.drain(FluxCreate.java:213) ~[reactor-core-3.5.0.jar:3.5.0]
        at reactor.core.publisher.FluxCreate$SerializedFluxSink.complete(FluxCreate.java:204) ~[reactor-core-3.5.0.jar:3.5.0]
        at org.neo4j.driver.internal.reactivestreams.InternalReactiveResult.lambda$createRecordConsumer$3(InternalReactiveResult.java:95) ~[neo4j-java-driver-5.2.0.jar:5.2.0-e0e5b6a8d95e38084c5cf9ac39e3ee8c9f5f05d0]
        at org.neo4j.driver.internal.handlers.pulln.BasicPullResponseHandler.complete(BasicPullResponseHandler.java:252) ~[neo4j-java-driver-5.2.0.jar:5.2.0-e0e5b6a8d95e38084c5cf9ac39e3ee8c9f5f05d0]
        at org.neo4j.driver.internal.handlers.pulln.BasicPullResponseHandler.onSuccess(BasicPullResponseHandler.java:119) ~[neo4j-java-driver-5.2.0.jar:5.2.0-e0e5b6a8d95e38084c5cf9ac39e3ee8c9f5f05d0]
        at org.neo4j.driver.internal.handlers.RoutingResponseHandler.onSuccess(RoutingResponseHandler.java:55) ~[neo4j-java-driver-5.2.0.jar:5.2.0-e0e5b6a8d95e38084c5cf9ac39e3ee8c9f5f05d0]
        at org.neo4j.driver.internal.async.inbound.InboundMessageDispatcher.handleSuccessMessage(InboundMessageDispatcher.java:87) ~[neo4j-java-driver-5.2.0.jar:5.2.0-e0e5b6a8d95e38084c5cf9ac39e3ee8c9f5f05d0]
        at org.neo4j.driver.internal.messaging.common.CommonMessageReader.unpackSuccessMessage(CommonMessageReader.java:68) ~[neo4j-java-driver-5.2.0.jar:5.2.0-e0e5b6a8d95e38084c5cf9ac39e3ee8c9f5f05d0]
        at org.neo4j.driver.internal.messaging.common.CommonMessageReader.read(CommonMessageReader.java:50) ~[neo4j-java-driver-5.2.0.jar:5.2.0-e0e5b6a8d95e38084c5cf9ac39e3ee8c9f5f05d0]
        at org.neo4j.driver.internal.async.inbound.InboundMessageHandler.channelRead0(InboundMessageHandler.java:81) ~[neo4j-java-driver-5.2.0.jar:5.2.0-e0e5b6a8d95e38084c5cf9ac39e3ee8c9f5f05d0]
        at org.neo4j.driver.internal.async.inbound.InboundMessageHandler.channelRead0(InboundMessageHandler.java:37) ~[neo4j-java-driver-5.2.0.jar:5.2.0-e0e5b6a8d95e38084c5cf9ac39e3ee8c9f5f05d0]
        at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.85.Final.jar:4.1.85.Final]
        at org.neo4j.driver.internal.async.inbound.MessageDecoder.channelRead(MessageDecoder.java:42) ~[neo4j-java-driver-5.2.0.jar:5.2.0-e0e5b6a8d95e38084c5cf9ac39e3ee8c9f5f05d0]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.85.Final.jar:4.1.85.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.85.Final.jar:4.1.85.Final]
        at java.base/java.lang.Thread.run(Thread.java:1589) ~[na:na]
@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Nov 30, 2022
@scottfrederick
Copy link
Contributor

This has also been reported at spring-projects/spring-data-neo4j#2635.

@x-wing
Copy link
Author

x-wing commented Dec 1, 2022

Thanks for noting! Hope the attached project can help the investigation. Health endpoint is critical to our infrastructure which relies on the health status to automate deployments. This error gives a false signal.

@philwebb philwebb added type: regression A regression from a previous release and removed status: waiting-for-triage An issue we've not yet triaged labels Dec 1, 2022
@philwebb philwebb self-assigned this Dec 1, 2022
@philwebb philwebb added this to the 3.0.x milestone Dec 1, 2022
@philwebb philwebb changed the title actuator/health for neo4j shows status DOWN with error java.util.NoSuchElementException: Source was empty Actuator health endpoint for neo4j shows status DOWN caused by NoSuchElementException Dec 1, 2022
@philwebb philwebb changed the title Actuator health endpoint for neo4j shows status DOWN caused by NoSuchElementException Actuator health endpoint for neo4j throws NoSuchElementException and always returns Status.DOWN Dec 1, 2022
@hadiyarajesh hadiyarajesh mentioned this issue Dec 1, 2022
Closed
@philwebb philwebb modified the milestones: 3.0.x, 3.0.1 Dec 1, 2022
philwebb added a commit that referenced this issue Dec 1, 2022
@philwebb
Add Neo4jReactiveHealthIndicatorIntegrationTests
06c6daf 
Add a full integration test for `Neo4jReactiveHealthIndicator`
to ensure that it works against a real server.

See gh-33428
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@philwebb philwebb

Labels
type: regression A regression from a previous release
Projects
None yet
Milestone
3.0.1
Development

No branches or pull requests
4 participants
@scottfrederick @philwebb @x-wing @spring-projects-issues