Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ElasticApmTracer crashes on JDK8 #400

Closed
agseekda opened this issue Dec 27, 2018 · 4 comments
Closed

ElasticApmTracer crashes on JDK8 #400

agseekda opened this issue Dec 27, 2018 · 4 comments
Assignees

Comments

@agseekda
Copy link

When running the elastic-apm-agent with our application on JDK8 we occasionally get exceptions like:

Exception in thread "iub7" java.lang.NoSuchMethodError: java.nio.CharBuffer.clear()Ljava/nio/CharBuffer;
at co.elastic.apm.impl.transaction.Db$2.recycle(Db.java:50)
at co.elastic.apm.impl.transaction.Db$2.recycle(Db.java:47)
at co.elastic.apm.objectpool.impl.QueueBasedObjectPool.recycle(QueueBasedObjectPool.java:70)
at co.elastic.apm.impl.transaction.Db.resetState(Db.java:181)
at co.elastic.apm.impl.context.SpanContext.resetState(SpanContext.java:59)
at co.elastic.apm.impl.transaction.Span.resetState(Span.java:106)
at co.elastic.apm.objectpool.impl.Resetter$ForRecyclable.recycle(Resetter.java:37)
at co.elastic.apm.objectpool.impl.Resetter$ForRecyclable.recycle(Resetter.java:28)
at co.elastic.apm.objectpool.impl.QueueBasedObjectPool.recycle(QueueBasedObjectPool.java:70)
at co.elastic.apm.impl.ElasticApmTracer.recycle(ElasticApmTracer.java:292)
at co.elastic.apm.impl.transaction.Span.recycle(Span.java:116)
at co.elastic.apm.report.ApmServerReporter.report(ApmServerReporter.java:113)
at co.elastic.apm.impl.ElasticApmTracer.endSpan(ElasticApmTracer.java:281)
at co.elastic.apm.impl.transaction.Span.doEnd(Span.java:100)
at co.elastic.apm.impl.transaction.AbstractSpan.end(AbstractSpan.java:210)
at co.elastic.apm.impl.transaction.AbstractSpan.end(AbstractSpan.java:197)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:227)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1256)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1231)
at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:730)

After a bit of digging we found that this is caused by a JDK9 incompatibility - see answer in https://stackoverflow.com/questions/48693695/java-nio-buffer-not-loading-clear-method-on-runtime

Seems like all that is needed to fix it is to cast CharBuffer to Buffer.

@eyalkoren
Copy link
Contributor

Hi. Thanks for trying and providing this important feedback!
Looking into this.
Just so I better understand the symptom- the header of the issue says it crashes, but is it? Also, please let me know which Java agent version you are using.

@agseekda
Copy link
Author

Hi, thanks for the quick response.

With "it" I mean the thread not the agent itself - seems the thread died and caused our application to slow down because the worker threads where gone.

We are using version 1.1.0

@eyalkoren
Copy link
Contributor

You can upgrade to agent version 1.2.0 where it should at least not let the thread die. However, I fear that until I build a fix for that, the Elasticsearch queries are going to be appended to one another... They cannot grow forever a there is a guard for that, but it's quite a high limit.
In any case, fixing now and will probably take a bit before I release a new build. What I suggest now is that you register on our repo to get notified about new releases and in the meantime either upgrade and see what happens in 1.2.0 or turn off the elasticsearch-restclient instrumentation through the disable_instrumentation configuration option

@agseekda
Copy link
Author

OK - we updated the agent to 1.2.0 which is better than turning off instrumentation. Once the new version is out with the proper fix we will upgrade again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants