Implement logs retriever pagination using new Elasticsearch Java client #315

cyrille-leclerc · 2022-02-09T09:29:57Z

Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
Ensure that the pull request title represents the desired changelog entry
Please describe what you did
Link to relevant issues in GitHub or Jira
Link to relevant pull requests, esp. upstream and downstream changes
Ensure you have provided tests - that demonstrates feature works or fixes the issue

.../java/io/jenkins/plugins/opentelemetry/backend/elastic/ElasticsearchLogStorageRetriever.java

src/test/java/io/jenkins/plugins/opentelemetry/backend/elastic/ElasticsearchRetrieverIT.java

cyrille-leclerc · 2022-02-10T13:50:08Z

...jenkins/plugins/opentelemetry/backend/elastic/ElasticsearchLogStorageScrollingRetriever.java

+            ScrollRequest scrollRequest = ScrollRequest.of(builder -> builder.scrollId(context.scrollId));
+            ScrollResponse<ObjectNode> scrollResponse = this.elasticsearchClient.scroll(scrollRequest, ObjectNode.class);
+            hits = scrollResponse.hits().hits();
+            newScrollId = context.scrollId; // TODO why doesn't the scroll response hold a new scrollId? scrollResponse.scrollId();


@swallez why is scrollResponse.scrollId null? I expected it to hold the scrollIdI should use next time so I don't have to understand if the backend has decided to reuse the sqame id or create a new one, similarly to the point in time API..

Have a look at the documentation. The way scroll works is as follows:

send a normal search request with the scroll parameter set that indicates a TTL,

for the next "pages", call client.scroll() to fetch the next batch.

cyrille-leclerc · 2022-02-10T13:52:19Z

...jenkins/plugins/opentelemetry/backend/elastic/ElasticsearchLogStorageScrollingRetriever.java

+            hits = scrollResponse.hits().hits();
+            newScrollId = context.scrollId; // TODO why doesn't the scroll response hold a new scrollId? scrollResponse.scrollId();
+        }
+        complete = hits.size() != PAGE_SIZE; // TODO is there smarter?


@swallez what's the best way to understand if the search request returned all the entries or if I should scroll more? as scrollResponse.scrollId is always null, I end up doing one wasted query if the number of entries of the response is a multiple of PAGE_SIZE

...jenkins/plugins/opentelemetry/backend/elastic/ElasticsearchLogStorageScrollingRetriever.java

cyrille-leclerc · 2022-02-10T21:43:48Z

java.lang.ClassNotFoundException: org.glassfish.json.JsonProviderImpl
	at jenkins.util.AntClassLoader.findClassInComponents(AntClassLoader.java:1387)
	at jenkins.util.AntClassLoader.findClass(AntClassLoader.java:1342)
	at jenkins.util.AntClassLoader.loadClass(AntClassLoader.java:1089)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:315)
	at jakarta.json.spi.JsonProvider.provider(JsonProvider.java:72)
Caused: jakarta.json.JsonException: Provider org.glassfish.json.JsonProviderImpl not found
	at jakarta.json.spi.JsonProvider.provider(JsonProvider.java:75)
	at co.elastic.clients.json.jackson.JsonValueParser.<init>(JsonValueParser.java:39)
	at co.elastic.clients.json.jackson.JacksonJsonpParser.getValue(JacksonJsonpParser.java:233)
	at co.elastic.clients.json.JsonData.from(JsonData.java:96)
	at co.elastic.clients.json.JsonpDeserializer$2.deserialize(JsonpDeserializer.java:112)
	at co.elastic.clients.elasticsearch._types.ErrorCause.lambda$setupErrorCauseDeserializer$0(ErrorCause.java:391)
	at co.elastic.clients.json.ObjectDeserializer.parseUnknownField(ObjectDeserializer.java:205)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:174)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:137)
	at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:85)
	at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:48)
	at co.elastic.clients.json.JsonpDeserializerBase$ArrayDeserializer.deserialize(JsonpDeserializerBase.java:320)
	at co.elastic.clients.json.JsonpDeserializerBase$ArrayDeserializer.deserialize(JsonpDeserializerBase.java:285)
	at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:75)
	at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:72)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:176)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:137)
	at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:75)
	at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
	at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
	at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:72)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:176)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:137)
	at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:75)
	at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
	at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
	at co.elastic.clients.transport.rest_client.RestClientTransport.getHighLevelResponse(RestClientTransport.java:276)
	at co.elastic.clients.transport.rest_client.RestClientTransport.performRequest(RestClientTransport.java:144)
	at co.elastic.clients.elasticsearch.ElasticsearchClient.search(ElasticsearchClient.java:1487)
	at io.jenkins.plugins.opentelemetry.backend.elastic.ElasticsearchLogStorageScrollingRetriever.overallLog(ElasticsearchLogStorageScrollingRetriever.java:113)
	at io.jenkins.plugins.opentelemetry.job.log.OtelLogStorage.overallLog(OtelLogStorage.java:98)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.getLogText(WorkflowRun.java:1057)
	at hudson.model.Run.writeLogTo(Run.java:1554)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)

swallez · 2022-02-11T13:48:53Z

I implemented log retrieval using point in time in swallez@e454add (test works AFAICT)

Feel free to pull these changes here.

cyrille-leclerc · 2022-02-11T14:36:21Z

thanks, can you by any chance do a PR?

swallez · 2022-02-11T14:50:49Z

@cyrille-leclerc I'll let you take it from there, I'm happy I could help but don't have much time to dedicate to this.

cyrille-leclerc · 2022-02-15T12:35:07Z

.../java/io/jenkins/plugins/opentelemetry/backend/elastic/ElasticsearchLogStorageRetriever.java

+                    "logPosition=(end:now,start:now-1d,streamLive:!f)&" +
+                    "logFilter=(language:kuery,query:%27trace.id:" + traceId + "%27)&";
+                if (pageNo == 0) {
+                    w.write("View logs in Kibana: " + logsVisualizationUrl + "\n\n");


@kuisathaverat do you know how to produce an HTML link? I guess it's related to the code of io.jenkins.plugins.opentelemetry.job.log.ConsoleNotes

it depends on the render you have configured for Jenkins, my experience on showing an URL in the console log is that will work if the URL is simple, but in the case of the Kibana URL is not simple, it has many parameters with delimiters and so on, in my test I could make the link work in all the cases (probably URL encoded works better, but I did not test it), and the link is ugly because is too long. Because of that, I changed to use an Action in the previous plugin.

cyrille-leclerc · 2022-02-15T12:41:09Z

@kuisathaverat I don't understand how the API works, I don't succeed in having any kind of pagination. We have 3 invocations OtelLogStorage.overallLog() then OverallLog.writeHtmlTo()invoked on the last AnnotatedLargeText so I restricted the retrieval of logs from Elasticsearch to the first 100 lines and then I add a link to Kibana.

kuisathaverat · 2022-02-15T12:44:24Z

@kuisathaverat I don't understand how the API works, I don't succeed in having any kind of pagination. We have 3 invocations OtelLogStorage.overallLog() then OverallLog.writeHtmlTo()invoked on the last AnnotatedLargeText so I restricted the retrieval of logs from Elasticsearch to the first 100 lines and then I add a link to Kibana.

I think is the best approach, it limits the possibility of issues in Jenkins when you have huge logs, you always can go to Elasticsearch to check the whole log.

cyrille-leclerc · 2022-02-15T12:50:34Z

src/main/java/io/jenkins/plugins/opentelemetry/backend/custom/CustomLogStorageRetriever.java

+import java.util.HashMap;
+import java.util.Map;
+
+public class CustomLogStorageRetriever implements LogStorageRetriever<CustomLogStorageRetriever.CustomLogsQueryContext> {


@kuisathaverat I want to introduce a generic LogStorageRetriever that just print a hyperlink to the visualization backend

cyrille-leclerc added 3 commits February 9, 2022 10:27

Implement logs retriever pagination using new Elasticsearch Java client

dbf59df

WIP

3f43bf8

WIP

c8278d4

cyrille-leclerc commented Feb 9, 2022

View reviewed changes

src/test/java/io/jenkins/plugins/opentelemetry/backend/elastic/ElasticsearchRetrieverIT.java Outdated Show resolved Hide resolved

cyrille-leclerc added 2 commits February 10, 2022 09:28

extract integration tests configuration from code

65ea92d

WIP

e54e639

cyrille-leclerc commented Feb 10, 2022

View reviewed changes

...jenkins/plugins/opentelemetry/backend/elastic/ElasticsearchLogStorageScrollingRetriever.java Outdated Show resolved Hide resolved

WIP

de31247

cyrille-leclerc added 3 commits February 15, 2022 00:47

WIP

0f589a9

WIP

688f58d

fix findbugs

5e745b3

cyrille-leclerc commented Feb 15, 2022

View reviewed changes

cyrille-leclerc marked this pull request as ready for review February 15, 2022 12:41

cyrille-leclerc requested a review from kuisathaverat as a code owner February 15, 2022 12:41

better docs

5762236

cyrille-leclerc commented Feb 15, 2022

View reviewed changes

WIP

7cd6d22

cyrille-leclerc merged commit 6af398d into send-build-logs-to-otel Feb 15, 2022

cyrille-leclerc deleted the implement-pagination-using-new-elasticsearch-client branch February 15, 2022 15:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement logs retriever pagination using new Elasticsearch Java client #315

Implement logs retriever pagination using new Elasticsearch Java client #315

cyrille-leclerc commented Feb 9, 2022

cyrille-leclerc Feb 10, 2022

swallez Feb 10, 2022

cyrille-leclerc Feb 10, 2022

cyrille-leclerc commented Feb 10, 2022

swallez commented Feb 11, 2022

cyrille-leclerc commented Feb 11, 2022

swallez commented Feb 11, 2022

cyrille-leclerc Feb 15, 2022

kuisathaverat Feb 15, 2022

cyrille-leclerc commented Feb 15, 2022

kuisathaverat commented Feb 15, 2022 •

edited

Loading

cyrille-leclerc Feb 15, 2022

Implement logs retriever pagination using new Elasticsearch Java client #315

Implement logs retriever pagination using new Elasticsearch Java client #315

Conversation

cyrille-leclerc commented Feb 9, 2022

cyrille-leclerc Feb 10, 2022

Choose a reason for hiding this comment

swallez Feb 10, 2022

Choose a reason for hiding this comment

cyrille-leclerc Feb 10, 2022

Choose a reason for hiding this comment

cyrille-leclerc commented Feb 10, 2022

swallez commented Feb 11, 2022

cyrille-leclerc commented Feb 11, 2022

swallez commented Feb 11, 2022

cyrille-leclerc Feb 15, 2022

Choose a reason for hiding this comment

kuisathaverat Feb 15, 2022

Choose a reason for hiding this comment

cyrille-leclerc commented Feb 15, 2022

kuisathaverat commented Feb 15, 2022 • edited Loading

cyrille-leclerc Feb 15, 2022

Choose a reason for hiding this comment

kuisathaverat commented Feb 15, 2022 •

edited

Loading