Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send pipeline logs to OpenTelemetry endpoint #304

Merged
merged 36 commits into from
Feb 20, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
7e3af1f
Send Pipeline Logs to OTel
cyrille-leclerc Jan 25, 2022
fdf81db
WIP
cyrille-leclerc Jan 25, 2022
cc45c2c
Disable OtelLogStorage if Otel SDK is not configured to export logs
cyrille-leclerc Jan 25, 2022
f319d24
Add trace context
cyrille-leclerc Jan 26, 2022
ddb2fec
WIP
cyrille-leclerc Jan 26, 2022
1224500
WIP
cyrille-leclerc Jan 26, 2022
8063fdb
Simplify code
cyrille-leclerc Jan 26, 2022
fd2d0cf
Simplify
cyrille-leclerc Jan 26, 2022
c4c0ea9
Fix dependency injection problems
cyrille-leclerc Jan 26, 2022
f398b1d
Add Elasticsearch log retriever
kuisathaverat Feb 6, 2022
4a35569
Merge branch 'master' into send-build-logs-to-otel
cyrille-leclerc Feb 6, 2022
6978007
Extract more elasticsearch code (#310)
cyrille-leclerc Feb 6, 2022
ed1594d
Extract logs retriever API (#311)
cyrille-leclerc Feb 7, 2022
6af398d
Implement logs retriever pagination using new Elasticsearch Java clie…
cyrille-leclerc Feb 15, 2022
a2e058a
Skip empty log messages
cyrille-leclerc Feb 15, 2022
a517bf2
Fix log rendering
cyrille-leclerc Feb 15, 2022
db37a26
WIP
cyrille-leclerc Feb 16, 2022
4b1e4db
WIP
cyrille-leclerc Feb 16, 2022
933f4d7
WIP
cyrille-leclerc Feb 17, 2022
8260804
Code cleanup
cyrille-leclerc Feb 17, 2022
fbb798a
Cleanup
cyrille-leclerc Feb 17, 2022
530967c
Prefix root span name of build executions by "BUILD"
cyrille-leclerc Feb 17, 2022
2d62535
Clock adjustment between the Jenkins Controller and the Jenkins Agents
cyrille-leclerc Feb 18, 2022
52d63fd
WIP
cyrille-leclerc Feb 18, 2022
1919856
Fix unit tests
cyrille-leclerc Feb 18, 2022
4cbe00f
WIP
cyrille-leclerc Feb 18, 2022
9231afa
WUP
cyrille-leclerc Feb 18, 2022
647d84a
WIP
cyrille-leclerc Feb 19, 2022
15c286d
Support disabling the instrumentation with traces of the Jenkins Web…
cyrille-leclerc Feb 19, 2022
8227106
Cleaner disabling of unit tests
cyrille-leclerc Feb 19, 2022
db5f908
WIP
cyrille-leclerc Feb 19, 2022
988f40c
WIP
cyrille-leclerc Feb 19, 2022
56403dc
fix "NoClassDefFoundError: org/parboiled/common/StringUtils"
cyrille-leclerc Feb 19, 2022
16af0ed
Cleanup
cyrille-leclerc Feb 20, 2022
73cba42
fix "NoClassDefFoundError: org/parboiled/common/StringUtils"
cyrille-leclerc Feb 20, 2022
6b32491
Switch back to the deprecated elasticsearch-rest-high-level-client in…
cyrille-leclerc Feb 20, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,6 @@ private/
agent.jar
local.pid

.env


2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ You can look at this video tutorial to get started: [![Tracing Your Jenkins Pipe
|------------------------------------------|-------|------------|
| otel.instrumentation.jenkins.job.dsl.collapse.job.name | Boolean, default `false` | When using Job DSL generated jobs, make the pipeline run root span name a low cardinality name using the name "Job from seed '${job.jobDslSeedJob.fullName}'" rather than using "${job.fullName}". Useful when the Job DSL plugin creates a lot of jobs |
| otel.instrumentation.jenkins.job.matrix.expand.job.name | Boolean, default `false` | When using Matrix Projects, the name of the combination jobs is by default collapsed to "${matrix-job-name}/execution" rather than using the full name that is generated joining the axis values of the combination |

| otel.instrumentation.jenkins.web.enabled | Boolean, default `true` | Disable the instrumentation of Jenkins web requests (ie the instrumentation of Jenkins Stapler) |


## Configuration as code
Expand Down
1 change: 0 additions & 1 deletion opentelemetry-collector/.gitignore

This file was deleted.

58 changes: 0 additions & 58 deletions opentelemetry-collector/opentelemetry-collector.yaml

This file was deleted.

61 changes: 0 additions & 61 deletions opentelemetry-collector/run-opentelemetry-collector.sh

This file was deleted.

66 changes: 49 additions & 17 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@
<gitHubRepo>jenkinsci/${project.artifactId}-plugin</gitHubRepo>
<opentelemetry.version>1.11.0</opentelemetry.version>
<opentelemetry-alpha.version>1.11.0-alpha</opentelemetry-alpha.version>
<useBeta>true</useBeta>
<elasticstack.version>7.17.0</elasticstack.version>
</properties>
<name>OpenTelemetry Plugin</name>
<description>Publish Jenkins metrics to an OpenTelemetry endpoint, including distributed traces of job executions and health metrics of the controller.</description>
Expand Down Expand Up @@ -59,16 +61,6 @@
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<!--
conflict between io.opentelemetry:opentelemetry-sdk-extension-aws and io.jenkins.configuration-as-code:test-harness
-->
<groupId>com.fasterxml.jackson</groupId>
<artifactId>jackson-bom</artifactId>
<version>2.13.1</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<!-- okhttp:4.9.3 -> okio:2.8.0 versus okhttp:4.9.3 -> okio:2.8.0 - kotlin-stdlib-->
<groupId>org.jetbrains.kotlin</groupId>
Expand Down Expand Up @@ -191,12 +183,15 @@
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-logging</artifactId>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-otlp-logs</artifactId>
</dependency>

<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_httpserver</artifactId>
</dependency>

<dependency>
<groupId>io.jenkins.plugins</groupId>
<artifactId>okhttp-api</artifactId>
Expand All @@ -223,6 +218,10 @@
<artifactId>jsr305</artifactId>
<version>3.0.2</version>
</dependency>
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>jackson2-api</artifactId>
</dependency>
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>credentials</artifactId>
Expand Down Expand Up @@ -334,6 +333,21 @@
<version>1.1.19</version>
<optional>true</optional>
</dependency>
<dependency>
<!--
Use the old `org.elasticsearch.client:elasticsearch-rest-high-level-client` waiting for
`co.elastic.clients:elasticsearch-java` to fix https://github.com/elastic/elasticsearch-java/issues/163
-->
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>${elasticstack.version}</version>
<exclusions>
<exclusion>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch-cli</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk-testing</artifactId>
Expand Down Expand Up @@ -413,6 +427,11 @@
<artifactId>configuration-as-code</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>ssh-slaves</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.jenkins.configuration-as-code</groupId>
<artifactId>test-harness</artifactId>
Expand All @@ -426,12 +445,6 @@
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusion>
<!--
<exclusion>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>jackson2-api</artifactId>
</exclusion>
-->
<exclusion>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-json-org</artifactId>
Expand Down Expand Up @@ -460,6 +473,22 @@
<version>1.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<version>1.16.3</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<build>
<plugins>
Expand All @@ -470,6 +499,9 @@
<!-- JENKINS-50520: since we need a custom version of Guava -->
<maskClasses>com.google.common.</maskClasses>
<minimumJavaVersion>8</minimumJavaVersion>
<loggers>
<io.jenkins.plugins.opentelemetry.job.log>FINE</io.jenkins.plugins.opentelemetry.job.log>
</loggers>
</configuration>
</plugin>
<plugin>
Expand Down
7 changes: 0 additions & 7 deletions prometheus/prometheus.yml

This file was deleted.

26 changes: 0 additions & 26 deletions prometheus/run-prometheus.sh

This file was deleted.

Loading