Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot make statsd work #122

Open
larsskj opened this issue May 20, 2022 · 5 comments
Open

Cannot make statsd work #122

larsskj opened this issue May 20, 2022 · 5 comments
Labels
bug

Comments

@larsskj
Copy link

larsskj commented May 20, 2022

Jenkins and plugins versions report

Environment 
Jenkins: 2.348
OS: Linux - 5.10.104-linuxkit
---
ace-editor:1.1
ant:475.vf34069fef73c
antisamy-markup-formatter:2.7
apache-httpcomponents-client-4-api:4.5.13-1.0
bootstrap5-api:5.1.3-6
bouncycastle-api:2.26
branch-api:2.1046.v0ca_37783ecc5
build-timeout:1.20
caffeine-api:2.9.3-65.v6a_47d0f4d1fe
checks-api:1.7.4
cloudbees-folder:6.722.v8165b_a_cf25e9
cobertura:1.17
code-coverage-api:2.0.4
command-launcher:81.v9c2cb_cb_db_392
credentials:1126.ve05618c41e62
credentials-binding:523.vd859a_4b_122e6
data-tables-api:1.11.4-4
display-url-api:2.3.6
durable-task:496.va67c6f9eefa7
echarts-api:5.3.2-1
email-ext:2.88
font-awesome-api:6.0.0-1
forensics-api:1.13.0
git:4.11.3
git-client:3.11.0
git-server:1.11
github:1.34.3
github-api:1.303-400.v35c2d8258028
github-autostatus:3.6.2
github-branch-source:1628.vb_2f51293cb_78
gradle:1.38
handlebars:3.0.8
jackson2-api:2.13.3-285.vc03c0256d517
jacoco:3.3.2
javax-activation-api:1.2.0-3
javax-mail-api:1.6.2-6
jaxb:2.3.6-1
jdk-tool:1.5
jjwt-api:0.11.2-71.v2722b_b_06a_2a_f
jquery3-api:3.6.0-3
jsch:0.1.55.2
junit:1.63
ldap:2.9
mailer:414.vcc4c33714601
matrix-auth:3.1.2
matrix-project:771.v574584b_39e60
momentjs:1.1.1
okhttp-api:4.9.3-105.vb96869f8ac3a
pam-auth:1.8
pipeline-build-step:2.18
pipeline-github-lib:36.v4c01db_ca_ed16
pipeline-graph-analysis:195.v5812d95a_a_2f9
pipeline-input-step:448.v37cea_9a_10a_70
pipeline-milestone-step:101.vd572fef9d926
pipeline-model-api:2.2081.v3919681ffc1e
pipeline-model-definition:2.2081.v3919681ffc1e
pipeline-model-extensions:2.2081.v3919681ffc1e
pipeline-rest-api:2.24
pipeline-stage-step:293.v200037eefcd5
pipeline-stage-tags-metadata:2.2081.v3919681ffc1e
pipeline-stage-view:2.24
plain-credentials:1.8
plugin-util-api:2.16.0
popper2-api:2.11.5-1
resource-disposer:0.19
scm-api:608.vfa_f971c5a_a_e9
script-security:1172.v35f6a_0b_8207e
snakeyaml-api:1.30.1
ssh-credentials:277.v95c2fec1c047
ssh-slaves:1.814.vc82988f54b_10
sshd:3.228.v4c9f9e652c86
structs:318.va_f3ccb_729b_71
timestamper:1.17
token-macro:293.v283932a_0a_b_49
trilead-api:1.57.v6e90e07157e1
variant:1.4
workflow-aggregator:578.vf9a_f99755f4a_
workflow-api:1153.vb_912c0e47fb_a_
workflow-basic-steps:948.v2c72a_091b_b_68
workflow-cps:2692.v76b_089ccd026
workflow-cps-global-lib:581.ve633085a_8a_87
workflow-durable-task-step:1139.v252a_e12e8463
workflow-job:1181.va_25d15548158
workflow-multibranch:712.vc169a_1387405
workflow-scm-step:400.v6b_89a_1317c9a_
workflow-step-api:625.vd896b_f445a_f8
workflow-support:820.vd1a_6cc65ef33
ws-cleanup:0.42

What Operating System are you using (both controller, and any agents involved in the problem)?

I'm trying to make a test setup working with statsd/graphite by running the official Jenkins Docker image and the docker-graphite-statsd image. Everything updated to latest version as of writing.

Reproduction steps

I've created a testjob in Jenkins, and according to the Jenkins log, the plugin submits stats as expected:

New StatsD client created. 768177101
May 20, 2022 1:28:32 PM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$2 run
Logging value 1 for key pipeline.kilroy.stage.greetings.status.completedsuccess
May 20, 2022 1:28:32 PM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$3 run
Logging duration 392 for key pipeline.kilroy.stage.greetings.duration
May 20, 2022 1:28:33 PM INFO org.jenkinsci.plugins.workflow.job.WorkflowRun finish
Kilroy #6 completed: SUCCESS
May 20, 2022 1:28:33 PM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$2 run
Logging value 1 for key pipeline.kilroy.job.status.completedsuccess
May 20, 2022 1:28:33 PM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$3 run
Logging duration 3749 for key pipeline.kilroy.job.duration
May 20, 2022 1:28:33 PM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$3 run
Logging duration 0 for key pipeline.kilroy.job.blocked_duration

The stats doesn't show up in Graphite, though.

Plugin configuration points to the Docker IP of the statsd container - 172.17.0.2 in this case.

Unfortunately, the official Jenkins Docker image doesn't include the nc utility, but I've tested from other Docker containers on the same Docker daemon, and with these the statsd/graphite combo works fine.

Expected Results

Stats logged in statsd/graphite.

Actual Results

No data logged.

Anything else?

No response
@larsskj larsskj added the bug label May 20, 2022
@jeffpearce
Copy link
Contributor

jeffpearce commented May 20, 2022 via email

@larsskj
Copy link
Author

larsskj commented May 23, 2022

Sorry for the late reply: My family required my presence during the weekend.

Hm - I'm afraid I don't have much more than shown above: The standard Jenkins Docker image doesn't produce a lot of logs.

Shown here is a snippet of the current system log - <jenkins-url>/log/all - with Jenkins left idle for several hours and then a single build executed. There is no more information available, I believe - the same log is returned using docker logs.

May 23, 2022 10:05:56 AM WARNING hudson.security.csrf.CrumbFilter doFilter

Found invalid crumb 8f88b53a418191cfeaa0a7da50a03b1bd8f324ead74dc7d598288e5d0d6fd52f. If you are calling this URL with a script, please use the API Token instead. More information: https://www.jenkins.io/redirect/crumb-cannot-be-used-for-script

May 23, 2022 10:05:56 AM WARNING hudson.security.csrf.CrumbFilter doFilter

No valid crumb was included in request for /ajaxExecutors by admin. Returning 403.

May 23, 2022 10:06:01 AM WARNING hudson.security.csrf.CrumbFilter doFilter

Found invalid crumb 8f88b53a418191cfeaa0a7da50a03b1bd8f324ead74dc7d598288e5d0d6fd52f. If you are calling this URL with a script, please use the API Token instead. More information: https://www.jenkins.io/redirect/crumb-cannot-be-used-for-script

May 23, 2022 10:06:01 AM WARNING hudson.security.csrf.CrumbFilter doFilter

No valid crumb was included in request for /ajaxBuildQueue by admin. Returning 403.

May 23, 2022 10:06:01 AM WARNING hudson.security.csrf.CrumbFilter doFilter

Found invalid crumb 8f88b53a418191cfeaa0a7da50a03b1bd8f324ead74dc7d598288e5d0d6fd52f. If you are calling this URL with a script, please use the API Token instead. More information: https://www.jenkins.io/redirect/crumb-cannot-be-used-for-script

May 23, 2022 10:06:01 AM WARNING hudson.security.csrf.CrumbFilter doFilter

No valid crumb was included in request for /ajaxExecutors by admin. Returning 403.

May 23, 2022 10:07:37 AM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$2 run

Logging value 1 for key pipeline.kilroy.stage.greetings.status.completedsuccess

May 23, 2022 10:07:37 AM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$3 run

Logging duration 476 for key pipeline.kilroy.stage.greetings.duration

May 23, 2022 10:07:38 AM INFO org.jenkinsci.plugins.workflow.job.WorkflowRun finish

Kilroy #7 completed: SUCCESS

May 23, 2022 10:07:38 AM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$2 run

Logging value 1 for key pipeline.kilroy.job.status.completedsuccess

May 23, 2022 10:07:38 AM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$3 run

Logging duration 3625 for key pipeline.kilroy.job.duration

May 23, 2022 10:07:38 AM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$3 run

Logging duration 0 for key pipeline.kilroy.job.blocked_duration

The crumb error is repeated over and over, I'm not sure whether this is related.

@larsskj
Copy link
Author

larsskj commented May 23, 2022

Just to see something work, I tried adding an InfluxDB to the setup as well.

Unfortunately, neither does this seem to work: The plugin config recommends InfluxDB version 2 - but it seems it attempts to write to Influx using version 1 format even though config is set to version 2?

Could not write to InfluxDB - HTTP/1.1 401 Unauthorized

May 23, 2022 12:36:24 PM WARNING org.jenkinsci.plugins.githubautostatus.notifiers.InfluxDbNotifier log

InfluxDB URL - http://172.17.0.5:8086/write?db=MYDB

May 23, 2022 12:36:24 PM WARNING org.jenkinsci.plugins.githubautostatus.notifiers.InfluxDbNotifier log

Series - stage,owner=none,repo=Kilroy,stagename=Greetings,result=CompletedSuccess jobname="Kilroy#10",branch="none",stagetime=436,passed=1,buildurl="job/Kilroy/10/",buildnumber=10,trigger="Started by user Administrator"

May 23, 2022 12:36:24 PM WARNING org.jenkinsci.plugins.githubautostatus.notifiers.InfluxDbNotifier log

{"code":"unauthorized","message":"Unauthorized"}

May 23, 2022 12:36:24 PM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$2 run

Logging value 1 for key pipeline.kilroy.stage.greetings.status.completedsuccess

May 23, 2022 12:36:24 PM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$3 run

Logging duration 436 for key pipeline.kilroy.stage.greetings.duration

May 23, 2022 12:36:25 PM INFO org.jenkinsci.plugins.workflow.job.WorkflowRun finish

Kilroy #10 completed: SUCCESS

May 23, 2022 12:36:25 PM WARNING org.jenkinsci.plugins.githubautostatus.notifiers.InfluxDbNotifier log

Could not write to InfluxDB - HTTP/1.1 401 Unauthorized

May 23, 2022 12:36:25 PM WARNING org.jenkinsci.plugins.githubautostatus.notifiers.InfluxDbNotifier log

InfluxDB URL - http://172.17.0.5:8086/write?db=MYDB

May 23, 2022 12:36:25 PM WARNING org.jenkinsci.plugins.githubautostatus.notifiers.InfluxDbNotifier log

Series - job,owner=none,repo=Kilroy,result=CompletedSuccess jobname="Kilroy#10",branch="none",blocked=0,jobtime=2266,blockedtime=0,passed=1,buildurl="job/Kilroy/10/",buildnumber=10,trigger="Started by user Administrator"

May 23, 2022 12:36:25 PM WARNING org.jenkinsci.plugins.githubautostatus.notifiers.InfluxDbNotifier log

{"code":"unauthorized","message":"Unauthorized"}

May 23, 2022 12:36:25 PM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$2 run

Logging value 1 for key pipeline.kilroy.job.status.completedsuccess

May 23, 2022 12:36:25 PM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$3 run

Logging duration 2266 for key pipeline.kilroy.job.duration

May 23, 2022 12:36:25 PM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient$3 run

Logging duration 0 for key pipeline.kilroy.job.blocked_duration

May 23, 2022 12:40:45 PM INFO org.jenkinsci.plugins.githubautostatus.StatsdClient newClient

New StatsD client created. 161850950

@larsskj
Copy link
Author

larsskj commented May 23, 2022

This probably deserves a bug on its own?

@larsskj
Copy link
Author

larsskj commented May 23, 2022

Just to add something positive: Using InfluxDB 1.8 works.

@larsskj larsskj mentioned this issue May 24, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jeffpearce @larsskj